Netmon for Debugging

On the project I'm working on instead of working on my MUD (since I haven't felt inspired about it lately) I'm in need of doing some protocol analysis. Yeah, network protocol analysis. I bet lots of people swear by Wireshark, but my tool of choice is Netmon, since I've used so much of it at work. Netmon has great parsers that let me drill in deeply to all the protocols involved in my project. I'm developing a server, and want to capture network traffic to see if my implementation is correct.

One problem is that Netmon doesn't work well when capturing loopback traffic. I do a lot of my development on the bus, with limited or no Internet access, so I tend to start both the server and client on my laptop. How, then, can I get my traffic into Netmon to use its cool interface to analyze it?

The method is pretty simple, really. There are three steps: have my code dump traffic to a file; consolidate those files into a Netmon capture file; and add to Netmon's parsers to recognize my new format.

Logging my traffic to files is pretty straightforward. In my case, I want to separate out the "messages", so I log only one per file. I order the files with a unique number, plus an indicator about whether it is a read from the network or a write to the network."

Then I wrote a program to read in each file and write it into the Netmon capture format instead. Netmon provides a really great API set, which has a reference included in the free download of Netmon.

This program was pretty straightforward. I created a capture file with NmCreateCaptureFile, read each file in turn into a buffer, and added it using NmBuildRawFrameFromBuffer and NmAddFrame.

I added a little "special sauce", if you will. Or if you won't. Whatever. In Netmon, the "frame parser" is the top-level parser invoked on every frame. It multiplexes the contents of the frame to different sub-parsers (Ethernet, Wifi, etc.) based on the NDIS_MEDIUM MediaType value stored in the frame (which is a parameter to NmBuildRawFrameFromBuffer). I don't want my frames to be parsed as Ethernet or Wifi or whatever, so I supplied my own value for MediaType.

We're now getting to the part about making Netmon recognize my format. The Frame parser has a big switch statement specifying how to parse the body of the Frame according to MediaType. I just need to insert a clause for my custom media type.

// in frame.npl
switch(MediaType)
{
    case 0:
    case 1:
        ETHERNET Ethernet;
    case 6:
        WiFi WiFi;
    case 0xFFFC:
        PayloadHeader PayloadHeader;
    case 2:
        TOKENRING TokenRing;
...
    case 0xFFFB:
        NetworkInfoEx NetworkInfoEx;

    case 0x2323:
        MungeTLS MungeTLS;
}

This works, and multiplexes to my custom parser, which I will show momentarily. But first, note that this method requires modifying or overriding the built-in frame parser in frame.npl. That's not very nice. The good people on the Netmon team must have had the exact same thought, because they added support into the parser language to insert case clauses into the parent parser's code to redirect it to a subparser--exactly what I did here, but in a non-invasive way!

[ RegisterAfter(Frame.NetworkInfoEx, MungeTLS, 0x2323) ]
Protocol MungeTLS
{
...

The first parameter to RegisterAfter is the clause in the switch statement after which the clause for handling MungeTLS should be inserted. The second parameter is the name of the field/variable to be inserted into the parent structure (the data type is implied by the protocol definition that immediately follows). The last parameter is the value to be used in the case clause to trigger redirecting to this sub-parser.

The model is a little specific, but there is a prescribed way to write these parsers, and it works very well.

Finally we get to the description of my actual "protocol", which I have lovingly called MungeTLS, and which I will expound on more in a later blog post. The main feature is that it wraps a proper TLS message and attaches a little metadata to the front. So far, the only metadata it has is the "directionality" of the frame (which of course is indicated by the "r" and "w" suffixes on the filenames I mentioned early in this post).

When my program generates the Netmon capture from the set of files, it writes the "direction" of the traffic into an extra first byte in the frame, which is parsed by the MungeTLS Netmon parser and used to indicate it in the UI.

const MediumMungeTLS = 0x2323;
const Flags_Receive = 0x0;
const Flags_Send = 0x1;
const Flags_Mask_SendReceive = 0x1;

[ RegisterAfter(Frame.NetworkInfoEx, MungeTLS, MediumMungeTLS) ]
Protocol MungeTLS
{
    [
      Property.Source = MTLS_Source(Flags & Flags_Mask_SendReceive),
      Property.Destination = MTLS_Destination(Flags & Flags_Mask_SendReceive)
    ]
    UINT8 Flags
    {
        UINT8 Unused:7;
        UINT8 SendReceive:1;
    }

    TLS InnerTLS;
}

Table MTLS_Source
{
    switch (value)
    {
        case Flags_Receive : "Client";
        case Flags_Send : "Server";
    }
}

Table MTLS_Destination
{
    switch (value)
    {
        case Flags_Receive : "Server";
        case Flags_Send : "Client";
    }
}

The Source and Destination properties are shown in a special place in the frame summary window in Netmon, usually occupied by things like the source/dest IP or MAC address. In this case, the strings "Client" and "Server" are shown instead, which is good enough for me!