Occasionally, I get to do something that I haven’t done in such a long time, that I don’t know how to do it using my current toolset.

Such is an example of kernel debugging.

Kernel debugging isn’t entirely new to me, but also one of those things that I do so infrequently that it takes a little bit of ramp up time to get back in the swing of it. Additionally, I had since moved to OSX. My objective was to diagnose a GINA problem with a component. GINA hooks allow you to add customization to the Windows Login screen. In Windows Vista+, the API changed to a Credential Provider, but this particular case was Windows XP. GINA hooks run in kernel space, so we need to attach a kernel debugger.

We need to troubleshoot this problem as Windows is starting up, so we need another machine. Typically the way I’ve done this before is to have two workstations hooked together via a Serial Port. The Kernel debugger works across a serial port.

My first problem was, I don’t have a physical workstation anymore. All of my Windows work is done in Virtual Machines now. Fortunately, what I discovered is that Parallels for Mac can do that just fine. Like the workstations, we will need two virtual machines: the debugger and the debugee.

My debugee is Windows XP SP3.

In Parallels, open up the Configuration for the Virtual Machine and switch to the Hardware Tab. Click the “+” to add a serial port. For the Source, select “New Socket”, and give your socket a name. Set the Mode to “Server” and then you are all set. Make sure the “connected” checkbox is checked as well.

That’s all it takes from Parallels for the serial port. Now would be a good time to start the debugee to enable debugging.

From here, it’s all pretty straight forward as if you were doing it on a real machine. Open up “msconfig” go to BOOT.INI, check “/DEBUG”, and set the debug port to COM1 and the baud rate to 115200. Click OK, and OK again. It may prompt you to restart, but lets not do that yet. Shut it down for now.

Now for the debugger. I’m using Windows 7 x64 for debugging, and WinDbg as my debugger tool. Before we get there, we need to add a serial port to the debugger Virtual Machine.

Shut down the debugger instance, open up the VM configuration and go to the “Hardware” tag. Click “+” and add a Serial Port. For the source, select the port that you created for the debugee instance, and set the mode to “Client”, and make sure it’s Connected.

Start the debugger instance, and get WinDbg fired up. Make sure to get your symbol source fixed up, and start Kernel debugging over COM1.

Once it’s waiting, go ahead and start the debugee operating system.

Success! We have a kernel debugging session across virtual machines in Parallels.