The Reverse Engineering Mindset – Is it Possible to Suspend a Kernel Thread from the Debugger?

Hello friends - I’m still compiling content for the upcoming REP series but I have an “interrupt to service" before the REP series drops on INT 3. Last week a member on my team was in a pinch which resulted in a thought-provoking team discussion over email. This is an outstanding way to start the morning.  His question was, “ I’m live debugging a system using WinDbg. Is it possible to suspend a kernel thread directly from the debugger?” Surprisingly a quick search on the web showed other developer types asking the same question with the absence of any reasonable answers. The lack of any solution motivated me to post a video response on INT 3.

The short answer is - “no”. The long answer is – “yes (a pseudo-suspend), with ebfe”. I’ll explain.

Video: This is an edge case scenario but it provides an opportunity to walk through fundamental reverse engineering techniques. In the video we'll attach to a Hyper-V virtual machine using Windbg, set a breakpoint in the debugger, and step through code with the end goal of hot patching the assembly with the ebfe opcode. If you enjoy the content, please join our INT 3 YouTube channel.

The scenario my teammate is debugging involves a reference count problem on a kernel object. One application makes a driver call to look for the object in the kernel. If the object is instantiated, the reference count is incremented. If the reference count drops to zero, the Object Manager destroys the object as you would expect. The problem is a second application utilizing the services from the same kernel object is concurrently decrementing the object’s reference count. This causes the object to be destroyed while the first application is using the object.

His intent was to test various behavioral patterns of the kernel code and he wanted to suspend the first application’s thread while it was in the kernel, prior to the reference count increment operation. While this single thread was suspended, he planned to terminate the second application before the decrement operation, and then resume the suspended thread from the first application.

He is debugging with a Debug Crash Cart. The “cart” consists of a Target computer and a second computer referred to as the Host where the Debugging Tools for Windows are installed. The two systems are attached via an Ethernet cable but there are other options such as a null-modem cable, 1394 cable, or a special USB cable. I discussed configuring this type of setup in an old article located here - https://blogs.msdn.microsoft.com/ntdebugging/2013/01/31/how-to-setup-a-debug-crash-cart-to-prevent-your-server-from-flat-lining

Warning: Only Do This in a Pinch
This technique is useful when you find yourself in the proverbial pinch on a lab machine but this is not intended to be used for thread synchronization on a production machine. If this target thread owns a synchronization object, such as a mutex or ERESOURCE, it can lead to blocking if a calling thread tries to obtain a synchronization object owned by the thread we’re hot patching. Also the target thread will spin in a tight loop creating implications if the IRQL is at DISPATCH level or above. I don’t recommend this technique on a production system but it conveys the “reverse engineering mindset”.

Here are the steps we used:
1. We used the ‘eb’ Windbg command to hot patch the assembly for the target thread with a ebfe opcode. This ‘JMP to self-opcode’ causes the target thread to loop forever. Make note of the original code so it can be replaced later.
2. Then we typed ‘g’ in the debugger command window to continue execution of the system. The target thread will loop on the one instruction.
3. Do you dirty work (e.g. kill the second application, test your scenario).
4. Then break in to the debugger to replace the JMP instruction with the original opcode.
5. Hit g and you're on your way.

-Ron