Wednesday, October 10, 2007

DebugActiveProcess and DEBUG_ONLY_THIS_PROCESS

Then attaching to a process to debug it the default debugging flags is DEBUG_ONLY_THIS_PROCESS, meaning that if the process calls CreateProcess the child process will not be debugged. But if the process is started by the debugger the process can be started with DEBUG_PROCESS meaning that both the new process and any child processes created will be debugged by the same debugger.

I have been locking for a way to change this DEBUG_ONLY_THIS_PROCESS flag since Application Inspector is able to attach to a process and I want it to monitor all child processes as well.

A couple of days ago I stumbled on a comment on a Microsoft descrition on that was new in Windows XP, talking about Dynamic Control over Debug-child Flag. The description did not describe how to change the flag so I had to investigate.

After some debugging of windbg and dbgeng.dll, that can change the setting using the command .childdbg I found the kernel call to do that I want. It's the undocumented NtSetInformationProcess that is able to change the debug flags.

Here is the code to set the flag.


//
// Process options.
//

// Indicates that the debuggee process should be
// automatically detached when the debugger exits.
// A debugger can explicitly detach on exit or this
// flag can be set so that detach occurs regardless
// of how the debugger exits.
// This is only supported on some system versions.
#define DEBUG_PROCESS_DETACH_ON_EXIT 0x00000001
// Indicates that processes created by the current
// process should not be debugged.
// Modifying this flag is only supported on some
// system versions.
#define DEBUG_PROCESS_ONLY_THIS_PROCESS 0x00000002

typedef enum _PROCESSINFOCLASS {
ProcessDebugFlags=31 // From ntddk.h
} PROCESSINFOCLASS;

typedef DWORD (CALLBACK * NTSETINFORMATIONPROCESS)(
IN HANDLE ProcessHandle,
IN PROCESSINFOCLASS ProcessInformationClass,
IN PVOID ProcessInformation,
IN ULONG ProcessInformationLength
);

static DWORD DebugSetProcessOptions(HANDLE hProcess,ULONG DebugFlags) {
static NTSETINFORMATIONPROCESS g_NtSetInformationProcess = NULL;

if (g_NtSetInformationProcess == NULL) {
HINSTANCE hNtDll;

hNtDll = LoadLibrary(_T("ntdll.dll"));
if (hNtDll == NULL) {
return GetLastError();
}

g_NtSetInformationProcess = (NTSETINFORMATIONPROCESS)GetProcAddress(hNtDll,"NtSetInformationProcess");
if (g_NtSetInformationProcess) {
return GetLastError();
}
}

return g_NtSetInformationProcess(hProcess,ProcessDebugFlags,&DebugFlags,sizeof(DebugFlags));
}