A Walk To Design An Anti-Debug Technique

In this post, i will try to design an anti-debug technique targeting the debugged process heap behavior. Most of us know that process heaps under debuggers are different from those under normal execution.

For example:
1) Debugged heaps are filled with byte patterns like 0xbaadf00d and 0xfeeefeee, especially those allocated by the "RtlAllocateHeap" function variations e.g. the "LocalAlloc" and "HeapAlloc" functions.

N.B. Memory allocated by the "malloc" function is, in most cases, filled with 0xCDCDCDCD even under normal execution.

2) The Lookaside Lists. For more info about the Lookaside lists, i advise you to read this paper (Practical Windows XP/2003 Heap Exploitation by John McDonald and Chris Valasek).

The Lookaside lists don't exist for the debugged heap when:
1) The "_NO_DEBUG_HEAP" environment variable is not present or not set to 1.
2) The "DisableHeapLookAside" registry value found under "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\xxx.exe" is set to 1, where xxx.exe is the executable file name.

So, if my code traverses all heaps under the current process and for each heap, the code traverses the lookaside lists to determine the number of free blocks, the total number of free blocks can indicate the presence of a debugger.

If the total number of free blocks is zero, then the process is being debugged or the registry value is "DisableHeapLookAside" set. and here is the code

• struct LOOKASIDE
• {
•     LOOKASIDE* pNext;
•     unsigned long* A,B,C,D,E,F,G,H,I,J,K;
• };

• unsigned long* GetHeaps(unsigned long* pNum)
• {
• unsigned long* x;
•          __asm
•         {
•                  mov eax,dword ptr fs:[0x30]
•                  mov x,eax
•         }
•         *pNum=*(x+0x22);
•         return (unsigned long*)*(x+0x24);
• }
• LOOKASIDE* GetLookaside(unsigned long* HeapBase)
• {
•     return (LOOKASIDE*)(*(unsigned long*)((unsigned char*)HeapBase+0x580));
• }

• //return no. of blocks in lookaside lists of all process heaps
• int calc_lookaside()
• {
•     unsigned long c;
•     unsigned long* p=GetHeaps(&c);
•     LOOKASIDE* px;
•     int total=0;
•     while(c--)
•     {
•         if(px=GetLookaside((unsigned long*)*p++))
•         {
•             px+=2;
•             for(int i=2;i<128;i++)
•             {
•                 int tot=0;
•                 LOOKASIDE* pxx=px;
•                 while(pxx->pNext)
•                 {
•                     tot++;
•                     pxx=pxx->pNext;
•                 }
•                 total+=(tot*i);
•                 px++;
•             }
•         }
•     }
•     return total;
• }

• void main()
• {
•     if(!calc_lookaside())
•     {
•         MessageBox(0,"Debugger present",0,0);
•         goto end;
•     }
•     MessageBox(0,"No Debugger",0,0);
• end:
•     Sleep(10000);
• }
•  

N.B. The trick was designed for Windows XP (32Bit).
 
N.B. OllyAdvanced is aware of this trick.

Final word, this article is just one step towards an anti-debug technique.

Many thanks to my friend, Amr Thabet, for his note about the "malloc" function behavior.