Saturday, October 27, 2012

Virtual PC vs. Resume Flag

In this post i will show you another weird behavior of Virtual PC 2007. I encountered this weird behavior while playing with Virtual PC 2007 with Windows XP SP3 installed inside. The behavior is all about how a Windows XP Virtual PC virtual machine handles the Resume Flag.

For those who don't know, the Resume Flag (Flag no. 16 in the EFLAGS register) is used to temporarily disable Hardware Breakpoints exceptions for one instruction. Without it, a Hardware-Breakpoint-On-Execution would infinitely trigger an EXCEPTION_SINGLE_STEP exception.

According to @osxreverser, Windows XP does not support the Resume Flag (RF). I was also amazed to see that also WinDbg and OllyDbg v1.10 don't use the resume flag. They use the Trap Flag (TF) instead.

Running a simple executable that on purpose makes use of the Resume Flag inside an XP Virtual PC Virtual Machine, i found out that execution flows normally as if XP supports the resume flag.

Given the finding above, i created a small executable that tries to detect if it is running inside Virtual PC 2007.
You can find it here and its source code from here.

I guess the finding above only applies if the host operating system itself supports the resume flag e.g. Windows 7 or later.

N.B. This topic is still under research.

Please don't hesitate to leave a comment.
You can also follow me on Twitter @waleedassar

2 comments:

  1. If Olly isn't issuing the RF, then either it's clearing the breakpoint and restoring it after the step, or the kernel is issuing it on Olly's behalf. Otherwise, you'd see the infinite exception loop.
    btw RF is only for break-on-execute, it has no effect on break-on-access.

    ReplyDelete
  2. >>If Olly isn't issuing the RF, then either it's clearing
    >>the breakpoint and restoring it after the step, or the
    >>kernel is issuing it on Olly's behalf. Otherwise, you'd
    >>see the infinite exception loop.
    OllyDbg v1.10 does not use the RF flag at all, i can't see that.
    Once OllyDbg v1.10 receives the EXCEPTION_SINGLE_STEP and ensures it is due to a HARDWARE-BREAKPOINT-ON-EXECUTION, it clears the corresponding local flag (L0, L1, L2, or L3 bit) in DR7 for the debug register (DR0, DR1, DR2, or DR3) and sets the Trap Flag (TF). Once it receives TF's EXCEPTION_SINGLE_STEP exception, it re-enables (L0, L1, L2, or L3) in DR7.

    ReplyDelete