Everyone who spent a short while playing with debuggers knows very well what Step into and Step over mean.
Step into is used to execute one instruction at a time, giving the chance to check registers, stack, etc, it is something called Single Stepping. Step over does the same job except when it comes to call or rep instructions, as Step over does not single step them, it simply jumps over but still executes them. How is that accomplished?
The answer is definitely software breakpoints. When you Step over a call instruction, OllyDbg or any other debugger places an int3 software breakpoint just after the call instruction. In other words, Step over is exactly the same as manually pressing F2 just after the call instruction and then pressing F9.
After we have seen how debuggers implement "Step over", how can we use this for our advantage?
We can easily use this fact as an anti-debug trick by inserting a small macro in the prologue of each function in our application. The macro should extract the return address from the stack and check the first byte pointed to by the return address. If it is 0xCC, it means that someone is trying to step over the protected function and it should escape execution as such.
Walied, Stop talking. Show me some damn code!
In OllyDbg, if we step over func1, func2, or func3, we will get this error message.
You can imagine how ridiculous it could be for a novice reverse engineer to debug your application as he or she is less likely to step into every single procedure in the application.
The demo mentioned above can be found here. You can also find its source code here.
An example demonstrating how Step over rep movsb can be used as anti-debug trick can be found here.
The interesting thing about rep instructions e.g. rep movsb is that reverse engineers usually step over them since rep movsb instructions often involve copying long strings or large chunks of data.
Everything mentioned in this post has been tested on OllyDbg v1.10, but as far as i know, it should work anywhere else.
You can follow me on Twitter @waleedassar
Step into is used to execute one instruction at a time, giving the chance to check registers, stack, etc, it is something called Single Stepping. Step over does the same job except when it comes to call or rep instructions, as Step over does not single step them, it simply jumps over but still executes them. How is that accomplished?
The answer is definitely software breakpoints. When you Step over a call instruction, OllyDbg or any other debugger places an int3 software breakpoint just after the call instruction. In other words, Step over is exactly the same as manually pressing F2 just after the call instruction and then pressing F9.
After we have seen how debuggers implement "Step over", how can we use this for our advantage?
We can easily use this fact as an anti-debug trick by inserting a small macro in the prologue of each function in our application. The macro should extract the return address from the stack and check the first byte pointed to by the return address. If it is 0xCC, it means that someone is trying to step over the protected function and it should escape execution as such.
Walied, Stop talking. Show me some damn code!
If we compile the code in the image above and load the resulting executable in OllyDbg, we will get something like this.
You can imagine how ridiculous it could be for a novice reverse engineer to debug your application as he or she is less likely to step into every single procedure in the application.
The demo mentioned above can be found here. You can also find its source code here.
An example demonstrating how Step over rep movsb can be used as anti-debug trick can be found here.
The interesting thing about rep instructions e.g. rep movsb is that reverse engineers usually step over them since rep movsb instructions often involve copying long strings or large chunks of data.
Everything mentioned in this post has been tested on OllyDbg v1.10, but as far as i know, it should work anywhere else.
You can follow me on Twitter @waleedassar
