I was recently somewhat surprised to find that there is really no C++ way to resolve a virtual function to its address at run-time. Admittedly, there is no good reason why anybody would morally need to do this, but when you’ve already lowered yourself to patching another process’s own code without consent, it seems like a very small crime.

Pioneers of such hackery have already established concrete methods for calling virtual functions from inline assembly, but these methods don’t quite stretch to getting the address in pointer form. So, if for no reason other than to convince you that it’s a lot of hassle, I present a miserable bit-chop hack to do just this.

Before you try to compile this, and crash your computer, here’s the health-and-safety information:

• Again, 32-bit only.
• I’ve only run tests on Microsoft Visual C++ 2005 Express Edition, but it’s not unrealistic that other MS compilers would produce the same code for virtual-table invocation. If you aren’t scared of your debugger, it could be adapted to any other compiler, in theory.
• We rely on a function returning with a misaligned stack, so debug builds will get nasty unless runtime stack-frame checks are disabled (from Properties/Configuration Properties/C++/Code Generation/Basic Runtime Checks).
• There isn’t really an easy way to parametrise this, what with C++’s strong-typing, so you’ll probably have to compile it once for each virtual function needing resolution.
• Excuse all the ‘__asm JMP’ing (I’m not pretending not to use goto), but C++’s usual flow control is often sugared in ways that would cost some of what little portability is left.
• This may have been better written in assembly alone, but people tend to find that a lot more intimidating than C++. Admittedly, there’s an obvious juxtaposition between the nature of the task and its implementation (C++ casts and all) but if you have to dance with the devil, it’s best to sing to your own tune.

typedef unsigned char uchar;
 
void* GetAddressOfVirtualFunction() {
	unsigned long call_size = 2;
	unsigned long stack_pointer = 0;
	unsigned long call_ptr = 0;
	void* address = NULL;
 
	__asm MOV stack_pointer, ESP
	__asm JMP skip_call;
do_call:
 
	//// Put a single call to the virtual function here 
	some_object->SomeVirtualFunction();
	//// and nothing else
 
	__asm JMP done_call;
skip_call:
	__asm MOV EAX, skip_call
	__asm MOV call_ptr, EAX
	call_ptr -= (call_size + 2);
	uchar call_opcode = *reinterpret_cast<uchar>(call_ptr);
	uchar call_register = *reinterpret_cast<uchar>(call_ptr + 1);
 
	if (call_opcode != 0xFF) {
		// Whatever we've just found isn't a CALL <Register>
		__asm JMP failed;
	}
 
	DWORD old_protect = 0;
	DWORD new_protect = PAGE_EXECUTE_READWRITE;
	if (VirtualProtect(reinterpret_cast&lt;void*&gt; (call_ptr), 
		2, 
		new_protect, 
		&old_protect) == FALSE) 
	{
		return NULL;
	}
 
 
	*reinterpret_cast&lt;unsigned short*&gt; (call_ptr) = 0x9090;
	__asm JMP do_call;
 
done_call:
	__asm PUSHAD
	switch (call_register) {
		case 0xD0:
			// EAX
			__asm POPAD
			__asm MOV address, EAX
			break;
		case 0xD1:
			// ECX
			__asm POPAD
			__asm MOV address, ECX
			break;
		case 0xD2:
			// EDX
			__asm POPAD
			__asm MOV address, EDX
			break;
		case 0xD3:
			// EBX
			__asm POPAD
			__asm MOV address, EBX
			break;
		default:
			// Unexpected register - fail
			address = NULL;
	}
 
	*reinterpret_cast&lt;uchar*&gt; (call_ptr) = call_opcode;
	*reinterpret_cast&lt;uchar*&gt; (call_ptr + 1) = call_register;
 
	VirtualProtect(reinterpret_cast&lt;void*&gt; (call_ptr), 
		2, 
		old_protect, 
		&new_protect);
 
	// Succeeded
	__asm MOV ESP, stack_pointer
	return address;
 
	// Failed
failed:
	__asm MOV ESP, stack_pointer
	return NULL;
}

If you want to see exactly how this works, then it would be quickest for you to trace through it in a debugger, but here’s the gist:

That call to some_object->SomeVirtualFunction in the middle of the snippet compiles to a vTable lookup that puts the address in one of the four general-purpose registers, and then calls it from there. Our code jumps over that call (so it knows exactly how far to track back) and locates the final CALL R32 instruction. In x86, this is two bytes long, and looks like FF Dx. The value of the second byte determines the register that is being called. So after a VirtualProtect, we patch over that call with a pair of NOPs and let the whole thing run. The virtual function is never actually called (so any arguments you provide need only fool the compiler). By the time we reach done_call, one of the four registers contains the address of the virtual function. We use a switch statement to find out which one, and extract the address to a safe location accordingly. Another quick patch and a call to VirtualProtect returns the memory to its initial state, whence we return.

You’ll notice that some magic is being done with the ESP. This is just to ensure that we don’t corrupt the stack, after bailing out of the function-call at the last moment. It is best done this way as we can’t easily determine the number of parameters pushed onto the stack (and never popped off, as virtual functions use the thiscall convention, similar to stdcall).

So if you haven’t already vomited by the point, it should be pretty clear that this code won’t port very far at all. For this reason, you’re strongly advised to keep it well away from anything that needs to me maintained. On the other hand, if you manage to get it to compile and run then the resulting executable is safe for general consumption, as all the liabilities are all ousted during linking.

Posted: Thursday, November 29th, 2007, 9:00 am
Filed under: Game Programming, RCE.