Hi there!
Calling conventions describe how, at the assembly level, one routine must call another. For example, how should function parameters be passed? By putting them in registers or pushing them in the stack? In which order? For C/C++ programmers, this is usually not a problem since the compiler takes care of it. But if your try to interface some assembly code with C routines, you must now at least one calling convention: the cdecl.
Cdecl is the C calling convention for x86 architecture and is the default for most compilers. Let’s describe it a little with an example. I’ll bee using two routines: Foo() an Bar(). Bar() takes 3 integers as arguments. The following C code will bee our road map and I’ll be showing the assembly equivalent (using the NASM syntax, what else?).
int Bar(int a, int b, int c);
void Foo(void)
{
/* Some stuff here */
Bar(42, 21, 84);
}
int Bar(int a, int b, int c)
{
int loc;
/* Some stuff here */
return 1337;
}
The return value will be stored in the eax register so the caller first have to push it’s current value.
Foo:
push eax
The caller pushes parameters in reverse order.
push 84
push 21
push 42
The caller calls the routine. Doing the call will push the return address (current eip) on the stack.
call Bar
The callee sets up a new stack frame. This is done by saving the ebp register and then setting it with the current content of the esp register.
Bar:
push ebp
mov ebp, esp
The callee saves any register that will be used later by pushing their values on the stack.
push ebx
The callee allocates room on the stack for local variables. This is done by decrementing the esp register.
sub esp, 4
The callee does what it have to do. Here's a diagram of what the stack looks like by now:
The callee stores the return value in the eax register.
mov eax, 1337
The callee releases allocated space on the stack by incrementing the esp register.
add esp, 4
The callee restores the registers content, including the ebp register.
pop ebx
pop ebp
The callee returns (this will pop the old value of eip).
ret
The caller must clean up the stack (i.e, remove the parameters by incrementing esp).
add esp, 12
By doing this correctly, you can use assembly routines in your C code or the opposite. Here's the final assembly code:
Foo:
; Some stuff here
push eax
push 84
push 21
push 42
call Bar
add esp, 12
; And continue what it was doing, return value stored in eax
Bar:
push ebp
mov ebp, esp
push ebx
sub esp, 4
; Some stuff here
mov eax, 1337
add esp, 4
pop ebx
pop ebp
ret
Other x86 conventions are used, like the stdcall convention (used, for example, by the Windows 32 API), the syscall a.k.a. pascal convention (Linux system calls, …), the thiscall convention (in C++ when calling an object’s member function), the non-standard fastcall convention, ...
Hopes this brings you some help ;)
