The cdecl calling convention

Hi there!

Calling conventions describe how, at the assembly level, one routine must call another. For example, how should function parameters be passed? By putting them in registers or pushing them in the stack? In which order? For C/C++ programmers, this is usually not a problem since the compiler takes care of it. But if your try to interface some assembly code with C routines, you must now at least one calling convention: the cdecl.

Cdecl is the C calling convention for x86 architecture and is the default for most compilers. Let’s describe it a little with an example. I’ll bee using two routines: Foo() an Bar(). Bar() takes 3 integers as arguments. The following C code will bee our road map and I’ll be showing the assembly equivalent (using the NASM syntax, what else?).

int Bar(int a, int b, int c);

void Foo(void)
{
  /* Some stuff here */
  Bar(42, 21, 84);
}

int Bar(int a, int b, int c)
{
  int loc;

  /* Some stuff here */
  return 1337;
}

The return value will be stored in the eax register so the caller first have to push it’s current value.

Foo:
    push eax

The caller pushes parameters in reverse order.

push 84
push 21
push 42

The caller calls the routine. Doing the call will push the return address (current eip) on the stack.

call Bar

The callee sets up a new stack frame. This is done by saving the ebp register and then setting it with the current content of the esp register.

Bar:
    push ebp
    mov ebp, esp

The callee saves any register that will be used later by pushing their values on the stack.

push ebx

The callee allocates room on the stack for local variables. This is done by decrementing the esp register.

sub esp, 4

The callee does what it have to do. Here's a diagram of what the stack looks like by now:

The callee stores the return value in the eax register.

mov eax, 1337

The callee releases allocated space on the stack by incrementing the esp register.

add esp, 4

The callee restores the registers content, including the ebp register.

pop ebx
pop ebp

The callee returns (this will pop the old value of eip).

ret

The caller must clean up the stack (i.e, remove the parameters by incrementing esp).

add esp, 12

By doing this correctly, you can use assembly routines in your C code or the opposite. Here's the final assembly code:

Foo:
; Some stuff here
    push eax

    push 84
    push 21
    push 42
    call Bar
    add esp, 12
; And continue what it was doing, return value stored in eax

Bar:
    push ebp
    mov ebp, esp

    push ebx
    sub esp, 4

; Some stuff here

    mov eax, 1337
    add esp, 4

    pop ebx
    pop ebp
    ret

Other x86 conventions are used, like the stdcall convention (used, for example, by the Windows 32 API), the syscall a.k.a. pascal convention (Linux system calls, …), the thiscall convention (in C++ when calling an object’s member function), the non-standard fastcall convention, ...

Hopes this brings you some help ;)