« Back to home

Radare2 - Using Emulation To Unpack Metasploit Encoders

Radare2 is an open source reverse engineering framework, and is quickly becoming one of my favourite tools when picking apart malware or looking at CTF binaries.

I was recently introduced to Radare’s ESIL (Evaluable Strings Intermediate Language), which is a way of representing instructions in a forth like language, and allows emulation of machine instructions in Radare’s ESIL VM. To help understand this functionality, lets look at some examples from the radare2 book:

push ebp

If we take this x86 instruction, we find that it can be translated to the following ESIL representation:


I won’t go through the syntax of ESIL, as that isn’t too important for what we are trying to achieve today, but if you are interested there is plenty of documentation available in the Radare2 book here.

If you are a visitor to /r/netsec, you may have recently seen this post from the radare.today blog on unpacking Metasploit’s “shaketa-ga-nai” encoder. What I liked was the power of emulating instructions during a disassembly without having to revert to debugging, and I wanted to apply the same concepts to other Metasploit encoding techniques to see just how easy this was.

Starting Easy - x86/countdown

To start with, we will look at the “x86/countdown” encoder, which is described as a “Single-byte XOR Countdown Encoder”. We can find the source code for the decoder in the Metasploit github repo here.

Reviewing the Ruby code from the ‘decoder_stub’ method, we find the following:

decoder =
    state.buf.length - 1,
    state.badchars) +
  "\xe8\xff\xff\xff" +  # call $+4
  "\xff\xc1" +          # inc ecx
  "\x5e" +              # pop esi
  "\x30\x4c\x0e\x07" +  # xor_loop: xor [esi + ecx + 0x07], cl
  "\xe2\xfa"            # loop xor_loop

This decoder stub looks quite straight forward, it is a basic XOR decoding method with a decrementing key, used to deobfuscate the original payload. Let’s encode a simple payload and see what the resulting disassembly looks like in Radare2:

msfvenom -p linux/x86/exec CMD=ls -e x86/countdown -f elf -o payload_countdown

As we can see, our disassembly matches the decoder in terms of byte values. You may notice however that the disassembly description looks slightly odd, this is due to the way in which the ‘call $+4’ jumps into the middle of an instruction. To stay on track, we can ignore this for now, however “e asm.middle=true” setting will help you to spot these kinds of tricks in future by adding a “middle” comment alongside these kinds of instructions.

Lets set up our ESIL VM and start stepping through the encoder which will help us to understand how this works, and what we must do to extract the raw payload. We can do this with the following Radare2 commands:

  • aei - Used to initialise the ESIL VM
  • aeim 0xffffd000 0x2000 stack - Used to initialise a 0x2000 byte stack for our ESIL VM to use
  • aeip - Sets the instruction pointer of the ESIL VM to our current location
  • e io.cache=true - Allows modification of the file (required by our decoder routine) without persisting to disk

Once we have set up the ESIL VM, we can check the emulated registers values before we start stepping through the decoder with the “aer” command:

[0x08048054]> aer
oeax = 0x00000000
eax = 0x00000000
ebx = 0x00000000
ecx = 0x00000000
edx = 0x00000000
esi = 0x00000000
edi = 0x00000000
esp = 0xffffe000
ebp = 0xffffe000
eip = 0x08048054
eflags = 0x00000000

Looks good, we have a stack and our EIP value is set to our current function. Now we can switch into Visual mode and step through the code to watch the magic happen:

Comparing this to the original disassembly, we can see that we are now presented with a different set of bytes after the ‘loop’ instruction. This is the unencoded version of “linux/x86/exec” payload.

Knowing this, we can look to port this into an r2pipe python script which will complete the following steps:

  1. Set up the ESIL VM
  2. Emulate instructions until we land after the ‘loop’ opcode
  3. Dump the unencoded payload after the ‘loop’

The final python script can be found here: https://gist.github.com/xpn/9dbc8aea2ea53d92f9fca08f0a1e4fa7

Lets do that again - x86/jmp_call_additive

Lets look at another encoder, “x86/jmp_call_additive”, and see if we can apply the same concepts as those above to decode a payload. First we look at the source of the decoder:


We find the decoder stub as:

"\xfc"                + # cld
"\xbbXORK"            + # mov ebx, key
"\xeb\x0c"            + # jmp short 0x14
"\x5e"                + # pop esi
"\x56"                + # push esi
"\x31\x1e"            + # xor [esi], ebx
"\xad"                + # lodsd
"\x01\xc3"            + # add ebx, eax
"\x85\xc0"            + # test eax, eax
"\x75\xf7"            + # jnz 0xa
"\xc3"                + # ret
"\xe8\xef\xff\xff\xff", # call 0x8

As before, we generate our payload:

msfvenom -p linux/x86/exec CMD=ls -e x86/jmp_call_additive -f elf -o payload_countdown

..then disassemble:

.. and finally, initialise and run against the ESIL VM:

Again, we can see that our payload is decoded, and the original payload is placed at the end of the decoder. This means we can simply amend our previous r2pipe python script to execute the ESIL VM until we have passed the final ‘call’ instruction, and then dump the unencoded payload.

The final script can be found here: https://gist.github.com/xpn/83c0b6b45a260d0d24408377ecd8bb55

Something a little more complicated - x86/alpha_mixed

Now that we have the basics, lets move onto something slightly more difficult, the “x86/alpha_mixed” encoder. This encoder takes a payload, and converts the binary into an ASCII charset.

Again, lets take a look at the decoder stub found at https://github.com/rapid7/rex-encoder/blob/master/lib/rex/encoder/alpha2/alpha_mixed.rb:

"jA" +          # push 0x41
"X" +           # pop eax
"P" +           # push eax
"0A0" +         # xor byte [ecx+30], al
"A" +           # inc ecx                         10       |
"2AB" +         # xor al, [ecx + 42]                 |
"2BB" +         # xor al, [edx + 42]                 |
"0BB" +         # xor [edx + 42], al                 |
"A" +           # inc ecx                            |
"B" +           # inc edx                            |
"X" +           # pop eax                            |
"P" +           # push eax                           |
"8AB" +         # cmp [ecx + 42], al                 |
"uJ" +          # jnz short -------------------------
"I"             # first encoded char, fixes the above J

As before, we generate an encoded payload using:

msfvenom -p linux/x86/exec CMD=ls -e x86/alpha_mixed -f elf -o payload_alpha

As you can see, the disassembly has an extra set of bytes above what we were expecting, including FPU instructions. If we refer back to the Radare2 article in which the shaketa-ga-nai encoder was unpacked, we know that FPU instructions are not yet supported by ESIL, which means we are forced to mock these instructions to have our ESIL VM work correctly.

This is actually a lot easier than it sounds, as in this case, the ‘fnstenv’ FPU opcode is simply being used to retrieve the current instruction pointer. All our mock code has to do is to retrieve the EIP pointer and add this to the stack to emulate this FPU instruction.

Another odd thing that you will notice, is that the final ‘jne’ instruction at address 0x0804808d jumps forward within our disassembly, whereas the source code of the decoder stub clearly shows this should be a jump back into the decoder. This is one of the cool things about emulating code, strange things like this become clear without the need to fire up a debugger.

Again, lets set up our ESIL VM and begin emulating the decoder:

aeim 0xffffd000 0x2000 stack
e io.cache=true

We first step through the code until our EIP value points to just after the “fnstenv” opcode. At this point, we want to put the address of the last FPU instruction on the stack, which in this case is the ‘fld’ opcode at 0x08048056. We can do this with the following r2 command:

wv 0x08048056 @ 0xffffe000

Once we have mocked this instruction, we can continue on with our stepping. You will quickly notice that after the first round of ‘xor’ instructions, the ‘jne’ location has been updated to something more recognisable:

This was due to the “4a” byte value which was updated by the decoder to “e9” to fit in with the ‘alpha_upper’ requirement, and also explains the comment in the decoder source that we saw:

"uJ" +          # jnz short -------------------------
"I"             # first encoded char, fixes the above J

Our final r2pipe script can be found here: https://gist.github.com/xpn/da4a497288d6e1ed066d47ff1b2ce2d7.

Hopefully the above gives you some idea over the power of ESIL, and Radare2’s emulation capability. If you have any feedback, feel free to comment in the usual places.