PE - Portable Executable File

Hello, folks!

I'm here again bringing an overview about PE files. I think it's unproductive to write an extensive technical post about PE files since there is a lot of information about that in the web. I linked in the end of the post very good references about it.

So the idea here is just to give a general idea of the PE file, then you will be able to do your own researches, you can use the reference as a start. I think it's more easy to learn when you can grab a general idea then start to digging more into each topic.

Anything that I wrote here is in the reference, so you can use a reference to a more in depth reading. Any questions you are free to ask me I would be glad to help if a can.

Tools used in this post:

• Resource Hacker
• x64dbg
• CFF Explorer
• IDA Free

PE File Format

Let's start thinking about the PE file with this poor analogy. PE file it's some kind of recipe with all the ingredients inside of it. So you have all steps to execute the recipe with all ingredients included in it or with a reference to it.

This format is inherited from the COFF (Common Object File Format) format that came along with VAX/VMS architecture, since Microsoft came from Digital Equipment Corporations that used COFF format file. These formats serves as base for the loaders to read an executable on the system. So to quickly migrate to Windows NT, the developers maintained the original format and enhanced it to PE (Portable Executable).

The PE format is used on Windows to execute programs and is the standard format, i.e. way of organize the data inside a file that make possible to all flavors of Windows to read it, load it and execute it. There aren't almost no difference between 32-bit and 64-bit PE files, the difference resides mostly on field's size.

DLL and EXE files uses the same PE Format and differs just in some values of some fields, mainly in the "File Header->Characteristics". DLL is basically the same for OCX and even CPL files. Once you know the structure of the PE File you know how the executable is disposed on the memory when it's executed, therefore the loader will decide which parts of the file on disk will be mapped into the memory. Let's see a little overview on the pic below:

PE File - File on Disk and In Memory

All the information the loader will be map into the memory will be in the PE file itself. And all the information about how to translate the offset of the file on disk to the file mapped into the memory will be accessible in the file. Next is a SVG image from Wikipedia has a nice view of the PE:

• PE - Portable Executable Structure

When the PE file is loaded into memory it is known as module as all the other PE files is imported by it. The beginning address of a PE file is know as HMODULE as is referenced in Microsoft's API. Differently when the file is on disk, in memory we have the concept of virtual memory, i.e. we don't access the real physical memory of our computer, the OS creates a virtual memory space to allocate everything, it acts mapping/translating the virtual addresses to the real physical memory. The OS then can control better the memory management and security. Some regions of virtual memory space are protected by the Windows Memory Manager (Windows Component) that is specified in the section header of the PE format to read-only, read/write and execute.

MS-DOS Header

Came in handy in the first version of windows, because windows machines isn't so common like in nowadays. So the executable could at least print some messaging asking for Windows to run it. This header and in the executables in general always starts with e_magic field or IMAGE_DOS_SIGNATURE, it's important to remeber this. The most important field here is the e_lfanew that have the offset to NT Header where all the useful information resides.

PE Sections

PE file sections are used to split the data in the file. Some sections represents code and other data. There is some kinds of data like, spaces to read and write information, API import, function export, resources and so on. Every section in the PE file specifies what is in it. Commonly PE file has two type of section, code and data.

Windows Loader grab the information on the section header to properly load the section in memory. There is a code section and other data sections. Each section has it's own attributes like which type of data and if this section is read only or read/write in memory, all specified on the field Characteristics in the Section Header. In some cases the section can be shared between process if it specified.

Section names is just a way to better identify what is within, for the operating system it doesn't matter the name itself just the field Characteristics that is indicating the type of the section.

It's important to remember that since the operating system uses virtual memory protection the Optional Header->SectionAlignment value (space between sections) in memory would be different from the file on disk. In disk the default value is multiple of 200h (hex) (so the offset in disk would be like 200h, 400h, 600h...), but in memory the loader maps the sections in a way that each section starts at the beginning of a memory page (which inherit the security flag read-only or read/write specified in the section header). Windows 32-bit has a page memory size of 4Kb and 64-bit 8kb. So for each architecture this would be the alignment of the section, anyway you can always check in the field in the Optional Header->FileAlignment.

Relative virtual address (RVA)

The RVA it's an important piece in the PE file. It's used to located objects after the file is loaded in memory. When a PE file is loaded in memory it starts at some determined address that we call ImageBase (this address will be the HMODULE) it's located in the OpitionalHeader->ImageBase. To simplify everything let's see in the CFF Explorer (PE Viewer) how they are expressed:

PE File - Section Header / Virtual Address

So to locate the .text section in-memory we have to use the VirtualAddress that is the Relative Virtual Address of the section in-memory, it's relative because the final address to the section depends on the ImageBase address. So to locate any section in-memory we need to add: ImageBase + RVA. If we have the ImageBase 0x400000 and the .text section RVA 0x1000, the final address would be 0x401000, is where the .text section starts in-memory.

So in the header we have both the RVA(VirtualAddress) and the Offset(RawAddress) in disk. If we don't map the PE file on the memory we will use only the Raw Address, if the PE file is loaded we will use only the ImageBase+VirtualAddress.

Data Directories

Data directories are data structure used keep information that the PE files need. For example the imports section have a data structure that contains all the information necessary to the Windows Loader when loading the PE file in memory. So it can load the imports before it starts to execute the code.

Examples of data directories is imports, export and resources. So in the PE file we have a header to located each one of these structures. In the nex image is the Header of the Data Directories. here you have the RVA for each Directory in memory and it's size.

PE File - Data Directories

Importing Functions

In the PE file we have an Directory containing all the information about the imported functions. Which functions from which DLLs, then the Loader can load and locate all the symbols it need to run the module.

When using imported functions from DLLs the compiler automatically compiles and generates the PE file specifying in the import section which DLLs is been used inside the file, so the Windows Loader can properly load the DLL and prepare it to be used by the file in run-time. Note that in my source-code I didn't imported all these functions, but the compiler did. I used the GCC and as you can see it imported lot of functions for internal purposes like security and so on.

PE File - Import Directory

The PE file keep an array of data structures with all DLL's imported. Each data structures have two arrays known as Import Address Table (IAT) and Import Name Table (INT). In the previous image we can see that each of these data structures have the name of the DLL (ModuleName) along with two arrays OFT (OriginalFirstThunk / INT) and FT (FirstThunk / IAT).

The tricky part here is that both arrays has the "same" structure, because the  structure itself is an union that could be any of the values defined in the structure. I recommend you to read the references to get a more comprehensive understanding about it, take time reading and exploring the PE file. Though the tricky part, in general, the FirstThunk field generally points to the IAT array that is overwritten by the Windows Loader with all the API function addresses and OriginalFirstThunk is an array with 2 fields Hint and Name. The Hint field it's the name of the imported function and hint is the ordinal of the function API might be.

PE File - Import Descriptor

Once the Windows Loader loaded the DLLs and overwritten the Import Address Table (IAT) with all addresses that the PE file need to import, all the calls to imported symbols (function API) is redirected to the IAT and finally to the real API address.

In the run-time if the call to the imported symbol is redirected to a JMP instruction, then it's accessing the IAT before reach the API. If the call doesn't passes through any JMP then probably it's going directly to the API.

Malloc CALL (IDA View)

IAT Jumping To Imported Malloc (IDA View)

In future post I will go in more details here, doing a manual DLL hijacking overwriting IAT. Stay tuned. :)

Exporting Functions

Exports is another data directory containing all the information about everything the PE file exports. We refer to this exports as "symbols", for example the API LoadLibraryA is an export symbol of kernel32.dll. This directory is a little tricky, because it have some confusing pointers and rules, I will try to keep it simple and objective, but for a in-depth information please check the references.

When exporting functions or data to others modules all the information must be in the Export Directory, because it's this information the Windows Loader searches for when the other module is importing the symbols in this PE file. Symbols it's a term that includes anything that could be exported. Generally when some module exports symbols, the name of these symbols is the same as was originally coded on the source file. Let's have a look inside the export directory:

PE File - Export Directory

When we are consuming some DLL and we need to import it's function, generally we call the GetProcAddress to give us the address to that function. When we do that, internally the Windows Loader goes into the array Export Name Table (ENT / Field AddressOfNames) gets the index of this function in the array and then access the same index in the array pointed by field AddressOfNameOrdinals, the Loader saves the ordinal in the array at the index before mentioned (ENT). The ordinal is actually the real index used to get the RVA (Relative Virtual Address) of the imported function. In the field AddressOfFunctions has an array of all the exported functions each index of the array is a RVA that points to the function. The tricky part is the field Base that is used with the ordinal, so to find the real index we need to add the field Base+Ordinal resulting in the index for the AddressOfFunctions array. Generally this field is 0x00000001 and all symbol is in order.

I think this part is the most important of all, then I will make it more detailed debugging the GetProcAddress. I think it is interesting to see how the things works. GetProcAddress is a API imported from kernel32.dll subsystem that uses the kernelbase.dll that uses native API ntdll.dll (undocumented).

If you want to try it I will let the code in my github so you can compile the DLL and the code that consumes it.

• Github - PE Portable File
• Look for Dll Consume in README

Remember at this point to be objective, follow the address that matter to you. I used the x64dbg is a very good debugger has a lot of functionality and has a great community developing it. After the LoadLibraryA our dll is is already loaded in memory, you can see it in the tab Memory Map (inside x64dbg). In my case it was loaded in the address 0x6C300000. So this is the address that matter to us. Let's debug it. I breakpoint the GetProcAddress:

Breakpoint - GetProcAddress

After the breakpoint I steped into until I found the begning of the process where the "Windows Loader" begins to search the "NONAME" symbol in the Export Directory in our DLL. I will not make this part too long so I will get right to the point. Debugging you can see that it received the BaseAddress of our DLL, then it got the NT Header, checked if it is a valid PE (OptionalHeader->Magic value), got the Export Directory RVA and Export Directory Size, and now the ntdll.dll is inside our export directory. As you can see in the next image EAX already have the AddressOfNames(0x6C30602C), then it start to compare if the name provided in the GetProcAddress is the same as the exported from the DLL.

EAX=AddressOfNames // ECX=PointerTo_NONAME_Function // EDX=NameToCompare

After the confirmation that is the same function and it's the index 0, because was the first function of the AddressOfNames. In the next image we will see that now it got the value in the index 0 of the field array AddressOfNameOrdinals and with this value it was able to sought the function address in the array field AddressOfFunctions.

Getting the AddressOfNameOrdinals

Getting the NONAME symbol Address

Well, I hope that I could be clear engough explaining all the process to you and introducing you to the Windows Loader, basics of the hierarchy process of subsystem and native API, and how important is the PE format to the Operating System. Any question just let me know, you can email me or call me on twitter.

Some PE files use only the ordinal value of the symbol to export it's symbols. Ordinal it's just an index of the symbol as a mentioned. So when some module try to import some symbol by ordinal in the Import Section will be specified which ordinal the Loader must search in the Export Section of the module imported.

Resource files

The resources files is another data directory in the PE file. Generally have it's own section (.rsrc), but it is not a rule. As I mentioned before the PE file have all ingredients included in the recipe, so anything the PE file needs it can include in itself. Any type of file can be included in the PE file, after all any kind of files are just binary.

The resources are just embed files. There is some ways to get this resources in the run-time, advanced ways and basic ways. Generally we use the Windows API to load the resources. I pretend to introduce to this methods in the future, for now let's see how it works.

PE File - Resource Directory

The resource directory it's a little confusing if you have to read these structures, but using a PE reader it's very easy. Works as a chain of structures, the main structure is the IMAGE_RESOURCE_DIRECTORY that contains some fields. There are only two important fields NumberOfNamedEntries and NumberOfIdEntries, these two fields values has the size of the array of the next structure, IMAGE_RESOURCE_DIRECTORY_ENTRY.

The IMAGE_RESOURCE_DIRECTORY_ENTRY structure has two fields, Name and OffsetToData. Now come the tricky part. If the most significant bit of the field Name is set (differ from 0), then the remaining bits is the offset to the name of the resource, if it's not set then it's a ID for the resource. If the most significant bit of the field OffsetToData is set, then the remain bits is the offset to another IMAGE_RESOURCE_DIRECTORY.  If this field is not set, then the offset points to the resource itself. It's important to remember that this offset is always relative to the beginning of the resource section.

In malware analysis it's a good place to keep additional parts of the malware to drop (droppers) on the machine. Generally malware do this trying to bypass antivirus alert, because lots of antivirus if not all of them do a heuristics analysis searching for malicious behavior. So in many cases the malware (PE file) in the resources are packed, when the main program is executed it unpack the file from resource section and drops it in some folder. To help us we have some tools like CFF Explorer and Resource Hacker.

To make it more clear let's compile a program with another executable inside of it. I will let the link to the source used here. I compiled everything with GCC. Follow the link:

• Github - PE Portable File
• Look for Resource Files in README

Well as you can see I used an icon and another source, follow the steps in the github to compile the first source inversing.c that will be inserted inside our main source blog_resource.c.

After you compiled the inversing.c move the executable to the blog_resource.c folder. In the res.rc you can find the files names to compile, I used "inversing.exe" you can use whataver you want, just change the name in the .rc file. So you first have to compile the res.rc to a object file, for this we use the tool windres from GCC. Read the README that will have all commands to compile, any questions please contact me.

After you have compiled the blog_resources.exe you can execute it to see that the code gets the first two bytes from the inversing.exe file that is {"MZ"} or (big-endian{0x4D, 0x5A}, little-endian{0x5A, 0x4D}), these two bytes represents the IMAGE_DOS_SIGNATURE of all executable files. We can see in the Resource Hacker.

Resource Hacker - inversing.exe inside blog_resources.exe

It's confusing, but you can read more about it on the reference. In future posts I want to bring a more detailed post on how a dropper could work using Windows API and without any API (it's possible too). Anyway in the reference you can read more about the functions I used to find the resource file.

.NET Header

This header is present in PE files compiled in the .NET Framework (obviously). This section is needed for specific information about .NET compilation such as metadata and intermediate language (IL). Differently from directly assembly compiled languages like C/C++, .NET has it's entry point on MSCOREE.DLL which is the DLL that will starts the execution based on the information from the .NET header.

To make a little clear in the below image you can see a little overview on the .NET execution flow:

.NET Framework - Book Eldad Eilam - Reversing: Secrets of Reverse Engineering

The assembly part in this case is from the .NET Framework, the application itself is the Intermediate Language (IL) interpreted by the Common Language Runtime (CLR). In this case the disassembler tool utilized is for the IL.

Conclusion

First of all I do apologizes any mistake that I've been made and I would appreciate that if I did any, please contact me. Any question you may have I would be glad to help if I can. Well, as I already said I tried to keep it simple and objective. I hope this can be useful for anyone. This post is one of the series of introduction topics to my future posts with more technical text.

I linked all references in the bottom. So as I already said, the response to any questions that maybe arises certainly is in the references, anyway feel free to contact me. My post is simple and objective to give a direction about the PE Files, for more in-depth information the references is the way. Thanks! :)

References

• Matt Pietrek - Peering Inside the PE: A Tour of the Win32 Portable Executable File Format
• Matt Pietrek - In-Depth Look into the Win32 Portable Executable File Format
• PE Format Overview (Wikipedia)
• MSDN - PE Format
• Windows Executable Files
• Eldad Eilam - Reversing: Secrets of Reverse Engineering
• Setting ICONS for Windows programs with GCC
• FindResource Reference
• LoadResource Reference
• Resource Types

Basics of Assembly

Introduction to Assembly

Hello everyone, it's me again bringing some basic stuff. Even being a basic stuff I hope that can help anyone. I am making this basic posts to my incoming ones, I will bring more technical analysis. Sorry for any mistakes that I maybe did on this post and if I did any, please send me a feedback.
Well, the assembly code it's the "only" way on reversing engineering. Or you interpret assembly mnemonics or you analyze opcodes (machine code). As said in the previous introductory topic (if you didn't read, I do recommend) the assembly code it's generated by the compiler for every code language that you use. So, understanding it is vital to RE. Assembly language doesn't is standart for all assemblers, the mnemonics(instructions) are different deppending of your processor architecture. Since the processor is made of several circuits and each processor has your own cricuits, the logic behind it's processing differs from each other. So the instructions that they "understand" is different from each processor architecture. In this blog I will give the approach of IA-32 architecture. I will not "teach", is more like an approach of what it is and how we work with. So knowing how the computer "works", as Memory-Data X Instruction-Operation. Logically the computer stocks all data on memory and all this data is interpreted and executed by the CPU.

 

Computer / Memory, CPU, IO Devices and BUS

The processor have some pointers to help the CPU keep track of what it need to do with what data. Then we have the instruction pointer and the data pointer. Through the posts we will see it in practice. So the instruction pointer points to the memory block that represents an instruction and the data pointer to a memory block that represents the data, then the CPU executes the instruction with the data that is pointed.

 

CPU Units X Memory

These pointers allocate the memory offsets in registers. As the processor executes these instructions the instruction pointer goes to the next instruction and the data pointer too. The instruction has between 1-3 bytes and is called opcode (operation code). So basically assembly has three "parts", opcode mnemonics, data sections and directives.

Opcode

Mnemonic code is the "english" representation of the instruction code, e. g. the '89' instruction is the 'mov' mnemonic. Different assembly types represent instructions differently.

 

OllyDbg / Right Assembly, Left OpCodes

Data

The data sections is the space used to store the data which the instrunction will use to execute, so this data can be in some memory section or it can use the stack (memory area, more later). All the data is stored in the hex representation and is referenced by it's memory address. So every data stored on the system has a memory address, it can be a immediate constant in the assembly code or it can be stored on the stack frame.

Directives

Directives are the elements used in assembly to tell the assembler (which compiles the assembly) how to interpret this type of data, data includes everything, like code and values. For example if you want to store a float value the assembler needs to know, then it can reserve memory properly. One of the important directives of assembly is the ".section" directive. This directive creates sections on memory for each type of data.

Sections

We can have any kind of section we want, but all programs have this by default:

• .text:
• All the code instructions are alocated in this section. No data is allowed here, except some fixed data of variables like a = 5, but that depends of the programmer on low-level programming and depends of the compiler on high-level programming.
• .data:
• This section is responsible for stores all data that the .text request. It will be referenced as an address in the .text section, like program.ADDRESS.
• .bss:
• This section generally used to unitialized data. I think the name might change from language to language, don't sure.

The IA-32 Architecture

The IA-32 architecture was designed for pentium processors by Intel. I don't know for sure if it is the most used nowdays, but it's a very known one and have a lot of documentation about it. When you learn assembly basis in one architecture, makes easy to learn in another one, because the base still the same. Some assembly let you do more others let do less, I think it's the basic difference. In my last topic I tried to wrote about computer in general form and I think that you already know how it is. I don't want to make this part too much extensive. So basically IA-32 is divided in 4 parts:

• Control unit:
  • Control is responsible for bring all the information from memory, data and instructions. Then it decodes this instructions into micro-operations and pass to execution unit. The result of the operation is passed back for control unit that stores the result.
• Execution unit:
  • Responsible for execute all the micro-operations.
• Registers:
  • Registers are responsible to keep track of data that are been used. This registers are internal memory of the processor. Having this little memory inside the processor make it much more faster than going outside (from processor itself) searching data in RAM memory and retrieve it.
  • I want to keep it objective so I will not write about all of them. You can read in the book listed on the reference, I do recommend. So there are basically four types of registers (there are more like I said):
    • General Purposes: 8 32-bits registers. They are used to work with data.
    • Segment: 6 16-bit registers. Used to memory access.
    • Intruction Pointer: 1 32-bit register. Points to the next instruction to be executed.
    • Floating-point: 8 80-bit register. It used to work with floating-point numbers.
• Flags:
  • Flags are used to keep control of the operations executed by processor. It's a way to know if some operation worked or not. There are specific type of flags to specific operations. We will see it.

Registers

General Purposes

These registers are mainly used to work with data as the code is been executed. All data used in the instructions are stored in these registers. They are 32-bits longer at the "top" level (32-bits), but it can be sliced in minor parts (16-bits, 8-bits) that stores minor data.

• EAX (32-bits):
  • AH (16-bits):
    • AL (8-bits)
• EBX (32-bits):
  • BH (16-bits):
    • BL (8-bits)
• ECX (32-bits):
  • CH (16-bits):
    • CL (8-bits)
• EDX (32-bits):
  • DH (16-bits):
    • DL (8-bits)
• EDI (32-bits):
  • DI (16-bits)
• ESI (32-bits):
  • SI (16-bits)
• EBP (32-bits):
  • BP (16-bits)
• ESP (32-bits):
  • SP (16-bits)

OllyDbg / Registers and Flags

These are the 8 32-bits registers that we will work with a lot. So it's important to say that modifing a top level register you will modify the low level register. For example if you put any value at AL and then put a new value on EAX the AL value will be overrided. Some of these registers are used in the default way. EBP and ESP for example are used to control the stack frame. Stack frame is a block of memory used to control some local variables of function/method. But you can use the stack when you need to call a method/function you push the parameter onto the stack so the call instructions can grab these parameters to call the method/function. As we analyze artifacts we will see the pattern of it's usage. Because of this it's important to know programming. The good way to learn it's to code and analyze it. On the later topics I will bring much more pratical examples with C language. It's amazing what we can do (I mean, you are managing energy, bro!! lol).

Segment

The segment registers are used to identify where data is located. Each segment register has a pointer to the section where it suposed to grab the data. The segment registers are:

• CS: Code Segment.
• DS: Data Segment
• SS: Stack Segment
• ES: Extra Segment
• FS: Extra Segment
• GS: Extra Segment

So each one is used in specific cases. For example, if you have a address memory on EAX (of data section) register and you have some data on EBX and want to save it on the .data section you maybe see that:

mov ds:[EAX], EBX

So you are moving the data on the EBX to where [EAX] pointing in the DS section. The [] brackets indicating that is a pointer to some memory address.

 

OllyDbg / Code acessing SS segment

We will see much more in pratical examples. The best way to learn.

Flags

The flags are maintained in a single 32-bits register called EFLAGS and each flag is represented by each bit. So as I said the flags are used to control if the operations that the processor executed worked or not. For example for conditional jumps through the code execution it checks if the Zero Flag (depends on the jump, others can be used) was set so it can know whether it will jump or not. There is an image as you already saw in registers topic. The flags is divided in three groups:

• Status Flags
• Control Flags
• System Flags

I will only discuss about status. If you want to know more, there are books listed in the reference, I do recommend to read. The status flags are used to sign the result of mathematical operations executed by the processor. The flags are:

• CF: Carry Flag.
  • Carry flag is used to manage the carry or borrow out in mathematical operations. It means that occured an overflow, i. e. there is some remaining data. Used in unsigned arithmetic.
• PF: Parity Flag.
  • Is set when the result of the operation have sum of the 1's bit even number.
• AF: Adjust Flag.
  • Used in Binary Coded Decimal (BCD), is set when the result of an operation is a borrow or carry. BCD it's a nice feature to work with decimals, if you want to know more at reference has what you need.
• ZF: Zero Flag.
  • Is set when the result of an operation is 0.
• SF: Sign Flag.
  • Is set in the most significant bit, when the operation results in a negative number.
• OF: Overflow Flag.
  • Is used in signed integer operations and is set when the operation result is too large for positive numbers representation or too small in the negative numbers representation.

Stack

So now we are going to talk about the stack. Stack is very important in the reversing due it's common usage in the assembly world. The stack is a memory area where the program in execution uses to store short-term data. So the stack is generally used to:

• Save register values
  • You can save the register in the stack to use it for another operation and then retrieve the old value to it.
• Store local variables
  • Store the local variables at function scope. Like I said before, when doing it the variable is accessed directly using SS segment to access the offset in the stack.
• Passing function parameters
  • To call a function generally you push all parameters on the stack from the right to left and call the function. For example, f(p1,p2,p3), you will pass p3,p2,p1.
• Store the return address of the function call
  • The address of the next instruction after the call instruction, doing that the program will know where to back when it executes the retn instruction inside the function called.

Storing locally means that when you enter in a function scope all the local variables generally uses the stack to keep your data. In general everything is on memory, on the address space that the operating system gave us. The stack is just an area on that address space where the program uses to store data. So to put values on the stack you uses push instruction and to retrieve it you uses pop instruction. Everytime you enter a function a stack frame is set. Stack frame is set of addresses reserved to the function in execution, that set is limited by EBP(Base pointer) and ESP ('Top' Stack pointer). Generally this is how you see the stack frame been set:

PUSH EBP
MOV EBP,ESP
SUB ESP, SIZE

 First you save the old EBP, for when you back to the previous function will be able to reset the old stack frame, then you set the new base pointer and add the size (how many addresses) you need to that function. Stack are managed as LIFO type, last in first out. So the last element that you pushed onto the stack is the first element that will be poped out. But you can access this data directly if you need it, remember the SS segement ? So you can access using it together with the ESP (points to the top of the stack) register, like:

mov EAX, PTR SS:[ESP+4]

In the last instruction you moved the value of "ESP (plus) 4" pointing at to the general register EAX.

 Some important things about stack is that it grows up to lower addresses on IA-32 architecture. So bigger the stack is, lower addresses it access as we can see:

 

How stack works basically

In the image we can see the heap too and it's our next topic. This post I am bringing a theorical approach but in the next we will discuss it all in a pratical way. For now lets just abstract.

Heap

Heap is a memory area where the program uses for dynamic allocation. The heap is managed generally by the OS. So when the programmer need some space to store data that is bigger than the stack could manage, then the program store it in the heap memory. So the memory heap is passed to the program when the OS is loading it on memory. When you have a literal expression on your code like:

char newText[] = "Testing how code storing data.";

Regardless the data was inside some function the compiler generally uses the .data section and give to the instruction that will use this data the constant immediate address of the data, something like program.ADDRESS. But when you are loading something external for example, you don't know the size you going to need so the size is dynamic, then the program will have a routine that allocate the size you need in the heap.

 Heap it's important in RE because many program uses it to allocate data and sometimes identify which routine allocates the memory can be useful.

Now I think you are able to understand the incoming new posts that I will bring here. As I post new topics I will try to discuss a little more on technical topics. On the pratical part we will have a better view.

References

• Google
• Eldad Eilam - Reversing: Secrets of Reverse Engineering
• Reverse Engineering Code With IDA Pro
• Professional Assembly Language - Richard Blum

Reverse Engineering - Didactic point of view ?

Hello folks!

Well, I'm starting this blog to produce content to the community and as a way to study about it. Sorry for any and all english mistakes :). Since I'm starting this blog I thought to bring an introductory text and increase difficulty as I make new ones, anyway I want to bring my parallel studies too. My goal it's generate information to everyone, trying making objective and didactic content. I'll try to translate all my posts from Portuguese to English. My focus is Portuguese, because our community is lack of Portuguese content. I hope you enjoy the content generated here.

If you have malware samples and wish to share with me before trash it, please send it to my e-mail pimptechh@gmail.com. I would be grateful. My intention is to bring the samples analyzed here. 

Reverse engineering is a complicated and polemic subject, because evolve many legal questions and for this reason is a theme that must be treated carefully. I will using the concept in software as it is a very embracing and could be referred in any engineered artifact. Basically it is a "deconstruction" of something that already is built. Many of us, human beings, use this knowledge to break software, in general known as "Crackers". Using this power to create a certain chaos on the system. Anyway this is very important process in the industry nowadays, software security, maintenance of software, software performance and so on. It's like a view on how it works and how can we make it better.
In reverse engineering is fundamental to have curiosity, since it is basically from where it arise the idea of reversing things. "Well, how would this was made ?" or "Well, why this is happening ?". A simple debugging, even with the source code on hands, would fit in this process. Trying to find some value or bug in the system. Therefore to understand a reverse process we must primarily have some basis on how this process or artifact was built in first place. If there is a need to revert a software, it's necessary to know at least how build one. Understand how a software works it's indispensable, since before just throw it in the reversing tools we must to know the basis. The fundamentals it's always important, I will try to dissertate the best way that I could to show the importance of the fundamentals by my studies.

Everything began, well I don't really know where it all began (still finding out), but we know the "first" big invent, or something like it of computer models that we have today, the ENIAC. This project already have a little collaboration of Neumann(creator of the actual computer model), helping solving some mathematical logic problem and right after ENIAC they start it's sucessor EDVAC. ENIAC basically did complex arithmetic calculations, although it didn't the concept of memory and programs. In the EDVAC version this concept started to be explored.

The modern computer "started" with John Von Neumann, a exceptional mind and memory. He could recite whole book chapters using exactly the same writed words without changing anything. Even reciting and translating simultaneously from German to English. In 1927 and 1928 published some papers about mathematical fundamentals of quantum theory and probability in quantum statistics, so if you don't believe in quantum mechanics or even doubt it, turn of you computer and throw it away haha. In these papers he demonstrated physicists phenomenal knowledge. The abstraction came from this extremely complex problems in which Von Neumann as was an expert. Clearly he was very ahead of his time. The history it's very interesting, worth read about it. There is a book "The computer and the brain" in which he tries to explain the guesses and concepts for the model designed by him. In this book he tries to do some analogy of our nervous system in a mathematical point-of-view using concepts of statistics and logic. Human body, biological machine.

When we thought about the magic computer makes,  it's amazing. Storage energy as the same time storage information and with that information solve problems fruits of intellect. The computer had it's necessity helping solving problems in a much faster way. Making complex calculations in unbelievable speed. Basically you insert an input, this value is stored in memory as energy/tension, this tension is differentiated between 0 and 1. All this stored within your computer and assisted by boolean algebra that generates our output, whatever it is, all depends of the problem you want to solve. When I say "problems" it's necessary an abstraction of what you consider a problem.
I don't know how used to be the conversation between Von Neumann and his mathematical friends, but would be very crazy for sure :P.

We solve a lot of problems every single day by the simple fact that we have stored energy in our computers. One of the programming concepts is here, abstraction, as higher is the layer you are more you abstracts the computation. When you need to talk to someone, all you need to do is choose your IM, send a message and solve the problem(talk to someone), just an example. Don't realizing this makes the paradigm breaking difficult. Math calculations like (2+2), you look at the operands, you realize that is a number 2, stored this information, added more one number 2 and then makes the logic addition (+) operation. You just reads the information stores it and operate it logically, just like and computer do. But how make all of this turn into information ? Rising the representation layers. We have in this case 0 and 1 stored as energy in our computer, this numbers can represent many thins like on and off, full and empty and so on. When we put blocks of 0 and 1 we have a lot of possible ways to represent information like number and letters.

All the information are disposed as 0 and 1, puting all them together forms blocks of information. All depends the way you want to interpret this information. Therefore by been 2 possible values we can assume that is base 2 representation. I'm not super math fan, but I can see the beauty. Anyway, how can we abstracts information from number base 2 like that ? Lets start by decimal base-10. Everybody, I do believe, saw on school about greatness, ways to represent this greatness like 10, 100, 1000 and so on. You implicitly understand these representation like ten, one hundred, one thousand and you can do that because the additional 0 at the end of each greatness. When we put the decimal digit together we are able to count (0,1,2,3.. 10,11,12...), each greatness break we have a new representation.

Is the same with binaries, but in binaries we change 0's and 1's places inside an information block. Here is an example of binary counting:

We can represent in this block-form or we can rise the way of representation to decimal numbers(friendly to us), converting base 2 to base 10 like follows:

Now, how is possible make operations with this primitive way of representation ? With boolean algebra. Inside the CPU have a lot of microcircuits of complex logic in which our information is treated. One of these microcircuits is the Half-Adder and I believe it is the smaller one:

In this circuit runs two inputs (A, B) of tension, "A" can be 0 or 1, the same for the "B" input. Now how this circuit really works ? Inside of microcircuit there is the logical gates like OR, Exclusive OR (XOR), AND, NOT and so on. This circuit have two outputs, sum and carry, like we saw in the previous image. With this we can add two bits like we do with numbers. Lets take this example (5+5=10), in this case sum is the value 0 and the carry is the value 1. The same with bits, e. g. like our Half-Adder we can "sum" two bits 1+1.

Doing that we realize another problem, because we know that the number 2 in binary is "10", right ? The Half-Adder did it's job, now we have to take care of this carry output, for this reason we have the Full-Adder (image below), it can receive three inputs A,B and the additional carry. Do you realize the problem abstraction ? Inside the Full-Adder we have the Half-Adder. Our CPU have a lot of extremely complex abstraction layers or microcircuits within each other, a lot of hours studying to really know all of it, however we will not design microcircuits we just want to know what our computer is made of and how it works, know the fundamentals. If is your desire to learn more, I'd recommend read about computational logic. There is many microcircuits embed in each other and the CPU is an arrangement of extremely complex circuits.

Right, now we have our 0's and 1's represented as information in blocks and been treated by the logic operators, making our machine generate our output. Even converting our information to decimal and after to hexadecimal, we will see it later, we have to understand that computer borned to solve complex mathematical operations and to do so it need to understand "complex" numbers. I'm not a mathematician, so sorry any mistakes. We have topics to see before get into hexadecimals, it's the main way to work with data in our field. Machine needs to interpret negative numbers, for this we have the Signed number system (is good topic to write about it later desmistify Signed and Unsigned in language C). With this system we use a most significant bit (msb) to represent the sign of the number (0)0000001, in this case the decimal number is "1" positive. 0 stands for positive and 1 to negative. Using the first bit or msb we are unable to represent the decimal 0 to 255, with that one bit less we can now represent only -127 to +127, because the msb was cut from our 8bit block. Have the solution and not having it, right ? Because, how do we will operate this logically ? Let's to the example 2 + (-3):

As we can see isn't the desired result, right ? To solve this we have the "Complement". The first version of the complement is the "One's complement", basically we have to flip all the bits in the block for it's opposite like this, "00000010"(+2) to "11111101"(-2). But the problem was solved partially since our result isn't the correct value yet as we can see in the next example:

Just to remember that the carry from the previous operation is carried out of the 
8bit result, in the third position from right to left then operation generates an carry that is carried to left until it's is carried out. As we can see the result is -1 of the correct result, would must to be "00000010" but instead we get "00000001". How would be possible to fix this ? If we add 1 bit to this result it would solve our problem. So the "Two's complement" came to help us doing exactly that. Let's see in the next example:

The "Two's complement" came to solve the "One's complement" problem. So let's suppose the same type of operation, but this time we want to do it with a negative number been bigger than the positive. Let's see 4 + (-5):

Well, as we can see in the previous example the value isn't the correct when the number negative is bigger. When this happen it's necessary to take this result and applies the complement on it again, flipping the 0's to 1 and the 1's to 0, then add 1 bit to it and grabbing that last carry to the new result. The last carry will sign our new number. If we do that our new result would be signed "10000001" (-1). The CPU uses flags to do this type of operation, in this case it uses the (S)ign Flag, but let's talk about it in another text.

Let's proceed to the base-10 conversion that is the number notation we use on daily basis. However we will see base-16 too and this particularly is very important since we use it a lot in reverse engineering. But before we do that it's important to know the bit-block sizes or rotulation it's very used on RE too. Each bit-block have a size or "name" to differentiate how long is the information inside of it. Let's see in the next example:

4 bits = 1 Nibble

8 bits = 1 Byte 

16 bits = WORD 

32 bits = DWORD

64 bits = QWORD

Now with hexadecimals we can encapsulate a larger quantity of bits in only one digit of hexadecimal. Let's take the block "1111" as example, in decimal the value is "15". Converting this previous value to hexadecimal we have "F", just it, one digit of hexadecimal and with two digits we can represent 8bit values. Let's see an example how count in hexadecimal:

There is many sites to convert hexadecimals, some examples:

• http://string-functions.com/hex-string.aspx
• http://string-functions.com/hex-decimal.aspx

To do all this we need to store all that information to work with it. In nowadays the way to do this is using Memory RAM. Memory RAM stands for Random Access Memory and are cells that stores energy/tension or information, call as you want, this type of memory are volatile, i. e. when the energy power is cut all the memory is lost. This mean that the CPU have to check this memory every cycle to maintain the energy already stored there, avoiding energy leaking.

So, now we have the energy stored, the CPU working logically with information, but how these information flows through the computer ? Well, to do this job we have the BUS of data. When we think about BUS of data it's necessary to abstracts the term to make it easy to understand. Just imagine a tunnel where all 0's and 1's passes freely. But how we manage this information passing through the channel ? I mean, how to address origin and destination of data ? For this we use the Multiplexer and Demultiplexer. These are circuits that manage from where to where, they pick the required information from the tunnel. It's like encode and decode the information. It's a complex topic, so if you interested I recommend to read about it. In the reference listed there is some good information.

So far we know how the computer interacts and how it operates all the information stored in it, however I would like to remember that all this I'm writing here is very basic, our computer is much more complex than that. Anyway the fundamentals is always the same, information is the same, bits are the same, how information is stored remains the same. I see this text as a way to think about computation. So everything is possible! We just have to visualize knowing how it works. Ok, how to interact with this incredible machine ? The answer is OS (Operating System) he manage everything for us, all the memory, all the low-level part of the machine. So everything passes through the OS, all the tasks uses the OS to do something. In reverse engineering we have to keep this in mind, you want to figure how something works ? First figures how it will be interact with the OS, starts from there. The OS encapsulates all the methods to manage your machine in different layers of abstraction. Want to know how it works ? Look for OS internals books. Worth the time reading I guarantee.

Well, now we know how the machine operates and what helps us to operate it. In the user layer of OS we already are in the tops of the abstraction layer everything is prettier and easier visually. So do the programming languages, knowing how programming it's essential. Many people may say that you don't really know how programming, but I ensure you that everything it's more easier when you know how programming. Our entire computer do what we say because of it. The only language computer know is machine language with all it's operators and operands. Any language that you use turns into machine language in the end. Nowadays we have what we call High-Level languages, they are called like that because your commands are close to our formal language in "real" life.

There many types of high-level languages, but there are two of them that is most used. Java and C# they are high used in daily basis. It let us solve problems in a much faster way, but that not implies performance because of encapsulation and abstraction. Higher you are in abstraction layer more technology you have under you, this means many things that you can't control. In general these languages runs within a virtual machine, so in this case the OS doesn't interact direct with your program, but instead with their virtual machines. Let's see an example of languages life cycle:

High-Level => Compiler =>Assembly => Machine Language

As we could see this cycle stands for languages that are directly compiled. Java and C# doesn't runs directly, instead they use a virtual machines like I already said. The cycle I've shown it's for languages like C/C++ that are compiled and runs directly with OS, there is no virtual machine between them. At low-level layer we have the assembly language that in fact is the machine code, but translated to assembly. Assembly language basically are the mnemonics from real machine language. Java have the JVM (Java Virtual Machine) and the C# have the CLR (Common Language Runtime), they both are virtual machine that interacts with the OS. These languages doesn't generates machine code, instead they generate bytecodes and only can be interpreted by their virtual machines. So to reverse it it's a little different, more easier I would say, because bytecode is a type of machine code but in high-level and more readable.

In reverse engineering we must be conscious about fundamentals, because we work directly with machine code/assembly. And this language works directly with the machine, with memory, deal directly with CPU and it's architecture like any high-level language but with a little different concepts. When you change the language you have to know how iteract with it, same with assembly and CPU architecture, but in general operations are the same. "Any" language have loops, variables, constants and so on. The same is for CPU and it's architecture. So I freeze that is important to programming and if you want to go down this path, you have to learn C/C++. If my opinion worth something, go for C :). When you understand low-level world well and how to interact well with C, then you should maybe migrate to C++ or don't hehe. Anyway OS is the key to reverse anything that is interacting with it. Let's see the below image and try to abstract your machine after read all this text:

Operating System | Third Layer
|||
Instruction Set Architecture | Second Layer
|||
Microarchitecture (Hardware) | First Layer

Conclusion

Well, this is my first text about the introductory part of reverse engineering, I hope that could help anyone. It's a magical world, in fact very wonderful world! The question is how far down the rabbit hole do you want to go ? I want to keep this blog always updated, trying to help the community to grown as I do. I will try to be most objective and didactic as possible to help everyone to understand the concepts. In the next texts I wish to bring more technical stuff like, malware analysis, solving puzzles, programming assembly/c and so on. Feel free to ask and criticize. My only goal is to learn and help the community. Let's share, let's grown together.

Regards from Pimptechh and nice studies.

References:

• Google
• Andrew S. Tanenbaum - Structured Computer Organization
• Eldad Eilam - Reversing: Secrets of Reverse Engineering
• Richard Blum - Professional Assembly Language (2005)