Willkommen auf MasterOf13FPS! MasterOf13FPS

Registriere dich noch heute oder melde dich an, falls du bereits Mitglied bist und verpasse keinen coolen Content mehr :)
  • Hey Guest, wir haben nun einen Telegram-Channel!
    Klicke hier, um zu joinen: Channel beitreten

[LUA] FFI course

Aui

New member
Old Team
Premium
Developer
Joined
Jul 9, 2020
Messages
44
Reaction score
7
Points
0
Location
Augsburg
Website
auitism.us
shoutout to owen.


[ Introduction ]

Hello! I have decided to write an in-depth FFI crash course to provide a place for anyone regardless of your experience to learn without looking through the dozens of possibly confusing scripts released here on the forum, which are generally undocumented and some are quite complex. This thread will act as a place for you to ask questions and learn FFI's basics, plus some tips and tricks from me and others to become a more experienced and competent Lua developer.

A quick explanation:
Throughout the first section of this thread, I will be providing pseudo-C code snippets, as C comes built in with dedicated methods for certain, very important operations to us and is also the language that FFI provides an interface to, so knowledge of how it operates is completely necessary. The Lua equivalents of these operations require a more abstract thought process, as Lua does not provide explicit operations for the aforementioned functionality. I will be explaining these concepts in pseudo-C and then explaining how the same can be achieved in Lua, while calling back to earlier terminology we will have covered by then.

What is FFI?:
FFI - Also known (although rarely referred to) as the Foreign Function Interface - is an interface between different languages allowing them to interact with each other. FFI is generally used to create bindings to C/C++/<Insert other systems programming languages here> libraries to higher level languages such as Python, Ruby but more importantly in our case, Lua. Bindings are just functions in the higher level languages that reach down into the lower level libraries and call their functions.

FFI in CSGO:
In CSGO, FFI is most commonly used to provide an interface into the CS:GO processes' memory, which allows a Lua developer to interact directly with the game, such as calling functions, without relying on a middleman such as the Lua API to do all the work. Since FFI provides an interface between Lua and C, using FFI requires some knowledge about low-level memory and how low level languages like C interact with it - we will be covering this later.

[ Memory and How languages interact with it ]
In low-level languages such as C/C++ (I will be focusing on C as C++ provides some extra features for memory/pointer management that are far beyond the scope of this thread), memory can be interacted with directly, and all low-level/system software programming languages must provide us with a feature to do so. C provides us with a feature called pointers, these are essentially memory addresses, but ideally they will point to a specific variable, and this variable can be of any type (integer, long integer, floating point, string, even another pointer) and contain any value.

The two types of pointers:
The reason I said ideally in the previous paragraph is because a pointer may not always point to a variable, pointers that do not point to variable or any address are known as null pointers. This is where the divide between the two different types of pointers begins to show, they are known as SAFE and UNSAFE pointers. I will be covering safe pointers first as I find them easier to understand as there is less ambiguity, however when using FFI you will be seeing mostly unsafe pointers, however knowing how safe pointers work will make learning about unsafe pointers much easier. Below is an example of how a safe pointer is created.

Code:
float floating_point = 100.f; // Create a variable (randomly chosen float type)
float* floating_pointer = &floating_point; // Create a pointer by using the & operator.
The variable floating_pointer now points to the variable floating_point (meaning it contains the address of the variable floating_point). It is called a safe pointer because we can say that the variable floating_point will always have an address in memory, and that the variable floating_pointer will always point to that variable.

Reading memory and dereferencing:
Lets say we only have access to the variable floating_pointer, yet we want to retrieve its value. This can be done through dereferencing. Dereferencing is simply the act of reading the memory at the address stored in the pointer. Lets say that the address of the variable floating_point in memory was 0x1000, and now our pointer floating_pointer contains that value. In C we would dereference the variable using the * operator, placing it before the variable name as such:

Code:
float original_value = *floating_pointer;
The variable original_value will now contain 100.f.

Unsafe pointers:
Unsafe pointers are a much more complex subject, hence why they're getting their own subheading. An unsafe pointer is a pointer that we cannot say for sure if it points to a value of a given type. If we were to create a pointer with the type char* (char* is the type used for strings in C, the pointer points to the address of the first character in the string) and assign it some arbitrary address instead of an address we have retrieved from something that we know has to exist, we cannot say for sure that the address we assigned that pointer actually points to a string as we are able to give this pointer absolutely any numerical value, which may cause instability or a crash if we attempt to read from the pointer by dereferencing. Below is an example of an unsafe pointer in use.

Code:
char* our_string = (char*)0x1234;
char first_character = *our_string;
This code, if compiled and ran will undoubtedly cause the application to crash as 0x1234 is almost certainly not a valid address.
You can also see, that we never use the & operator that was discussed before, it is only used in the creation of safe pointers in C.

Why unsafe pointers?:
Because of the nature of what we are doing with FFI, we only have access to the memory, not the higher level source code abstraction that would allow us to create safe pointers. Unsafe pointers are simply a way of telling our code how to interpret a memory address. Throughout this post I have said variable, but remember that these variables can be of any type (functions, engine defined structures, custom user defined types, and basic integral types). In the Lua API, we are provided with some features that allow us to easily scan memory and retrieve important engine structures and classes, these are client.find_signature and client.create_interface and both of these concepts will be explored later in the upcoming section.

A quick introduction into Memory type punning:
Type punning is the act of bypassing the type system that is implemented inside of programming languages. Type punning is more generally talked about in statically typed programming languages, not memory but although we are not dealing with the type system directly, we are dealing with the memory variables once managed by the type system. Look at this snippet from a hex editor, this was taken from a random executable file containing an XML string although this isn't important.

rjl7Ezy.png


From this snippet, we can clearly see from the ASCII representation on the right-hand-side of the image that it is the beginning of a XML string, however if i wanted to read the first 4 bytes of this string as a 4 byte integer, I could do this:

Code:
int* int_pointer = (int*)0x405060;
int first_4_bytes = *int_pointer;

// This could be tied into one line.
int first_4_bytes = *(int*)0x405060;
The value of the variable first_4_bytes is now 0x6D783F3C (notice how its just the value of each of the first 4 bytes but going backwards).
The brackets and the int type name before the number is whats known as casting, im essentially telling the programming language how to interpret this memory address. This can be used on existing variables as well, given that they are numerical or are already a pointer just of a different type. This will be touched upon further in the next section. Remember! Memory is just a massive sequence of bytes, and it can be represented however you want it to, this is where a lot of the unsafety comes in.

Pointer arithmetic:
Pointer arithmetic is just like normal arithmetic like 1+2, but it takes something else into account. In all programming languages, types have size. This means that each type (byte, boolean, integer, long integer, string, floating point, etc.) have a specific size and some - like strings - can be dynamic in size. Pointer arithmetic simply takes the size of the type of the pointer into account when performing arithmetic operations (+ and -, multiplication and division are not used) on pointers. Here is an example of traversing an array using pointer arithmetic:

Code:
int our_array[10] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 0};
int* our_array_ptr = our_array; // Both of these are identical, but im doing this for clarity to show that our_array really is a pointer.
int our_1st_value = *(our_array + 1);
You would be right in assuming that the variable our_1st_value now contains 1, but theres more going on behind the scenes, let me demonstrate:

kdHyHYx.png


This is what our array looks like in memory, notice how each value is takes up 4 bytes. This is because the size of integers in C is 4 bytes, but our numbers are so small that they only use the first byte of the 4 they are given. Remember that our_array now contains the address 0x14001f068, and you may expect adding one to produce 0x14001f069, but remember that the next value in the array is 4 bytes away, so adding 1 really gives you 0x14001f06C as the size of the type has been taken into account.

[ Getting practical with FFI ]
In this section we will cover actually using FFI and some of the features the Lua API provides us that makes our lives a lot easier. However, before reading on I must admit that in Lua, since it is not really intended to be a low-level language it does not come built in with some features that I would like which can make things awkward sometimes, but I will be covering ways of how we achieve what we need in the second subsection.

Getting started with FFI:
To actually start using FFI, you must include the library. This is done using the require keyword:

Code:
local ffi = require("ffi")
FFI can then be used as ffi.* throughout your script. I have linked the official Lua FFI documentation, so you can refer to that for some more functionality provided that I will not be covering. NOTE: functions such as ffi.c have been blocked for security reasons.

How to achieve things in Lua:
Lua does not provide us with a dedicated operator for dereferencing pointers like C does, but we're in luck because we can treat our pointer like an array with only one element, because in memory an array is really just a pointer to the first element of the array (like strings are pointers to the first character in C). So if we wish to access the first (and only) element of our "array" we can use... [0]! Look at the following example:

Code:
byte our_array[5] = {0x74, 0x20, 0x62, 0x65, 0x20};
In the above snippet, we see an array getting declared and initialised with 5 elements, I have provided an example of what it would look like in memory if this was to be stored directly in data (as a global variable in a C program).

ZEyeY8S.png


You can see that each value has been stored as a single byte, and that our_array has been given the address 0x400060. Now as I have said before, to us, our_array is just an array, but what really happens is it is turned into a pointer of type byte* and trying to do certain things, such as printing this array will instead print the address of the array. Reading from the array using [n] simply adds n to the pointer, while taking Pointer Arithmetic into account and dereferencing, so these are all equivalent:

Code:
int first_value = *our_array; // Dereferencing will just give the first value
int first_value = our_array[0]; // Same as regular dereferencing, as we are just adding 0.
So we can imagine, that if we have a pointer it is, in memory, identical to an array containing one value, both are variables that contain an address that points to one single value, so we use [0] to dereference in Lua.

So now we know how to read from our pointers, but how do we create them? Well, the Lua API provided to us comes with 2 important functions that have been briefly mentioned before.

Signatures and signature scanning:
Signatures are simply sequences of bytes in memory, they consist of a signature and a mask, the mask tells us which values can be wildcards (they can be any value, I'll explain why this is important later) and which must be present. The signature is the actual pattern in memory. They are most commonly represented in 2 ways, raw byte strings and IDA's style, however esoterik decided to be a cretin and use some unloved god-child mix of the two. Below are some examples of each:

Code:
IDA Style: 55 8B EC 83 E4 ? ? ? ? 53 56 57 8B F1 E8 - The ?'s represent the wildcards.
Code Style: \x55\x8B\xEC\x83\xE4\x00\x00\x00\x00\x53\x56\x57\x8B\xF1\xE8, xxxxx????xxxxxx - The ?'s in the second part are the wildcards.
GameSense Style: \x55\x8B\xEC\x83\xE4\xCC\xCC\xCC\xCC\x53\x56\x57\x8B\xF1\xE8 - The \xCC's represent the wildcards.
For IDA users, this modified version of Aidan Khoury's SigMaker can be used to create GameSense style code signatures:
SigMaker extension by infirms1337

Here is a quick example of how signatures can be used to find a function:
In the Source Engine, there is a function in the C_CSPlayer class called SetAbsAngles. Now the class C_CSPlayer gets updated a lot so therefore the index of this function (think of functions in classes like an array for now, I'll talk more about this in a future update) may change a lot so I'm going to sig this function. Using IDA or some other means, I can locate the function and use the SigMaker plugin to generate the signature 55 8B EC 83 E4 F8 83 EC 64 53 56 57 8B F1 E8.
If we are to open client.dll in IDA and use the "search for sequence of bytes" search method and input the that signature - We will see:

phkGjrn.png


This confirms that our signature is correct and will provide us with an address, but you might wonder how does this actually work? Look at this screenshot taken from IDA of the disassembly of that function:

nUPcsfF.png


Ignoring the actual disassembly, take a look at the numbers in grey on the left, these are the opcodes of the x86 assembly. What IDA as well as client.find_signature will do is scan memory for the pattern you provide it, until it matches completely and it will return the address in which the first match was found, which is why you must ensure that either there is only one match for your signature, or all the matches are completely identical.

So for example, we could do this in Lua as such:

Code:
-- Specify the DLL that the signature is inside of, as well as handling an error if the signature wasn't found, this is always good practice.
local signature_match = client.find_signature("client.dll", "\x55\x8B\xEC\x83\xE4\xF8\x83\xEC\x64\x53\x56\x57\x8B\xF1\xE8") or error("Signature was not found")

-- If everything went to plan, the address we now have in signature_match is a pointer to a function of type void(void*, float*).
-- This could also be done by using ffi.cdef, but global typedefs should be avoided.
local set_abs_angles = ffi.cast("void(__thiscall*)(void*, float*)", signature_match)

-- You can now call set_abs_angles providing it with the correct arguments.
...
A lot is used here, ffi.cdef and ffi.cast will be talked about in more detail soon.

Source engine's interface system:
The Source Engine is comprised of many different DLL's that must interact with each other, to do this, the engine has a system called interfaces. Each DLL exposes a function to other modules called CreateInterface. What this allows other modules to do is call that function and give a specific interface name, and be provided with a pointer to that class, containing all the functions it needs. As you can imagine we can do this too as anyone can call these functions. The Lua API has provided us with a quick and easy way to do this, through client.create_interface. Interface names can be sound by looking through the leaked CS:GO source code, specifically file names beginning with i such as istudiorender.h. Working with interfaces is arguably more complex than signatures, as the structure of abstract classes is a bit more complex. The structure is as follows:

Code:
-- This ffi.cdef is put here purely for readability, still avoid using global typedefs, especially if you plan on having many users.
ffi.cdef[[
typedef struct _our_interface {
void** virtual_function_table;
} our_interface;
]]

local our_interface = ffi.cast("our_interface*", _client.create_interface(...))

-- If you look at the example interface I provided earlier in the Source SDK, you can see many function labelled virtual,
-- these functions are stored as pointers in an array which is held in the first member of every interface, to access this
-- you can do what I have done and defined a helper struct to make accessing this easier, or cast it to something like
-- void** and dereference to get the address of this array.
local our_virtual_function_table = our_interface.virtual_function_table

-- I have defined the member virtual_function_table as void** because as I mentioned earlier, arrays are represented as pointers in memory,
-- so we can access this list of functions like an array, and I have also made it an array of void*, as void* represents any pointer type.

-- Lets say index 5 is a function pointer of type void(int) that we want to call.
ffi.cast("void(*)(int)", our_virtual_function_table[5])(10)
sapphyrus has just told be about some undocumented functions that the Lua API also provides to make virtual function table traversal easier,
vtable_bind and vtable_thunk can be used to quickly attach to a VFT which isn't directly exposed, and retrieve functions from it safely and quickly:

Code:
-- Bind to the NetChannelInfo interface, which is retrieved from the GetNetChannelInfo function from the VEngineClient interface.
local native_GetNetChannelInfo = vtable_bind("engine.dll", "VEngineClient014", 78, "void*(__thiscall*)(void*)")

-- Lookup functions in the currently bound VFT.
local native_GetLatency = vtable_thunk(9, "float(__thiscall*)(void*, int)")
local native_GetAvgLatency = vtable_thunk(10, "float(__thiscall*)(void*, int)")
...
Casting:
Casting is extremely useful and can be achieved through the ffi.cast function. I spoke briefly about casting earlier where I told the programming language that a number was actually meant to be interpreted as a pointer. This is the same in Lua, through the use of FFI, a number retrieved from client.find_signature or client.create_interface can be casted into a pointer value, allowing you to read and modify memory as you please. Some examples using ffi.cast are located below and throughout the rest of the post.

Some examples:
This example is taken from my Netvar Proxy hooking library, and it shows you how to call a function from an interface received from client.create_interface.

Code:
-- Use client.create_interface to find the exported interface from client.dll
local chl_client = cast("void***", client.create_interface("client.dll", "VClient018")) or error('ChlClient is nil.')

-- Dereference once to get the address of the VFT, which points to the first entry.
local chl_client_function_table = chl_client[0]

-- Access index 8 into the function table as that is the index of the function I need.
local get_all_classes = cast(ffi.typeof("ntv_ClientClass*(__thiscall*)(void*)"), chl_client_function_table[8])

-- When calling virtual functions, they will always require the first parameter to be the this pointer, which is just a pointer
-- To the class they belong to, like this.
local client_class = get_all_classes(chl_client)
Thank you for reading! I hope this helped you to get started with FFI and begin on your journey, I will be updating this post with examples and adding more sections in the future about more complex ideas and abstractions of memory that are commonly used in FFI development such as safe pointers in Lua through FFI, as well as independent reverse engineering methods to help you keep yourself up to date, instead of relying on others. Stay tuned!

Further reading:
C Typedefs
FFI Documentation
Official FFI tutorial
 
shape1
shape2
shape3
shape4
shape5
shape6
Top