Impacket Developer Guide. Part 1. RPC Deep Dive
Hello everybody, my name is Michael Zhmailo and I am a penetration testing expert in the CICADA8 team.
We’re starting a series of articles that will describe how to develop using Impacket. This is part one.
Introduction
We believe that our story should start with the basics. Therefore, the first (and, spoiler, the second) parts will pay a lot of attention to the basics of RPC, rather than directly programming. However, I will actively show examples of C++ code for ease of understanding. In addition, the text of the article contains many references to research and material from other authors. This way, you will get a full and detailed understanding of RPC before moving on to the issues of developing tools using Impacket. We’ll learn Impacket from scratch to the point where you can write a custom tool for Lateral Movement!
What’s RPC?
So, RPC stands for Remote Procedure Call and allows us to access the functionality of server programs remotely. Moreover, RPC can work on top of many different transports. All supported RPC transport can be divided into two categories:
- Connectionless — UDP, IPX and more..
- Connection — TCP, SMB, HTTP, NetBIOS over TCP and more...
In RPC you have a server that hosts RPC functionality and you have a client that accesses that functionality. I can even show you some simple pseudocode.
# Server.cpp
int Add(int a, int b)
{
return a+b;
}# Client.cpp
int main()
{
rpc_h handle = RpcConnectToServer("10.10.10.10","1337");
int result = RpcInvokeMethod(handle, Add, 5, 2);
std::cout << result << std::endl; # 7
return 0;
}Interfaces, Stubs and Marshalling
One of the most important parts of RPC in Windows is the interfaces. It is in them that all the functionality that the server provides to the client is written. These files contain a unique interface identifier (UUID), as well as a set of methods that the client can use. Interfaces are described using a special Interface Definition Language, and interface files have the extension .idl. Let’s check the example.
[
uuid(AB4ED934-1293-10DE-BC12-AE18C48DEF33),
version(1.0),
]
interface Calculator
{
int Add(
[in] handle_t hBinding,
[in] int a,
[in] int b
);
}Interface files correspond to a special description language MIDL (Microsoft Interface Definition Language). It is worth noting that interface files are needed not only by us as developers, but also by a special compiler called the MIDL compiler. The MIDL compiler is needed to generate so-called stubs. Stubs are needed to pass parameters from the client process to the server process and back. This process is called marshalling.
Why is marshalling needed?
Often, the functions provided by the RPC server are much more complex than the ones in my example. Such complex functions accept many arguments, including large data structures: lists, dictionaries, entire classes and structures. To correctly transfer this type of data, you need to convert it into a format suitable for transmission over the network. This is what marshalling is responsible for.
Thus, if you have an IDL file describing the functionality of the RPC server, you can generate all the necessary stubs to correctly transmit the necessary arguments of the RPC functions over the network.
Generation is automatic and usually is done using the midl.exe program.
midl.exe /app_config MyInterface.idlAfter that, several files will appear in the current folder. These are the stubs. We will see how to use them in the program a bit later.
MyInterface_c.c— Client-Side stubs;MyInterface_s.c— Server-Side stubs;MyInterface_h.h— Header file.
If you look inside these files, you will see many functions whose names start with Ndr*. For example, NdrClientCall2() or NdrServerCall3(). Here we need to get acquainted with one more mechanism. It is called Network Data Representation (NDR) Engine. It is responsible for marshalling the data transmitted via RPC.
Let’s summarize. We have a client and a server, there is a file with the server interface definition. We can compile this interface using MIDL, get stubs and use stubs to call functions on the server. Passing arguments will be done using the marshalling procedure, which works using NDR mechanism. There is also RPC Runtime. This is a set of libraries that implement the transport of marshalled data and all the necessary RPC functions. An example of this is rpcrt4.dll.
RPC Binding
Let me tell you about two more terms. The process of connecting a client to a server is called RPC Binding. The server exports available interfaces, and the client, knowing their UUID, can connect to them. To be precise, the client doesn’t connect to an interface, but to an endpoint using special string called RPC String Binding. And then, if the interface the client needs is configured on this endpoint, the necessary RPC methods are called. An RPC String Binding is simply a string sequence that points to a specific endpoint.
# Connect over TCP
ncacn_ip_tcp:192.168.0.24[49664]
# Connect over SMB
ncacn_np:\\DESKTOP-0PRT7UI[\pipe\lsass]The RPC String Binding itself is not as simple as it may seem at first glance. This string has the following format.
InterfaceUUID@ProtocolSequence:NetworkAddress[Endpoint,Option]ObjectUUID— UUID of the interface that will be used to call the methods. That is, it is the RPC Interface ID. Often not specified;ProtocolSequence— the protocol used by RPC to transfer data over the network. It iscombination of the RPC protocol (e.g.ncacn), transport protocol (e.g.TCP), and network protocol (e.g.IP). MSRPC supports the following types. Here is a breakdown of some of the most common RPC protocol combinations.
NCACN (Network Computing Architecture CoNnection oriented protocol): RPC over a TCP connection
NCADG (Network Computing Architecture Datagrame Protocol): RPC over a UDP connection
NCALRPC (Network Computing Architecture Local Remote Procedure): RPC through a local connectionNetworkAddress— the remote address of the system that will receive the RPC call. The format and contents of this field depends on the specifiedProtocolSequence.
Endpoint,Options— optional fields, used to specify the endpoint more precisely. More detailed documentation is provided at the MSDN in each specificProtocolSequence.
Once we have the RPC String Binding we can connect to the RPC server. If the connection is successful, we will have an RPC Binding Handle through which we can interact with the target system.
Client-Server Connection Algorithm
Top-level
The client connects to the RPC server using string binding, calls a remote procedure, marshals the parameters and additional information in the message, and sends the message to the server. Upon receiving the message, the server unmarshals its contents, performs the requested operation, and sends the result back to the client. The server stub and the client stub take care of the marshalling and unmarshalling of the parameters. The mechanism of remote procedure calls is shown in the figure below.
+-------------------------------+ +--------------------------------+
| Client Machine | | Server Machine |
+-------------------------------+ +--------------------------------+
| | | |
| +-------------------------+ | | +--------------------------+ |
| | Client | | | | Server | |
| | +-------------------+ | | | | +--------------------+ | |
| | | Return | Call | | | | | | Call | Return | | |
| | +-------------------+ | | | | +--------------------+ | |
| +------------|------------+ | | +------------|-------------+ |
| | | | | |
| +------------|------------+ | | +------------|-------------+ |
| | Client stub | | | | Server stub | |
| | +--------+ +--------+ | | | | +--------+ +--------+ | |
| | | Unpack | | Pack | | | | | | Unpack | | Pack | | |
| | +--------+ +--------+ | | | | +--------+ +--------+ | |
| +------------|------------+ | | +------------|-------------+ |
| | | | | |
| +------------|------------+ | | +------------|-------------+ |
| | RPC Runtime | | | | RPC Runtime | |
| | +--------+ +--------+ | | | | +--------+ +--------+ | |
| | |Receive | | Send | | | | | |Receive | | Send | | |
| | | | | | | | | | | | | | | |
| | +--------+ +--------+ | | | | +--------+ +--------+ | |
| +------------|------------+ | | +------------|-------------+ |
| | | | | |
+---------------|----------------+ +---------------|----------------+
| |
| Call Packet |
|------------------------------------->|
| |
| Result Packet |
|<-------------------------------------|
| |
+---------------|----------------+ +---------------|----------------+Low-level
- First, the server developers create the functions that it wants to be available via RPC.
- Then, the developers create an interface file (.idl) so that clients can generate stubs. This interface file is passed to the client. It is worth noting that the client itself, if it applies a little RPC server analysis skills, can create an IDL file for itself, but more on that later.
- The server registers its interfaces in the RPC Runtime Library using the RpcServerRegisterIf2() function
# MyInterface_s.c (Generated by MIDL)
static const RPC_SERVER_INTERFACE MyInterface___RpcServerInterface =
{
sizeof(RPC_SERVER_INTERFACE),
{{0xd6b1ad2b,0xb550,0x4729,{0xb6,0xc2,0x16,0x51,0xf5,0x84,0x80,0xc3}},{1,0}},
{{0x8A885D04,0x1CEB,0x11C9,{0x9F,0xE8,0x08,0x00,0x2B,0x10,0x48,0x60}},{2,0}},
(RPC_DISPATCH_TABLE*)&MyInterface_v1_0_DispatchTable,
0,
0,
0,
&MyInterface_ServerInfo,
0x06000000
};
RPC_IF_HANDLE MyInterface_v1_0_s_ifspec = (RPC_IF_HANDLE)& MyInterface___RpcServerInterface;
# MyInterface_h.h (Generated by MIDL)
extern RPC_IF_HANDLE MyInterface_v1_0_s_ifspec;
# RpcServer.cpp
RpcServerRegisterIf2(
MyInterface_v1_0_s_ifspec,
NULL,
NULL,
0,
RPC_C_LISTEN_MAX_CALLS_DEFAULT,
(unsigned)-1,
NULL); - The server fills in information about the endpoints through which the functionality described in the interfaces can be accessed. This is done using the RpcServerUseProtseq() function. Here’s example of TCP RPC Server.
RPC_WSTR pszProtSeq = (RPC_WSTR)L"ncacn_ip_tcp";
RPC_WSTR pszTCPPort = (RPC_WSTR)L"1337";
RpcServerUseProtseqEp(
pszProtSeq,
RPC_C_PROTSEQ_MAX_REQS_DEFAULT,
pszTCPPort,
NULL
);Now we have two options. We can set up authentication information and start waiting for clients, or we can look at the Dynamic Endpoint mechanism. Since I am trying to make the most comprehensive review, I suggest looking at what RPC Dynamic Endpoint is.
— — LOW-LEVEL PAUSE — —
Types of RPC Endpoints
There are two main types of endpoints: RPC Well-Known and RPC Dynamic Endpoint. An RPC Well-Known endpoint is one that the client connects to immediately, the client immediately knows the full address of the endpoint. An example of such a point: ncacn_ip_tcp:10.10.10.10[4444] or ncacn_np:\\10.10.10.10[\pipe\lsass]
However, there is also a second type of point — dynamic. In this case, the client only has the Protocol Sequence used and the address. Example: ncacn_ip_tcp:10.10.10.10 or ncacn_np:\\10.10.10.10
In this case, the client needs to contact a special Endpoint Mapper service to get a full RPC String Binding. First, the server registers in this service, and it assigns a free endpoint with the required Protocol Sequence. After that, the server starts listening. When the client wants to connect to the server, it contacts the epmapper service, provides it with the server interface, and the service returns the data for the connection. For example, the missing TCP port.
Let’s look at some example code.
Here’s RPC Well-Known registration.
RPC_STATUS rpcStatus;
// Interface Registration
rpcStatus = RpcServerRegisterIf2(...);
if (rpcStatus != RPC_S_OK)
return;
// Well-Known Endpoint
RpcServerUseProtseqEp(
(RPC_WSTR)L"ncacn_ip_tcp",
RPC_C_PROTSEQ_MAX_REQS_DEFAULT,
(RPC_WSTR)L"1337",
NULL
);
// Start Listening
RpcServerListen(
1,
RPC_C_LISTEN_MAX_CALLS_DEFAULT,
FALSE
);And here’s RPC Dynamic Endpoint registration.
RPC_STATUS rpcStatus;
// Interface Registration
rpcStatus = RpcServerRegisterIf2(...);
if (rpcStatus != RPC_S_OK)
return;
// Well-Known Endpoint
RpcServerUseProtseqEp(
(RPC_WSTR)L"ncacn_ip_tcp",
RPC_C_PROTSEQ_MAX_REQS_DEFAULT,
NULL // port missing (that's NULL for security descriptor)
);
// Registration in the epmapper service
RPC_BINDING_VECTOR* pbindingVector = 0;
RpcServerInqBindings(&pbindingVector);
RpcEpRegister(
MyInterface_v1_0_s_ifspec,
pbindingVector,
0,
(RPC_WSTR)L"MyDynamicIFace"
);
// Start Listening
RpcServerListen(
1,
RPC_C_LISTEN_MAX_CALLS_DEFAULT,
FALSE
);As you can see, a call to the RpcServerInqBindings() and RpcEpRegister() functions has been added, which are used to register the Dynamic Endpoint.
— — LOW-LEVEL CONTINUE — —
- After registering endpoints, the server can optionally configure authentication using RpcServerRegisterAuthInfo().
RPC authentication is provided by special providers called RPC Security Packages. Here’s list of the available providers.
unsigned long dwAuthnSvc = RPC_C_AUTHN_WINNT;
RpcServerRegisterAuthInfo(null, dwAuthnSvc, NULL, NULL);By the way, if my memory serves me right, this is the function that is called from CoInitializeSecurity(). And the CoInitializeSecurity() function was used in the RemoteKrbRelay tools to abuse Kerberos tickets relay. In fact, in this function we can give an arbitrary SPN, and the client will come to us with this SPN. Find code here.
- Finally, the server can start listening for clients using RpcServerListen().
RpcServerListen(
1,
RPC_C_LISTEN_MAX_CALLS_DEFAULT,
FALSE
);Now let’s go to the Client Side!
- The client creates a Binding Handle via the RpcStringBindingCompose() and RpcBindingFromStringBinding() functions
// Glues into one line
RpcStringBindingComposeW(RPC_UUID, (RPC_WSTR)L"ncacn_np", (RPC_WSTR)host, InterfaceAddress, NULL, &StringBinding);
// Create Variable
RPC_WSTR bindingHandle = 0;
RpcBindingFromStringBindingW(StringBinding, &bindingHandle);- The client calls the RPC Runtime library, which makes a call to the RPC server and requests information about endpoints from the Endpoint Mapper service. It is worth noting that this behavior is only for dynamic endpoints. This operation is performed by the RPC Runtime Library. We don’t need to call any methods. However, if you want, you can call RpcEpResolveBinding()
RpcEpResolveBinding(
bindingHandle,
MyInterface_v1_0_c_ifspec
);- The client can specify authentication parameters on the endpoint via RpcBindingSetAuthInfo();
- The client makes a call to some RPC method. RPC methods are declared in the interface file;
RpcTryExcept {
Add(bindingHandle, 2 ,3);
}- The client RPC Runtime library marshals the parameters passed to the method into the NDR format and sends it to the server;
- The server RPC Runtime receives data and unmarshals it in order to pass it to the RPC method code;
- When the called method on the server returns the control flow, the server stub receives the value of the parameters declared as
[out]and[in,out](defined in the IDL file), as well as the return value of the method, marshals them and sends the marshalled data back to the client.
Success : )
RPC Handle Binding Types
Before I look at some code examples, I’d like to introduce you to some more basic terms. So, in RPC there are still different types of Handle Binding. Globally there are Explicit Handle Binding and Implicit Handle Binding.
The binding type is defined in idl files. In the examples above we used Explicit Handle Binding.
# THAT'S EXPLICIT
[
uuid(AB4ED999-1233-10AA-BC67-BB12C48GHM33),
version(1.0),
]
interface Calculator
{
int Add(
[in] handle_t hBinding,
[in, string] int a,
[in] int b
);
}
RpcTryExcept {
Add(bindingHandle, 2 ,3);
}
# THAT'S IMPLICIT
[
uuid(AB4ED999-1233-10AA-BC67-BB12C48GHM33),
version(1.0),
implicit_handle(handle_t hBinding)
]
interface Calculator
{
int Add(
[in, string] int a,
[in] int b
);
}
RpcTryExcept {
Add(2 ,3);
}Do you see the difference?
So, Explicit Binding requires explicit passing of Binding Handle on each RPC call. This allows to use one Binding Handle in multithreaded applications.
Implicit Binding is used in single-threaded applications. It does not pass Binding Handle on each function call.
NDR Engine Deep Dive
You might want to take a deeper look at how NDR Engine works. In that case, you’ll encounter a real horror! A huge number of nested structures and classes, pointers and a complete mess :) However, a long time ago in X (thx to sixtyvividtails) i saw this beautiful visualization. Now I share it with you.
In fact, this is a description of what happens and what structures are used inside Ndr-functions.
RPC Examples
Here is an example of a client-server program using Well-Known Endpoints. So we have the following interface.
[
uuid(AB4ED934-1293-10DE-BC12-AE18C48DEF33),
version(1.0),
]
interface Calculator
{
int Add(
[in] handle_t hBinding,
[in] int a,
[in] int b
);
}We go to Visual Studio, name the project Server, add this file to the project, name it interface.idl. Then we create another project in the project. Name it Client. It should look like this
Paste this code into Client.cpp and Server.cpp.
// Client.cpp
#include <windows.h>
#include <stdio.h>
#include "interface_h.h"
#pragma comment(lib, "rpcrt4.lib")
int main() {
RPC_STATUS status;
RPC_WSTR stringBinding = NULL;
int result;
handle_t hBinding = NULL;
status = RpcStringBindingCompose(
NULL,
(RPC_WSTR)L"ncacn_ip_tcp",
(RPC_WSTR)L"localhost",
(RPC_WSTR)L"1337",
NULL,
&stringBinding);
if (status) {
printf("RpcStringBindingCompose failed: %d\n", status);
return status;
}
status = RpcBindingFromStringBinding(stringBinding, &hBinding);
if (status) {
printf("RpcBindingFromStringBinding failed: %d\n", status);
return status;
}
RpcTryExcept{
result = Add(hBinding, 5, 3);
printf("5 + 3 = %d\n", result);
result = Add(hBinding, 10, 20);
printf("10 + 20 = %d\n", result);
}
RpcExcept(1)
{
printf("Runtime reported exception %d\n", RpcExceptionCode());
}
RpcEndExcept;
RpcStringFree(&stringBinding);
RpcBindingFree(&hBinding);
return 0;
}
void* __RPC_USER midl_user_allocate(size_t size) {
return malloc(size);
}
void __RPC_USER midl_user_free(void* p) {
free(p);
}// Server.cpp
#include <windows.h>
#include <stdio.h>
#include "interface_h.h"
#pragma comment(lib, "rpcrt4.lib")
int Add(handle_t hBinding, int a, int b) {
printf("Received numbers: %d and %d\n", a, b);
return a + b;
}
int main() {
RPC_STATUS status;
RPC_BINDING_VECTOR* bindingVector = NULL;
status = RpcServerUseProtseqEp(
(RPC_WSTR)L"ncacn_ip_tcp",
RPC_C_PROTSEQ_MAX_REQS_DEFAULT,
(RPC_WSTR)L"1337",
NULL
);
if (status) {
printf("RpcServerUseProtseq failed: %d\n", status);
return status;
}
status = RpcServerRegisterIf2(
Calculator_v1_0_s_ifspec,
NULL,
NULL,
0,
RPC_C_LISTEN_MAX_CALLS_DEFAULT,
(unsigned)-1,
NULL);
if (status) {
printf("RpcServerRegisterIf2 failed: %d\n", status);
return status;
}
printf("Calculator Server is listening...\n");
status = RpcServerListen(1, RPC_C_LISTEN_MAX_CALLS_DEFAULT, FALSE);
if (status) {
printf("RpcServerListen failed: %d\n", status);
return status;
}
return 0;
}
void* __RPC_USER midl_user_allocate(size_t size) {
return malloc(size);
}
void __RPC_USER midl_user_free(void* p) {
free(p);
}// interface_h.h (FOR REVIEW ONLY!! DO NOT ATTACH TO THE PROJECT!! U'LL COMPILE IT FROM IDL FILE)
/* this ALWAYS GENERATED file contains the definitions for the interfaces */
/* File created by MIDL compiler version 8.01.0628 */
/* at Tue Jan 19 08:14:07 2038
*/
/* Compiler settings for interface.idl:
Oicf, W1, Zp8, env=Win64 (32b run), target_arch=AMD64 8.01.0628
protocol : all , ms_ext, c_ext, robust
error checks: allocation ref bounds_check enum stub_data
VC __declspec() decoration level:
__declspec(uuid()), __declspec(selectany), __declspec(novtable)
DECLSPEC_UUID(), MIDL_INTERFACE()
*/
/* @@MIDL_FILE_HEADING( ) */
/* verify that the <rpcndr.h> version is high enough to compile this file*/
#ifndef __REQUIRED_RPCNDR_H_VERSION__
#define __REQUIRED_RPCNDR_H_VERSION__ 500
#endif
#include "rpc.h"
#include "rpcndr.h"
#ifndef __RPCNDR_H_VERSION__
#error this stub requires an updated version of <rpcndr.h>
#endif /* __RPCNDR_H_VERSION__ */
#ifndef __interface_h_h__
#define __interface_h_h__
#if defined(_MSC_VER) && (_MSC_VER >= 1020)
#pragma once
#endif
#ifndef DECLSPEC_XFGVIRT
#if defined(_CONTROL_FLOW_GUARD_XFG)
#define DECLSPEC_XFGVIRT(base, func) __declspec(xfg_virtual(base, func))
#else
#define DECLSPEC_XFGVIRT(base, func)
#endif
#endif
/* Forward Declarations */
#ifdef __cplusplus
extern "C"{
#endif
#ifndef __Calculator_INTERFACE_DEFINED__
#define __Calculator_INTERFACE_DEFINED__
/* interface Calculator */
/* [version][uuid] */
int Add(
/* [in] */ handle_t hBinding,
/* [in] */ int a,
/* [in] */ int b);
extern RPC_IF_HANDLE Calculator_v1_0_c_ifspec;
extern RPC_IF_HANDLE Calculator_v1_0_s_ifspec;
#endif /* __Calculator_INTERFACE_DEFINED__ */
/* Additional Prototypes for ALL interfaces */
/* end of Additional Prototypes */
#ifdef __cplusplus
}
#endif
#endif
#interface_c.c (FOR REVIEW ONLY!! DO NOT ATTACH TO THE PROJECT!! U'LL COMPILE IT FROM IDL FILE)
https://gist.github.com/MzHmO/1d7cb961771a03065364a916cacbbed8#interface_s.c (FOR REVIEW ONLY!! DO NOT ATTACH TO THE PROJECT!! U'LL COMPILE IT FROM IDL FILE)
https://gist.github.com/MzHmO/56370b75a25417482789d788b53589eaThen click on the button to build the solution. And in your home directory there will be files that were generated by the midl compiler.
Include the _s file in the server project, the _c file in the client project, and include the _h header file in both projects.
Finally, run the project and check that everything works.
I also recommend this excellent repository, which shows quite a few different RPC clients and servers, both with Implicit and Explicit bindings. In addition, the examples include authentication setup. We’ll talk about RPC security in Part 2. However, you can try to study the code optionally!
I also share with you a repository of RPC clients on C#. And pay attention to SharpSystemTriggers. It shows how to work with MIDL representation of interfaces in C#.
If you are interested in developing in Powershell, I recommend this repository. It is great because it contains the implementation of all RPC mechanisms at the deepest possible level.
Some Troubleshooting
U can see successfully created MyInteface_c.c and MyInterface_s.c stubs from IDL file, however when trying to compile you will get errors: C2011, LNK2001.
To solve LNK2001, add these lines to your code.
#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#include <rpc.h>
void __RPC_FAR* __RPC_API midl_user_allocate(size_t cBytes);
void __RPC_API midl_user_free(void __RPC_FAR* p);
void __RPC_FAR* __RPC_API midl_user_allocate(size_t cBytes)
{
return HeapAlloc(GetProcessHeap(), HEAP_ZERO_MEMORY, cBytes);
}
void __RPC_API midl_user_free(void __RPC_FAR* p)
{
HeapFree(GetProcessHeap(), 0, p);
}If it writes that it cannot find some Ndr function, then it is enough to link the program withrpcrt4.lib.
#pragma comment(lib, "Rpcrt4.lib")The C2011 issue is caused by type redefinitions in header files. You should try to simply remove all data structures that are redefined in the _h.h file.
If you have any other compilation error, please see this documentation.
Conclusion
In this article, we learned the basics of RPC and got one step closer to developing programs using Impacket. In the next part, we will take a closer look at RPC Security, as well as learn how to explore available RPC servers on the system. Subscribe to us in X!
