I’m watching you! How to spy Windows users via MS UIA

Hello everybody, my name is Michael Zhmailo and I am a penetration testing expert in the MTS Innovation Center CICADA8 team.

We’re sure you’ll agree that keeping track of users has always been an interesting challenge. It’s like exploring their secrets, finding passwords, and discovering attack scenarios that depend only on the human factor.
We’re excited to share our new research about using a User Interface Automation framework for user spying. In this article, we’ll take you on a journey through the Windows Component Object Model, create our own event handlers, and learn about the Windows graphical representation tree, among other things.

We have also developed a POC called Spyndicapped.

Tool help

What’s MS UIA?

MS UIA (Microsoft User Interface Automation) is a special framework designed to automate the use of the Windows GUI. With its help you can read any text values on the screen, open menus, close windows.

In general, you can do everything a user does in front of a monitor, just as if a hacker were sitting at a desktop.

The framework was originally designed for handicapped people, hence the name of the tool — Spyndicapped.

MSAA (Microsoft Active Accessibility) was heavily used in Windows prior to MS UIA, but it has been completely replaced by MS UIA. I should note that MS UIA is supported by almost all modern systems, starting with Windows XP, so I don’t see any point in diving into MSAA.

U can read more information about MSAA and UIA comparsion here.

Basic Structure

Elements, Properties and so on

So, let’s take a look at the basic structure.

The framework itself is made up of several main components: elements, properties, events, and patterns.

Highlighting all elements

Highliting all properties

Pattern calling

Appearance of UIA_Window_WindowOpenedEventId event

Elements are everything you see on the screen, like text, buttons, menus, and anything else there. Properties are what make each element special. For example, the Name property of an element contains the NAME of the element, and the Value property is the value. Flags are various additional flags. Hopefully, you get the idea.

It’s a bit like the built-in graphics engine in Windows, but for UIA. You can handle any events related to the GUI, like entering data, changing a property, or opening a window.

And finally, patterns provide the opportunity for live interaction with elements, which is really cool. For example, the Scroll pattern allows you to scroll Scroll Bars using program code, and the Invoke pattern simulates a function call that occurs when a button is clicked.

Graphical representation of elements in Windows

(aka Automation Tree or Control Tree)

All graphical elements in Windows have a strict hierarchical structure. The root of the hierarchy is the current desktop. The descendants of the desktop are the displayed windows. Descendants of windows are controls inside the window. In turn, inside the controls there may be other buttons, menus and other mechanisms with which the user interacts.

Windows automation tree view in inspect.exe (we’ll explore this tool later)

Button in the control tree

This tree structure is incredibly convenient for searching for items, traversing the tree, or handling structure changes. In Spyndicapped there are quite a few methods related to working with a tree, you can find them in a special class MyTreeWalker.

Example methods to work with tree

Let’s dive into COM!

Work with UIA is available only on top of the corresponding COM classes. You can develop a program also in CSharp, but I did it in C++, because I like them and I like to make a lot of mistakes in memory management :).

Globally, we need to use multiple interfaces:
- IUIAutomation — allows us to get started with COM. It can be obtained through the call of CoCreateInstance() function;
- IUIAutomationElement — points to a specific graphical element. It can be obtained by processing UIA events or through tree traversal (more on this later);
- IUIAutomationCondition — allows you to create conditions used when searching for the desired element. For example, you can create a condition to find an element with the name Misha123 or the value Cicada8;
- IUIAutomationTreeWalker — is needed for traversing the tree of the graphical representation.

It is worth noting that the user only sees the IUIAutomatomatioElement on the screen. Everything else is auxiliary interfaces.

This is all an IUIAutomationElement that we can interact with

Almost all of our code will be based on processing events, getting the necessary IUIAutomationElement and outputting their values.

The methods of these interfaces will help us with this, but let’s first familiarize ourselves with event handlers.

Event handling

Globally on top of all those entities I described earlier, there are two more roles: the UIA client and the UIA provider. The UIA client subscribes to and processes events that are received from UIA providers. You can generate your own UIA events if you write a provider. But we will write a client, since it is sufficient for POC to handle events that are provided by the built-in Windows UIA providers.

There are quite a large number of different events that can be handled. And the processing must be done with the help of a suitable provider.

Events example (accevent.exe. we’ll explore this tool later)

The IUIAutomationEventHandler is used to handle basic UIA events. These events include, for example, opening windows, calling functions. In general, all constants that start with UIA_ and end with EventId.

UIA Events example

When registering a handler, you must explicitly specify which events you want to track. If you do not specify any events, you will not be able to receive notifications. Handler registration is done using the AddAutomationEventHandler() function from the IUIAutomationEventHandler interface, to which you pass the address of the handler function and the events you want to handle.

Here’s registration example.

 std::vector<EVENTID> eventIds = {
   UIA_AutomationPropertyChangedEventId,
   UIA_Text_TextSelectionChangedEventId,
   UIA_Text_TextChangedEventId,
   UIA_Invoke_InvokedEventId,
   UIA_Window_WindowOpenedEventId,
 };

 for (size_t i = 0; i < eventIds.size(); i++)
 {
  HRESULT hr = pAutomation->AddAutomationEventHandler(eventIds[i], pAutomationElement, TreeScope_Subtree, NULL, (IUIAutomationEventHandler*)pMyAutomationEventHandler);
  if (FAILED(hr)) {
   Log(L"Failed to add event handler for event ID: " + std::to_wstring(eventIds[i]), WARNING);
   PrintErrorFromHRESULT(hr);
   return E_ABORT;
  }
 }

After that the handler function will be called. Here’s handler example.

HRESULT STDMETHODCALLTYPE MyAutomationEventHandler::HandleAutomationEvent(IUIAutomationElement* pAutomationElement, EVENTID eventID)
{
  ...
}

• pAutomationElement — the element that generated the event. For example, it will be an input field if it started receiving data;
• eventID — triggered event.

The Spyndicapped tool also registers a property handler (IUIAutomationPropertyChangedEventHandler). It allows you to track property changes that are registered with specific programs, such as chrome.exe or keepass.exe.

Adding such a handler is possible via AddPropertyChangedEventHandler() or AddPropertyChangedHandlerNativeArray().

Here’s registration example.

std::vector<int> propertyIds = {
 UIA_NamePropertyId,
 UIA_ValueValuePropertyId,
 UIA_SelectionItemIsSelectedPropertyId,
};


SAFEARRAY* propertyArray = SafeArrayCreateVector(VT_I4, 0, propertyIds.size());
if (!propertyArray)
{
 Log(L"Can't create property arrray", WARNING);
 return E_ABORT;
}

for (size_t i = 0; i < propertyIds.size(); i++)
{
 long index = static_cast<long>(i);
 SafeArrayPutElement(propertyArray, &index, &propertyIds[i]);
}

hr = pAutomation->AddPropertyChangedEventHandler(pAutomationElement, TreeScope_Subtree, NULL, pMyPropertyChangedEventHandler, propertyArray);

if (FAILED(hr)) {
 Log(L"Failed to add property changed event handler" , WARNING);
 PrintErrorFromHRESULT(hr);
 return E_ABORT;
}

And handler’s function.

HRESULT STDMETHODCALLTYPE MyPropertyChangedEventHandler::HandlePropertyChangedEvent(IUIAutomationElement* pAutomationElement, PROPERTYID propId, VARIANT vVar)
{
  ...
}

• pAutomationElement — the element that generated the event. For example, it will be an input field if it started receiving data;
• propId — id of the modified property;
• vVar — new value in VARIANT format.

These are by no means all handlers. You can create handlers for notifications, changing focus, changing text position and much more. Here is the full list:

• IUIAutomationChangesEventHandler — any changes;
• IUIAutomationActiveTextPositionChangedEventHandler — text position changes;
• IUIAutomationTextEditTextChangedEventHandler — text modification through autocomplete and other mechanisms for the visually impaired;
• IUIAutomationFocusChangedEventHandler — focus changed;
• IUIAutomationStructureChangedEventHandler — changes in the control tree (adding a element for example);
• IUIAutomationNotificationEventHandler — notification handling.

Development assistance using UIA

How to understand which event occurs when a key is pressed? Or which graphic element should be accessed to solve our problem? Several tools (including the aforementioned inspect.exe and accevent.exe) can help us here.

WinSpy

Link

This is an extended version of the well-known Spy++ for debugging graphical elements. To view the UIA tree, click on the View Automation Tree button:

Usage example

Inspect.exe

Link

I like Inspect.exe better than WinSpy. This tool comes bundled with the Windows SDK. It displays all the necessary data about the Windows graphical tree.

inspect.exe demo

You can click on these buttons and make the item light up, which is very handy for locating the right button!

Help buttons

Yellow highlighting example

U can learn more about inspect.exe usage here.

accevent.exe

Link

With this application, you can keep track of all UIA events in Windows. In fact, it is a universal UIA Client! Usage is so simple. Firstly, click on the mode and select «UIA Events».

Then click on button «Settings» and select the desired events and the properties you want to receive about the event.

Settings example

Then go to «Events» -> «Start Listening»

And u’ll start receiving events.

Events capture

Then click on the «Stop Listening» button and start analyzing events.

How to make your own stealth logger

During my research I identified the main events that occur when typing or working with text data, after which I created two handlers:
- MyAutomationEventHandler.cpp;
- MyPropertyChangedEventHandler.cpp.

The first one allows to handle general UIA events, and the second one — property changed events.

Then i learned to identify from an event which process it came from. I did this with the help of functions:
- Finder::GetPIDByUIAutomationElement();
- Finder::GetModuleNameFromPid();

After that I implemented a simple routing mechanism on lambda functions. Depending on the process name, the corresponding function is called from the file named <HandlerName>Apps.cpp

 if (g_IgnoreHandlers)
 {
  HandleOther(pAutomationElement, wsProcName, wsEventString, wsDate, eventID);
 }
 else {
  std::unordered_map<std::wstring, std::function<void()>> handlers = {
   { L"firefox.exe", [this, pAutomationElement, wsProcName, wsEventString, wsDate, eventID]() { HandleFirefox(pAutomationElement, wsProcName, wsEventString, wsDate, eventID); } },
   { L"explorer.exe", [this, pAutomationElement, wsProcName, wsEventString, wsDate, eventID]() { HandleExplorer(pAutomationElement, wsProcName, wsEventString, wsDate, eventID); } }
  };

  auto it = handlers.find(Helpers::ConvertToLower(wsProcName));

  if (it != handlers.end()) {
   it->second();
  }
  else {
   HandleOther(pAutomationElement, wsProcName, wsEventString, wsDate, eventID);
  }
 }

For example, if the event was received in the MyAutomationEventHandler.cpp handler, the function from MyAutomationEventHandlerApps.cpp will be called.

Then inside each-function handler I created unique logic for each process.

For example, if there were changes in the firefox.exe process, the domain name of the current tab is checked. If it is web.whatsapp.com or app.slack.com, then it processes that site to find the contents of the message and the name of the recipient.

std::unordered_map<std::wstring, std::function<void()>> handlers = {
   { L"web.whatsapp.com", [this, pAutomationElement, wsProcName, wsDate]() { HandleWhatsAppFF(pAutomationElement, wsProcName, wsDate); } },
   { L"app.slack.com", [this, pAutomationElement, wsProcName, wsDate]() { HandleSlackFF(pAutomationElement, wsProcName, wsDate); } }
 };

 auto it = handlers.find(Helpers::ConvertToLower(wsDomain));

 if (it != handlers.end()) {
  it->second();
 }
 else {
  HandleOther(pAutomationElement, wsProcName, wsEventString, wsDate, eventID);
 }

As a result, you have the ability to intercept almost any potentially interesting text content.

Slack looting example

Whatsapp web looting example

Among other things, I added a separate handler for the KeePass.exe process. If Spyndicapped detects an open KeePass window, it automatically retrieves all passwords from the current tab by quickly emulating the “Copy Password” button. All this is done only by MS UIA tools, which potentially makes the tool quite stealthy from EDR.

Keepass looting example

My tool utilizes almost all of the concepts necessary for successful development with MS UIA. You can easily check the source code and modify Spyndicapped to automate tracking of almost ANY application on Windows!

Conclusion

Thank you for reading this article. Feedback is very important to me. If you liked the article, you can give it a star on Github or retwit it on the X!

Merry Christmas.