Techniques for evading automated analysis

Many exploit tools, such as LOIC and the BOMBA exploit pack, evade detection in a sandbox by employing message boxes for activation.

Figure 1. Malware using MessageBox for activation.

The MessageBox and MessageBoxEx APIs display dialog boxes, sets of buttons and specific messages. As shown in Figure 1, the function returns an integer value indicating that a button has been clicked (and which one). The rest of the code will only execute after this integer value has been returned (i.e. after a response has been registered).

In the case of an automated analysis system, since there is no human interaction (no buttons being clicked), the malicious code will remain dormant. However, in a real life scenario, a user is likely to click the button, thus activating the malware.

Many pieces of malware, such as Trojan UpClicker, employ hooking of the mouse to evade sandbox analysis. As shown in Figure 2, UpClicker calls the SetWindowsHookExA() function with 0Eh as a parameter. This is used to hook the mouse. Function fn is the pointer to the callback procedure in the code, i.e. when mouse activity takes place, the code pointed to by the function fn will be called.

Figure 2. Upclicker code showing hooking of the mouse.

The code pointed to by fn (see Figure 3) reveals that the function monitors mouse movements. Specifically, it monitors the press of the left mouse button down and up.

Figure 3. Code pointed to by fn.

As can be seen from the code shown in Figure 3, the UnhookWindowsHookEx() function is only called when the left mouse button moves up. The call to this function will unhook the malicious code from the mouse, after which it makes a call to the sub_401170() function, which executes the malicious code. So until the left mouse button is released (which won’t happen in an automated analysis system as there is no human interaction/mouse activity), the code will remain dormant, making it immune from automated analysis in a sandbox.

VMware is widely used for creating a virtualized environment for automated analysis. In order to detect the presence of VMware, one technique employed by malware is to check for specific services used only by VMware. Some of these are: vmicheatbeat, vmci, vmdebug, vmmouse, vmscis, VMTools, vmware, vmx86, vmhgfs and vmxnet.

One of the methods for detecting such services is to use the RegOpenKeyExA() function and check for the services (see Figure 4). If the RegOpenKeyExA() function succeeds, the return value will be a non zero error code.

Figure 4. Malware using the RegOpenKeyExA() function to check for the presence of VMware.

(Click here to view a larger version of Figure 4.)

Another way to detect the presence of VMware is to check for the presence of files used by the program. For example, malware may use the GetFileAttributeA() function with the files used by VMware as its parameter (see Figure 5).

Figure 5. Malware using GetFileAttributeA() to determine the presence of vmmouse.

(Click here to view a larger version of Figure 5.)

The GetFileAttributeA() function will retrieve the system attributes for a specified file or directory. As shown in Figure 5, after the function call, the code ‘cmp eax, 0FFFFFFFh’ checks if the value returned is -1. This denotes that the function is unable to retrieve the attributes of the file (in this case vmmouse.sys), hence in this case the environment in which the code is executing is not a VMware virtualized environment.

VMware uses the VMX port to communicate with the virtual machine. The port is checked by malware to detect the presence of VMware. The flow of instructions is shown in Figure 6.

Figure 6. Malware using IO ports to detect VMware.

The instruction ‘move eax, “VMXh”’ loads the value 0x564D5868 into the EAX register. In the subsequent instruction, EBX is loaded with any value, after which ECX is set to 0Ah. This value (0Ah) will get the VMware version. The DX register is set to the port ‘VX’ which, as discussed above, allows interfacing with VMware. The instruction ‘in eax, dx’ is then called to read from the port into EAX. In the presence of VMware the call will succeed, while if VMware is absent an exception will occur. Once a malicious program has detected the presence of VMware, it will stop execution.

Besides checking the execution environment for VirtualBox, VMware and VPC images, many malicious programs also check for proprietary automated malware analysis systems such as Sandboxie or Joe Sandbox, Anubis and CWSandbox. One of the commonly used methods to check for the presence of these is to check for a unique product key.

As shown in Figure 7, to check the product ID, the registry key HKLM\Software\Microsoft\Windows\CurrentVersion\ is opened using the RegOpenKeyExA() function. After that, a call is made to the RegQueryValueEx() function. The RegQueryValueEx() function retrieves the type and data for the specified value associated with the open registry key. The data value returned by the function is then compared with known product IDs. If the value matches the product ID for Joe Sandbox, CWSandbox or the Anubis sandbox then the presence of these automated analysis systems can be detected.

Figure 7. Malware checking specific product ID.

(Click here to view a larger version of Figure 7.)

Another way to detect proprietary automated analysis systems is to check for the presence of processes and DLLs that are specific to the systems. As shown in Figure 8, the malware uses the GetModuleHandleA() function to get the handle of the file ‘sbiedl.dll’. This is a DLL that is loaded within the sbiectrl.exe process of Sandboxie, a proprietary sandbox used for automated analysis.

Figure 8. Malware checking for the presence of Sandboxie.

If the GetModuleHandleA() function is able to get the handle of the sbiedl.dll module, then the malicious code will know that it is executing inside Sandboxie.

As shown in Figure 9, malware checks for the presence of Joe Sandbox by checking for processes specific to the system including joeboxserver.exe and joeboxcontrol.exe. If any of the processes are present, the malware will not execute.

Figure 9. Malware checking for Joe Sandbox.

Virtualized environments are set to execute malware within a given time frame. One way for malware to evade automatic analysis is to use a sleep call with a long delay. As shown in Figure 10, sleep calls accept parameters in milliseconds. The execution of the code is suspended for a number of milliseconds.

Figure 10. Malware using sleep calls.

Since the virtualized environment is set to capture the behaviour of a piece of malware within a given time frame, a long sleep call will delay the execution of the malware, thus bypassing analysis by the automated system.