Writing Custom Event Correlation Rules in Elastic Security

As an exercise in security engineering, I will be taking a scenario involving dynamic file downloads and writing custom Event Correlation rules around the activity. In the scenario, users download a single dynamic link library (DLL) file using powershell quite often as part of development operations from an Amazon S3 bucket. The DLL file is always downloaded to the same folder on the host and is always 8 characters long. Additionally, because the file always comes from an S3 bucket, the DNS request is always the same.

Using Rule Exceptions

Elastic Security allows for “Rule Exceptions” and as of at least 8.8, allows for wildcards. The issue is, sometimes the field data is not in the right form. From the documentation, “the selected Field data type must be keyword, text, or wildcard.”

The fields from the alerts depend drastically on the configuration and ingestion pipelines. In the most default configuration, these fields are “unknown.” As such, exceptions are often difficult. To get around this, I’ve taken to re-writing custom rules in Elastic Security to allow for finer tuning utilizing their intuitive Event Query Language (EQL).

Writing the Custom Rule

To start, I will navigate to the rule I would like to replace with my custom rule. In this case, it is “Remote File Download via Powershell.” To find this rule, I can navigate to the “Manage” menu under “Security” and select “Rules.”

After making the selection, I simply search for the rule I want. Then I can duplicate that rule from the 3 dotted menu.

Next the option appears to duplicate the rule, the rule and all exceptions (including expired), or the rule and all active exceptions. I chose only the rule, as I had no exceptions written anyway. Then it’s off to the rule settings. It opens to the “Definition,” but I’ll return to this in a moment. For now, I configure the rule “About” tab. I’ve appended “Custom Rule” to the end of the title, to better keep track. Under “Advanced Settings” things like false positives can be listed, investigation guides can be better fleshed out, and MITRE ATT&CK techniques can be appended.

Next we’ll take a look at the query, the impetus for this exercise. Here I find the following EQL query:

sequence by host.id, process.entity_id with maxspan=30s
  [network where host.os.type == "windows" and process.name : ("powershell.exe", "pwsh.exe", "powershell_ise.exe") and network.protocol == "dns" and
   not dns.question.name : ("localhost", "*.microsoft.com", "*.azureedge.net", "*.powershellgallery.com", "*.windowsupdate.com", "metadata.google.internal") and
   not user.domain : "NT AUTHORITY"]
    [file where host.os.type == "windows" and process.name : "powershell.exe" and event.type == "creation" and file.extension : ("exe", "dll", "ps1", "bat") and
   not file.name : "__PSScriptPolicy*.ps1"]

While it may look intimidating, this query simply tracks two events, a DNS request and a file creation event. These events must be on the same host and by the same process within 30 seconds of each other. (`sequence by host.id, process.entity_id with maxspan=30s`)

The DNS event must be on a Windows host and done by PowerShell. The question name cannot be for the localhost, several Microsoft domains, or a Google domain. The user must not be a part of the NT AUTHORITY domain.

The file creation event must also be on a Windows host and the process must be PowerShell. Additionally, the file must have the extension “exe”, “dll”, “ps1”, or “bat.” The file must not start with the string “__PSScriptPolicy” and end in “.ps1” with any amount of characters in between.

By modifying these two events, I can allows file creations as described in the test scenario from an S3 bucket. To start, I’ll modify the DNS event. There are already domains that are allow-listed in the following part of the query:

not dns.question.name : ("localhost", "*.microsoft.com", "*.azureedge.net", "*.powershellgallery.com", "*.windowsupdate.com", "metadata.google.internal")

To add an S3 bucket, I simply add “s3.us-west-1.amazonaws.com” to this list.

Note: this could be replaced with “s3.*.amazonaws.com” to allow for any region.

To modify the file event, I will add a negating clause containing the file information described in the test scenario. This clause will appear as follows:

[file where host.os.type == "windows" and process.name : "powershell.exe" and event.type == "creation" and file.extension : ("exe", "dll", "ps1", "bat") and
   not file.name :"__PSScriptPolicy*.ps1" and not (file.name:"????????.dll" and file.path:"C:\\Users\\*\\Desktop\\test\\*")]

Elastic’s Event Query Language supports two wildcard operators, ‘?’ and ‘*’. The ‘?’ character will match any single character and ‘*’ will match zero or more characters in between.

The file name: ????????.dll uses wildcards from Elastic’s EQL to specify any 8 character long file name ending in “.dll.” The file path C:\\Users\\*\\Desktop\\test\\* allows any user to download this file to the ‘test’ folder, or subfolders, on their Desktop.

Note: Even wrapped in quotation marks, backslashes need to be escaped.

The full query will look like this:

sequence by host.id, process.entity_id with maxspan=30s
  [network where host.os.type == "windows" and process.name : ("powershell.exe", "pwsh.exe", "powershell_ise.exe") and network.protocol == "dns" and
   not dns.question.name : ("localhost", "*.microsoft.com", "*.azureedge.net", "*.powershellgallery.com", "*.windowsupdate.com", "metadata.google.internal", "s3.us-west-1.amazonaws.com") and
   not user.domain : "NT AUTHORITY"]
    [file where host.os.type == "windows" and process.name : "powershell.exe" and event.type == "creation" and file.extension : ("exe", "dll", "ps1", "bat") and
   not file.name :"__PSScriptPolicy*.ps1" and not (file.name:"????????.dll" and file.path:"C:\\Users\\*\\Desktop\\test\\*")]

Now to test it in Elastic!

On a Windows host in my home lab, I used the following PowerShell script to generate one of four random DLLs in two locations of varying lengths.

# Generate a random directory name
$randomDirName = -join ((65..90) + (97..122) | Get-Random -Count 10 | % { [char]$_ })
$randomDllName = -join ((65..90) + (97..122) | Get-Random -Count 8 | % { [char]$_ })
$randomDll2Name = -join ((65..90) + (97..122) | Get-Random -Count 10 | % { [char]$_ })

# Create the directory
New-Item -Path . -Name $randomDirName -ItemType Directory

# Set the URL for the file download
$url = "https://s3.us-west-1.amazonaws.com/[REDACTED_FOR_SECURITY]"

# Download the file to the newly created directory
Invoke-WebRequest -Uri $url -OutFile ".\$randomDirName\$randomDllName.dll"
Write-Host "C:\Users\administrator\Desktop\test\$randomDirName\$randomDllName.dll"


Invoke-WebRequest -Uri $url -OutFile ".\$randomDirName\$randomDll2Name.dll"
Write-Host 
"C:\Users\administrator\Desktop\test\$randomDirName\$randomDll2Name.dll"


Invoke-WebRequest -Uri $url -OutFile "C:\Users\administrator\Desktop\$randomDll2Name.dll"
Write-Host "C:\Users\administrator\Desktop\$randomDll2Name.dll"


Invoke-WebRequest -Uri $url -OutFile "C:\Users\administrator\Desktop\$randomDllName.dll"
Write-Host "C:\Users\administrator\Desktop\$randomDllName.dll"

I simply commented out and re-ran as needed. This is important to consider when testing due to the rule sequencing by the host and the process entity ID. As such, it will only alert to the first suspicious DLL created by any process. Creating all files from the same process will cause only one alert to be generated for the first triggering file created.

This produced the following results:

Notice, the yellow shaded alerts, these are building block alerts. These are the DNS query, and file download events. Elastic does not show these by default, and will instead show a single alert with the offending rule name. This is due to the event correlation tracking two or more events. Elastic will deduplicate these events into a single alert, however, many details appear to be stripped in the shown alert from the deduplicated events. This is seemingly to not confuse end users, combining details from different actions such as a process executions and network traffic.

These are the files alerted to:
C:\Users\administrator\Desktop\test\MgNOGraTyt\LiMQGOaDKl.dll,
C:\Users\administrator\Desktop\pLoqihVKnA.dll, and
C:\Users\administrator\Desktop\tILNJDlB.dll
The first is over 8 characters long, the next is both over 8 characters long and not in the test folder, and the last is 8 characters, but not in the test folder. The file “C:\Users\administrator\Desktop\test\hzSfRwqlCW\tILNJDlB.dll” was not alerted to!

Thus, I’ve successfully written a custom rule to circumvent data labeling issues and still allow for rule tuning. This type of creative engineering and innovation is applied to find proper work-arounds and efficiently get the job done without sacrifice.