Splunk: Exploring SPL
Learn and explore the basics of the Search Processing Language.
medium
120 min
To access material, start machines and answer questions login.
Splunk is a powerful SIEM solution that provides the ability to search and explore machine data. Search Processing Language (SPL) is used to make the search more effective. It comprises various functions and commands used together to form complex yet effective search queries to get optimized results.
This room will dive deep into some key fundamentals of searching capability, like
chaining SPL queries to construct simple to complex queries.
Learning Objectives
This room will teach the following topics:
- What are Search processing Language?
- How to apply filters to narrow down results.
- Using transformational commands.
- Changing the order of the results.
Room Prerequisites
- This room is based on the SIEM concepts covered in Intro to SIEM and Splunk: Basics rooms. Complete these rooms and continue to the next task.
Room Machine
Before moving forward, deploy the machine. You can access this lab in the AttackBox or click https://LAB_WEB_URL.p.thmlabs.com/ to start the lab in your browser when the machine is fully started. The machine will take up to 3-5 minutes to start.
Note: For this room, we will work on the index Windowslogs.
What is the name of the host in the Data Summary tab?
Search & Reporting App is the default interface used to search and analyze the data on the Splunk Home page. It has various functionalities that assist analysts in improving the search experience.
Some important functionalities present in the search App are explained below:
1) Search Head:
Search Head is where we use search processing language queries to look for the data.
2) Time Duration:
This tab option provides multiple options to select the time duration for the search. All-time will display the events in real-time. Similarly, the last 60 minutes will display all the events captured in the last hour.
3) Search History:
This tab saves the search queries that the user has run in the past along with the time when it was run. It lets the user click on the past searches and look at the result. The filter option is used to search for the particular query based on the term.
4) Data Summary:
This tab provides a summary of the data type, the data source, and the hosts that generated the events as shown below. This tab is very important feature used to get a brief idea about the network visibility.
5) Field Sidebar:
The Field Sidebar can be found on the left panel of Splunk search. This sidebar has two sections showing selected fields and interesting fields. It also provides quick results, such as top values and raw values against each field.
Some important points to understand about the sidebar are explained below:
| 1- Selected Fields | Splunk extracts the default fields like source, sourcetype, and host, which appear in each event, and places them under the selected fields column. We can select other fields that seem essential and add them to the list. |
| 2- Interesting Fields | Pulls all the interesting fields it finds and displays them in the left panel to further explore. |
| 3- Alpha-numeric fields 'α' | This alpha symbol shows that the field contains text values. |
| 4- Numeric fields '#' | This symbol shows that this field contains numerical values. |
| 5- Count | The number against each field shows the number of events captured in that timeframe. |
In the left field panel, which Source IP has recorded max events?
How many events are returned when we apply the time filter to display events on 04/15/2022 and Time from 08:05 AM to 08:06 AM?
Splunk Search Processing Language comprises of multiple functions, operators and commands that are used together to form a simple to complex search and get the desired results from the ingested logs. Main components of SPL are explained below:
Search Field Operators
Splunk
field operators are the building blocks used to construct any search
query. These field operators are used to filter, remove, and narrow down
the search result based on the given criteria. Common field operators
are Comparison operators, wildcards, and boolean operators.
Comparison Operators
These operators are used to compare the values against the fields. Some common comparisons operators are mentioned below:
| Field Name | Operator | Example | Explanation |
| Equal | = | UserName=Mark | This
operator is used to match values against the field. In this example, it
will look for all the events, where the value of the field UserName is
equal to Mark. |
| Not Equal to | != | UserName!=Mark | This operator returns all the events where the UserName value does not match Mark. |
| Less than | < | Age < 10 | Showing all the events with the value of Age less than 10. |
| Less than or Equal to | <= | Age <= 10 | Showing all the events with the value of Age less than or equal to 10. |
| Greater than | > | Outbound_traffic > 50 MB | This will return all the events where the Outbound traffic value is over 50 MB. |
| Greater Than or Equal to | >= | Outbound_traffic >= 50 MB | This will return all the events where the Outbound traffic value is greater or equal to 50 MB. |
Lets use the comparison operator to display all the event logs from the index "windowslogs", where AccountName is not Equal to "System"
Search Query: index=windowslogs AccountName !=SYSTEM
Boolean Operators
Splunk supports the following Boolean operators, which can be very handy in searching/filtering and narrowing down results.
| Operator | Syntax | Explanation |
| NOT | field_A NOT value | Ignore the events from the result where field_A contain the specified value. |
| OR | field_A=value1 OR field_A=value2 | Return all the events in which field_A contains either value1 or value2. |
| AND | field_A=value1 AND field_B=value2 | Return all the events in which field_A contains value1 and field_B contains value2. |
To understand how boolean operator works in SPL, lets add the condition to show the events from the James account.
Search Query: index=windowslogs AccountName !=SYSTEM AND AccountName=James
Wild Card
Splunk supports wildcards to match the characters in the strings.
| Wildcard symbol | Example | Explanation |
* | status=fail* | It will return all the results with values like status=failed status=failure |
Search Query: index=windowslogs DestinationIp=172.*
How many events are observed with Destination IP 172.18.39.6 AND destination Port 135?
What is the Source IP with highest count returned with this Search query?
Search Query: index=windowslogs Hostname="Salena.Adam" DestinationIp="172.18.38.5"
In the index windowslogs, search for all the events that contain the term cyber how many events returned?
Now search for the term cyber*, how many events are returned?
Our network generates thousands of logs each minute, all ingesting into our SIEM solution. It becomes a daunting task to search for any anomaly without using filters. SPL allows us to use Filters to narrow down the result and only show the important events that we are interested in. We can add or remove certain data from the result using filters. The following commands are useful in applying filters to the search results.
Fields
| Command | fields |
| Explanation | Fields
command is used to add or remove mentioned fields from the search
results. To remove the field, minus sign ( - ) is used before the
fieldname and plus ( + ) is used before the fields which we want to
display. |
| Syntax | | fields <field_name1> <field_name2> |
| Example | | |
Let's use the fields command to only display host, User, and SourceIP fields using the following syntax.
Search Query: index=windowslogs | fields + host + User + SourceIp
Note: Click on the More field to display the fields if some fields are not visible.
Search
| Command | search |
| Explanation | This command is used to search for the raw text while using the chaining command |
| Syntax | | search <search_keyword> |
| Example | | |
Use the search command to show all the events containing the term
Powershell. This will return all the events that contain the term "Powershell".
Search Query: index=windowslogs | search Powershell
Dedup
| Command | dedup |
| Explanation | Dedup is the command used to remove duplicate fields from the search results. We often get the results with various fields getting the same results. These commands remove the duplicates to show the unique values. |
| Syntax | | dedup <fieldname> |
| Example | | |
We can use the dedup command to show the list of unique EventIDs from a particular hostname.
Search Query: index=windowslogs | table EventID User Image Hostname | dedup EventID
Rename
| Command | rename |
| Explanation | It allows us to change the name of the field in the search results. It is useful in a scenario when the field name is generic or log, or it needs to be updated in the output. |
| Syntax | | rename <fieldname> |
| Example | | |
Let's rename the User field to Employees using the following search query.
Search Query: index=windowslogs | fields + host + User + SourceIp | rename User as Employees
What is the third EventID returned against this search query?
Search Query:index=windowslogs | table _time EventID Hostname SourceName | reverse
Use the dedup command against the Hostname field before the reverse command in the query mentioned in Question 1. What is the first username returned in the Hostname field?
SPL provides various commands to bring structure or order to the search
results. These sorting commands like head, tail, and sort can be very useful during logs investigation. These ordering commands are explained below:
Table
| Explanation | Each event has multiple fields, and not every field is important to display. The Table command allows us to create a table with selective fields as columns. |
| Syntax | | table <field_name1> <fieldname_2> |
| Example | | | |
This search query will create a table with three columns selected and ignore all the remaining columns from the display.
Search Query: index=windowslogs | table EventID Hostname SourceName
| Explanation | The head command returns the first 10 events if no number is specified. |
| Syntax | | head <number> |
| Example | | | |
The following search query will show the table containing the mentioned fields and display only the top 5 entries.
Search Query: index=windowslogs | table _time EventID Hostname SourceName | head 5
Tail
| Explanation | The Tail command returns the last 10 events if no number is specified. |
| Syntax | | tail <number> |
| Example | | | tail 20 # will return the last 20 events from the result list |
The following search query will show the table containing the mentioned fields and
display only 5 entries from the bottom of the list.
Search Query: index=windowslogs | table _time EventID Hostname SourceName | tail 5
sort
| Explanation | The Sort command allows us to order the fields in ascending or descending order. |
| Syntax | | |
| Example | | |
The following search query will sort the results based on the Hostname field.
Search Query: index=windowslogs | table _time EventID Hostname SourceName | sort Hostname
Reverse
| Explanation | The reverse command simply reverses the order of the events. |
| Syntax | | reverse |
Example |
|
index=windowslogs | table _time EventID Hostname SourceName | reverse
What is the last EventID returned when the query in question 1 is updated with the tail command?
Sort the above query against the SourceName. What is the top SourceName returned?
Transformational commands are those commands that change the result into a data structure from the field-value pairs. These commands simply transform specific values for each event into numerical values which can easily be utilized for statistical purposes or turn the results into visualizations. Searches that use these transforming commands are called transforming searches. Some of the most used transforming commands are explained below.
General Transformational Commands
Top
| Command | top |
| Explanation | This command returns frequent values for the top 10 events. |
| Syntax | | top <field_name> | top limit=6 <field_name> |
| Example |
|
The following command will display the top 7 Image ( representing Processes) captured.
Search Query: index=windowslogs | top limit=7 Image
Rare
| Command | rare |
| Explanation | This command does the opposite of top command as it returns the least frequent values or bottom 10 results. |
| Syntax | | rare <field_name> | rare limit=6 <field_name> |
| Example |
|
The following command will display the rare 7 Image (Processes) captured.
Search Query:index=windowslogs | rare limit=7 Image
Highlight
| Command | highlight |
| Explanation | The highlight command shows the results in raw events mode with fields highlighted. |
| Syntax | highlight <field_name1> <field_name2> |
| Example |
|
The following command will highlight the three mentioned fields in the raw logs
index=windowslogs | highlight User, host, EventID, Image
STATS Commands
SPL supports various stats commands that help in calculating statistics on the values. Some common stat commands are:
| Command | Explanation | Syntax | Example |
| Average | This command is used to calculate the average of the given field. | stats avg(field_name) | stats avg(product_price) |
| Max | It will return the maximum value from the specific field. | stats max(field_name) | stats max(user_age) |
| Min | It will return the minimum value from the specific field. | stats min(field_name) | stats min(product_price) |
| Sum | It will return the sum of the fields in a specific value. | stats sum(field_name) | stats sum(product_cost) |
| Count | The count command returns the number of data occurrences. | stats count(function) AS new_NAME | stats count(source_IP) |
Splunk Chart Commands
These are very important types of transforming commands that are used to present the data in table or visualization form. Most of the chart commands utilize various stat commands.
Chart
| Command | chart |
| Explanation | The chart command is used to transform the data into tables or visualizations. |
| Syntax | | chart <function> |
| Example | | |
Search Query: index=windowslogs | chart count by User
Timechart
| Command | timechart |
| Explanation | The timechart command returns the time series chart covering the field following the function mentioned. Often combined with STATS commands. |
| Syntax | | timechart function <field_name> |
| Example | | |
The following query will display the Image chart based on the time.
index=windowslogs | timechart count by Image
Using the rare command, identify the user with the least number of activities captured?
Create a pie-chart using the chart command - what is the count for the conhost.exe process?
Ready to learn Cyber Security? Create your free account today!
TryHackMe provides free online cyber security training to secure jobs & upskill through a fun, interactive learning environment.
Already have an account? Log in