Develop and manage ASIM parsers

Azure Sentinel is a scalable, cloud-native, security information event management (SIEM) and security orchestration automated response (SOAR) solution. Azure Sentinel delivers intelligent security analytics and threat intelligence across the enterprise, providing a single solution for alert detection, threat visibility, proactive hunting, and threat response. Critical to its operation is the ability to ingest data from various sources, which is where Azure Sentinel Information Model (ASIM) parsers come into play.

Understanding the Role of ASIM Parsers in Azure Sentinel

ASIM parsers standardize and normalize data from various sources, making it easier for security analysts to query, visualize, and analyze the data. The ASIM normalization process converts disparate log formats into a common schema, enabling cross-platform security analytics. It allows analysts to write queries once and apply them across various data sources without the need to account for the peculiarities of each source.

Developing an ASIM Parser

To develop an ASIM parser, you’ll follow these steps:

1. Understand the Schema: Familiarize yourself with the ASIM schema, which includes common fields like EventTime, SourceIP, DestinationIP, EventID, and more. This schema forms the foundation of the data normalization process.
2. Identify the Data Source: Determine which data source you will be parsing. Each source has its own log format, so you need a deep understanding of the log structure as provided by the data producer.
3. Write the Parser: Using Kusto Query Language (KQL), you’ll write parsing logic to map fields from the original log schema to the ASIM schema. The parsing function should deal with data type conversions, field extractions, and any necessary transformations.
4. Test the Parser: Before deployment, thoroughly test the parser with diverse datasets to ensure it accurately normalizes logs under various situations.
5. Deploy and Manage the Parser: Once tested, deploy the parser within Azure Sentinel. Management includes ongoing maintenance, such as updating the parser for changes in source log formats or improvements in the ASIM schema.

Managing an ASIM Parser

Managing ASIM parsers involves regularly reviewing your parsers to ensure they continue to operate as expected. These tasks often include:

• Monitoring Performance: Check the performance of your parsers to ensure they are operating efficiently. Slow parsers can bottleneck the ingestion process.
• Updating Parsers: When log formats change or when ASIM schema updates are released, you’ll need to modify and redeploy your parsers.
• Error Handling: Create robust error handling within your parsers to deal with corrupt or unexpected log formats.

Example: Developing a Simple ASIM Parser

Imagine you are developing an ASIM parser for a fictitious web server’s logs. Here’s an outline of the process:

1. The schema of web server logs has fields like timestamp, source IP, request type, URL, user-agent, etc.
2. Analyze a sample web server log to understand the log structure.
3. Write a KQL-based parser function that maps each field from the web server log to the corresponding ASIM fields.
4. The parser might look something like this:

create function with (folder = “ASIM”, docstring=”Web Server ASIM Parser”, skipvalidation=”true”)
WebServer_ASIM_Parser()
{
WebServerLogs // Your table where logs are initially ingested
| extend EventTime = todatetime(timestamp),
SourceIP = tostring(src_ip),
HttpRequestMethod = tostring(request_type),
Url = tostring(url),
UserAgent = tostring(user_agent)
// … more field mappings
}

5. Use sample logs to test your parser, checking that each field is correctly normalized.
6. Once tests pass, deploy the parser in Azure Sentinel for use.

Managing parsers entails:

• Regularly checking the logs for parsing anomalies,
• Periodic review of the source logs format, and
• Tuning for performance and efficacy.

In conclusion

ASIM parsers are pivotal in creating a unified security posture within the Azure Sentinel platform. Through careful development and ongoing management, these parsers enable security analysts to work more efficiently and with greater confidence in their data, ultimately contributing to enhanced security operations and incident response efforts.