Configure and use MSTICPy in notebooks

It’s designed to support security analysts during investigations by providing a range of tools and functions for data analysis and visualization. Using MSTICPy within Jupyter or Azure notebooks can enhance the capabilities of analysts studying for the SC-200 Microsoft Security Operations Analyst exam, enabling them to perform advanced data analysis and threat hunting.

Prerequisites

Before configuring MSTICPy, you need:

• Python 3.6 or later.
• A Jupyter or Azure notebook environment setup.
• An understanding of Python and notebook operations.
• Access to data sources you will analyze (e.g., Log Analytics).

Installation of MSTICPy

To install MSTICPy, use pip, the Python package installer. In your notebook, you can run the following command:

!pip install msticpy

Configuration of MSTICPy

After installing MSTICPy, you need to create a configuration file that includes settings for various data providers. The easiest way to create this is by using MSTICPy’s in-built configuration tool which can be started with the following command:

from msticpy.config import MpConfigEdit
MpConfigEdit()

This will display a configuration UI in your notebook where you can input connection details for services such as Azure Sentinel, and other data providers.

Setting up Authentication

Authentication credentials are necessary to access your data sources. MSTICPy supports different authentication methods, including Azure CLI, Managed Identity, and Device Code. To set up authentication, add the following:

from msticpy.data import data_providers
data_providers.init_notebook()

This will initialize the data providers configured in your settings file, prompting you for authentication if necessary.

Importing Data

With your configuration set, you can begin accessing data sources. For example, to query data from Azure Sentinel, you can use the following code:

from msticpy.data import QueryProvider
qry_prov = QueryProvider(“AzureSentinel”)

workspace_id = “your-workspace-id”
tenant_id = “your-tenant-id”
qry_prov.connect(connection_str=f”workspace_id={workspace_id};tenant_id={tenant_id}”)

# Run a query
query = “SecurityEvent | take 10”
result_df = qry_prov.exec_query(query)

Data Analysis and Visualization

Once you have data in a Pandas DataFrame, you can use MSTICPy’s analysis and visualization functions. For example, to visualize Geo-locations of IPs in a dataframe:

from msticpy.analysis import geo_ip
from msticpy.vis import foliummap

df_with_ip = result_df[result_df[“ColumnNameWithIP”].notnull()]
ip_entities = geo_ip.lookup_ips(df_with_ip, “ColumnNameWithIP”)

# Generate map
folium_map = foliummap.FoliumMap()
for _, row in ip_entities.iterrows():
folium_map.add_ip_cluster(ip_entity=row[‘IpAddress’],
popup_content=str(row[‘AdditionalData’]))
folium_map.folium_map

This generates an interactive map directly in the notebook showing the geographical distribution of IP addresses.

Time Series Analysis

MSTICPy has functions to support time series analysis, which can be particularly useful for identifying trends and patterns over time:

from msticpy.analysis import timeseries

# Assuming ‘result_df’ has a datetime column named ‘TimeGenerated’
timeseries.display_timeseries_anomolies(result_df, time_column=’TimeGenerated’, data_column=’SomeDataColumn’)

This function will display a time series plot with detected anomalies highlighted.

Investigation Tools

For more in-depth investigations, MSTICPy offers classes and functions to help dissect complex security events:

from msticpy.sectools import eventcluster

# Conduct event clustering
clusters = eventcluster.dbcluster_events(data=result_df, time_column=’TimeGenerated’, cluster_columns=[‘Col1’, ‘Col2’])

Here we perform clustering on security events to identify patterns or outliers.

Summary

In conclusion, MSTICPy provides a comprehensive toolkit for security analysts working within Jupyter or Azure notebooks. From connecting to data sources and authenticating, to performing complex data analysis and visualization, MSTICPy supports analysts in various stages of the threat investigation process. By learning to configure and use MSTICPy, candidates preparing for the SC-200 exam can greatly enhance their practical skills in identifying and mitigating cyber threats.