Office 365 Log Ingester#

Gravwell provides an ingester for Microsoft Office 365 logs. The ingester can process all supported log types. In order to configure the ingester, you will need to register a new application within the Azure Active Directory management portal; this will generate a set of keys which can be used to access the logs. You will need the following information:

  • Client ID: A UUID generated for your application via the Azure management console

  • Client secret: A secret token generated for your application via the Azure console

  • Azure Directory ID: A UUID representing your Active Directory instance, found in the Azure Active Directory dashboard. This is sometimes called a Tenant ID.

  • Tenant Domain: The domain of your Office 365 domain, e.g. “mycorp.onmicrosoft.com”

  • Plan: The type of Office 365 subscription

  • Options are Enterprise, GCC Government, GCC High Government, DOD Government

Office 365 API#

The Office 365 API provides a limited set of audit activity for clients. This is not a high fidelity dump of all user activity, nor is it an introspection into email content or activity. The data provided by the API can provide high level views into basic API activity for accounts.

The Microsoft Office 365 Management Activity API documentation can provide more concrete details on what you can and cannot see via this API.

Note

Your subscription tier heavily influences the depth and quantity of data available in the API. Low-end plans provide very little, while more expensive plans like GCC High Government can provide much more detailed audit activity.

Creating an Azure Application#

To create a new Azure Application for our ingester you will need to visit the Azure Portal and log in with administrative O365 credentials. Then you will need to go to the App registrations section.

Within the App registrations portal create a “New registration”.

Provide a human friendly name so that you remember why you created this application. Select an appropriate account type and specify a valid Redirect URI (the ingester will not use this redirect URI but it should be valid and owned by your organization).

Then you will need to grant this application permission to use the Office 365 Management APIs. Specifically, add permissions for:

  • Microsoft Graph User.Read

  • Office 365 Management APIs ActivityFeed.Read

  • Office 365 Management APIs ActivityFeed.ReadDlp

These allow the ingester to request audit data from the management APIs, including activity and DLP data. If you do not wish to ingest the DLP data, you can remove the DLP log permission.

Note that you will need to click the “Grant admin consent” button on this page to actually enable the management API methods. If you are not an administrator in your O365 tenant, you will need to get an administrator to grant consent. Refer to the Microsoft documentation for more information about permissions and admin consent.

Finally you will need to go to “Certificates & secrets” and request a new application secret. This will provide a “Secret ID” and “Secret Value”. The Secret Value is what is needed for the ingester configuration.

Installation and configuration#

To install the Debian package, make sure the Gravwell Debian repository is configured as described in the quickstart. Then run the following command as root:

apt update && apt install gravwell-o365

To install the Redhat package, make sure the Gravwell Redhat repository is configured as described in the quickstart. Then run the following command as root:

yum install gravwell-o365

To install via the standalone shell installer, download the installer from the downloads page, then run the following command as root, replacing X.X.X with the appropriate version:

bash gravwell_o365_installer_X.X.X.sh

You may be prompted for additional configuration during the installation.

There is currently no Docker image for this ingester

You will now need to open the /opt/gravwell/etc/o365_ingest.conf configuration file and set it up for your Office 365 account, replacing the placeholder fields and modifying tags as desired. Once you have modified the configuration as described below, start the service with the command systemctl start gravwell_o365_ingest.service

Basic Configuration#

The O365 ingester uses the unified global configuration block described in the ingester section. Like most other Gravwell ingesters, the O365 ingester supports multiple upstream indexers, TLS, cleartext, and named pipe connections, a local cache, and local logging.

The configuration file is at /opt/gravwell/etc/o365_ingest.conf. The ingester will also read configuration snippets from its configuration overlay directory (/opt/gravwell/etc/o365_ingest.conf.d).

Global Configuration Parameters#

Parameter

Type

Default Value

Description

Client-ID

UUID

Identifier generated for your application via the Azure management console

Client-Seecret

String

Secret token generated for your application via the Azure console

Directory-ID

UUID

Identifier representing your Active Directory instance, found in the Azure Active Directory dashboard. This is sometimes called a Tenant ID

Tenant-Domain

FQDN

The domain of your Office 365 domain, e.g. “mycorp.onmicrosoft.com”

Plan-Name

String

Enterprise

The type of Office 365 subscription in use. Options are Enterprise, GCC Government, GCC High Government, DOD Government

Content-Type Configuration Parameters#

Parameter

Type

Default Value

Description

Content-Type

String

The log category to fetch from the Microsoft Office 365 audit API

Tag-Name

String

The Gravwell tag to associate with the fetched log category

Example Configuration#

The example below shows a sample configuration which connects to an indexer on the local machine (note the Pipe-Backend-target setting) and feeds it logs from all supported log types:

[Global]
Ingest-Secret = IngestSecrets
Connection-Timeout = 0
Pipe-Backend-target=/opt/gravwell/comms/pipe #a named pipe connection, this should be used when ingester is on the same machine as a backend
Log-Level=ERROR #options are OFF INFO WARN ERROR
State-Store-Location=/opt/gravwell/etc/o365_ingest.state

Client-ID="79fb8690-109f-11ea-a253-2b12a0d35073"
Client-Secret="<secret>"
Directory-ID="e8b7895e-109f-11ea-9dcc-93fb14b5dab5"
Tenant-Domain="acme.gov"
Plan-Name="GCC Government"

[ContentType "azureAD"]
	Content-Type="Audit.AzureActiveDirectory"
	Tag-Name="365-azure"

[ContentType "exchange"]
	Content-Type="Audit.Exchange"
	Tag-Name="365-exchange"

[ContentType "sharepoint"]
	Content-Type="Audit.SharePoint"
	Tag-Name="365-sharepoint"

[ContentType "general"]
	Content-Type="Audit.General"
	Tag-Name="365-general"

[ContentType "dlp"]
	Content-Type="DLP.All"
	Tag-Name="365-dlp"