Search Library

REST API located at /api/library

The search library API is used to store and retrieve saved search queries. The search library is a useful system to build up valuable queries with names, notes, tags, and anything else that might be valuable in day-to-day operations.

The search library is a permissioned API, which means each user owns their search library and can optionally share them with groups. Each library entry also has a global flag, which means any user can read the library entry. ONLY admins can set the global flag.

Only owners can delete library entries, even if another user has access to a library entry through group membership, they cannot delete the entry.

Admin Operations

Administrators interact with the search library in the same way as all other users. If an admin wants administrative access to the library API, whether to list, delete, or modify library entries outside of their ownership they must append the admin flag to requests.

For example, performing a GET request on /api/library will return only the calling users library entries (admin or not). Performing the same GET request on /api/library?admin=true will return all library entries for all users. The admin flag is ignored for non-admin users.

Basic API Overview

The Library API is rooted at /api/library and responds to the following request methods:

method	description	Supports Admin Calls
GET	Retrieve a list of available library entries	TRUE
POST	Add a new library entry	FALSE
PUT	Update an existing library entry	TRUE
DELETE	Delete an existing library entry	TRUE

Every search library entry has both a “ThingUUID” and a “GUID” associated with it. The “ThingUUID” is always unique: there will only ever be one search library entry with a given ThingUUID on the system. The “GUID”, on the other hand, is an ID which is used to refer to a search library entry from other things like dashboards or actionables; when you install a kit, all the search library entries will have the same GUID as on the kit creator’s system, allowing cross-linking. Each user could potentially have a search library entry with the same GUID, but each of those entries will have a unique ThingUUID.

The DELETE and PUT method require that the “ThingUUID” or “GUID” of a specific library entry be appended to the URL. The ThingUUID and the GUID can be used interchangeably in the API, but be aware that in “admin mode” there may be multiple accessible items with the same GUID. For example, to update the entry with the GUID of 5f72d51e-d641-11e9-9f54-efea47f6014a a PUT request would be issued against /api/library/5f72d51e-d641-11e9-9f54-efea47f6014a with a complete entry structure encoded into the body of the request.

The GET method can optionally append a GUID to request a specific library entry, if not GUID is present the GET method returns a list of all available entries. If a users does not have access to a specific entry specified by the GUID the webserver will return a response of 403.

The structure of a library entry is as follows:

struct {
	ThingUUID   uuid.UUID
	GUID        uuid.UUID
	UID         int32
	GIDS        []int32
	Global      boolean
	Name        string
	Description string
	Query       string
	Labels      []string
	Metadata    RawObject
}

The structure members are:

Member	Description	Omitted if Empty
ThingUUID	Unique identifier on the local system
GUID	Global “name”; persists across kit installation. Multiple users may have search library entries with the same GUID.
UID	Owners system user id
GIDs	List of group ids the entry is shared with	X
Global	Boolean indicating whether the entry is globally readable
Name	A human readable name for the query
Description	A human readable description of the query
Query	The query string
Labels	A list of human readable labels used for categorizing queries	X
Metadata	An opaque JSON blob used for arbitrary parameter storage, just be valid JSON but does not have any particular structure	X

Here is an example entry that is owned by the user with UID 1 and shared with 3 groups. The UID and GID values must be mapped back to user and group names using the user API:

{
	"ThingUUID": "69755a85-d5b1-11e9-89c2-0242ac130005",
	"GUID": "ae132ecc-88dd-11ea-a6aa-373f4c2439d4",
	"UID": 1,
	"GIDs": [1, 3, 5],
	"Global": false,
	"Name": "syslog counting query",
	"Description": "A simple chart that shows total syslog over time",
	"Query": "tag=syslog syslog Appname | stats count by Appname | chart count by Appname",
	"Labels": [
		"syslog",
		"chart",
		"trending"
	],
	"Metadata": {}
}

Examples API Interaction

This section contains example interactions with the search library API endpoint. These examples were generated using the Gravwell CLI using the -debug flag.

Creating a New Entry

Request:

POST /api/library
{
	"GIDs": [1, 2],
	"Global": false,
	"Name": "netflow agg",
	"Description": "Total traffic using netflow data",
	"Query": "tag=netflow netflow Bytes | stats sum(Bytes) as TotalTraffic | chart TotalTraffic",
	"Labels": [
		"netflow",
		"traffic",
		"aggs"
	]
}

Note

Because the “GUID” field is not set here, the system will assign one. You may also include the GUID in the request.

Response:

{
	"ThingUUID": "c9169d15-d643-11e9-99d3-0242ac130005",
	"GUID": "ae132ecc-88dd-11ea-a6aa-373f4c2439d4",
	"UID": 1,
	"GIDs": [1, 2],
	"Global": false,
	"Name": "netflow agg",
	"Description": "Total traffic using netflow data",
	"Query": "tag=netflow netflow Bytes | stats sum(Bytes) as TotalTraffic | chart TotalTraffic",
	"Labels": [
		"netflow",
		"traffic",
		"aggs"
	]
}

Retrieving Entries

Request:

GET http://172.19.0.5:80/api/library

Response:

[
	{
		"ThingUUID": "0b5a66cb-d642-11e9-931c-0242ac130005",
		"GUID": "ae132ecc-88dd-11ea-a6aa-373f4c2439d4",
		"UID": 1,
		"Global": false,
		"Name": "netflow agg",
		"Description": "Total traffic using netflow data",
		"Query": "tag=netflow netflow Bytes | stats sum(Bytes) as TotalTraffic | chart TotalTraffic",
		"Labels": [
			"netflow",
			"traffic",
			"aggs"
		],
		"Metadata": {
			"value": 1,
			"extra": "some extra field value"
		}
	},
	{
		"ThingUUID": "69755a85-d5b1-11e9-89c2-0242ac130005",
		"GUID": "d57611be-88dd-11ea-a94d-df6bfb56a8a8",
		"UID": 1,
		"Global": false,
		"Name": "test2",
		"Description": "testing second",
		"Query": "tag=foo grep bar",
		"Labels": [
			"foo",
			"bar",
			"baz"
		],
	}
]

Requesting Specific Entry

Request:

GET http://172.19.0.5:80/api/library/ae132ecc-88dd-11ea-a6aa-373f4c2439d4

Response:

{
	"ThingUUID": "0b5a66cb-d642-11e9-931c-0242ac130005",
	"GUID": "ae132ecc-88dd-11ea-a6aa-373f4c2439d4",
	"UID": 1,
	"Global": false,
	"Name": "netflow agg",
	"Description": "Total traffic using netflow data",
	"Query": "tag=netflow netflow Bytes | stats sum(Bytes) as TotalTraffic | chart TotalTraffic",
	"Labels": [
		"netflow",
		"traffic",
		"aggs"
	],
	"Metadata": {
		"value": 1,
		"extra": "some extra field value"
	}
}

Note that you would get the same response from api/library/0b5a66cb-d642-11e9-931c-0242ac130005 too.

Updating an Entry

Request:

PUT /api/library/69755a85-d5b1-11e9-89c2-0242ac130005
{
	"ThingUUID": "69755a85-d5b1-11e9-89c2-0242ac130005",
	"GUID": "d57611be-88dd-11ea-a94d-df6bfb56a8a8",
	"Global": false,
	"Name": "SyslogAgg",
	"Description": "Updated Syslog aggregate",
	"Query": "tag=syslog length | stats sum(length) | chart sum",
	"Labels": [
		"syslog",
		"agg",
		"totaldata"
	],
	"Metadata": {}
}

Response:

{
	"ThingUUID": "69755a85-d5b1-11e9-89c2-0242ac130005",
	"GUID": "d57611be-88dd-11ea-a94d-df6bfb56a8a8",
	"UID": 1,
	"Global": false,
	"Name": "SyslogAgg",
	"Description": "Updated Syslog aggregate",
	"Query": "tag=syslog length | stats sum(length) | chart sum",
	"Labels": [
		"syslog",
		"agg",
		"totaldata"
	]
}

Deleting an Entry

Request:

DELETE /api/library/69755a85-d5b1-11e9-89c2-0242ac130005

Admin Deleting an Entry

If an non-admin appends the admin flag the webserver will ignore the flag, if the non-admin is the owner of the specified entry the action (DELETE in this case) still works. If the non-admin user does NOT own the entry the webserver responds with a 403 StatusForbidden.

When performing an admin deletion, always use the ThingUUID in the URL parameter or else the wrong item may be deleted.

Request:

DELETE /api/library/69755a85-d5b1-11e9-89c2-0242ac130005?admin=true