* [Event Log] Added KQL queries support for Event Log API. * refactored to use core.elasticsearch.client * Fixed tests * removed get index pattern for event log * Fixed tests * Fixed due to comments. |
||
---|---|---|
.. | ||
common | ||
generated | ||
scripts | ||
server | ||
jest.config.js | ||
kibana.json | ||
README.md | ||
tsconfig.json |
Event Log
Overview
The purpose of this plugin is to provide a way to persist a history of events occuring in Kibana, initially just for the Make It Action project - alerts and actions.
Basic Usage - Logging Events
Follow these steps to use eventLog
in your plugin:
- Declare
eventLog
as a dependency inkibana.json
:
{
...
"requiredPlugins": ["eventLog"],
...
}
- Register provider / actions, and create your plugin's logger, using service
API provided in the
setup
stage:
...
import { IEventLogger, IEventLogService } from '../../event_log/server';
interface PluginSetupDependencies {
eventLog: IEventLogService;
}
...
public setup(core: CoreSetup, { eventLog }: PluginSetupDependencies) {
...
eventLog.registerProviderActions('my-plugin', ['action-1, action-2']);
const eventLogger: IEventLogger = eventLog.getLogger({ event: { provider: 'my-plugin' } });
...
}
...
- To log an event, call
logEvent()
on theeventLogger
object you created:
...
eventLogger.logEvent({ event: { action: 'action-1' }, tags: ['fe', 'fi', 'fo'] });
...
Testing
Unit tests
Documentation: https://www.elastic.co/guide/en/kibana/current/development-tests.html#_unit_testing
yarn test:jest x-pack/plugins/event_log --watch
API Integration tests
None yet!
Background
For the Make It Action alerting / action plugins, we will need a way to persist data regarding alerts and actions, for UI and investigative purposes. We're referring to this persisted data as "events", and will be persisted to a new elasticsearch index referred to as the "event log".
Example events are actions firing, alerts running their scheduled functions, alerts scheduling actions to run, etc.
This functionality will be provided in a new NP plugin eventLog
, and will
provide server-side plugin APIs to write to the event log, and run limited
queries against it. For now, access via HTTP will not be available, due to
security concerns and lack of use cases.
The current clients for the event log are the actions and alerting plugins, however the event log currently has nothing specific to them, and is general purpose, so can be used by any plugin to "log events".
We currently assume that there may be many events logged, and that (some) customers may not be interested in "old" events, and so to keep the event log from consuming too much disk space, we'll set it up with ILM and some kind of reasonable default policy that can be customized by the user. This implies also the use of rollver, setting a write index alias upon rollover, and that searches for events will be done via an ES index pattern / alias to search across event log indices with a wildcard.
The shape of the documents indexed into the event log index is a subset of ECS properties with a few Kibana extensions. Over time the subset is of ECS and Kibana extensions will likely grow.
Basic example
When an action is executed, an event should be written to the event log.
Here's a kbn-action
command to
execute a "server log" action (writes a message to the Kibana log):
$ kbn-action execute 79b4c37e-ef42-4421-a0b0-b536840f930d '{level:info message:hallo}'
{
"status": "ok"
}
Here's the event written to the event log index:
{
"_index": ".kibana-event-log-000001",
"_type": "_doc",
"_id": "d2CXT20BPOpswQ8vgXp5",
"_score": 1,
"_source": {
"event": {
"provider": "actions",
"action": "execute",
"start": "2019-12-09T21:16:43.424Z",
"end": "2019-12-09T21:16:43.425Z",
"duration": 1000000
},
"kibana": {
"saved_objects": [
{
"type": "action",
"id": "79b4c37e-ef42-4421-a0b0-b536840f930d"
}
]
},
"message": "action executed successfully: 79b4c37e-ef42-4421-a0b0-b536840f930d - .server-log - server-log",
"@timestamp": "2019-12-09T21:16:43.425Z",
"ecs": {
"version": "1.3.1"
}
}
}
The shape of the document written to the index is a subset of ECS with an
extended field of kibana
with some Kibana-related properties contained within
it.
The ES mappings for the ECS data, and the config-schema for the ECS data, are generated by a script, and available here:
It's anticipated that these interfaces will grow over time, hopefully adding more ECS fields but adding Kibana extensions as required.
Since there are some security concerns with the data, we are currently restricting access via known saved object ids. That is, you can only query history records associated with specific saved object ids.
API
// IEvent is a TS type generated from the subset of ECS supported
// the NP plugin returns a service instance from setup() and start()
export interface IEventLogService {
registerProviderActions(provider: string, actions: string[]): void;
isProviderActionRegistered(provider: string, action: string): boolean;
getProviderActions(): Map<string, Set<string>>;
getLogger(properties: IEvent): IEventLogger;
}
export interface IEventLogger {
logEvent(properties: IEvent): void;
startTiming(event: IEvent): void;
stopTiming(event: IEvent): void;
}
The plugin exposes an IEventLogService
object to plugins that pre-req it.
Those plugins need to call registerProviderActions()
to indicate the values
of the event.provider
and event.action
values they will be using
when logging events.
The pre-registration helps in two ways:
- dealing with misspelled values
- preventing index explosion on those fields
Once the values are registered, the plugin will get an IEventLogger
instance
by passing in a set of default properties to be used for all it's logging,
to the getLogger()
method. For instance, the actions
plugin creates a
logger with event.provider
set to actions
, and provides event.action
values when writing actual entries.
The IEventLogger
object can be cached at the plugin level and accessed by
any code in the plugin. It has a single method to write an event log entry,
logEvent()
, which is passed specific properties for the event.
The final data written is a combination of the data passed to getLogger()
when
creating the logger, and the data passed on the logEvent()
call, and then
that result is validated to ensure it's complete and valid. Errors will be
logged to the server log.
The logEvent()
method returns no values, and is itself not asynchronous.
It's a "call and forget" kind of thing. The method itself will arrange
to have the ultimate document written to the index asynchronously. It's designed
this way because it's not clear what a client would do with a result from this
method, nor what it would do if the method threw an error. All the error
processing involved with getting the data into the index is handled internally,
and logged to the server log as appropriate.
The startTiming()
and stopTiming()
methods can be used to set the timing
properties start
, end
, and duration
in the event. For example:
const loggedEvent: IEvent = { event: { action: 'foo' } };
// sets event.start
eventLogger.startTiming(loggedEvent);
longRunningFunction();
// sets event.end and event.duration
eventLogger.stopTiming(loggedEvent);
eventLogger.logEvent(loggedEvent);
It's anticipated that more "helper" methods like this will be provided in the future.
Stored data
The elasticsearch index for the event log will have ILM and rollover support, as customers may decide to only keep recent event documents, wanting indices with older event documents deleted, turned cold, frozen, etc. We'll supply some default values, but customers will be able to tweak these.
The index template, mappings, config-schema types, etc for the index can be found in the generated directory. These files are generated from a script which takes as input the ECS properties to use, and the Kibana extensions.
See ilm rollover action docs for more info on the is_write_index
, and index.lifecycle.*
properties.
Of particular note in the mappings
:
- all "objects" are
dynamic: 'strict'
implies users can't add new fields - all the
properties
are indexed
We may change some of that before releasing.
ILM setup
We'll want to provide default ILM policy, this seems like a reasonable first attempt:
PUT _ilm/policy/event_log_policy
{
"policy": {
"phases": {
"hot": {
"actions": {
"rollover": {
"max_size": "50GB",
"max_age": "30d"
}
}
},
"delete": {
"min_age": "90d",
"actions": {
"delete": {}
}
}
}
}
}
This means that ILM would "rollover" the current index, say
.kibana-event-log-8.0.0-000001
by creating a new index .kibana-event-log-8.0.0-000002
,
which would "inherit" everything from the index template, and then ILM will
set the write index of the the alias to the new index. This would happen
when the original index grew past 50 GB, or was created more than 30 days ago.
After rollover, the indices will be removed after 90 days to avoid disks to fill up.
For more relevant information on ILM, see: getting started with ILM doc and write index alias behavior: