* bump to a pre-8.0 version * export KibanaClient from /lib sub-folder * workaround the problem of the absence of estypes * update es client usage in pacakges * export estypes from another path * import errors from root * import errors from root 2 * update transport import * update import path for /api/types * update import path for /api/types * import errors from top export * use TransportResult instead if ApiResponse * fix errors in client_config * fix src/core/server/saved_objects/migrationsv2/actions/integration_tests/actions.test.ts * use KibanaClient in mock. we dont export the original Client * fix client mocks * fix errors on SO * fix remaining core errors * update estype import path * fix errors in data plugin * fix data_views * fix es_ui_shared * fix errors in interactive_setup * fix errors in ./test folder * add @elastic/transport to the runtime deps * fix errors in packages * fix erros in src/core * fix errors in test/ * fix an error in actions plugin * woraround and fix errors in APM plugin * fix errors in canvas * fix errors in event_log * fix errors in fleet * fix errors in ILM * fix errors in infra * fix errors in ingest_pipeline * fix errors in lens * fix errors in license_management * fix errors in licensing * fix errors in logstash * fix errors in ml * fix errors in monitoring * fix errors in observability * fix errors in rule_registry * fix errors in reporting * fix errors in rule_registry * fix errors in security * fix errors in security_solution * fix errors in snapshot_restore * fix errors in transform * fix errors in UA * fix errors in uptime * fix errors in x-pack/test * fix eslint errors * fix new errors * use default HTTP Connection. Undici does not support agent config options keepAlive and maxSockets * create does not accept require_alias option * update deps * use transport types exported from ES client package * fix ErrorCause | string errors * do not use enum * fix errors in data plugin * update x-pack code * fix transport * fix apm search request * do not crash on reporting * fix kbn-test build * mute reporting error to start * fix ftr build * another attempt * update import path * address or mute new errors * REMOVE me. pin transport version temporarily. * remove deep imports from transport package * fix jest crash * fix product check tests * remove unnecessary ts-expect-error * fix a few failed unit tests * bump to canary 24 * remove unnecessary ts-expect-error * remove dependency on transport * fix types in tests * mute errors in xpack tests * product check doesn;t spam in logs anymore * filterPath --> filter_path * ignoreUnavailable --> ignore_unavailable * ignoreUnavailable --> ignore_unavailable * trackScores --> track_scores * trackTotalHits --> track_total_hits * fix es-arcives * fix data plugin crashes * fix watcher test utils * rollback unnecessary changes * fix another problem in es-archiver * fix scroll. for whatever reason scroll fails when request scroll_id in body * add meta: true in kbn-securitysolution-es-utils * bump client to canary 25 * fix errors in accordance with the es client spec * update securityscolution-es-utils * unify scroll api in reporting and fix tests * fix unit tests in watcher * refactor APM to abort request with AbortController API * fix missing es client calls in tests * fix missing meta in detection engine FTR tests * fix another bunch of errors in js tests * fix wrong coercion * remove test-grep pattern * fix apm unit test * rename terminateAfter to terminate_after in infra plugin * rename terminateAfter to terminate_after in uptime plugin * rename terminateAfter to terminate_after in apm plugin * fix security roles FTR tests * fix reference * fix post_privilidges test * fix post_privilidges * bump client to 26 * add meta for index_management test helpers * remove ts-expect-error caused by bad type in reason * bump client to 27 * REMOVE me. workaround until fixed in the es client * fix incorrect type casting * swtich from camelCase params * use `HttpConnection` for FTR-related clients * bump client to 29 * Revert "REMOVE me. workaround until fixed in the es client" This reverts commit |
||
---|---|---|
.. | ||
common | ||
docs | ||
scripts/generate_ecs_fieldmap | ||
server | ||
jest.config.js | ||
kibana.json | ||
README.md | ||
tsconfig.json |
Rule Registry
The rule registry plugin aims to make it easy for rule type producers to have their rules produce the data that they need to build rich experiences on top of a unified experience, without the risk of mapping conflicts.
The plugin installs default component templates and a default lifecycle policy that rule type producers can use to create index templates.
It also exposes a rule data client that will create or update the index stream that rules will write data to. It will not do so on plugin setup or start, but only when data is written.
Configuration
By default, these indices will be prefixed with .alerts
. To change this, for instance to support legacy multitenancy, set the following configuration option:
xpack.ruleRegistry.index: 'myAlerts'
The above produces an alerts index prefixed .alerts-myAlerts
.
To disable writing entirely:
xpack.ruleRegistry.write.enabled: false
Setting up the index template
On plugin setup, rule type producers can create the index template as follows:
// get the FQN of the component template. All assets are prefixed with the configured `index` value, which is `.alerts` by default.
const componentTemplateName = plugins.ruleRegistry.getFullAssetName('apm-mappings');
// if write is disabled, don't install these templates
if (!plugins.ruleRegistry.isWriteEnabled()) {
return;
}
// create or update the component template that should be used
await plugins.ruleRegistry.createOrUpdateComponentTemplate({
name: componentTemplateName,
body: {
template: {
settings: {
number_of_shards: 1,
},
// mappingFromFieldMap is a utility function that will generate an
// ES mapping from a field map object. You can also define a literal
// mapping.
mappings: mappingFromFieldMap(
{
[SERVICE_NAME]: {
type: 'keyword',
},
[SERVICE_ENVIRONMENT]: {
type: 'keyword',
},
[TRANSACTION_TYPE]: {
type: 'keyword',
},
[PROCESSOR_EVENT]: {
type: 'keyword',
},
},
'strict'
),
},
},
});
// Install the index template, that is composed of the component template
// defined above, and others. It is important that the technical component
// template is included. This will ensure functional compatibility across
// rule types, for a future scenario where a user will want to "point" the
// data from a rule to a different index.
await plugins.ruleRegistry.createOrUpdateIndexTemplate({
name: plugins.ruleRegistry.getFullAssetName('apm-index-template'),
body: {
index_patterns: [plugins.ruleRegistry.getFullAssetName('observability.apm*')],
composed_of: [
// Technical component template, required
plugins.ruleRegistry.getFullAssetName(TECHNICAL_COMPONENT_TEMPLATE_NAME),
componentTemplateName,
],
},
});
// Finally, create the rule data client that can be injected into rule type
// executors and API endpoints
const ruleDataClient = new RuleDataClient({
alias: plugins.ruleRegistry.getFullAssetName('observability.apm'),
getClusterClient: async () => {
const coreStart = await getCoreStart();
return coreStart.elasticsearch.client.asInternalUser;
},
ready,
});
// to start writing data, call `getWriter().bulk()`. It supports a `namespace`
// property as well, that for instance can be used to write data to a space-specific
// index.
await ruleDataClient.getWriter().bulk({
body: eventsToIndex.flatMap((event) => [{ index: {} }, event]),
});
// to read data, simply call ruleDataClient.getReader().search:
const response = await ruleDataClient.getReader().search({
body: {
query: {},
size: 100,
fields: ['*'],
sort: {
'@timestamp': 'desc',
},
},
allow_no_indices: true,
});
Schema
The following fields are defined in the technical field component template and should always be used:
@timestamp
: the ISO timestamp of the alert event. For the lifecycle rule type helper, it is always the value ofstartedAt
that is injected by the Kibana alerting framework.event.kind
: signal (for the changeable alert document), state (for the state changes of the alert, e.g. when it opens, recovers, or changes in severity), or metric (individual evaluations that might be related to an alert).event.action
: the reason for the event. This might beopen
,close
,active
, orevaluate
.tags
: tags attached to the alert. Right now they are copied over from the rule.kibana.alert.rule.rule_type_id
: the identifier of the rule type, e.g.apm.transaction_duration
kibana.alert.rule.uuid
: the saved objects id of the rule.kibana.alert.rule.name
: the name of the rule (as specified by the user).kibana.alert.rule.category
: the name of the rule type (as defined by the rule type producer)kibana.alert.rule.consumer
: the feature which produced the alert (inherited from the rule producer field). Usually a Kibana feature id likeapm
,siem
...kibana.alert.instance.id
: the id of the alert instance, that is unique within the context of the rule execution it was created in. E.g., for a rule that monitors latency for all services in all environments, this might beopbeans-java:production
.kibana.alert.uuid
: the unique identifier for the alert during its lifespan. If an alert recovers (or closes), this identifier is re-generated when it is opened again.kibana.alert.status
: the status of the alert. Can beactive
orrecovered
.kibana.alert.start
: the ISO timestamp of the time at which the alert started.kibana.alert.end
: the ISO timestamp of the time at which the alert recovered.kibana.alert.duration.us
: the duration of the alert, in microseconds. This is always the difference between either the current time, or the time when the alert recovered.kibana.alert.severity
: the severity of the alert, as a keyword (e.g. critical).kibana.alert.evaluation.value
: The measured (numerical value).kibana.alert.threshold.value
: The threshold that was defined (or, in case of multiple thresholds, the one that was exceeded).kibana.alert.ancestors
: the array of ancestors (if any) for the alert.kibana.alert.depth
: the depth of the alert in the ancestral tree (default 0).kibana.alert.building_block_type
: the building block type of the alert (default undefined).
Alerts as data
Alerts as data can be interacted with using the AlertsClient api found in x-pack/plugins/rule_registry/server/alert_data_client/alerts_client.ts
This api includes public methods such as
[x] getFullAssetName [x] getAlertsIndex [x] get [x] update [ ] bulkUpdate (TODO) [ ] find (TODO)