[Security Solution] Migrates siem.notifications ruleAlertId to saved object references array (#113205)
## Summary Fixes https://github.com/elastic/kibana/issues/113276 * Migrates the legacy `siem.notifications` "ruleAlertId" to be within the references array * Adds code to serialize and de-serialize "ruleAlertId" from the saved object references array * Adds migration code to `kibana-alerting` to migrate on startup * Adds `legacy_saved_object_references/README.md` which describes how to test and what those files are for. * Updates earlier similar `signals/saved_object_references/README.md` after reviewing it during my work * Names these files the format of `legacy_foo` since this is all considered legacy work and will be removed once the legacy notification system is removed after customers have migrated. * Adds unit tests * Adds 2e2 tests We only migrate if we find these conditions and cases: * "ruleAlertId" is not `null`, `undefined` or malformed data * The"ruleAlertId" references do not already have an exceptionItem reference already found within it. We migrate on the common use case: * "ruleAlertId" exists and is a string We do these additional (mis-use) cases and steps as well. These should NOT be common things that happen but we safe guard for them here: * If the migration is run twice we are idempotent and do NOT add duplicates or remove items. * If the migration was partially successful but re-run a second time, we only add what is missing. Again no duplicates or removed items should occur. * If the saved object references already exists and contains a different or foreign value, we will retain the foreign reference(s) and still migrate. Before migration you should see data structures like this if you query: ```json # Get the alert type of "siem-notifications" which is part of the legacy system. GET .kibana/_search { "query": { "term": { "alert.alertTypeId": "siem.notifications" } } } ``` ```json "data..omitted": "data..omitted", "params" : { "ruleAlertId" : "933ca720-1be1-11ec-a722-83da1c22a481" <-- Pre-migration we had this Saved Object ID which is not part of references array below }, "actions" : [ { "group" : "default", "params" : { "message" : "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts" }, "actionTypeId" : ".slack", "actionRef" : "action_0" <-- Pre-migration this is correct as this work is already done within the alerting plugin }, "references" : [ { "id" : "879e8ff0-1be1-11ec-a722-83da1c22a481", "name" : "action_0", <-- Pre-migration this is correct as this work is already done within the alerting plugin "type" : "action" } ] ], "data..omitted": "data..omitted", ``` After migration you should see data structures like this: ```json "data..omitted": "data..omitted", "params" : { "ruleAlertId" : "933ca720-1be1-11ec-a722-83da1c22a481" <-- Post-migration this is not used but rather the serialized version references is used instead. }, "actions" : [ { "group" : "default", "params" : { "message" : "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts" }, "actionTypeId" : ".slack", "actionRef" : "action_0" }, "references" : [ { "id" : "879e8ff0-1be1-11ec-a722-83da1c22a481", "name" : "action_0", "type" : "action" }, { "id" : "933ca720-1be1-11ec-a722-83da1c22a481", <-- Our id here is preferred and used during serialization. "name" : "param:alert_0", <-- We add the name of our reference which is param:alert_0 similar to action_0 but with "param" "type" : "alert" <-- We add the type which is type of alert to the references } ] ], "data..omitted": "data..omitted", ``` ## Manual testing There are e2e and unit tests but for any manual testing or verification you can do the following: If you have a 7.14.0 system and can migrate it forward that is the most straight forward way to ensure this does migrate correctly and forward. You should see that the legacy notification system still operates as expected. If you are a developer off of master and want to test different scenarios then this section is for below as it is more involved and harder to do but goes into more depth: * Create a rule and activate it normally within security_solution * Do not add actions to the rule at this point as we are exercising the older legacy system. However, you want at least one action configured such as a slack notification. * Within dev tools do a query for all your actions and grab one of the `_id` of them without their prefix: ```json # See all your actions GET .kibana/_search { "query": { "term": { "type": "action" } } } ``` Mine was `"_id" : "action:879e8ff0-1be1-11ec-a722-83da1c22a481"`, so I will be copying the ID of `879e8ff0-1be1-11ec-a722-83da1c22a481` Go to the file `detection_engine/scripts/legacy_notifications/one_action.json` and add this id to the file. Something like this: ```json { "name": "Legacy notification with one action", "interval": "1m", <--- You can use whatever you want. Real values are "1h", "1d", "1w". I use "1m" for testing purposes. "actions": [ { "id": "879e8ff0-1be1-11ec-a722-83da1c22a481", <--- My action id "group": "default", "params": { "message": "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts" }, "actionTypeId": ".slack" <--- I am a slack action id type. } ] } ``` Query for an alert you want to add manually add back a legacy notification to it. Such as: ```json # See all your siem.signals alert types and choose one GET .kibana/_search { "query": { "term": { "alert.alertTypeId": "siem.signals" } } } ``` Grab the `_id` without the alert prefix. For mine this was `933ca720-1be1-11ec-a722-83da1c22a481` Within the directory of detection_engine/scripts execute the script: ```json ./post_legacy_notification.sh 933ca720-1be1-11ec-a722-83da1c22a481 { "ok": "acknowledged" } ``` which is going to do a few things. See the file `detection_engine/routes/rules/legacy_create_legacy_notification.ts` for the definition of the route and what it does in full, but we should notice that we have now: Created a legacy side car action object of type `siem-detection-engine-rule-actions` you can see in dev tools: ```json # See the actions "side car" which are part of the legacy notification system. GET .kibana/_search { "query": { "term": { "type": { "value": "siem-detection-engine-rule-actions" } } } } ``` But more importantly what the saved object references are which should be this: ```json # Get the alert type of "siem-notifications" which is part of the legacy system. GET .kibana/_search { "query": { "term": { "alert.alertTypeId": "siem.notifications" } } } ``` If you need to ad-hoc test what happens when the migration runs you can get the id of an alert and downgrade it, then restart Kibana. The `ctx._source.references.remove(1)` removes the last element of the references array which is assumed to have a rule. But it might not, so ensure you check your data structure and adjust accordingly. ```json POST .kibana/_update/alert:933ca720-1be1-11ec-a722-83da1c22a481 { "script" : { "source": """ ctx._source.migrationVersion.alert = "7.15.0"; ctx._source.references.remove(1); """, "lang": "painless" } } ``` If you just want to remove your your "param:alert_0" and it is the second array element to test the errors within the console then you would use ```json POST .kibana/_update/alert:933ca720-1be1-11ec-a722-83da1c22a481 { "script" : { "source": """ ctx._source.references.remove(1); """, "lang": "painless" } } ``` Check your log files and should see errors about the saved object references missing until you restart Kibana. Once you restart then it will migrate forward and you will no longer see errors. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
This commit is contained in:
Frank Hassanabad
GitHub
parent
0d9825d03c
commit
ba7bea456a
|
|
@ -1696,6 +1696,223 @@ describe('successful migrations', () => {
|
|||
},
|
||||
});
|
||||
});
|
||||
|
||||
test('security solution is migrated to saved object references if it has a "ruleAlertId"', () => {
|
||||
const migration7160 = getMigrations(encryptedSavedObjectsSetup, isPreconfigured)['7.16.0'];
|
||||
const alert = getMockData({
|
||||
alertTypeId: 'siem.notifications',
|
||||
params: {
|
||||
ruleAlertId: '123',
|
||||
},
|
||||
});
|
||||
|
||||
expect(migration7160(alert, migrationContext)).toEqual({
|
||||
...alert,
|
||||
attributes: {
|
||||
...alert.attributes,
|
||||
legacyId: alert.id,
|
||||
},
|
||||
references: [
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '123',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
test('security solution does not migrate anything if its type is not siem.notifications', () => {
|
||||
const migration7160 = getMigrations(encryptedSavedObjectsSetup, isPreconfigured)['7.16.0'];
|
||||
const alert = getMockData({
|
||||
alertTypeId: 'other-type',
|
||||
params: {
|
||||
ruleAlertId: '123',
|
||||
},
|
||||
});
|
||||
|
||||
expect(migration7160(alert, migrationContext)).toEqual({
|
||||
...alert,
|
||||
attributes: {
|
||||
...alert.attributes,
|
||||
legacyId: alert.id,
|
||||
},
|
||||
});
|
||||
});
|
||||
test('security solution does not change anything if "ruleAlertId" is missing', () => {
|
||||
const migration7160 = getMigrations(encryptedSavedObjectsSetup, isPreconfigured)['7.16.0'];
|
||||
const alert = getMockData({
|
||||
alertTypeId: 'siem.notifications',
|
||||
params: {},
|
||||
});
|
||||
|
||||
expect(migration7160(alert, migrationContext)).toEqual({
|
||||
...alert,
|
||||
attributes: {
|
||||
...alert.attributes,
|
||||
legacyId: alert.id,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test('security solution will keep existing references if we do not have a "ruleAlertId" but we do already have references', () => {
|
||||
const migration7160 = getMigrations(encryptedSavedObjectsSetup, isPreconfigured)['7.16.0'];
|
||||
const alert = {
|
||||
...getMockData({
|
||||
alertTypeId: 'siem.notifications',
|
||||
params: {},
|
||||
}),
|
||||
references: [
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '123',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
expect(migration7160(alert, migrationContext)).toEqual({
|
||||
...alert,
|
||||
attributes: {
|
||||
...alert.attributes,
|
||||
legacyId: alert.id,
|
||||
},
|
||||
references: [
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '123',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
test('security solution will keep any foreign references if they exist but still migrate other "ruleAlertId" references', () => {
|
||||
const migration7160 = getMigrations(encryptedSavedObjectsSetup, isPreconfigured)['7.16.0'];
|
||||
const alert = {
|
||||
...getMockData({
|
||||
alertTypeId: 'siem.notifications',
|
||||
params: {
|
||||
ruleAlertId: '123',
|
||||
},
|
||||
}),
|
||||
references: [
|
||||
{
|
||||
name: 'foreign-name',
|
||||
id: '999',
|
||||
type: 'foreign-name',
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
expect(migration7160(alert, migrationContext)).toEqual({
|
||||
...alert,
|
||||
attributes: {
|
||||
...alert.attributes,
|
||||
legacyId: alert.id,
|
||||
},
|
||||
references: [
|
||||
{
|
||||
name: 'foreign-name',
|
||||
id: '999',
|
||||
type: 'foreign-name',
|
||||
},
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '123',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
test('security solution is idempotent and if re-run on the same migrated data will keep the same items "ruleAlertId" references', () => {
|
||||
const migration7160 = getMigrations(encryptedSavedObjectsSetup, isPreconfigured)['7.16.0'];
|
||||
const alert = {
|
||||
...getMockData({
|
||||
alertTypeId: 'siem.notifications',
|
||||
params: {
|
||||
ruleAlertId: '123',
|
||||
},
|
||||
}),
|
||||
references: [
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '123',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
expect(migration7160(alert, migrationContext)).toEqual({
|
||||
...alert,
|
||||
attributes: {
|
||||
...alert.attributes,
|
||||
legacyId: alert.id,
|
||||
},
|
||||
references: [
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '123',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
test('security solution will not migrate "ruleAlertId" if it is invalid data', () => {
|
||||
const migration7160 = getMigrations(encryptedSavedObjectsSetup, isPreconfigured)['7.16.0'];
|
||||
const alert = {
|
||||
...getMockData({
|
||||
alertTypeId: 'siem.notifications',
|
||||
params: {
|
||||
ruleAlertId: 55, // This is invalid if it is a number and not a string
|
||||
},
|
||||
}),
|
||||
};
|
||||
|
||||
expect(migration7160(alert, migrationContext)).toEqual({
|
||||
...alert,
|
||||
attributes: {
|
||||
...alert.attributes,
|
||||
legacyId: alert.id,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test('security solution will not migrate "ruleAlertId" if it is invalid data but will keep existing references', () => {
|
||||
const migration7160 = getMigrations(encryptedSavedObjectsSetup, isPreconfigured)['7.16.0'];
|
||||
const alert = {
|
||||
...getMockData({
|
||||
alertTypeId: 'siem.notifications',
|
||||
params: {
|
||||
ruleAlertId: 456, // This is invalid if it is a number and not a string
|
||||
},
|
||||
}),
|
||||
references: [
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '123',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
expect(migration7160(alert, migrationContext)).toEqual({
|
||||
...alert,
|
||||
attributes: {
|
||||
...alert.attributes,
|
||||
legacyId: alert.id,
|
||||
},
|
||||
references: [
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '123',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('8.0.0', () => {
|
||||
|
|
|
|||
|
|
@ -54,6 +54,17 @@ export const isAnyActionSupportIncidents = (doc: SavedObjectUnsanitizedDoc<RawAl
|
|||
export const isSecuritySolutionRule = (doc: SavedObjectUnsanitizedDoc<RawAlert>): boolean =>
|
||||
doc.attributes.alertTypeId === 'siem.signals';
|
||||
|
||||
/**
|
||||
* Returns true if the alert type is that of "siem.notifications" which is a legacy notification system that was deprecated in 7.16.0
|
||||
* in favor of using the newer alerting notifications system.
|
||||
* @param doc The saved object alert type document
|
||||
* @returns true if this is a legacy "siem.notifications" rule, otherwise false
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
*/
|
||||
export const isSecuritySolutionLegacyNotification = (
|
||||
doc: SavedObjectUnsanitizedDoc<RawAlert>
|
||||
): boolean => doc.attributes.alertTypeId === 'siem.notifications';
|
||||
|
||||
export function getMigrations(
|
||||
encryptedSavedObjects: EncryptedSavedObjectsPluginSetup,
|
||||
isPreconfigured: (connectorId: string) => boolean
|
||||
|
|
@ -103,7 +114,11 @@ export function getMigrations(
|
|||
const migrateRules716 = createEsoMigration(
|
||||
encryptedSavedObjects,
|
||||
(doc): doc is SavedObjectUnsanitizedDoc<RawAlert> => true,
|
||||
pipeMigrations(setLegacyId, getRemovePreconfiguredConnectorsFromReferencesFn(isPreconfigured))
|
||||
pipeMigrations(
|
||||
setLegacyId,
|
||||
getRemovePreconfiguredConnectorsFromReferencesFn(isPreconfigured),
|
||||
addRuleIdsToLegacyNotificationReferences
|
||||
)
|
||||
);
|
||||
|
||||
const migrationRules800 = createEsoMigration(
|
||||
|
|
@ -574,6 +589,49 @@ function removeMalformedExceptionsList(
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* This migrates rule_id's within the legacy siem.notification to saved object references on an upgrade.
|
||||
* We only migrate if we find these conditions:
|
||||
* - ruleAlertId is a string and not null, undefined, or malformed data.
|
||||
* - The existing references do not already have a ruleAlertId found within it.
|
||||
* Some of these issues could crop up during either user manual errors of modifying things, earlier migration
|
||||
* issues, etc... so we are safer to check them as possibilities
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
* @param doc The document that might have "ruleAlertId" to migrate into the references
|
||||
* @returns The document migrated with saved object references
|
||||
*/
|
||||
function addRuleIdsToLegacyNotificationReferences(
|
||||
doc: SavedObjectUnsanitizedDoc<RawAlert>
|
||||
): SavedObjectUnsanitizedDoc<RawAlert> {
|
||||
const {
|
||||
attributes: {
|
||||
params: { ruleAlertId },
|
||||
},
|
||||
references,
|
||||
} = doc;
|
||||
if (!isSecuritySolutionLegacyNotification(doc) || !isString(ruleAlertId)) {
|
||||
// early return if we are not a string or if we are not a security solution notification saved object.
|
||||
return doc;
|
||||
} else {
|
||||
const existingReferences = references ?? [];
|
||||
const existingReferenceFound = existingReferences.find((reference) => {
|
||||
return reference.id === ruleAlertId && reference.type === 'alert';
|
||||
});
|
||||
if (existingReferenceFound) {
|
||||
// skip this if the references already exists for some uncommon reason so we do not add an additional one.
|
||||
return doc;
|
||||
} else {
|
||||
const savedObjectReference: SavedObjectReference = {
|
||||
id: ruleAlertId,
|
||||
name: 'param:alert_0',
|
||||
type: 'alert',
|
||||
};
|
||||
const newReferences = [...existingReferences, savedObjectReference];
|
||||
return { ...doc, references: newReferences };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function setLegacyId(
|
||||
doc: SavedObjectUnsanitizedDoc<RawAlert>
|
||||
): SavedObjectUnsanitizedDoc<RawAlert> {
|
||||
|
|
|
|||
|
|
@ -6,7 +6,6 @@
|
|||
*/
|
||||
|
||||
import { Logger } from 'src/core/server';
|
||||
import { schema } from '@kbn/config-schema';
|
||||
import { parseScheduleDates } from '@kbn/securitysolution-io-ts-utils';
|
||||
import {
|
||||
DEFAULT_RULE_NOTIFICATION_QUERY_SIZE,
|
||||
|
|
@ -15,12 +14,19 @@ import {
|
|||
} from '../../../../common/constants';
|
||||
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyNotificationAlertTypeDefinition } from './legacy_types';
|
||||
import {
|
||||
LegacyNotificationAlertTypeDefinition,
|
||||
legacyRulesNotificationParams,
|
||||
} from './legacy_types';
|
||||
import { AlertAttributes } from '../signals/types';
|
||||
import { siemRuleActionGroups } from '../signals/siem_rule_action_groups';
|
||||
import { scheduleNotificationActions } from './schedule_notification_actions';
|
||||
import { getNotificationResultsLink } from './utils';
|
||||
import { getSignals } from './get_signals';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { legacyExtractReferences } from './legacy_saved_object_references/legacy_extract_references';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { legacyInjectReferences } from './legacy_saved_object_references/legacy_inject_references';
|
||||
|
||||
/**
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
|
|
@ -36,9 +42,12 @@ export const legacyRulesNotificationAlertType = ({
|
|||
defaultActionGroupId: 'default',
|
||||
producer: SERVER_APP_ID,
|
||||
validate: {
|
||||
params: schema.object({
|
||||
ruleAlertId: schema.string(),
|
||||
}),
|
||||
params: legacyRulesNotificationParams,
|
||||
},
|
||||
useSavedObjectReferences: {
|
||||
extractReferences: (params) => legacyExtractReferences({ logger, params }),
|
||||
injectReferences: (params, savedObjectReferences) =>
|
||||
legacyInjectReferences({ logger, params, savedObjectReferences }),
|
||||
},
|
||||
minimumLicenseRequired: 'basic',
|
||||
isExportable: false,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,217 @@
|
|||
This is where you add code when you have rules which contain saved object references. Saved object references are for
|
||||
when you have "joins" in the saved objects between one saved object and another one. This can be a 1 to M (1 to many)
|
||||
relationship for example where you have a rule which contains the "id" of another saved object.
|
||||
|
||||
NOTE: This is the "legacy saved object references" and should only be for the "legacy_rules_notification_alert_type".
|
||||
The legacy notification system is being phased out and deprecated in favor of using the newer alerting notification system.
|
||||
It would be considered wrong to see additional code being added here at this point. However, maintenance should be expected
|
||||
until we have all users moved away from the legacy system.
|
||||
|
||||
|
||||
## How to create a legacy notification
|
||||
|
||||
* Create a rule and activate it normally within security_solution
|
||||
* Do not add actions to the rule at this point as we are exercising the older legacy system. However, you want at least one action configured such as a slack notification.
|
||||
* Within dev tools do a query for all your actions and grab one of the `_id` of them without their prefix:
|
||||
|
||||
```json
|
||||
# See all your actions
|
||||
GET .kibana/_search
|
||||
{
|
||||
"query": {
|
||||
"term": {
|
||||
"type": "action"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Mine was `"_id" : "action:879e8ff0-1be1-11ec-a722-83da1c22a481"`, so I will be copying the ID of `879e8ff0-1be1-11ec-a722-83da1c22a481`
|
||||
|
||||
Go to the file `detection_engine/scripts/legacy_notifications/one_action.json` and add this id to the file. Something like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "Legacy notification with one action",
|
||||
"interval": "1m", <--- You can use whatever you want. Real values are "1h", "1d", "1w". I use "1m" for testing purposes.
|
||||
"actions": [
|
||||
{
|
||||
"id": "879e8ff0-1be1-11ec-a722-83da1c22a481", <--- My action id
|
||||
"group": "default",
|
||||
"params": {
|
||||
"message": "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
|
||||
},
|
||||
"actionTypeId": ".slack" <--- I am a slack action id type.
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Query for an alert you want to add manually add back a legacy notification to it. Such as:
|
||||
|
||||
```json
|
||||
# See all your siem.signals alert types and choose one
|
||||
GET .kibana/_search
|
||||
{
|
||||
"query": {
|
||||
"term": {
|
||||
"alert.alertTypeId": "siem.signals"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Grab the `_id` without the alert prefix. For mine this was `933ca720-1be1-11ec-a722-83da1c22a481`
|
||||
|
||||
Within the directory of detection_engine/scripts execute the script:
|
||||
|
||||
```json
|
||||
./post_legacy_notification.sh 933ca720-1be1-11ec-a722-83da1c22a481
|
||||
{
|
||||
"ok": "acknowledged"
|
||||
}
|
||||
```
|
||||
|
||||
which is going to do a few things. See the file `detection_engine/routes/rules/legacy_create_legacy_notification.ts` for the definition of the route and what it does in full, but we should notice that we have now:
|
||||
|
||||
Created a legacy side car action object of type `siem-detection-engine-rule-actions` you can see in dev tools:
|
||||
|
||||
```json
|
||||
# See the actions "side car" which are part of the legacy notification system.
|
||||
GET .kibana/_search
|
||||
{
|
||||
"query": {
|
||||
"term": {
|
||||
"type": {
|
||||
"value": "siem-detection-engine-rule-actions"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
But more importantly what the saved object references are which should be this:
|
||||
|
||||
```json
|
||||
# Get the alert type of "siem-notifications" which is part of the legacy system.
|
||||
GET .kibana/_search
|
||||
{
|
||||
"query": {
|
||||
"term": {
|
||||
"alert.alertTypeId": "siem.notifications"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
I omit parts but leave the important parts pre-migration and post-migration of the Saved Object.
|
||||
|
||||
```json
|
||||
"data..omitted": "data..omitted",
|
||||
"params" : {
|
||||
"ruleAlertId" : "933ca720-1be1-11ec-a722-83da1c22a481" <-- Pre-migration we had this Saved Object ID which is not part of references array below
|
||||
},
|
||||
"actions" : [
|
||||
{
|
||||
"group" : "default",
|
||||
"params" : {
|
||||
"message" : "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
|
||||
},
|
||||
"actionTypeId" : ".slack",
|
||||
"actionRef" : "action_0" <-- Pre-migration this is correct as this work is already done within the alerting plugin
|
||||
},
|
||||
"references" : [
|
||||
{
|
||||
"id" : "879e8ff0-1be1-11ec-a722-83da1c22a481",
|
||||
"name" : "action_0", <-- Pre-migration this is correct as this work is already done within the alerting plugin
|
||||
"type" : "action"
|
||||
}
|
||||
]
|
||||
],
|
||||
"data..omitted": "data..omitted",
|
||||
```
|
||||
|
||||
Post migration this structure should look like this after Kibana has started and finished the migration.
|
||||
|
||||
```json
|
||||
"data..omitted": "data..omitted",
|
||||
"params" : {
|
||||
"ruleAlertId" : "933ca720-1be1-11ec-a722-83da1c22a481" <-- Post-migration this is not used but rather the serialized version references is used instead.
|
||||
},
|
||||
"actions" : [
|
||||
{
|
||||
"group" : "default",
|
||||
"params" : {
|
||||
"message" : "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
|
||||
},
|
||||
"actionTypeId" : ".slack",
|
||||
"actionRef" : "action_0"
|
||||
},
|
||||
"references" : [
|
||||
{
|
||||
"id" : "879e8ff0-1be1-11ec-a722-83da1c22a481",
|
||||
"name" : "action_0",
|
||||
"type" : "action"
|
||||
},
|
||||
{
|
||||
"id" : "933ca720-1be1-11ec-a722-83da1c22a481", <-- Our id here is preferred and used during serialization.
|
||||
"name" : "param:alert_0", <-- We add the name of our reference which is param:alert_0 similar to action_0 but with "param"
|
||||
"type" : "alert" <-- We add the type which is type of rule to the references
|
||||
}
|
||||
]
|
||||
],
|
||||
"data..omitted": "data..omitted",
|
||||
```
|
||||
|
||||
Only if for some reason a migration has failed due to a bug would we fallback and try to use `"ruleAlertId" : "933ca720-1be1-11ec-a722-83da1c22a481"`, as it was last stored within SavedObjects. Otherwise all access will come from the
|
||||
references array's version. If the migration fails or the fallback to the last known saved object id `"ruleAlertId" : "933ca720-1be1-11ec-a722-83da1c22a481"` does happen, then the code emits several error messages to the
|
||||
user which should further encourage the user to help migrate the legacy notification system to the newer notification system.
|
||||
|
||||
## Useful queries
|
||||
|
||||
This gives you back the legacy notifications.
|
||||
|
||||
```json
|
||||
# Get the alert type of "siem-notifications" which is part of the legacy system.
|
||||
GET .kibana/_search
|
||||
{
|
||||
"query": {
|
||||
"term": {
|
||||
"alert.alertTypeId": "siem.notifications"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If you need to ad-hoc test what happens when the migration runs you can get the id of an alert and downgrade it, then
|
||||
restart Kibana. The `ctx._source.references.remove(1)` removes the last element of the references array which is assumed
|
||||
to have a rule. But it might not, so ensure you check your data structure and adjust accordingly.
|
||||
```json
|
||||
POST .kibana/_update/alert:933ca720-1be1-11ec-a722-83da1c22a481
|
||||
{
|
||||
"script" : {
|
||||
"source": """
|
||||
ctx._source.migrationVersion.alert = "7.15.0";
|
||||
ctx._source.references.remove(1);
|
||||
""",
|
||||
"lang": "painless"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If you just want to remove your "param:alert_0" and it is the second array element to test the errors within the console
|
||||
then you would use
|
||||
```json
|
||||
POST .kibana/_update/alert:933ca720-1be1-11ec-a722-83da1c22a481
|
||||
{
|
||||
"script" : {
|
||||
"source": """
|
||||
ctx._source.references.remove(1);
|
||||
""",
|
||||
"lang": "painless"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## End to end tests
|
||||
See `test/alerting_api_integration/spaces_only/tests/alerting/migrations.ts` for tests around migrations
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { loggingSystemMock } from 'src/core/server/mocks';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyRulesNotificationParams } from '../legacy_types';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { legacyExtractReferences } from './legacy_extract_references';
|
||||
|
||||
describe('legacy_extract_references', () => {
|
||||
type FuncReturn = ReturnType<typeof legacyExtractReferences>;
|
||||
let logger = loggingSystemMock.create().get('security_solution');
|
||||
|
||||
beforeEach(() => {
|
||||
logger = loggingSystemMock.create().get('security_solution');
|
||||
});
|
||||
|
||||
test('It returns the references extracted as saved object references', () => {
|
||||
const params: LegacyRulesNotificationParams = {
|
||||
ruleAlertId: '123',
|
||||
};
|
||||
expect(
|
||||
legacyExtractReferences({
|
||||
logger,
|
||||
params,
|
||||
})
|
||||
).toEqual<FuncReturn>({
|
||||
params,
|
||||
references: [
|
||||
{
|
||||
id: '123',
|
||||
name: 'alert_0',
|
||||
type: 'alert',
|
||||
},
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
test('It returns the empty references array if the ruleAlertId is missing for any particular unusual reason', () => {
|
||||
const params = {};
|
||||
expect(
|
||||
legacyExtractReferences({
|
||||
logger,
|
||||
params: params as LegacyRulesNotificationParams,
|
||||
})
|
||||
).toEqual<FuncReturn>({
|
||||
params: params as LegacyRulesNotificationParams,
|
||||
references: [],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { Logger } from 'src/core/server';
|
||||
import { RuleParamsAndRefs } from '../../../../../../alerting/server';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyRulesNotificationParams } from '../legacy_types';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { legacyExtractRuleId } from './legacy_extract_rule_id';
|
||||
|
||||
/**
|
||||
* Extracts references and returns the saved object references.
|
||||
* NOTE: You should not have to add any new ones here at all, but this keeps consistency with the other
|
||||
* version(s) used for security_solution rules.
|
||||
*
|
||||
* How to add a new extracted references here (This should be rare or non-existent):
|
||||
* ---
|
||||
* Add a new file for extraction named: extract_<paramName>.ts, example: extract_foo.ts
|
||||
* Add a function into that file named: extract<ParamName>, example: extractFoo(logger, params.foo)
|
||||
* Add a new line below and concat together the new extract with existing ones like so:
|
||||
*
|
||||
* const legacyRuleIdReferences = legacyExtractRuleId(logger, params.ruleAlertId);
|
||||
* const fooReferences = extractFoo(logger, params.foo);
|
||||
* const returnReferences = [...legacyRuleIdReferences, ...fooReferences];
|
||||
*
|
||||
* Optionally you can remove any parameters you do not want to store within the Saved Object here:
|
||||
* const paramsWithoutSavedObjectReferences = { removeParam, ...otherParams };
|
||||
*
|
||||
* If you do remove params, then update the types in: security_solution/server/lib/detection_engine/notifications/legacy_rules_notification_alert_type.ts
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
* @param logger Kibana injected logger
|
||||
* @param params The params of the base rule(s).
|
||||
* @returns The rule parameters and the saved object references to store.
|
||||
*/
|
||||
export const legacyExtractReferences = ({
|
||||
logger,
|
||||
params,
|
||||
}: {
|
||||
logger: Logger;
|
||||
params: LegacyRulesNotificationParams;
|
||||
}): RuleParamsAndRefs<LegacyRulesNotificationParams> => {
|
||||
const legacyRuleIdReferences = legacyExtractRuleId({
|
||||
logger,
|
||||
ruleAlertId: params.ruleAlertId,
|
||||
});
|
||||
const returnReferences = [...legacyRuleIdReferences];
|
||||
|
||||
// Modify params if you want to remove any elements separately here. For legacy ruleAlertId, we do not remove the id and instead
|
||||
// keep it to both fail safe guard against manually removed saved object references or if there are migration issues and the saved object
|
||||
// references are removed. Also keeping it we can detect and log out a warning if the reference between it and the saved_object reference
|
||||
// array have changed between each other indicating the saved_object array is being mutated outside of this functionality
|
||||
const paramsWithoutSavedObjectReferences = { ...params };
|
||||
|
||||
return {
|
||||
references: returnReferences,
|
||||
params: paramsWithoutSavedObjectReferences,
|
||||
};
|
||||
};
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { loggingSystemMock } from 'src/core/server/mocks';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyRulesNotificationParams } from '../legacy_types';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { legacyExtractRuleId } from './legacy_extract_rule_id';
|
||||
|
||||
describe('legacy_extract_rule_id', () => {
|
||||
type FuncReturn = ReturnType<typeof legacyExtractRuleId>;
|
||||
let logger = loggingSystemMock.create().get('security_solution');
|
||||
|
||||
beforeEach(() => {
|
||||
logger = loggingSystemMock.create().get('security_solution');
|
||||
});
|
||||
|
||||
test('it returns an empty array given a "undefined" ruleAlertId.', () => {
|
||||
expect(
|
||||
legacyExtractRuleId({
|
||||
logger,
|
||||
ruleAlertId: undefined as unknown as LegacyRulesNotificationParams['ruleAlertId'],
|
||||
})
|
||||
).toEqual<FuncReturn>([]);
|
||||
});
|
||||
|
||||
test('logs expect error message if given a "undefined" ruleAlertId.', () => {
|
||||
expect(
|
||||
legacyExtractRuleId({
|
||||
logger,
|
||||
ruleAlertId: null as unknown as LegacyRulesNotificationParams['ruleAlertId'],
|
||||
})
|
||||
).toEqual<FuncReturn>([]);
|
||||
|
||||
expect(logger.error).toBeCalledWith(
|
||||
'Security Solution notification (Legacy) system "ruleAlertId" is null or undefined when it never should be. ,This indicates potentially that saved object migrations did not run correctly. Returning empty reference'
|
||||
);
|
||||
});
|
||||
|
||||
test('it returns the "ruleAlertId" transformed into a saved object references array.', () => {
|
||||
expect(
|
||||
legacyExtractRuleId({
|
||||
logger,
|
||||
ruleAlertId: '123',
|
||||
})
|
||||
).toEqual<FuncReturn>([
|
||||
{
|
||||
id: '123',
|
||||
name: 'alert_0',
|
||||
type: 'alert',
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { Logger, SavedObjectReference } from 'src/core/server';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyRulesNotificationParams } from '../legacy_types';
|
||||
|
||||
/**
|
||||
* This extracts the "ruleAlertId" "id" and returns it as a saved object reference.
|
||||
* NOTE: Due to rolling upgrades with migrations and a few bugs with migrations, I do an additional check for if "ruleAlertId" exists or not. Once
|
||||
* those bugs are fixed, we can remove the "if (ruleAlertId == null) {" check, but for the time being it is there to keep things running even
|
||||
* if ruleAlertId has not been migrated.
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
* @param logger The kibana injected logger
|
||||
* @param ruleAlertId The rule alert id to get the id from and return it as a saved object reference.
|
||||
* @returns The saved object references from the rule alert id
|
||||
*/
|
||||
export const legacyExtractRuleId = ({
|
||||
logger,
|
||||
ruleAlertId,
|
||||
}: {
|
||||
logger: Logger;
|
||||
ruleAlertId: LegacyRulesNotificationParams['ruleAlertId'];
|
||||
}): SavedObjectReference[] => {
|
||||
if (ruleAlertId == null) {
|
||||
logger.error(
|
||||
[
|
||||
'Security Solution notification (Legacy) system "ruleAlertId" is null or undefined when it never should be. ',
|
||||
'This indicates potentially that saved object migrations did not run correctly. Returning empty reference',
|
||||
].join()
|
||||
);
|
||||
return [];
|
||||
} else {
|
||||
return [
|
||||
{
|
||||
id: ruleAlertId,
|
||||
name: 'alert_0',
|
||||
type: 'alert',
|
||||
},
|
||||
];
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,74 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { loggingSystemMock } from 'src/core/server/mocks';
|
||||
import { SavedObjectReference } from 'src/core/server';
|
||||
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyRulesNotificationParams } from '../legacy_types';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { legacyInjectReferences } from './legacy_inject_references';
|
||||
|
||||
describe('legacy_inject_references', () => {
|
||||
type FuncReturn = ReturnType<typeof legacyInjectReferences>;
|
||||
let logger = loggingSystemMock.create().get('security_solution');
|
||||
const mockSavedObjectReferences = (): SavedObjectReference[] => [
|
||||
{
|
||||
id: '123',
|
||||
name: 'alert_0',
|
||||
type: 'alert',
|
||||
},
|
||||
];
|
||||
|
||||
beforeEach(() => {
|
||||
logger = loggingSystemMock.create().get('security_solution');
|
||||
});
|
||||
|
||||
test('returns parameters from a saved object if found', () => {
|
||||
const params: LegacyRulesNotificationParams = {
|
||||
ruleAlertId: '123',
|
||||
};
|
||||
|
||||
expect(
|
||||
legacyInjectReferences({
|
||||
logger,
|
||||
params,
|
||||
savedObjectReferences: mockSavedObjectReferences(),
|
||||
})
|
||||
).toEqual<FuncReturn>(params);
|
||||
});
|
||||
|
||||
test('returns parameters from the saved object if found with a different saved object reference id', () => {
|
||||
const params: LegacyRulesNotificationParams = {
|
||||
ruleAlertId: '123',
|
||||
};
|
||||
|
||||
expect(
|
||||
legacyInjectReferences({
|
||||
logger,
|
||||
params,
|
||||
savedObjectReferences: [{ ...mockSavedObjectReferences()[0], id: '456' }],
|
||||
})
|
||||
).toEqual<FuncReturn>({
|
||||
ruleAlertId: '456',
|
||||
});
|
||||
});
|
||||
|
||||
test('It returns params with an added ruleAlertId if the ruleAlertId is missing due to migration bugs', () => {
|
||||
const params = {} as LegacyRulesNotificationParams;
|
||||
|
||||
expect(
|
||||
legacyInjectReferences({
|
||||
logger,
|
||||
params,
|
||||
savedObjectReferences: [{ ...mockSavedObjectReferences()[0], id: '456' }],
|
||||
})
|
||||
).toEqual<FuncReturn>({
|
||||
ruleAlertId: '456',
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { Logger, SavedObjectReference } from 'src/core/server';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyRulesNotificationParams } from '../legacy_types';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { legacyInjectRuleIdReferences } from './legacy_inject_rule_id_references';
|
||||
|
||||
/**
|
||||
* Injects references and returns the saved object references.
|
||||
* How to add a new injected references here (NOTE: We do not expect to add more here but we leave this as the same pattern we have in other reference sections):
|
||||
* ---
|
||||
* Add a new file for injection named: legacy_inject_<paramName>.ts, example: legacy_inject_foo.ts
|
||||
* Add a new function into that file named: legacy_inject<ParamName>, example: legacyInjectFooReferences(logger, params.foo)
|
||||
* Add a new line below and spread the new parameter together like so:
|
||||
*
|
||||
* const foo = legacyInjectFooReferences(logger, params.foo, savedObjectReferences);
|
||||
* const ruleParamsWithSavedObjectReferences: RuleParams = {
|
||||
* ...params,
|
||||
* foo,
|
||||
* ruleAlertId,
|
||||
* };
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
* @param logger Kibana injected logger
|
||||
* @param params The params of the base rule(s).
|
||||
* @param savedObjectReferences The saved object references to merge with the rule params
|
||||
* @returns The rule parameters with the saved object references.
|
||||
*/
|
||||
export const legacyInjectReferences = ({
|
||||
logger,
|
||||
params,
|
||||
savedObjectReferences,
|
||||
}: {
|
||||
logger: Logger;
|
||||
params: LegacyRulesNotificationParams;
|
||||
savedObjectReferences: SavedObjectReference[];
|
||||
}): LegacyRulesNotificationParams => {
|
||||
const ruleAlertId = legacyInjectRuleIdReferences({
|
||||
logger,
|
||||
ruleAlertId: params.ruleAlertId,
|
||||
savedObjectReferences,
|
||||
});
|
||||
const ruleParamsWithSavedObjectReferences: LegacyRulesNotificationParams = {
|
||||
...params,
|
||||
ruleAlertId,
|
||||
};
|
||||
return ruleParamsWithSavedObjectReferences;
|
||||
};
|
||||
|
|
@ -0,0 +1,101 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { loggingSystemMock } from 'src/core/server/mocks';
|
||||
import { SavedObjectReference } from 'src/core/server';
|
||||
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { legacyInjectRuleIdReferences } from './legacy_inject_rule_id_references';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyRulesNotificationParams } from '../legacy_types';
|
||||
|
||||
describe('legacy_inject_rule_id_references', () => {
|
||||
type FuncReturn = ReturnType<typeof legacyInjectRuleIdReferences>;
|
||||
let logger = loggingSystemMock.create().get('security_solution');
|
||||
const mockSavedObjectReferences = (): SavedObjectReference[] => [
|
||||
{
|
||||
id: '123',
|
||||
name: 'alert_0',
|
||||
type: 'alert',
|
||||
},
|
||||
];
|
||||
|
||||
beforeEach(() => {
|
||||
logger = loggingSystemMock.create().get('security_solution');
|
||||
});
|
||||
|
||||
test('returns parameters from the saved object if found', () => {
|
||||
expect(
|
||||
legacyInjectRuleIdReferences({
|
||||
logger,
|
||||
ruleAlertId: '123',
|
||||
savedObjectReferences: mockSavedObjectReferences(),
|
||||
})
|
||||
).toEqual<FuncReturn>('123');
|
||||
});
|
||||
|
||||
test('returns parameters from the saved object if "ruleAlertId" is undefined', () => {
|
||||
expect(
|
||||
legacyInjectRuleIdReferences({
|
||||
logger,
|
||||
ruleAlertId: undefined as unknown as LegacyRulesNotificationParams['ruleAlertId'],
|
||||
savedObjectReferences: mockSavedObjectReferences(),
|
||||
})
|
||||
).toEqual<FuncReturn>('123');
|
||||
});
|
||||
|
||||
test('prefers to use saved object references if the two are different from each other', () => {
|
||||
expect(
|
||||
legacyInjectRuleIdReferences({
|
||||
logger,
|
||||
ruleAlertId: '456',
|
||||
savedObjectReferences: mockSavedObjectReferences(),
|
||||
})
|
||||
).toEqual<FuncReturn>('123');
|
||||
});
|
||||
|
||||
test('returns sent in "ruleAlertId" if the saved object references is empty', () => {
|
||||
expect(
|
||||
legacyInjectRuleIdReferences({
|
||||
logger,
|
||||
ruleAlertId: '456',
|
||||
savedObjectReferences: [],
|
||||
})
|
||||
).toEqual<FuncReturn>('456');
|
||||
});
|
||||
|
||||
test('does not log an error if it returns parameters from the saved object when found', () => {
|
||||
legacyInjectRuleIdReferences({
|
||||
logger,
|
||||
ruleAlertId: '123',
|
||||
savedObjectReferences: mockSavedObjectReferences(),
|
||||
});
|
||||
expect(logger.error).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('logs an error if found with a different saved object reference id', () => {
|
||||
legacyInjectRuleIdReferences({
|
||||
logger,
|
||||
ruleAlertId: '456',
|
||||
savedObjectReferences: mockSavedObjectReferences(),
|
||||
});
|
||||
expect(logger.error).toBeCalledWith(
|
||||
'The id of the "saved object reference id": 123 is not the same as the "saved object id": 456. Preferring and using the "saved object reference id" instead of the "saved object id"'
|
||||
);
|
||||
});
|
||||
|
||||
test('logs an error if the saved object references is empty', () => {
|
||||
legacyInjectRuleIdReferences({
|
||||
logger,
|
||||
ruleAlertId: '123',
|
||||
savedObjectReferences: [],
|
||||
});
|
||||
expect(logger.error).toBeCalledWith(
|
||||
'The saved object reference was not found for the "ruleAlertId" when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good "ruleAlertId" which might not work. "ruleAlertId" with its id being returned is: 123'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0; you may not use this file except in compliance with the Elastic License
|
||||
* 2.0.
|
||||
*/
|
||||
|
||||
import { Logger, SavedObjectReference } from 'src/core/server';
|
||||
// eslint-disable-next-line no-restricted-imports
|
||||
import { LegacyRulesNotificationParams } from '../legacy_types';
|
||||
|
||||
/**
|
||||
* This injects any legacy "id"'s from saved object reference and returns the "ruleAlertId" using the saved object reference. If for
|
||||
* some reason it is missing on saved object reference, we log an error about it and then take the last known good value from the "ruleId"
|
||||
*
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
* @param logger The kibana injected logger
|
||||
* @param ruleAlertId The alert id to merge the saved object reference from.
|
||||
* @param savedObjectReferences The saved object references which should contain a "ruleAlertId"
|
||||
* @returns The "ruleAlertId" with the saved object reference replacing any value in the saved object's id.
|
||||
*/
|
||||
export const legacyInjectRuleIdReferences = ({
|
||||
logger,
|
||||
ruleAlertId,
|
||||
savedObjectReferences,
|
||||
}: {
|
||||
logger: Logger;
|
||||
ruleAlertId: LegacyRulesNotificationParams['ruleAlertId'];
|
||||
savedObjectReferences: SavedObjectReference[];
|
||||
}): LegacyRulesNotificationParams['ruleAlertId'] => {
|
||||
const referenceFound = savedObjectReferences.find((reference) => {
|
||||
return reference.name === 'alert_0';
|
||||
});
|
||||
if (referenceFound) {
|
||||
if (referenceFound.id !== ruleAlertId) {
|
||||
// This condition should not be reached but we log an error if we encounter it to help if we migrations
|
||||
// did not run correctly or we create a regression in the future.
|
||||
logger.error(
|
||||
[
|
||||
'The id of the "saved object reference id": ',
|
||||
referenceFound.id,
|
||||
' is not the same as the "saved object id": ',
|
||||
ruleAlertId,
|
||||
'. Preferring and using the "saved object reference id" instead of the "saved object id"',
|
||||
].join('')
|
||||
);
|
||||
}
|
||||
return referenceFound.id;
|
||||
} else {
|
||||
logger.error(
|
||||
[
|
||||
'The saved object reference was not found for the "ruleAlertId" when we were expecting to find it. ',
|
||||
'Kibana migrations might not have run correctly or someone might have removed the saved object references manually. ',
|
||||
'Returning the last known good "ruleAlertId" which might not work. "ruleAlertId" with its id being returned is: ',
|
||||
ruleAlertId,
|
||||
].join('')
|
||||
);
|
||||
return ruleAlertId;
|
||||
}
|
||||
};
|
||||
|
|
@ -5,6 +5,8 @@
|
|||
* 2.0.
|
||||
*/
|
||||
|
||||
import { schema, TypeOf } from '@kbn/config-schema';
|
||||
|
||||
import {
|
||||
RulesClient,
|
||||
PartialAlert,
|
||||
|
|
@ -102,8 +104,8 @@ export type LegacyNotificationExecutorOptions = AlertExecutorOptions<
|
|||
export const legacyIsNotificationAlertExecutor = (
|
||||
obj: LegacyNotificationAlertTypeDefinition
|
||||
): obj is AlertType<
|
||||
AlertTypeParams,
|
||||
AlertTypeParams,
|
||||
LegacyRuleNotificationAlertTypeParams,
|
||||
LegacyRuleNotificationAlertTypeParams,
|
||||
AlertTypeState,
|
||||
AlertInstanceState,
|
||||
AlertInstanceContext
|
||||
|
|
@ -116,8 +118,8 @@ export const legacyIsNotificationAlertExecutor = (
|
|||
*/
|
||||
export type LegacyNotificationAlertTypeDefinition = Omit<
|
||||
AlertType<
|
||||
AlertTypeParams,
|
||||
AlertTypeParams,
|
||||
LegacyRuleNotificationAlertTypeParams,
|
||||
LegacyRuleNotificationAlertTypeParams,
|
||||
AlertTypeState,
|
||||
AlertInstanceState,
|
||||
AlertInstanceContext,
|
||||
|
|
@ -131,3 +133,19 @@ export type LegacyNotificationAlertTypeDefinition = Omit<
|
|||
state,
|
||||
}: LegacyNotificationExecutorOptions) => Promise<AlertTypeState | void>;
|
||||
};
|
||||
|
||||
/**
|
||||
* This is the notification type used within legacy_rules_notification_alert_type for the alert params.
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
* @see legacy_rules_notification_alert_type
|
||||
*/
|
||||
export const legacyRulesNotificationParams = schema.object({
|
||||
ruleAlertId: schema.string(),
|
||||
});
|
||||
|
||||
/**
|
||||
* This legacy rules notification type used within legacy_rules_notification_alert_type for the alert params.
|
||||
* @deprecated Once we are confident all rules relying on side-car actions SO's have been migrated to SO references we should remove this function
|
||||
* @see legacy_rules_notification_alert_type
|
||||
*/
|
||||
export type LegacyRulesNotificationParams = TypeOf<typeof legacyRulesNotificationParams>;
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
"interval": "1m",
|
||||
"actions": [
|
||||
{
|
||||
"id": "879e8ff0-1be1-11ec-a722-83da1c22a481",
|
||||
"id": "42534430-2092-11ec-99a6-05d79563c01a",
|
||||
"group": "default",
|
||||
"params": {
|
||||
"message": "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
|
||||
|
|
|
|||
|
|
@ -21,6 +21,26 @@ GET .kibana/_search
|
|||
}
|
||||
```
|
||||
|
||||
If you want to manually test the downgrade of an alert then you can use this script.
|
||||
```json
|
||||
# Set saved object array references as empty arrays and set our migration version to be 7.14.0
|
||||
POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c
|
||||
{
|
||||
"script" : {
|
||||
"source": """
|
||||
ctx._source.migrationVersion.alert = "7.14.0";
|
||||
ctx._source.references = []
|
||||
""",
|
||||
"lang": "painless"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Reload the alert in the security_solution and notice you get these errors until you restart Kibana to cause a migration moving forward. Although you get errors,
|
||||
everything should still operate normally as we try to work even if migrations did not run correctly for any unforeseen reasons.
|
||||
|
||||
For testing idempotentence, just re-run the same script above for a downgrade after you restarted Kibana.
|
||||
|
||||
## Structure on disk
|
||||
Run a query in dev tools and you should see this code that adds the following savedObject references
|
||||
to any newly saved rule:
|
||||
|
|
@ -141,4 +161,4 @@ Good examples and utilities can be found in the folder of `utils` such as:
|
|||
You can follow those patterns but if it doesn't fit your use case it's fine to just create a new file and wire up your new saved object references
|
||||
|
||||
## End to end tests
|
||||
At this moment there are none.
|
||||
See `test/alerting_api_integration/spaces_only/tests/alerting/migrations.ts` for tests around migrations
|
||||
|
|
|
|||
|
|
@ -257,5 +257,21 @@ export default function createGetTests({ getService }: FtrProviderContext) {
|
|||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('7.16.0 migrates security_solution (Legacy) siem.notifications with "ruleAlertId" to be saved object references', async () => {
|
||||
// NOTE: We hae to use elastic search directly against the ".kibana" index because alerts do not expose the references which we want to test exists
|
||||
const response = await es.get<{ references: [{}] }>({
|
||||
index: '.kibana',
|
||||
id: 'alert:d7a8c6a1-9394-48df-a634-d5457c35d747',
|
||||
});
|
||||
expect(response.statusCode).to.eql(200);
|
||||
expect(response.body._source?.references).to.eql([
|
||||
{
|
||||
name: 'param:alert_0',
|
||||
id: '1a4ed6ae-3c89-44b2-999d-db554144504c',
|
||||
type: 'alert',
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -451,4 +451,44 @@
|
|||
"updated_at": "2020-06-17T15:35:39.839Z"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
"type": "doc",
|
||||
"value": {
|
||||
"id": "alert:d7a8c6a1-9394-48df-a634-d5457c35d747",
|
||||
"index": ".kibana_1",
|
||||
"source": {
|
||||
"alert" : {
|
||||
"name" : "test upgrade of ruleAlertId",
|
||||
"alertTypeId" : "siem.notifications",
|
||||
"consumer" : "alertsFixture",
|
||||
"params" : {
|
||||
"ruleAlertId" : "1a4ed6ae-3c89-44b2-999d-db554144504c"
|
||||
},
|
||||
"schedule" : {
|
||||
"interval" : "1m"
|
||||
},
|
||||
"enabled" : true,
|
||||
"actions" : [ ],
|
||||
"throttle" : null,
|
||||
"apiKeyOwner" : null,
|
||||
"apiKey" : null,
|
||||
"createdBy" : "elastic",
|
||||
"updatedBy" : "elastic",
|
||||
"createdAt" : "2021-07-27T20:42:55.896Z",
|
||||
"muteAll" : false,
|
||||
"mutedInstanceIds" : [ ],
|
||||
"scheduledTaskId" : null,
|
||||
"tags": []
|
||||
},
|
||||
"type" : "alert",
|
||||
"migrationVersion" : {
|
||||
"alert" : "7.8.0"
|
||||
},
|
||||
"updated_at" : "2021-08-13T23:00:11.985Z",
|
||||
"references": [
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue