maxmustermann/kibana
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[Security Solution][Detections] EQL Validation (#77493)

Browse source 
* WIP: Adding new route for EQL Validation

This is mostly boilerplate with some rough parameter definitions; the
actual implementation of the validation is going to live in our
validateEql function.

A few tests are failing as the mocks haven't yet been implemented, I
need to see the shape of the responses first.

* Cherry-pick Marshall's EQL types

* Implements actual EQL validation

* Performs an EQL search
* filters out non-parsing errors, and returns what remains in the
  response
* Adds mocks for empty EQL responses (we don't yet have a need for
  mocked data, but we will when we unit-test validateEql)

* Adds validation calls to the EQL form input

* Adds EQL Validation response schema,mocks,tests
* Adds frontend function to call our validation endpoint
* Adds hook, useEqlValidation, to call the above function and return
  state
* Adds labels/help text for EQL Query bar
* EqlQueryBar consumes useEqlValidation and marks the field as invalid,
  but does not yet report errors.

* Do not call the validation API if query is not present

This causes a broader error that results in a 400 response; we can (and
do) handle the case of a blank query in the form itself.

* Remove EQL Help Text

It doesn't add any information for the user, and it currently looks bad
when combined with validation errors.

* Flesh out and use our popover for displaying validation errors

* Fixes issue where old errors were persisted after the user had made
  modifications

* Include verification_exception errors as validation errors

These include errors related to index fields and mappings.

* Generalize our validation helpers

We're concerned with validation errors; the source of those errors is an
implementation detail of these functions.

* Move error popover and EQL reference link to footer

This more closely resembles the new Eui Markdown editor, which places
errors and doc links in a footer.

* Fix jest tests following additional prop

* Add icon for EQL Rule card

* Fixes existing EqlQueryBar tests

These were broken by our use of useAppToasts and the EUI theme.

* Add unit tests around error rendering on EQL Query Bar

* Add tests for ErrorPopover

* Remove unused schema type

Decode doesn't do any additional processing, so we can use t.TypeOf here
(the default for buildRouteValidation).

* Remove duplicated header

* Use ignore parameter to prevent EQL validations from logging errors

Without `ignore: [400]` the ES client will log errors and then throw
them. We can catch the error, but the log is undesirable.

This updates the query to use the ignore parameter, along with updating
the validation logic to work with the updated response.

Adds some mocks and tests around these responses and helpers, since
these will exist independent of the validation implementation.

* Include mapping_exceptions during EQL query validation

These include errors for inaccessible indexes, which should be useful to
the rule writer in writing their EQL query.

* Display toast messages for non-validation messages

* fix type errors

This type was renamed.

* Do not request data in our validation request

By not having the cluster retrieve/send any data, this should saves us
a few CPU cycles.

* Move EQL validation to an async form validator

Rather than invoking a custom validation hook (useEqlValidation) at custom times (onBlur) in our EqlQueryBar
component, we can instead move this functionality to a form validation
function and have it be invoked automatically by our form when values
change. However, because we still need to handle the validation messages
slightly differently (place them in a popover as opposed to an
EuiFormRow), we also need custom error retrieval in the form of
getValidationResults.

After much pain, it was determined that the default behavior of
_.debounce does not work with async validator functions, as a debounced
call will not "wait" for the eventual invocation but will instead return
the most recently resolved value. This leads to stale validation
results and terrible UX, so I wrote a custom function (debounceAsync)
that behaves like we want/need; see tests for details.

* Invalidate our query field when index patterns change

Since EQL rules actually validate against the relevant indexes, changing
said indexes should invalidate/revalidate the query.

With the form lib, this is beautifully simple :)

* Set a min-height on our EQL textarea

* Remove unused prop from EqlQueryBar

Index corresponds to the value from the index field; now that our EQL
validation is performed by the form we have no need for it here.

* Update EQL overview link to point to elasticsearch docs

Adds an entry in our doclinks service, and uses that.

* Remove unused prop from stale tests

* Update docLinks documentation with new EQL link

* Fix bug where saved query rules had no type selected on Edit

* Wait for kibana requests to complete before moving between rule tabs

With our new async validation, a user can quickly navigate away from the
Definition tab before the validation has completed, resulting in the
form being invalidated. Any subsequent user actions cause the form to
correct itself, but until I can find a better solution here this really
just gives the validation time to complete and sidesteps the issue.
This commit is contained in:
Ryland Herrick 2020-10-02 13:19:02 -05:00 committed by GitHub
parent 374ccfd66f
commit 7cfdeaeede
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
41 changed files with 1075 additions and 29 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
docs/development/core/public/kibana-plugin-core-public.doclinksstart.links.md
View file

@ -91,6 +91,7 @@ readonly links: {
readonly gettingStarted: string;
};
readonly query: {
readonly eql: string;
readonly luceneQuerySyntax: string;
readonly queryDsl: string;
readonly kueryQuerySyntax: string;

2
docs/development/core/public/kibana-plugin-core-public.doclinksstart.md
View file

@ -17,5 +17,5 @@ export interface DocLinksStart
| --- | --- | --- |
| [DOC\_LINK\_VERSION](./kibana-plugin-core-public.doclinksstart.doc_link_version.md) | <code>string</code> | |
| [ELASTIC\_WEBSITE\_URL](./kibana-plugin-core-public.doclinksstart.elastic_website_url.md) | <code>string</code> | |
| [links](./kibana-plugin-core-public.doclinksstart.links.md) | <code>{</code><br/><code> readonly dashboard: {</code><br/><code> readonly drilldowns: string;</code><br/><code> readonly drilldownsTriggerPicker: string;</code><br/><code> readonly urlDrilldownTemplateSyntax: string;</code><br/><code> readonly urlDrilldownVariables: string;</code><br/><code> };</code><br/><code> readonly filebeat: {</code><br/><code> readonly base: string;</code><br/><code> readonly installation: string;</code><br/><code> readonly configuration: string;</code><br/><code> readonly elasticsearchOutput: string;</code><br/><code> readonly startup: string;</code><br/><code> readonly exportedFields: string;</code><br/><code> };</code><br/><code> readonly auditbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly metricbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly heartbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly logstash: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly functionbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly winlogbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly aggs: {</code><br/><code> readonly date_histogram: string;</code><br/><code> readonly date_range: string;</code><br/><code> readonly filter: string;</code><br/><code> readonly filters: string;</code><br/><code> readonly geohash_grid: string;</code><br/><code> readonly histogram: string;</code><br/><code> readonly ip_range: string;</code><br/><code> readonly range: string;</code><br/><code> readonly significant_terms: string;</code><br/><code> readonly terms: string;</code><br/><code> readonly avg: string;</code><br/><code> readonly avg_bucket: string;</code><br/><code> readonly max_bucket: string;</code><br/><code> readonly min_bucket: string;</code><br/><code> readonly sum_bucket: string;</code><br/><code> readonly cardinality: string;</code><br/><code> readonly count: string;</code><br/><code> readonly cumulative_sum: string;</code><br/><code> readonly derivative: string;</code><br/><code> readonly geo_bounds: string;</code><br/><code> readonly geo_centroid: string;</code><br/><code> readonly max: string;</code><br/><code> readonly median: string;</code><br/><code> readonly min: string;</code><br/><code> readonly moving_avg: string;</code><br/><code> readonly percentile_ranks: string;</code><br/><code> readonly serial_diff: string;</code><br/><code> readonly std_dev: string;</code><br/><code> readonly sum: string;</code><br/><code> readonly top_hits: string;</code><br/><code> };</code><br/><code> readonly scriptedFields: {</code><br/><code> readonly scriptFields: string;</code><br/><code> readonly scriptAggs: string;</code><br/><code> readonly painless: string;</code><br/><code> readonly painlessApi: string;</code><br/><code> readonly painlessSyntax: string;</code><br/><code> readonly luceneExpressions: string;</code><br/><code> };</code><br/><code> readonly indexPatterns: {</code><br/><code> readonly loadingData: string;</code><br/><code> readonly introduction: string;</code><br/><code> };</code><br/><code> readonly addData: string;</code><br/><code> readonly kibana: string;</code><br/><code> readonly siem: {</code><br/><code> readonly guide: string;</code><br/><code> readonly gettingStarted: string;</code><br/><code> };</code><br/><code> readonly query: {</code><br/><code> readonly luceneQuerySyntax: string;</code><br/><code> readonly queryDsl: string;</code><br/><code> readonly kueryQuerySyntax: string;</code><br/><code> };</code><br/><code> readonly date: {</code><br/><code> readonly dateMath: string;</code><br/><code> };</code><br/><code> readonly management: Record&lt;string, string&gt;;</code><br/><code> readonly visualize: Record&lt;string, string&gt;;</code><br/><code> }</code> | |
| [links](./kibana-plugin-core-public.doclinksstart.links.md) | <code>{</code><br/><code> readonly dashboard: {</code><br/><code> readonly drilldowns: string;</code><br/><code> readonly drilldownsTriggerPicker: string;</code><br/><code> readonly urlDrilldownTemplateSyntax: string;</code><br/><code> readonly urlDrilldownVariables: string;</code><br/><code> };</code><br/><code> readonly filebeat: {</code><br/><code> readonly base: string;</code><br/><code> readonly installation: string;</code><br/><code> readonly configuration: string;</code><br/><code> readonly elasticsearchOutput: string;</code><br/><code> readonly startup: string;</code><br/><code> readonly exportedFields: string;</code><br/><code> };</code><br/><code> readonly auditbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly metricbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly heartbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly logstash: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly functionbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly winlogbeat: {</code><br/><code> readonly base: string;</code><br/><code> };</code><br/><code> readonly aggs: {</code><br/><code> readonly date_histogram: string;</code><br/><code> readonly date_range: string;</code><br/><code> readonly filter: string;</code><br/><code> readonly filters: string;</code><br/><code> readonly geohash_grid: string;</code><br/><code> readonly histogram: string;</code><br/><code> readonly ip_range: string;</code><br/><code> readonly range: string;</code><br/><code> readonly significant_terms: string;</code><br/><code> readonly terms: string;</code><br/><code> readonly avg: string;</code><br/><code> readonly avg_bucket: string;</code><br/><code> readonly max_bucket: string;</code><br/><code> readonly min_bucket: string;</code><br/><code> readonly sum_bucket: string;</code><br/><code> readonly cardinality: string;</code><br/><code> readonly count: string;</code><br/><code> readonly cumulative_sum: string;</code><br/><code> readonly derivative: string;</code><br/><code> readonly geo_bounds: string;</code><br/><code> readonly geo_centroid: string;</code><br/><code> readonly max: string;</code><br/><code> readonly median: string;</code><br/><code> readonly min: string;</code><br/><code> readonly moving_avg: string;</code><br/><code> readonly percentile_ranks: string;</code><br/><code> readonly serial_diff: string;</code><br/><code> readonly std_dev: string;</code><br/><code> readonly sum: string;</code><br/><code> readonly top_hits: string;</code><br/><code> };</code><br/><code> readonly scriptedFields: {</code><br/><code> readonly scriptFields: string;</code><br/><code> readonly scriptAggs: string;</code><br/><code> readonly painless: string;</code><br/><code> readonly painlessApi: string;</code><br/><code> readonly painlessSyntax: string;</code><br/><code> readonly luceneExpressions: string;</code><br/><code> };</code><br/><code> readonly indexPatterns: {</code><br/><code> readonly loadingData: string;</code><br/><code> readonly introduction: string;</code><br/><code> };</code><br/><code> readonly addData: string;</code><br/><code> readonly kibana: string;</code><br/><code> readonly siem: {</code><br/><code> readonly guide: string;</code><br/><code> readonly gettingStarted: string;</code><br/><code> };</code><br/><code> readonly query: {</code><br/><code> readonly eql: string;</code><br/><code> readonly luceneQuerySyntax: string;</code><br/><code> readonly queryDsl: string;</code><br/><code> readonly kueryQuerySyntax: string;</code><br/><code> };</code><br/><code> readonly date: {</code><br/><code> readonly dateMath: string;</code><br/><code> };</code><br/><code> readonly management: Record&lt;string, string&gt;;</code><br/><code> readonly visualize: Record&lt;string, string&gt;;</code><br/><code> }</code> | |

2
src/core/public/doc_links/doc_links_service.ts
View file

@ -119,6 +119,7 @@ export class DocLinksService {
gettingStarted: `${ELASTIC_WEBSITE_URL}guide/en/security/${DOC_LINK_VERSION}/index.html`,
},
query: {
eql: `${ELASTICSEARCH_DOCS}eql.html`,
luceneQuerySyntax: `${ELASTICSEARCH_DOCS}query-dsl-query-string-query.html#query-string-syntax`,
queryDsl: `${ELASTICSEARCH_DOCS}query-dsl.html`,
kueryQuerySyntax: `${ELASTIC_WEBSITE_URL}guide/en/kibana/${DOC_LINK_VERSION}/kuery-query.html`,
@ -227,6 +228,7 @@ export interface DocLinksStart {
readonly gettingStarted: string;
};
readonly query: {
readonly eql: string;
readonly luceneQuerySyntax: string;
readonly queryDsl: string;
readonly kueryQuerySyntax: string;

1
src/core/public/public.api.md
View file

@ -539,6 +539,7 @@ export interface DocLinksStart {
readonly gettingStarted: string;
};
readonly query: {
readonly eql: string;
readonly luceneQuerySyntax: string;
readonly queryDsl: string;
readonly kueryQuerySyntax: string;

1
x-pack/plugins/security_solution/common/constants.ts
View file

@ -117,6 +117,7 @@ export const DETECTION_ENGINE_PREPACKAGED_URL = `${DETECTION_ENGINE_RULES_URL}/p
export const DETECTION_ENGINE_PRIVILEGES_URL = `${DETECTION_ENGINE_URL}/privileges`;
export const DETECTION_ENGINE_INDEX_URL = `${DETECTION_ENGINE_URL}/index`;
export const DETECTION_ENGINE_TAGS_URL = `${DETECTION_ENGINE_URL}/tags`;
export const DETECTION_ENGINE_EQL_VALIDATION_URL = `${DETECTION_ENGINE_URL}/validate_eql`;
export const DETECTION_ENGINE_RULES_STATUS_URL = `${DETECTION_ENGINE_RULES_URL}/_find_statuses`;
export const DETECTION_ENGINE_PREPACKAGED_RULES_STATUS_URL = `${DETECTION_ENGINE_RULES_URL}/prepackaged/_status`;

12
x-pack/plugins/security_solution/common/detection_engine/schemas/request/eql_validation_schema.mock.ts Normal file
View file

@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { EqlValidationSchema } from './eql_validation_schema';
export const getEqlValidationSchemaMock = (): EqlValidationSchema => ({
index: ['index-123'],
query: 'process where process.name == "regsvr32.exe"',
});

59
x-pack/plugins/security_solution/common/detection_engine/schemas/request/eql_validation_schema.test.ts Normal file
View file

@ -0,0 +1,59 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { pipe } from 'fp-ts/lib/pipeable';
import { left } from 'fp-ts/lib/Either';
import { exactCheck } from '../../../exact_check';
import { foldLeftRight, getPaths } from '../../../test_utils';
import { eqlValidationSchema, EqlValidationSchema } from './eql_validation_schema';
import { getEqlValidationSchemaMock } from './eql_validation_schema.mock';
describe('EQL validation schema', () => {
it('requires a value for index', () => {
const payload = {
...getEqlValidationSchemaMock(),
index: undefined,
};
const decoded = eqlValidationSchema.decode(payload);
const checked = exactCheck(payload, decoded);
const message = pipe(checked, foldLeftRight);
expect(getPaths(left(message.errors))).toEqual([
'Invalid value "undefined" supplied to "index"',
]);
expect(message.schema).toEqual({});
});
it('requires a value for query', () => {
const payload = {
...getEqlValidationSchemaMock(),
query: undefined,
};
const decoded = eqlValidationSchema.decode(payload);
const checked = exactCheck(payload, decoded);
const message = pipe(checked, foldLeftRight);
expect(getPaths(left(message.errors))).toEqual([
'Invalid value "undefined" supplied to "query"',
]);
expect(message.schema).toEqual({});
});
it('validates a payload with index and query', () => {
const payload = getEqlValidationSchemaMock();
const decoded = eqlValidationSchema.decode(payload);
const checked = exactCheck(payload, decoded);
const message = pipe(checked, foldLeftRight);
const expected: EqlValidationSchema = {
index: ['index-123'],
query: 'process where process.name == "regsvr32.exe"',
};
expect(getPaths(left(message.errors))).toEqual([]);
expect(message.schema).toEqual(expected);
});
});

18
x-pack/plugins/security_solution/common/detection_engine/schemas/request/eql_validation_schema.ts Normal file
View file

@ -0,0 +1,18 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import * as t from 'io-ts';
import { index, query } from '../common/schemas';
export const eqlValidationSchema = t.exact(
t.type({
index,
query,
})
);
export type EqlValidationSchema = t.TypeOf<typeof eqlValidationSchema>;

17
x-pack/plugins/security_solution/common/detection_engine/schemas/response/eql_validation_schema.mock.ts Normal file
View file

@ -0,0 +1,17 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { EqlValidationSchema } from './eql_validation_schema';
export const getEqlValidationResponseMock = (): EqlValidationSchema => ({
valid: false,
errors: ['line 3:52: token recognition error at: '],
});
export const getValidEqlValidationResponseMock = (): EqlValidationSchema => ({
valid: true,
errors: [],
});

59
x-pack/plugins/security_solution/common/detection_engine/schemas/response/eql_validation_schema.test.ts Normal file
View file

@ -0,0 +1,59 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { pipe } from 'fp-ts/lib/pipeable';
import { left } from 'fp-ts/lib/Either';
import { exactCheck } from '../../../exact_check';
import { foldLeftRight, getPaths } from '../../../test_utils';
import { getEqlValidationResponseMock } from './eql_validation_schema.mock';
import { eqlValidationSchema } from './eql_validation_schema';
describe('EQL validation response schema', () => {
it('validates a typical response', () => {
const payload = getEqlValidationResponseMock();
const decoded = eqlValidationSchema.decode(payload);
const checked = exactCheck(payload, decoded);
const message = pipe(checked, foldLeftRight);
expect(getPaths(left(message.errors))).toEqual([]);
expect(message.schema).toEqual(getEqlValidationResponseMock());
});
it('invalidates a response with extra properties', () => {
const payload = { ...getEqlValidationResponseMock(), extra: 'nope' };
const decoded = eqlValidationSchema.decode(payload);
const checked = exactCheck(payload, decoded);
const message = pipe(checked, foldLeftRight);
expect(getPaths(left(message.errors))).toEqual(['invalid keys "extra"']);
expect(message.schema).toEqual({});
});
it('invalidates a response with missing properties', () => {
const payload = { ...getEqlValidationResponseMock(), valid: undefined };
const decoded = eqlValidationSchema.decode(payload);
const checked = exactCheck(payload, decoded);
const message = pipe(checked, foldLeftRight);
expect(getPaths(left(message.errors))).toEqual([
'Invalid value "undefined" supplied to "valid"',
]);
expect(message.schema).toEqual({});
});
it('invalidates a response with properties of the wrong type', () => {
const payload = { ...getEqlValidationResponseMock(), errors: 'should be an array' };
const decoded = eqlValidationSchema.decode(payload);
const checked = exactCheck(payload, decoded);
const message = pipe(checked, foldLeftRight);
expect(getPaths(left(message.errors))).toEqual([
'Invalid value "should be an array" supplied to "errors"',
]);
expect(message.schema).toEqual({});
});
});

16
x-pack/plugins/security_solution/common/detection_engine/schemas/response/eql_validation_schema.ts Normal file
View file

@ -0,0 +1,16 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import * as t from 'io-ts';
export const eqlValidationSchema = t.exact(
t.type({
valid: t.boolean,
errors: t.array(t.string),
})
);
export type EqlValidationSchema = t.TypeOf<typeof eqlValidationSchema>;

3
x-pack/plugins/security_solution/common/detection_engine/utils.ts
View file

@ -19,5 +19,6 @@ export const hasNestedEntry = (entries: EntriesArray): boolean => {
export const isEqlRule = (ruleType: Type | undefined): boolean => ruleType === 'eql';
export const isThresholdRule = (ruleType: Type | undefined): boolean => ruleType === 'threshold';
export const isQueryRule = (ruleType: Type | undefined): boolean => ruleType === 'query';
export const isQueryRule = (ruleType: Type | undefined): boolean =>
ruleType === 'query' || ruleType === 'saved_query';
export const isThreatMatchRule = (ruleType: Type): boolean => ruleType === 'threat_match';

3
x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_custom.spec.ts
View file

@ -93,7 +93,7 @@ import {
goToScheduleStepTab,
waitForTheRuleToBeExecuted,
} from '../tasks/create_new_rule';
import { saveEditedRule } from '../tasks/edit_rule';
import { saveEditedRule, waitForKibana } from '../tasks/edit_rule';
import { esArchiverLoad, esArchiverUnload } from '../tasks/es_archiver';
import { loginAndWaitForPageWithoutDateRange } from '../tasks/login';
import { refreshPage } from '../tasks/security_header';
@ -290,6 +290,7 @@ describe('Custom detection rules deletion and edition', () => {
context('Edition', () => {
it('Allows a rule to be edited', () => {
editFirstRule();
waitForKibana();
// expect define step to populate
cy.get(CUSTOM_QUERY_INPUT).should('have.text', existingRule.customQuery);

2
x-pack/plugins/security_solution/cypress/screens/edit_rule.ts
View file

@ -5,3 +5,5 @@
*/
export const EDIT_SUBMIT_BUTTON = '[data-test-subj="ruleEditSubmitButton"]';
export const KIBANA_LOADING_INDICATOR = '[data-test-subj="globalLoadingIndicator"]';
export const KIBANA_LOADING_COMPLETE_INDICATOR = '[data-test-subj="globalLoadingIndicator-hidden"]';

6
x-pack/plugins/security_solution/cypress/tasks/edit_rule.ts
View file

@ -4,9 +4,13 @@
* you may not use this file except in compliance with the Elastic License.
*/
import { EDIT_SUBMIT_BUTTON } from '../screens/edit_rule';
import { EDIT_SUBMIT_BUTTON, KIBANA_LOADING_COMPLETE_INDICATOR } from '../screens/edit_rule';
export const saveEditedRule = () => {
cy.get(EDIT_SUBMIT_BUTTON).should('exist').click({ force: true });
cy.get(EDIT_SUBMIT_BUTTON).should('not.exist');
};
export const waitForKibana = () => {
cy.get(KIBANA_LOADING_COMPLETE_INDICATOR).should('exist');
};

31
x-pack/plugins/security_solution/public/common/hooks/eql/api.ts Normal file
View file

@ -0,0 +1,31 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { HttpStart } from '../../../../../../../src/core/public';
import { DETECTION_ENGINE_EQL_VALIDATION_URL } from '../../../../common/constants';
import { EqlValidationSchema as EqlValidationRequest } from '../../../../common/detection_engine/schemas/request/eql_validation_schema';
import { EqlValidationSchema as EqlValidationResponse } from '../../../../common/detection_engine/schemas/response/eql_validation_schema';
interface ApiParams {
http: HttpStart;
signal: AbortSignal;
}
export const validateEql = async ({
http,
query,
index,
signal,
}: ApiParams & EqlValidationRequest) => {
return http.fetch<EqlValidationResponse>(DETECTION_ENGINE_EQL_VALIDATION_URL, {
method: 'POST',
body: JSON.stringify({
query,
index,
}),
signal,
});
};

26
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/eql_overview_link.tsx Normal file
View file

@ -0,0 +1,26 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import React from 'react';
import styled from 'styled-components';
import { EuiLink, EuiText } from '@elastic/eui';
import { useKibana } from '../../../../common/lib/kibana';
import { EQL_OVERVIEW_LINK_TEXT } from './translations';
const InlineText = styled(EuiText)`
display: inline-block;
`;
export const EqlOverviewLink = () => {
const overviewUrl = useKibana().services.docLinks.links.query.eql;
return (
<EuiLink external href={overviewUrl} target="_blank">
<InlineText size="xs">{EQL_OVERVIEW_LINK_TEXT}</InlineText>
</EuiLink>
);
};

37
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/eql_query_bar.test.tsx
View file

@ -7,9 +7,12 @@
import React from 'react';
import { shallow, mount } from 'enzyme';
import { useFormFieldMock } from '../../../../common/mock';
import { TestProviders, useFormFieldMock } from '../../../../common/mock';
import { mockQueryBar } from '../../../pages/detection_engine/rules/all/__mocks__/mock';
import { EqlQueryBar, EqlQueryBarProps } from './eql_query_bar';
import { getEqlValidationError } from './validators.mock';
jest.mock('../../../../common/lib/kibana');
describe('EqlQueryBar', () => {
let mockField: EqlQueryBarProps['field'];
@ -27,7 +30,11 @@ describe('EqlQueryBar', () => {
});
it('sets the field value on input change', () => {
const wrapper = mount(<EqlQueryBar dataTestSubj="myQueryBar" field={mockField} />);
const wrapper = mount(
<TestProviders>
<EqlQueryBar dataTestSubj="myQueryBar" field={mockField} />
</TestProviders>
);
wrapper
.find('[data-test-subj="eqlQueryBarTextInput"]')
@ -44,4 +51,30 @@ describe('EqlQueryBar', () => {
expect(mockField.setValue).toHaveBeenCalledWith(expected);
});
it('does not render errors for a valid query', () => {
const wrapper = mount(
<TestProviders>
<EqlQueryBar dataTestSubj="myQueryBar" field={mockField} />
</TestProviders>
);
expect(wrapper.find('[data-test-subj="eql-validation-errors-popover"]').exists()).toEqual(
false
);
});
it('renders errors for an invalid query', () => {
const invalidMockField = useFormFieldMock({
value: mockQueryBar,
errors: [getEqlValidationError()],
});
const wrapper = mount(
<TestProviders>
<EqlQueryBar dataTestSubj="myQueryBar" field={invalidMockField} />
</TestProviders>
);
expect(wrapper.find('[data-test-subj="eql-validation-errors-popover"]').exists()).toEqual(true);
});
});

53
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/eql_query_bar.tsx
View file

@ -4,11 +4,24 @@
* you may not use this file except in compliance with the Elastic License.
*/
import React, { FC, useCallback, ChangeEvent } from 'react';
import React, { FC, useCallback, ChangeEvent, useEffect, useState } from 'react';
import styled from 'styled-components';
import { EuiFormRow, EuiTextArea } from '@elastic/eui';
import { FieldHook, getFieldValidityAndErrorMessage } from '../../../../shared_imports';
import { FieldHook } from '../../../../shared_imports';
import { useAppToasts } from '../../../../common/hooks/use_app_toasts';
import { DefineStepRule } from '../../../pages/detection_engine/rules/types';
import * as i18n from './translations';
import { EqlQueryBarFooter } from './footer';
import { getValidationResults } from './validators';
const TextArea = styled(EuiTextArea)`
display: block;
border: ${({ theme }) => theme.eui.euiBorderThin};
border-bottom: 0;
box-shadow: none;
min-height: ${({ theme }) => theme.eui.euiFormControlHeight};
`;
export interface EqlQueryBarProps {
dataTestSubj: string;
@ -17,14 +30,27 @@ export interface EqlQueryBarProps {
}
export const EqlQueryBar: FC<EqlQueryBarProps> = ({ dataTestSubj, field, idAria }) => {
const { addError } = useAppToasts();
const [errorMessages, setErrorMessages] = useState<string[]>([]);
const { setValue } = field;
const { isInvalid, errorMessage } = getFieldValidityAndErrorMessage(field);
const { isValid, message, messages, error } = getValidationResults(field);
const fieldValue = field.value.query.query as string;
useEffect(() => {
setErrorMessages(messages ?? []);
}, [messages]);
useEffect(() => {
if (error) {
addError(error, { title: i18n.EQL_VALIDATION_REQUEST_ERROR });
}
}, [error, addError]);
const handleChange = useCallback(
(e: ChangeEvent<HTMLTextAreaElement>) => {
const newQuery = e.target.value;
setErrorMessages([]);
setValue({
filters: [],
query: {
@ -41,19 +67,22 @@ export const EqlQueryBar: FC<EqlQueryBarProps> = ({ dataTestSubj, field, idAria
label={field.label}
labelAppend={field.labelAppend}
helpText={field.helpText}
error={errorMessage}
isInvalid={isInvalid}
error={message}
isInvalid={!isValid}
fullWidth
data-test-subj={dataTestSubj}
describedByIds={idAria ? [idAria] : undefined}
>
<EuiTextArea
data-test-subj="eqlQueryBarTextInput"
fullWidth
isInvalid={isInvalid}
value={fieldValue}
onChange={handleChange}
/>
<>
<TextArea
data-test-subj="eqlQueryBarTextInput"
fullWidth
isInvalid={!isValid}
value={fieldValue}
onChange={handleChange}
/>
<EqlQueryBarFooter errors={errorMessages} />
</>
</EuiFormRow>
);
};

50
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/errors_popover.test.tsx Normal file
View file

@ -0,0 +1,50 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import React from 'react';
import { shallow, mount } from 'enzyme';
import { ErrorsPopover } from './errors_popover';
describe('ErrorsPopover', () => {
let mockErrors: string[];
beforeEach(() => {
mockErrors = [];
});
it('renders correctly', () => {
const wrapper = shallow(<ErrorsPopover errors={mockErrors} />);
expect(wrapper.find('[data-test-subj="eql-validation-errors-popover"]')).toHaveLength(1);
});
it('renders the number of errors by default', () => {
mockErrors = ['error', 'other', 'third'];
const wrapper = mount(<ErrorsPopover errors={mockErrors} />);
expect(
wrapper.find('[data-test-subj="eql-validation-errors-popover"]').first().text()
).toContain('3');
});
it('renders the error messages if clicked', () => {
mockErrors = ['error', 'other'];
const wrapper = mount(<ErrorsPopover errors={mockErrors} />);
wrapper
.find('[data-test-subj="eql-validation-errors-popover-button"]')
.first()
.simulate('click');
expect(
wrapper.find('[data-test-subj="eql-validation-errors-popover"]').first().text()
).toContain('2');
const messagesContent = wrapper
.find('[data-test-subj="eql-validation-errors-popover-content"]')
.text();
expect(messagesContent).toContain('error');
expect(messagesContent).toContain('other');
});
});

55
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/errors_popover.tsx Normal file
View file

@ -0,0 +1,55 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import React, { FC, useCallback, useState } from 'react';
import { EuiButtonEmpty, EuiPopover, EuiPopoverTitle, EuiText } from '@elastic/eui';
import * as i18n from './translations';
export interface ErrorsPopoverProps {
ariaLabel?: string;
errors: string[];
}
export const ErrorsPopover: FC<ErrorsPopoverProps> = ({ ariaLabel, errors }) => {
const [isOpen, setIsOpen] = useState(false);
const handleToggle = useCallback(() => {
setIsOpen(!isOpen);
}, [isOpen]);
const handleClose = useCallback(() => {
setIsOpen(false);
}, []);
return (
<EuiPopover
data-test-subj="eql-validation-errors-popover"
button={
<EuiButtonEmpty
data-test-subj="eql-validation-errors-popover-button"
iconType="crossInACircleFilled"
size="s"
color="danger"
aria-label={ariaLabel}
onClick={handleToggle}
>
{errors.length}
</EuiButtonEmpty>
}
isOpen={isOpen}
closePopover={handleClose}
anchorPosition="downCenter"
>
<div data-test-subj="eql-validation-errors-popover-content">
<EuiPopoverTitle>{i18n.EQL_VALIDATION_ERRORS_TITLE}</EuiPopoverTitle>
{errors.map((message, idx) => (
<EuiText key={idx}>{message}</EuiText>
))}
</div>
</EuiPopover>
);
};

42
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/footer.tsx Normal file
View file

@ -0,0 +1,42 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import React, { FC } from 'react';
import styled from 'styled-components';
import { EuiFlexGroup, EuiFlexItem, EuiPanel } from '@elastic/eui';
import * as i18n from './translations';
import { ErrorsPopover } from './errors_popover';
import { EqlOverviewLink } from './eql_overview_link';
export interface Props {
errors: string[];
}
const Container = styled(EuiPanel)`
border-radius: 0;
background: ${({ theme }) => theme.eui.euiPageBackgroundColor};
padding: ${({ theme }) => theme.eui.euiSizeXS};
`;
const FlexGroup = styled(EuiFlexGroup)`
min-height: ${({ theme }) => theme.eui.euiSizeXL};
`;
export const EqlQueryBarFooter: FC<Props> = ({ errors }) => (
<Container>
<FlexGroup alignItems="center" justifyContent="spaceBetween" gutterSize="none">
<EuiFlexItem>
{errors.length > 0 && (
<ErrorsPopover ariaLabel={i18n.EQL_VALIDATION_ERROR_POPOVER_LABEL} errors={errors} />
)}
</EuiFlexItem>
<EuiFlexItem grow={false}>
<EqlOverviewLink />
</EuiFlexItem>
</FlexGroup>
</Container>
);

42
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/translations.ts Normal file
View file

@ -0,0 +1,42 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { i18n } from '@kbn/i18n';
export const EQL_VALIDATION_REQUEST_ERROR = i18n.translate(
'xpack.securitySolution.detectionEngine.eqlValidation.requestError',
{
defaultMessage: 'An error occurred while validating your EQL query',
}
);
export const EQL_VALIDATION_ERRORS_TITLE = i18n.translate(
'xpack.securitySolution.detectionEngine.eqlValidation.title',
{
defaultMessage: 'EQL Validation Errors',
}
);
export const EQL_VALIDATION_ERROR_POPOVER_LABEL = i18n.translate(
'xpack.securitySolution.detectionEngine.eqlValidation.showErrorsLabel',
{
defaultMessage: 'Show EQL Validation Errors',
}
);
export const EQL_QUERY_BAR_LABEL = i18n.translate(
'xpack.securitySolution.detectionEngine.eqlQueryBar.label',
{
defaultMessage: 'Enter an EQL Query',
}
);
export const EQL_OVERVIEW_LINK_TEXT = i18n.translate(
'xpack.securitySolution.detectionEngine.eqlOverViewLink.text',
{
defaultMessage: 'Event Query Language (EQL) Overview',
}
);

19
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/validators.mock.ts Normal file
View file

@ -0,0 +1,19 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { ValidationError } from '../../../../shared_imports';
import { ERROR_CODES } from './validators';
export const getEqlResponseError = (): ValidationError => ({
code: ERROR_CODES.FAILED_REQUEST,
message: 'something went wrong',
});
export const getEqlValidationError = (): ValidationError => ({
code: ERROR_CODES.INVALID_EQL,
messages: ['line 1: WRONG\nline 2: ALSO WRONG'],
message: '',
});

52
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/validators.test.ts Normal file
View file

@ -0,0 +1,52 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { debounceAsync } from './validators';
jest.useFakeTimers();
describe('debounceAsync', () => {
let fn: jest.Mock;
beforeEach(() => {
fn = jest.fn().mockResolvedValueOnce('first');
});
it('resolves with the underlying invocation result', async () => {
const debounced = debounceAsync(fn, 0);
const promise = debounced();
jest.runOnlyPendingTimers();
expect(await promise).toEqual('first');
});
it('resolves intermediate calls when the next invocation resolves', async () => {
const debounced = debounceAsync(fn, 200);
fn.mockResolvedValueOnce('second');
const promise = debounced();
jest.runOnlyPendingTimers();
expect(await promise).toEqual('first');
const promises = [debounced(), debounced()];
jest.runOnlyPendingTimers();
expect(await Promise.all(promises)).toEqual(['second', 'second']);
});
it('debounces the function', async () => {
const debounced = debounceAsync(fn, 200);
debounced();
jest.runOnlyPendingTimers();
debounced();
debounced();
jest.runOnlyPendingTimers();
expect(fn).toHaveBeenCalledTimes(2);
});
});

105
x-pack/plugins/security_solution/public/detections/components/rules/eql_query_bar/validators.ts Normal file
View file

@ -0,0 +1,105 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { isEmpty } from 'lodash';
import { FieldHook, ValidationError, ValidationFunc } from '../../../../shared_imports';
import { isEqlRule } from '../../../../../common/detection_engine/utils';
import { KibanaServices } from '../../../../common/lib/kibana';
import { DefineStepRule } from '../../../pages/detection_engine/rules/types';
import { validateEql } from '../../../../common/hooks/eql/api';
import { FieldValueQueryBar } from '../query_bar';
import * as i18n from './translations';
export enum ERROR_CODES {
FAILED_REQUEST = 'ERR_FAILED_REQUEST',
INVALID_EQL = 'ERR_INVALID_EQL',
}
/**
* Unlike lodash's debounce, which resolves intermediate calls with the most
* recent value, this implementation waits to resolve intermediate calls until
* the next invocation resolves.
*
* @param fn an async function
*
* @returns A debounced async function that resolves on the next invocation
*/
export const debounceAsync = <Args extends unknown[], Result extends Promise<unknown>>(
fn: (...args: Args) => Result,
interval: number
): ((...args: Args) => Result) => {
let handle: ReturnType<typeof setTimeout> | undefined;
let resolves: Array<(value?: Result) => void> = [];
return (...args: Args): Result => {
if (handle) {
clearTimeout(handle);
}
handle = setTimeout(() => {
const result = fn(...args);
resolves.forEach((resolve) => resolve(result));
resolves = [];
}, interval);
return new Promise((resolve) => resolves.push(resolve)) as Result;
};
};
export const eqlValidator = async (
...args: Parameters<ValidationFunc>
): Promise<ValidationError<ERROR_CODES> | void | undefined> => {
const [{ value, formData }] = args;
const { query: queryValue } = value as FieldValueQueryBar;
const query = queryValue.query as string;
const { index, ruleType } = formData as DefineStepRule;
const needsValidation = isEqlRule(ruleType) && !isEmpty(query);
if (!needsValidation) {
return;
}
try {
const { http } = KibanaServices.get();
const signal = new AbortController().signal;
const response = await validateEql({ query, http, signal, index });
if (response?.valid === false) {
return {
code: ERROR_CODES.INVALID_EQL,
message: '',
messages: response.errors,
};
}
} catch (error) {
return {
code: ERROR_CODES.FAILED_REQUEST,
message: i18n.EQL_VALIDATION_REQUEST_ERROR,
error,
};
}
};
export const getValidationResults = <T = unknown>(
field: FieldHook<T>
): { isValid: boolean; message: string; messages?: string[]; error?: Error } => {
const hasErrors = field.errors.length > 0;
const isValid = !field.isChangingValue && !hasErrors;
if (hasErrors) {
const [error] = field.errors;
const message = error.message;
if (error.code === ERROR_CODES.INVALID_EQL) {
return { isValid, message, messages: error.messages };
} else {
return { isValid, message, error: error.error };
}
} else {
return { isValid, message: '' };
}
};

6
x-pack/plugins/security_solution/public/detections/components/rules/select_rule_type/eql_search_icon.svg Normal file
View file

@ -0,0 +1,6 @@
<svg width="26" height="24" viewBox="0 0 26 24" fill="none" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" clip-rule="evenodd" d="M17.0699 15.5699C18.2787 14.0377 19 12.1031 19 10C19 5.02944 14.9706 1 10 1C5.02944 1 1 5.02944 1 10C1 14.9706 5.02944 19 10 19C11.7165 19 13.3208 18.5195 14.6856 17.6856L20 23H24.5L17.0699 15.5699Z" fill="white"/>
<path d="M17.0699 15.5699L16.6773 15.2602L16.402 15.6091L16.7163 15.9234L17.0699 15.5699ZM14.6856 17.6856L15.0392 17.332L14.7608 17.0537L14.4249 17.2589L14.6856 17.6856ZM20 23L19.6464 23.3536L19.7929 23.5H20V23ZM24.5 23V23.5H25.7071L24.8536 22.6464L24.5 23ZM18.5 10C18.5 11.9868 17.819 13.8131 16.6773 15.2602L17.4624 15.8796C18.7383 14.2623 19.5 12.2194 19.5 10H18.5ZM10 1.5C14.6944 1.5 18.5 5.30558 18.5 10H19.5C19.5 4.75329 15.2467 0.5 10 0.5V1.5ZM1.5 10C1.5 5.30558 5.30558 1.5 10 1.5V0.5C4.75329 0.5 0.5 4.75329 0.5 10H1.5ZM10 18.5C5.30558 18.5 1.5 14.6944 1.5 10H0.5C0.5 15.2467 4.75329 19.5 10 19.5V18.5ZM14.4249 17.2589C13.1363 18.0462 11.6219 18.5 10 18.5V19.5C11.8111 19.5 13.5052 18.9927 14.9463 18.1123L14.4249 17.2589ZM20.3536 22.6464L15.0392 17.332L14.332 18.0392L19.6464 23.3536L20.3536 22.6464ZM24.5 22.5H20V23.5H24.5V22.5ZM16.7163 15.9234L24.1464 23.3536L24.8536 22.6464L17.4234 15.2163L16.7163 15.9234Z" fill="black"/>
<circle cx="10" cy="10" r="5.5" stroke="black"/>
<path d="M8 11.2169V8.7831L10 7.5831L12 8.7831V11.2169L10 12.4169L8 11.2169Z" stroke="black"/>
</svg>
Side by side

After

Width:  |  Height:  |  Size: 1.4 KiB

5
x-pack/plugins/security_solution/public/detections/components/rules/select_rule_type/index.tsx
View file

@ -7,6 +7,7 @@
import React, { useCallback, useMemo } from 'react';
import { EuiCard, EuiFlexGrid, EuiFlexItem, EuiFormRow, EuiIcon } from '@elastic/eui';
import { Type } from '../../../../../common/detection_engine/schemas/common/schemas';
import { isMlRule } from '../../../../../common/machine_learning/helpers';
import {
isThresholdRule,
@ -18,7 +19,7 @@ import { FieldHook } from '../../../../shared_imports';
import { useKibana } from '../../../../common/lib/kibana';
import * as i18n from './translations';
import { MlCardDescription } from './ml_card_description';
import { Type } from '../../../../../common/detection_engine/schemas/common/schemas';
import EqlSearchIcon from './eql_search_icon.svg';
interface SelectRuleTypeProps {
describedByIds?: string[];
@ -144,7 +145,7 @@ export const SelectRuleType: React.FC<SelectRuleTypeProps> = ({
data-test-subj="eqlRuleType"
title={i18n.EQL_TYPE_TITLE}
description={i18n.EQL_TYPE_DESCRIPTION}
icon={<EuiIcon size="l" type="bullseye" />}
icon={<EuiIcon size="l" type={EqlSearchIcon} />}
isDisabled={eqlSelectableConfig.isDisabled && !eqlSelectableConfig.isSelected}
selectable={eqlSelectableConfig}
/>

5
x-pack/plugins/security_solution/public/detections/components/rules/step_define_rule/index.tsx
View file

@ -274,6 +274,10 @@ const StepDefineRuleComponent: FC<StepDefineRuleProps> = ({
isLoading: indexPatternsLoading,
dataTestSubj: 'detectionEngineStepDefineRuleEqlQueryBar',
}}
config={{
...schema.queryBar,
label: i18n.EQL_QUERY_BAR_LABEL,
}}
/>
) : (
<UseField
@ -281,6 +285,7 @@ const StepDefineRuleComponent: FC<StepDefineRuleProps> = ({
path="queryBar"
config={{
...schema.queryBar,
label: i18n.QUERY_BAR_LABEL,
labelAppend: (
<MyLabelButton
data-test-subj="importQueryFromSavedTimeline"

13
x-pack/plugins/security_solution/public/detections/components/rules/step_define_rule/schema.tsx
View file

@ -4,9 +4,9 @@
* you may not use this file except in compliance with the Elastic License.
*/
import { isEmpty } from 'lodash';
import { i18n } from '@kbn/i18n';
import { EuiText } from '@elastic/eui';
import { isEmpty } from 'lodash/fp';
import React from 'react';
import {
@ -25,6 +25,7 @@ import {
ValidationFunc,
} from '../../../../shared_imports';
import { DefineStepRule } from '../../../pages/detection_engine/rules/types';
import { debounceAsync, eqlValidator } from '../eql_query_bar/validators';
import {
CUSTOM_QUERY_REQUIRED,
INVALID_CUSTOM_QUERY,
@ -36,6 +37,7 @@ import {
export const schema: FormSchema<DefineStepRule> = {
index: {
fieldsToValidateOnChange: ['index', 'queryBar'],
type: FIELD_TYPES.COMBO_BOX,
label: i18n.translate(
'xpack.securitySolution.detectionEngine.createRule.stepAboutRule.fiedIndexPatternsLabel',
@ -69,12 +71,6 @@ export const schema: FormSchema<DefineStepRule> = {
],
},
queryBar: {
label: i18n.translate(
'xpack.securitySolution.detectionEngine.createRule.stepDefineRule.fieldQuerBarLabel',
{
defaultMessage: 'Custom query',
}
),
validations: [
{
validator: (
@ -120,6 +116,9 @@ export const schema: FormSchema<DefineStepRule> = {
}
},
},
{
validator: debounceAsync(eqlValidator, 300),
},
],
},
ruleType: {

14
x-pack/plugins/security_solution/public/detections/components/rules/step_define_rule/translations.tsx
View file

@ -71,6 +71,20 @@ export const ENABLE_ML_JOB_WARNING = i18n.translate(
}
);
export const QUERY_BAR_LABEL = i18n.translate(
'xpack.securitySolution.detectionEngine.createRule.stepDefineRule.fieldQuerBarLabel',
{
defaultMessage: 'Custom query',
}
);
export const EQL_QUERY_BAR_LABEL = i18n.translate(
'xpack.securitySolution.detectionEngine.createRule.stepDefineRule.EqlQueryBarLabel',
{
defaultMessage: 'EQL query',
}
);
export const THREAT_MATCH_INDEX_HELPER_TEXT = i18n.translate(
'xpack.securitySolution.detectionEngine.createRule.stepDefineRule.threatMatchingIcesHelperDescription',
{

1
x-pack/plugins/security_solution/public/shared_imports.ts
View file

@ -22,6 +22,7 @@ export {
useForm,
useFormContext,
useFormData,
ValidationError,
ValidationFunc,
VALIDATION_TYPES,
} from '../../../../src/plugins/es_ui_shared/static/forms/hook_form_lib';

5
x-pack/plugins/security_solution/server/lib/detection_engine/routes/__mocks__/request_context.ts
View file

@ -18,6 +18,7 @@ const createMockClients = () => ({
alertsClient: alertsClientMock.create(),
clusterClient: elasticsearchServiceMock.createLegacyScopedClusterClient(),
licensing: { license: licensingMock.createLicenseMock() },
newClusterClient: elasticsearchServiceMock.createScopedClusterClient(),
savedObjectsClient: savedObjectsClientMock.create(),
appClient: siemMock.createClient(),
});
@ -31,7 +32,9 @@ const createRequestContextMock = (
core: {
...coreContext,
elasticsearch: {
legacy: { ...coreContext.elasticsearch, client: clients.clusterClient },
...coreContext.elasticsearch,
client: clients.newClusterClient,
legacy: { ...coreContext.elasticsearch.legacy, client: clients.clusterClient },
},
savedObjects: { client: clients.savedObjectsClient },
},

27
x-pack/plugins/security_solution/server/lib/detection_engine/routes/__mocks__/request_responses.ts
View file

@ -15,8 +15,9 @@ import {
INTERNAL_RULE_ID_KEY,
INTERNAL_IMMUTABLE_KEY,
DETECTION_ENGINE_PREPACKAGED_URL,
DETECTION_ENGINE_EQL_VALIDATION_URL,
} from '../../../../../common/constants';
import { ShardsResponse } from '../../../types';
import { EqlSearchResponse, ShardsResponse } from '../../../types';
import {
RuleAlertType,
IRuleSavedAttributesSavedObjectAttributes,
@ -28,6 +29,7 @@ import { QuerySignalsSchemaDecoded } from '../../../../../common/detection_engin
import { SetSignalsStatusSchemaDecoded } from '../../../../../common/detection_engine/schemas/request/set_signal_status_schema';
import { getCreateRulesSchemaMock } from '../../../../../common/detection_engine/schemas/request/create_rules_schema.mock';
import { getListArrayMock } from '../../../../../common/detection_engine/schemas/types/lists.mock';
import { getEqlValidationSchemaMock } from '../../../../../common/detection_engine/schemas/request/eql_validation_schema.mock';
export const typicalSetStatusSignalByIdsPayload = (): SetSignalsStatusSchemaDecoded => ({
signal_ids: ['somefakeid1', 'somefakeid2'],
@ -145,6 +147,13 @@ export const getPrepackagedRulesStatusRequest = () =>
path: `${DETECTION_ENGINE_PREPACKAGED_URL}/_status`,
});
export const eqlValidationRequest = () =>
requestMock.create({
method: 'post',
path: DETECTION_ENGINE_EQL_VALIDATION_URL,
body: getEqlValidationSchemaMock(),
});
export interface FindHit<T = RuleAlertType> {
page: number;
perPage: number;
@ -577,6 +586,22 @@ export const getEmptySignalsResponse = (): SignalSearchResponse => ({
},
});
export const getEmptyEqlSearchResponse = (): EqlSearchResponse<unknown> => ({
hits: { total: { value: 0, relation: 'eq' }, events: [] },
is_partial: false,
is_running: false,
took: 1,
timed_out: false,
});
export const getEmptyEqlSequencesResponse = (): EqlSearchResponse<unknown> => ({
hits: { total: { value: 0, relation: 'eq' }, sequences: [] },
is_partial: false,
is_running: false,
took: 1,
timed_out: false,
});
export const getSuccessfulSignalUpdateResponse = () => ({
took: 18,
timed_out: false,

69
x-pack/plugins/security_solution/server/lib/detection_engine/routes/eql/helpers.mock.ts Normal file
View file

@ -0,0 +1,69 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { ApiResponse } from '@elastic/elasticsearch';
import { ErrorResponse } from './helpers';
export const getValidEqlResponse = (): ApiResponse['body'] => ({
is_partial: false,
is_running: false,
took: 162,
timed_out: false,
hits: {
total: {
value: 1,
relation: 'eq',
},
sequences: [],
},
});
export const getEqlResponseWithValidationError = (): ErrorResponse => ({
error: {
root_cause: [
{
type: 'verification_exception',
reason:
'Found 2 problems\nline 1:1: Unknown column [event.category]\nline 1:13: Unknown column [event.name]',
},
],
type: 'verification_exception',
reason:
'Found 2 problems\nline 1:1: Unknown column [event.category]\nline 1:13: Unknown column [event.name]',
},
});
export const getEqlResponseWithValidationErrors = (): ErrorResponse => ({
error: {
root_cause: [
{
type: 'verification_exception',
reason:
'Found 2 problems\nline 1:1: Unknown column [event.category]\nline 1:13: Unknown column [event.name]',
},
{
type: 'parsing_exception',
reason: "line 1:4: mismatched input '<EOF>' expecting 'where'",
},
],
type: 'verification_exception',
reason:
'Found 2 problems\nline 1:1: Unknown column [event.category]\nline 1:13: Unknown column [event.name]',
},
});
export const getEqlResponseWithNonValidationError = (): ApiResponse['body'] => ({
error: {
root_cause: [
{
type: 'other_error',
reason: 'some other reason',
},
],
type: 'other_error',
reason: 'some other reason',
},
});

58
x-pack/plugins/security_solution/server/lib/detection_engine/routes/eql/helpers.test.ts Normal file
View file

@ -0,0 +1,58 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { getValidationErrors, isErrorResponse, isValidationErrorResponse } from './helpers';
import {
getEqlResponseWithNonValidationError,
getEqlResponseWithValidationError,
getEqlResponseWithValidationErrors,
getValidEqlResponse,
} from './helpers.mock';
describe('eql validation helpers', () => {
describe('isErrorResponse', () => {
it('is false for a regular response', () => {
expect(isErrorResponse(getValidEqlResponse())).toEqual(false);
});
it('is true for a response with non-validation errors', () => {
expect(isErrorResponse(getEqlResponseWithNonValidationError())).toEqual(true);
});
it('is true for a response with validation errors', () => {
expect(isErrorResponse(getEqlResponseWithValidationError())).toEqual(true);
});
});
describe('isValidationErrorResponse', () => {
it('is false for a regular response', () => {
expect(isValidationErrorResponse(getValidEqlResponse())).toEqual(false);
});
it('is false for a response with non-validation errors', () => {
expect(isValidationErrorResponse(getEqlResponseWithNonValidationError())).toEqual(false);
});
it('is true for a response with validation errors', () => {
expect(isValidationErrorResponse(getEqlResponseWithValidationError())).toEqual(true);
});
});
describe('getValidationErrors', () => {
it('returns a single error for a single root cause', () => {
expect(getValidationErrors(getEqlResponseWithValidationError())).toEqual([
'Found 2 problems\nline 1:1: Unknown column [event.category]\nline 1:13: Unknown column [event.name]',
]);
});
it('returns multiple errors for multiple root causes', () => {
expect(getValidationErrors(getEqlResponseWithValidationErrors())).toEqual([
'Found 2 problems\nline 1:1: Unknown column [event.category]\nline 1:13: Unknown column [event.name]',
"line 1:4: mismatched input '<EOF>' expecting 'where'",
]);
});
});
});

35
x-pack/plugins/security_solution/server/lib/detection_engine/routes/eql/helpers.ts Normal file
View file

@ -0,0 +1,35 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import get from 'lodash/get';
import has from 'lodash/has';
const PARSING_ERROR_TYPE = 'parsing_exception';
const VERIFICATION_ERROR_TYPE = 'verification_exception';
const MAPPING_ERROR_TYPE = 'mapping_exception';
interface ErrorCause {
type: string;
reason: string;
}
export interface ErrorResponse {
error: ErrorCause & { root_cause: ErrorCause[] };
}
const isValidationErrorType = (type: unknown): boolean =>
type === PARSING_ERROR_TYPE || type === VERIFICATION_ERROR_TYPE || type === MAPPING_ERROR_TYPE;
export const isErrorResponse = (response: unknown): response is ErrorResponse =>
has(response, 'error.type');
export const isValidationErrorResponse = (response: unknown): response is ErrorResponse =>
isErrorResponse(response) && isValidationErrorType(get(response, 'error.type'));
export const getValidationErrors = (response: ErrorResponse): string[] =>
response.error.root_cause
.filter((cause) => isValidationErrorType(cause.type))
.map((cause) => cause.reason);

46
x-pack/plugins/security_solution/server/lib/detection_engine/routes/eql/validate_eql.ts Normal file
View file

@ -0,0 +1,46 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { ElasticsearchClient } from '../../../../../../../../src/core/server';
import { getValidationErrors, isErrorResponse, isValidationErrorResponse } from './helpers';
export interface Validation {
isValid: boolean;
errors: string[];
}
export interface ValidateEqlParams {
client: ElasticsearchClient;
index: string[];
query: string;
}
export const validateEql = async ({
client,
index,
query,
}: ValidateEqlParams): Promise<Validation> => {
const response = await client.eql.search(
{
// @ts-expect-error type is missing allow_no_indices
allow_no_indices: true,
index: index.join(','),
body: {
query,
size: 0,
},
},
{ ignore: [400] }
);
if (isValidationErrorResponse(response.body)) {
return { isValid: false, errors: getValidationErrors(response.body) };
} else if (isErrorResponse(response.body)) {
throw new Error(JSON.stringify(response.body));
} else {
return { isValid: true, errors: [] };
}
};

53
x-pack/plugins/security_solution/server/lib/detection_engine/routes/eql/validation_route.test.ts Normal file
View file

@ -0,0 +1,53 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { ApiResponse, TransportRequestPromise } from '@elastic/elasticsearch/lib/Transport';
import { serverMock, requestContextMock } from '../__mocks__';
import { eqlValidationRequest, getEmptyEqlSearchResponse } from '../__mocks__/request_responses';
import { eqlValidationRoute } from './validation_route';
describe('validate_eql route', () => {
let server: ReturnType<typeof serverMock.create>;
let { clients, context } = requestContextMock.createTools();
beforeEach(() => {
server = serverMock.create();
({ clients, context } = requestContextMock.createTools());
clients.newClusterClient.asCurrentUser.eql.search.mockResolvedValue(
(getEmptyEqlSearchResponse() as unknown) as TransportRequestPromise<
ApiResponse<unknown, unknown>
>
);
eqlValidationRoute(server.router);
});
describe('normal status codes', () => {
test('returns 200 when doing a normal request', async () => {
const response = await server.inject(eqlValidationRequest(), context);
expect(response.status).toEqual(200);
});
test('returns the payload when doing a normal request', async () => {
const response = await server.inject(eqlValidationRequest(), context);
const expectedBody = {
valid: true,
errors: [],
};
expect(response.status).toEqual(200);
expect(response.body).toEqual(expectedBody);
});
test('returns 500 when bad response from cluster', async () => {
clients.newClusterClient.asCurrentUser.eql.search.mockImplementation(() => {
throw new Error('Test error');
});
const response = await server.inject(eqlValidationRequest(), context);
expect(response.status).toEqual(500);
expect(response.body).toEqual({ message: 'Test error', status_code: 500 });
});
});
});

49
x-pack/plugins/security_solution/server/lib/detection_engine/routes/eql/validation_route.ts Normal file
View file

@ -0,0 +1,49 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { IRouter } from '../../../../../../../../src/core/server';
import { eqlValidationSchema } from '../../../../../common/detection_engine/schemas/request/eql_validation_schema';
import { DETECTION_ENGINE_EQL_VALIDATION_URL } from '../../../../../common/constants';
import { buildRouteValidation } from '../../../../utils/build_validation/route_validation';
import { transformError, buildSiemResponse } from '../utils';
import { validateEql } from './validate_eql';
export const eqlValidationRoute = (router: IRouter) => {
router.post(
{
path: DETECTION_ENGINE_EQL_VALIDATION_URL,
validate: {
body: buildRouteValidation(eqlValidationSchema),
},
options: {
tags: ['access:securitySolution'],
},
},
async (context, request, response) => {
const siemResponse = buildSiemResponse(response);
const { query, index } = request.body;
const esClient = context.core.elasticsearch.client.asCurrentUser;
try {
const validation = await validateEql({
client: esClient,
query,
index,
});
return response.ok({
body: { valid: validation.isValid, errors: validation.errors },
});
} catch (err) {
const error = transformError(err);
return siemResponse.error({
body: error.message,
statusCode: error.statusCode,
});
}
}
);
};

4
x-pack/plugins/security_solution/server/routes/index.ts
View file

@ -34,6 +34,7 @@ import { createTimelinesRoute } from '../lib/timeline/routes/create_timelines_ro
import { updateTimelinesRoute } from '../lib/timeline/routes/update_timelines_route';
import { getDraftTimelinesRoute } from '../lib/timeline/routes/get_draft_timelines_route';
import { cleanDraftTimelinesRoute } from '../lib/timeline/routes/clean_draft_timelines_route';
import { eqlValidationRoute } from '../lib/detection_engine/routes/eql/validation_route';
import { SetupPlugins } from '../plugin';
import { ConfigType } from '../config';
import { installPrepackedTimelinesRoute } from '../lib/timeline/routes/install_prepacked_timelines_route';
@ -94,4 +95,7 @@ export const initRoutes = (
// Privileges API to get the generic user privileges
readPrivilegesRoute(router, security, usingEphemeralEncryptionKey);
// Route used by the UI for form validation
eqlValidationRoute(router);
};