Go to file
Ryland Herrick 7cfdeaeede
[Security Solution][Detections] EQL Validation (#77493)
* WIP: Adding new route for EQL Validation

This is mostly boilerplate with some rough parameter definitions; the
actual implementation of the validation is going to live in our
validateEql function.

A few tests are failing as the mocks haven't yet been implemented, I
need to see the shape of the responses first.

* Cherry-pick Marshall's EQL types

* Implements actual EQL validation

* Performs an EQL search
* filters out non-parsing errors, and returns what remains in the
  response
* Adds mocks for empty EQL responses (we don't yet have a need for
  mocked data, but we will when we unit-test validateEql)

* Adds validation calls to the EQL form input

* Adds EQL Validation response schema,mocks,tests
* Adds frontend function to call our validation endpoint
* Adds hook, useEqlValidation, to call the above function and return
  state
* Adds labels/help text for EQL Query bar
* EqlQueryBar consumes useEqlValidation and marks the field as invalid,
  but does not yet report errors.

* Do not call the validation API if query is not present

This causes a broader error that results in a 400 response; we can (and
do) handle the case of a blank query in the form itself.

* Remove EQL Help Text

It doesn't add any information for the user, and it currently looks bad
when combined with validation errors.

* Flesh out and use our popover for displaying validation errors

* Fixes issue where old errors were persisted after the user had made
  modifications

* Include verification_exception errors as validation errors

These include errors related to index fields and mappings.

* Generalize our validation helpers

We're concerned with validation errors; the source of those errors is an
implementation detail of these functions.

* Move error popover and EQL reference link to footer

This more closely resembles the new Eui Markdown editor, which places
errors and doc links in a footer.

* Fix jest tests following additional prop

* Add icon for EQL Rule card

* Fixes existing EqlQueryBar tests

These were broken by our use of useAppToasts and the EUI theme.

* Add unit tests around error rendering on EQL Query Bar

* Add tests for ErrorPopover

* Remove unused schema type

Decode doesn't do any additional processing, so we can use t.TypeOf here
(the default for buildRouteValidation).

* Remove duplicated header

* Use ignore parameter to prevent EQL validations from logging errors

Without `ignore: [400]` the ES client will log errors and then throw
them. We can catch the error, but the log is undesirable.

This updates the query to use the ignore parameter, along with updating
the validation logic to work with the updated response.

Adds some mocks and tests around these responses and helpers, since
these will exist independent of the validation implementation.

* Include mapping_exceptions during EQL query validation

These include errors for inaccessible indexes, which should be useful to
the rule writer in writing their EQL query.

* Display toast messages for non-validation messages

* fix type errors

This type was renamed.

* Do not request data in our validation request

By not having the cluster retrieve/send any data, this should saves us
a few CPU cycles.

* Move EQL validation to an async form validator

Rather than invoking a custom validation hook (useEqlValidation) at custom times (onBlur) in our EqlQueryBar
component, we can instead move this functionality to a form validation
function and have it be invoked automatically by our form when values
change. However, because we still need to handle the validation messages
slightly differently (place them in a popover as opposed to an
EuiFormRow), we also need custom error retrieval in the form of
getValidationResults.

After much pain, it was determined that the default behavior of
_.debounce does not work with async validator functions, as a debounced
call will not "wait" for the eventual invocation but will instead return
the most recently resolved value. This leads to stale validation
results and terrible UX, so I wrote a custom function (debounceAsync)
that behaves like we want/need; see tests for details.

* Invalidate our query field when index patterns change

Since EQL rules actually validate against the relevant indexes, changing
said indexes should invalidate/revalidate the query.

With the form lib, this is beautifully simple :)

* Set a min-height on our EQL textarea

* Remove unused prop from EqlQueryBar

Index corresponds to the value from the index field; now that our EQL
validation is performed by the form we have no need for it here.

* Update EQL overview link to point to elasticsearch docs

Adds an entry in our doclinks service, and uses that.

* Remove unused prop from stale tests

* Update docLinks documentation with new EQL link

* Fix bug where saved query rules had no type selected on Edit

* Wait for kibana requests to complete before moving between rule tabs

With our new async validation, a user can quickly navigate away from the
Definition tab before the validation has completed, resulting in the
form being invalidated. Any subsequent user actions cause the form to
correct itself, but until I can find a better solution here this really
just gives the validation time to complete and sidesteps the issue.
2020-10-02 13:19:02 -05:00
.ci [QA][Code Coverage] Upload the coverage static site before ingestion (#78695) 2020-09-29 08:29:22 -06:00
.github Move legacy plugins to appropriate teams (#79078) 2020-10-02 13:39:38 +02:00
common/graphql
config Remove unused elasticsearch.preserverHost setting (#78608) 2020-09-30 10:39:08 +02:00
data
docs [Security Solution][Detections] EQL Validation (#77493) 2020-10-02 13:19:02 -05:00
examples Add TS projects for src/plugins & x-pack/plugins (#78440) 2020-09-30 15:02:41 +02:00
licenses Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
packages upgrade @elastic/charts to v23.0.0 (#79226) 2020-10-02 19:48:40 +02:00
plugins [dev/cli] ensure plugins/ and all watch source dirs exist (#78973) 2020-09-30 10:20:44 -07:00
rfcs RFC: Improve saved object migrations (#66056) 2020-09-29 09:34:54 +02:00
scripts [QA][Code Coverage] Coverage teams lookup w/o Additional Config (#77111) 2020-09-28 15:27:02 -06:00
src [Security Solution][Detections] EQL Validation (#77493) 2020-10-02 13:19:02 -05:00
tasks chore(NA): assures a single version for the same dependency across the entire project (#78825) 2020-10-02 00:46:00 +01:00
test Adjust extend_es_archiver to handle additional cases (#79308) 2020-10-02 20:14:42 +02:00
typings [ts] enable "resolveJsonModule" and disable existing failures (#78855) 2020-09-30 15:44:43 -07:00
utilities
vars chore(NA): assures a single version for the same dependency across the entire project (#78825) 2020-10-02 00:46:00 +01:00
x-pack [Security Solution][Detections] EQL Validation (#77493) 2020-10-02 13:19:02 -05:00
.backportrc.json
.browserslistrc
.editorconfig
.eslintignore
.eslintrc.js Storybook 6 and config changes (#75357) 2020-09-29 19:34:05 -05:00
.fossa.yml
.gitattributes
.gitignore
.i18nrc.json
.node-version Update Node.js to version 10.22.1 (#77619) 2020-09-16 18:51:50 +02:00
.nvmrc
.prettierrc
.sass-lint.yml
.telemetryrc.json [Usage Collection] [schema] ui_metric (#78827) 2020-10-01 08:22:51 +01:00
.yarnrc
api-documenter.json
CONTRIBUTING.md
FAQ.md
github_checks_reporter.json
Gruntfile.js
Jenkinsfile
kibana.d.ts Remove legacy plugins support (#77599) 2020-09-23 09:52:51 +02:00
LICENSE.txt
NOTICE.txt
package.json upgrade @elastic/charts to v23.0.0 (#79226) 2020-10-02 19:48:40 +02:00
preinstall_check.js
README.md
renovate.json5 [renovate] looks like team slug might just be kibana-app 2020-09-24 10:10:17 -07:00
STYLEGUIDE.md Remove legacy optimizer (#73154) 2020-08-13 09:08:44 -07:00
tsconfig.base.json [ts] enable "resolveJsonModule" and disable existing failures (#78855) 2020-09-30 15:44:43 -07:00
tsconfig.browser.json
tsconfig.json Add TS projects for src/plugins & x-pack/plugins (#78440) 2020-09-30 15:02:41 +02:00
tsconfig.refs.json Add TS projects for src/plugins & x-pack/plugins (#78440) 2020-09-30 15:02:41 +02:00
tsconfig.types.json ui_actions service initial docs (#78902) 2020-09-30 16:44:29 +02:00
TYPESCRIPT.md
yarn.lock upgrade @elastic/charts to v23.0.0 (#79226) 2020-10-02 19:48:40 +02:00

Kibana

Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.

Getting Started

If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.

If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.

Using a Kibana Release

If you want to use a Kibana release in production, give it a test run, or just play around:

Building and Running Kibana, and/or Contributing Code

You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:

Documentation

Visit Elastic.co for the full Kibana documentation.

For information about building the documentation, see the README in elastic/docs.

Version Compatibility with Elasticsearch

Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.

Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.

Situation Example Kibana version Example ES version Outcome
Versions are the same. 5.1.2 5.1.2 💚 OK
ES patch number is newer. 5.1.2 5.1.5 ⚠️ Logged warning
ES minor number is newer. 5.1.2 5.5.0 ⚠️ Logged warning
ES major number is newer. 5.1.2 6.0.0 🚫 Fatal error
ES patch number is older. 5.1.2 5.1.0 ⚠️ Logged warning
ES minor number is older. 5.1.2 5.0.0 🚫 Fatal error
ES major number is older. 5.1.2 4.0.0 🚫 Fatal error

Questions? Problems? Suggestions?

  • If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
  • Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.