Go to file
Frank Hassanabad a4f37cd9e0
[SIEM] [Detection Engine] Adds filtering abilities to the KQL REST API (#49451)
## Summary

* Removes the older beginner KQL type of signal creation in favor of newer version with filtering
* Adds ability to create KQL or lucene queries that will work with the UI filters
* UI state with the filters are now savable to re-hydrate UI's on the front end
* Adds `saved_id` ability so the UI can tether dynamic saved queries with signals
* Changed `it` to `test` as `it` is not the alias we use for tests 
* Updated script which converts older saved searches to work with newer mechanism
* Fixed script to accept proper ndjson lines
* Adds validation unit tests for the endpoint
* Increases validation strictness of the endpoints
* Adds more data scripts for testing scenarios
* https://github.com/elastic/kibana/issues/47013


## Testing
* Run `./hard_reset.sh` script 
* Test with both algorithms through this toggle before starting kibana:
`export USE_REINDEX_API=true`
* Convert older saved searches to compatible new query filters by running:
`./convert_saved_search_to_signals.sh ~/projects/saved_searches /tmp/signals`
* Post them`./post_signal.sh /tmp/signals/*.json`
* Hard reset again
* Test smaller set of signals and REST endpoints using the typical scripts of:
```sh
./post_signal.sh
./read_signal.sh
./find_signals.sh
./update_signal.sh
./delete_signal.sh
```
or test using POSTMAN, etc... If you want to test validation. If you see any validation issues let me know as I have validation testing files and can easily fix them add another unit test to the growing large collection we have now. 

Change in your advanced settings of SIEM to use your signals index you configured for verification that the signals show up.

### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~

~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~

~~- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~

### For maintainers

~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~

~~- [ ] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~
2019-10-28 17:27:39 -06:00
.ci Re-add pipeline for flaky test runner job (#48781) 2019-10-25 12:57:46 -04:00
.github Allow plugins to register top nav menu items (regression fix) (#48542) 2019-10-28 10:40:21 +02:00
bin Update node options Initialization on our scripts (#40302) 2019-08-15 16:09:22 +01:00
common/graphql
config Fixing kibana.yml doc comments for elasticsearch.ssl.certificate/key (#49262) 2019-10-25 08:23:09 -07:00
data
docs Provide uiSettings service in NP (#48413) 2019-10-28 16:06:12 -04:00
licenses
packages Update moment related packages (#46849) 2019-10-28 12:07:35 -07:00
rfcs [RFC] Kibana Management Section Service (#43631) 2019-10-03 10:27:00 -05:00
scripts refactor failed_tests_reporter to use TS, no octokit (#46993) 2019-10-08 13:56:07 -07:00
src remove code-plugin specific build task and related node module (#49525) 2019-10-28 15:40:11 -07:00
style_guides Style guide cleanup (#46464) 2019-09-25 11:19:16 +02:00
tasks Remove rimraf (#48985) 2019-10-24 06:29:43 -07:00
test Provide uiSettings service in NP (#48413) 2019-10-28 16:06:12 -04:00
typings [Telemetry] Move to OSS (#45769) 2019-10-16 10:24:20 +03:00
utilities [mkdirp] remove in favor of recursive fs.mkdir (#47251) 2019-10-04 08:15:06 -07:00
vars Re-add pipeline for flaky test runner job (#48781) 2019-10-25 12:57:46 -04:00
webpackShims Add lru-cache shim (#49014) 2019-10-23 12:56:00 -05:00
x-pack [SIEM] [Detection Engine] Adds filtering abilities to the KQL REST API (#49451) 2019-10-28 17:27:39 -06:00
.backportrc.json Add 7.5 to .backportrc.json 2019-10-16 21:20:14 +02:00
.browserslistrc [autoprefixer] upgrade to 9.6.1, unify browserslist handling (#41038) 2019-07-15 09:33:33 -07:00
.editorconfig [editorconfig] disable insert_final_newline for package.json 2019-04-18 09:44:17 -07:00
.eslintignore Update eslint related packages (#48513) 2019-10-21 16:32:31 -07:00
.eslintrc.js Update eslint related packages (#48513) 2019-10-21 16:32:31 -07:00
.gitattributes [canvas] Color fixes + Storybook 5 (#34075) 2019-04-02 11:21:51 -05:00
.gitignore gitignore legacy optimize directory (#44314) 2019-08-28 15:01:24 -04:00
.i18nrc.json Allow plugins to register top nav menu items (regression fix) (#48542) 2019-10-28 10:40:21 +02:00
.node-version Bump node to 10.15.2 (#32200) 2019-02-28 10:48:42 -06:00
.nvmrc Bump node to 10.15.2 (#32200) 2019-02-28 10:48:42 -06:00
.prettierrc
.sass-lint.yml [feature][Canvas] Share Workpads in other Websites (#46278) 2019-10-11 12:15:41 -05:00
.yarnrc
CONTRIBUTING.md Adding "setting up prettier" in CONTRIBUTING.md (#46310) 2019-10-14 11:02:06 -07:00
FAQ.md
github_checks_reporter.json implementing github checks - second attempt (#35757) 2019-05-01 16:02:33 -05:00
Gruntfile.js
Jenkinsfile Re-add pipeline for flaky test runner job (#48781) 2019-10-25 12:57:46 -04:00
kibana.d.ts Update eslint related packages (#48513) 2019-10-21 16:32:31 -07:00
LICENSE.txt
NOTICE.txt standardize notice-comment usage (#48677) 2019-10-21 11:54:45 -07:00
package.json Provide uiSettings service in NP (#48413) 2019-10-28 16:06:12 -04:00
preinstall_check.js
README.md Update paths to ui, server, deprecation, plugin_discovery in src/legacy 2019-02-11 10:41:37 -05:00
renovate.json5 Provide uiSettings service in NP (#48413) 2019-10-28 16:06:12 -04:00
STYLEGUIDE.md Add TypeScript rules to STYLEGUIDE [skip ci] (#47125) 2019-10-03 10:15:31 +02:00
tsconfig.browser.json
tsconfig.json Es ui shared updates (#46570) 2019-09-26 17:06:33 +02:00
tsconfig.types.json [Core] Move Saved objects files to core (#38771) 2019-06-18 13:10:23 +02:00
TYPESCRIPT.md Fix path references into and out of x-pack/legacy 2019-06-20 13:34:48 -04:00
yarn.lock remove code-plugin specific build task and related node module (#49525) 2019-10-28 15:40:11 -07:00

Kibana

Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.

Getting Started

If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.

If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.

Using a Kibana Release

If you want to use a Kibana release in production, give it a test run, or just play around:

Building and Running Kibana, and/or Contributing Code

You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:

Documentation

Visit Elastic.co for the full Kibana documentation.

For information about building the documentation, see the README in elastic/docs.

Version Compatibility with Elasticsearch

Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.

Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.

Situation Example Kibana version Example ES version Outcome
Versions are the same. 5.1.2 5.1.2 💚 OK
ES patch number is newer. 5.1.2 5.1.5 ⚠️ Logged warning
ES minor number is newer. 5.1.2 5.5.0 ⚠️ Logged warning
ES major number is newer. 5.1.2 6.0.0 🚫 Fatal error
ES patch number is older. 5.1.2 5.1.0 ⚠️ Logged warning
ES minor number is older. 5.1.2 5.0.0 🚫 Fatal error
ES major number is older. 5.1.2 4.0.0 🚫 Fatal error

Questions? Problems? Suggestions?

  • If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
  • Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.