Commit graph

28848 commits

Author SHA1 Message Date
Frank Hassanabad
a4f37cd9e0
[SIEM] [Detection Engine] Adds filtering abilities to the KQL REST API (#49451)
## Summary

* Removes the older beginner KQL type of signal creation in favor of newer version with filtering
* Adds ability to create KQL or lucene queries that will work with the UI filters
* UI state with the filters are now savable to re-hydrate UI's on the front end
* Adds `saved_id` ability so the UI can tether dynamic saved queries with signals
* Changed `it` to `test` as `it` is not the alias we use for tests 
* Updated script which converts older saved searches to work with newer mechanism
* Fixed script to accept proper ndjson lines
* Adds validation unit tests for the endpoint
* Increases validation strictness of the endpoints
* Adds more data scripts for testing scenarios
* https://github.com/elastic/kibana/issues/47013


## Testing
* Run `./hard_reset.sh` script 
* Test with both algorithms through this toggle before starting kibana:
`export USE_REINDEX_API=true`
* Convert older saved searches to compatible new query filters by running:
`./convert_saved_search_to_signals.sh ~/projects/saved_searches /tmp/signals`
* Post them`./post_signal.sh /tmp/signals/*.json`
* Hard reset again
* Test smaller set of signals and REST endpoints using the typical scripts of:
```sh
./post_signal.sh
./read_signal.sh
./find_signals.sh
./update_signal.sh
./delete_signal.sh
```
or test using POSTMAN, etc... If you want to test validation. If you see any validation issues let me know as I have validation testing files and can easily fix them add another unit test to the growing large collection we have now. 

Change in your advanced settings of SIEM to use your signals index you configured for verification that the signals show up.

### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~

~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~

~~- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~

### For maintainers

~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~

~~- [ ] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~
2019-10-28 17:27:39 -06:00
Spencer
69f47a9b90
remove code-plugin specific build task and related node module (#49525) 2019-10-28 15:40:11 -07:00
Mikhail Shustov
b6591eb0bd
Provide uiSettings service in NP (#48413)
* provide ui settins client via context

* update mocks

* update types and expose setDefaults to plugins

* move ui settings routes to NP

* add typings fro test kbn server

* move integration test & improve typings

* hide client private methods, update tests

* add unit tests for get_upgradable_config

* inline writeErrors into createOrUpgradeConfig to simplify testing

* regen docs

* add functional tests for ui_settings service

* unify test suites

* add types for sipertest in core_plugin tests

* tsify core_plugins tests

* add test for empty saved config

* update renovate

* rename get/setDefaults to reguster

* regen docs

* regen docs

* Update src/core/MIGRATION.md

Co-Authored-By: Josh Dover <me@joshdover.com>
2019-10-28 16:06:12 -04:00
renovate[bot]
5b375df8e9 Update moment related packages (#46849)
* Update moment related packages

* unify moment version in yarn.lock

* adapt ro pluralization fixes

* update some bad pluralization in fixtures/snapshots
2019-10-28 12:07:35 -07:00
Chris Davies
6d23793489
[Lens] Pass date range to the query bar (#49485) 2019-10-28 15:04:03 -04:00
Ben Skelker
e3b6da981b
Fixes title capitalisations (#49445) 2019-10-28 20:26:15 +02:00
Josh Dover
23a9058cc4
Migrate ui/registry/feature_catalogue to New Platform plugin (#48818) 2019-10-28 13:08:30 -05:00
Josh Dover
2d1c239fdd
Migrate necessary ui/autoload functionality to NP (#48689) 2019-10-28 12:58:29 -05:00
Aleh Zasypkin
57efead954
Correctly process paths in map_of and record_of. Do not swallow and use indent for nested one_of error messages. (#49457) 2019-10-28 19:39:48 +02:00
Matthias Wilhelm
df2a99631b Fix missing html formatting in Doc_Viewer (#49326)
* Add detection if a value has been formatted, conditional rendering

* Use markup by formatters, it's escaped for dangerouslySetInnerHTML

* Enable dangerouslySetInnerHTML for displaying values

* Use regex for replace
2019-10-28 18:31:20 +01:00
Steph Milovic
bd265d72b0
[SIEM] optimize hosts query (#49409) 2019-10-28 12:23:33 -05:00
Tim Roes
15831b289a
Improve "Create new" UI in dashboard (#49189)
* Change create new UI

* Implement review feedback

* Fix types in tests

* Remove obsolete translations

* Remove unnecessary flexbox
2019-10-28 18:12:37 +01:00
patrykkopycinski
93269e58e5
[SIEM] Add Authentications histogram (#48260) 2019-10-28 17:43:14 +01:00
Christiane (Tina) Heiligers
c2672aeaa8
Bug fix:Adds a migration for transforming TSVB vis split_filters (#49000)
* Bug fix:Adds a migration for transforming TSVB visualization split_filters from strings to query:language objects to 7.5

* Changes hard coded migration version to get the version number from the saved object's migrationVersion in failing functional test

* Changes migration number from '7.5.0' to '7.4.2', fixes typo, changes test expectations to more explicit ones
2019-10-28 09:42:14 -07:00
Court Ewing
0b82cfeae7
Remove experimental code app (#49404)
* Fix erroneous code plugin import in apm

* Stop running legacy code plugin

* Stop testing code app and apis

* Remove code plugin source and deprecate config

* Remove code plugin docs

* Remove xpack.code translations

* Remove code import on api_integration

* Remove code privilege from security test

* remove two more mentions of code

* remove code es_archives

* remove code doc images
2019-10-28 11:40:27 -04:00
Dmitry Lemeshko
d9a5acf971
FTR: clear browser storage between test suites (#48683)
* clear browser storage on test suite completion

* [logstash/pipeline_list] fix tests dependency

* always load new index data
2019-10-28 09:37:34 -04:00
patrykkopycinski
d01099778b
[SIEM] Refactor hosts routing (#47459) 2019-10-28 14:16:49 +01:00
Jean-Louis Leysens
49804a2645
[Console] Restore completer behaviour (#49422)
* Restore completer behaviour

* Move ace logic into shim and update getCursor -> getCursorPosition
2019-10-28 13:54:47 +01:00
Maryia Lapata
fb903f4f9f
[Vis: Default editor] Unit tests for number list (#48412)
* Add unit tests for number list

* Remove unused dependency

* Fix code review comments

* Refactor NumberList to set model validity in one place
2019-10-28 15:50:39 +03:00
Maryia Lapata
17464f8612
[Vis: Default editor] Fix validation for bounds margin (#48216)
* Fix validation for bounds margin

* Update custom_extents_options.tsx

* Rename isValid property

* Revert boundsMargin type
2019-10-28 15:23:18 +03:00
Shahzad
d9d8398fb1
[Uptime] Issue/48845 Disable filter and Add link to location docs when no location exists (#49175)
* disable search text if no item exists

* update snaps
2019-10-28 16:52:33 +05:00
Jean-Louis Leysens
f868fa66fa
[Console][Legacy Editor] Ace Range implementation not being used (#49352)
* Use ace Range implementation, not interface, for instantiating

* Add simple smoke test for use of Ace ranges

* Update test title
2019-10-28 11:47:59 +01:00
Marjorie Jones
cbe6fd3e3d
[DOCS] Changes to support the split of the Metrics Guide and the Logs Guide (#48633)
* Linking to new Metrics and Logs documents

* Linking to new Metrics and Logs documents
2019-10-28 09:39:53 +00:00
Liza Katz
6bb30e7190
Allow plugins to register top nav menu items (regression fix) (#48542)
* Move top nav menu to its own plugin
Allow registering additional options from other plugins
Added demo plugin

* Added functional test to validate top nav registration

* Improved names

* Rename array

* Fixed lens tests

* Deleted old NavBarExtensionsRegistryProvider

* Fixed top nav menu test

* Attempt fixing test by clearing ui_actions on stop

* temporary disable test
2019-10-28 10:40:21 +02:00
patrykkopycinski
795d1caa5a
[SIEM] Fix host details page redirect (#48720) 2019-10-27 06:23:51 -07:00
Dmitry Lemeshko
f9b0a46dfd
[kbn-expect] add optional error message (#48895)
* [kbn-expect] add optional error message

* review feedback: replace error message with provided one

* add optional message for contain/string
2019-10-26 15:51:45 +02:00
Catherine Liu
9482ba5765
Fixed typo in formatnumber example (#49379) 2019-10-26 06:25:09 -07:00
Nick Peihl
3e89cb54de
Remove mapping type from example url (#49425) 2019-10-25 17:35:16 -07:00
Mikhail Shustov
11d3445b5d
watch ignores resolved to plugins internals (#48629) 2019-10-26 02:06:50 +02:00
Justin Kambic
e7d82f29d9
[Uptime] Do not drop edge buckets (#48157)
* Checkpoint first uptime bucket

* Add code to specify interval for histogram chart, delete obsolete formatting code.

* Revert file to master version.

* Update usages of modified helper function.

* Delete obslete snapshot.

* Modify outdated tests and refresh snapshots.

* Update API test fixtures and several tests to avoid flakiness.

* Remove unneeded size field.

* Rename a type.

* Split concerns to two separate functions.

* Update chart formatting label stops and casing to fit latest charts.

* Remove addition of interval to x domain.

* Update tests to accommodate new changes.
2019-10-25 18:42:31 -04:00
renovate[bot]
dba5481181 Update dependency @elastic/charts to ^13.5.9 (#49390) 2019-10-25 17:22:34 -05:00
Michael Marcialis
7003c2c6d6
[SIEM] Map Docs Link & Intrinsic Ratios (#49267)
* rough out markup and intrinsic ratio poc

* reorganize comps

* media queries for aspect ratios

* disable ratio when error; add translations

* unit tests, translations & cleanup

* update copy per ben’s suggestions

* snapshots and translations

* move paddingSize prop inline

* change panel selector

* fix FormattedMessage id

* update snapshots
2019-10-25 17:43:10 -04:00
Wylie Conlon
ed07fb4c06
[Expressions] [Lens] Implement a loading state and error state in the ExpressionRenderer (#48841)
* Add loading indicator to Lens workspace panel

* [Expressions] [Lens] Handle loading and errors in ExpressionRenderer

* Using loading$ observable and improve tests

* Using CSS and to handle layout of expression renderer

Added TODO for using chart loader when area is completely empty

* Improve error handling and simplify code

* Fix cleanup behavior

* Fix double render and prevent error cases in xy chart

* Fix context for use in dashboards

* Remove className from expression rendere component

* Improve handling of additional interpreter args

* More layout fixes

- Hide chart if Empty not Loading
- Fix relative positioning for progress bar since className is no longer passed (super hacky)
2019-10-25 16:32:59 -04:00
gchaps
9bd8f74ff8
[DOCS] Updates Snapshot docs to include retention (#49007)
* [DOCS] Updates Snapshot docs to include retention

* [DOCS] Creates tutorial for snapshot & restore

* [DOCS] Incorporates review comments for Snapshot doc

* [DOCS] Fixes typo

* [DOCS] Adds x-pack label
2019-10-25 13:27:55 -07:00
Tim Sullivan
e0cf748a1d
[Reporting] Fix scroll timeout logging bug (#49111)
* [Reporting] Fix scroll timeout logging bug

* test cancellation token

* test time out
2019-10-25 12:46:09 -07:00
gchaps
243a9815e3
[DOCS] Adds docs for API Keys UI (#49135)
* [DOCS] Adds docs for API Keys UI

* [DOCS] Incorporates review comments into API keys doc

* [DOCS] Fixes typo
2019-10-25 11:59:01 -07:00
Steph Milovic
4e8878d5da
[SIEM] N-flow ecs bug fix (#49266) 2019-10-25 13:55:30 -05:00
James Gowdy
467dfcc577
[ML] Fixing advanced detector field selects enablement (#49348) 2019-10-25 18:37:34 +01:00
Tim Schnell
a4d2638d48
[skip ci] Function Reference Docs Update (#48791)
* setting up function reference pattern

* adding functions A-F

* grammar fixes

* Apply suggestions from code review

Co-Authored-By: Catherine Liu <catherineqliu@outlook.com>

* more review updates
2019-10-25 12:27:11 -05:00
Brian Seeders
9da6c07059
Re-add pipeline for flaky test runner job (#48781)
* Revert "Revert "Add pipeline for flaky test runner job (#46740)""

This reverts commit 7d96a13fad.

Also reconcile changes to Jenkinsfile since original revert happened

* Fix param parsing and add missed change

* Add missing variable
2019-10-25 12:57:46 -04:00
Wylie Conlon
58dbb130dc
[task manager] Kibana should start without task manager (#48568)
* [task manager] Kibana should not fail to start without task manager

* Check for task manager in maps

* Lower log level

* Update task registration
2019-10-25 12:42:31 -04:00
renovate[bot]
c7a1c895b4 Update dependency @elastic/charts to ^13.5.8 (#49327) 2019-10-25 09:41:21 -07:00
Tyler Smalley
373320a3a0
[npm] Removes react-clipboard.js and redux-test-utils (#49283)
* [npm] Removes react-clipboard.js

Added in the original commit of APM, but does not appear to still be used.

* [npm] Remove redux-test-utils

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-10-25 09:38:13 -07:00
Tyler Smalley
b24e44c5b8
Removes unused babel-plugin-inline-react-svg package (#49271)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-10-25 09:23:04 -07:00
Tyler Smalley
fcc9ea369a
[npm] Removes unused dependencies (#49287)
* [npm] Remove mochawesome-report-generator
* [npm] Remove babel-plugin-transform-react-remove-prop-types
* [npm] Removing popper.js
* [npm] Remove stream-stream
* [npm] Remove react-redux-request

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-10-25 09:22:14 -07:00
Wylie Conlon
91a5664b4f
[Lens] Text updates (#48964)
* [Lens] Text updates

* Update per comments

* Update text for datapanel

* Fix tests
2019-10-25 12:09:41 -04:00
Joe Reuter
8499cfd0f9
ensure loading order (#49316) 2019-10-25 17:53:45 +02:00
Brandon Kobel
9acc18095b
Fixing kibana.yml doc comments for elasticsearch.ssl.certificate/key (#49262) 2019-10-25 08:23:09 -07:00
James Gowdy
909d28fe38
[ML] Fixing edit buttons in advanced wizard summary (#49324) 2019-10-25 15:59:31 +01:00
Tyler Smalley
7ecf5ac935
[npm] Removes stream-to-observable (#49272)
Added in https://github.com/elastic/kibana/pull/18885 as part of the
RxJS 6 upgrade and used in the screenshot stictcher, however it is not
longer used.

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-10-25 07:40:27 -07:00