Commit graph

9677 commits

Author SHA1 Message Date
Liza Katz f290c68696
Improve search typescript (#69333)
* [search] Refactor the way search strategies are registered/retrieved on the server

* Fix types and tests and update docs

* Fix failing test

* Fix build of example plugin

* Fix functional test

* Make server strategies sync

* Move strategy name into options

* docs

* Remove FE strategies

* TypeScript of hell
delete search explorer

* Fix search interceptor OSS tests

* typos

* test cleanup

* Delete search example
fix interceptor async tests to use fake timers

* docs

* fix

* return search wrapper

* Update search interceptor tests and abort utils

* ts

* jest test fix

* code review

* change how logs consume search API

Co-authored-by: Lukas Olson <olson.lukas@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-08 00:34:28 +03:00
Steph Milovic 3f5ebdda7a
[Security Solution] [Timeline] Timeline manager tweaks (#69988) 2020-07-07 15:27:00 -06:00
Paul Tavares e99974d8e4
[Endpoint] Support redirect from Policy Details to Ingest when user initiates Edit Policy from Datasource Edit page (#70874)
* allow Policy Details to support route state
* Functional Tests cases that cover ingest navigation
2020-07-07 16:55:40 -04:00
Søren Louv-Jansen cb6fbc0e1b
[APM] Add API tests (#70740) 2020-07-07 22:40:28 +02:00
Yara Tercero 37c2c925d3
[Security Solution][Exceptions] - Tie server and client code together (#70918)
## Summary

This PR tries to start to tie together the server and client changes for exceptions lists. 

- Updates graphql types to allow UI access to a rule's `exceptions_list` property
- Updates the exception viewer component to now dynamically take the rule `exceptions_list`, up until now we just had an empty array in it's place
- Updates the viewer logic to check if a rule has an endpoint list associated with it. If it does, then it displays both detections and endpoint UIs (in the viewer), if it does not, then it only displays the detections UI
- Updates the viewer UI to better deal with spacing when an exception list item only has one or two entries (before the and badge with the antennas was stretching passed the exception items to fill the space)
- Updates the detections engine exceptions logic to fetch list items using an exception list's `id` as opposed to it's `list_id`, this now aligns with the UI using the same params on its end
- Adds exception list `type` to information kept by the rule for exception lists
- Updates the exception list type from `string` to `endpoint | detection`
- Updates the exception list _item_ type from `string` to `simple`
- Adds unit tests for the detection engine server side util that fetches the exception list items
2020-07-07 15:49:43 -04:00
Mikhail Shustov aeff8c154b
[Audit Logging] Add AuditTrail service (#69278)
* add generic audit_trail service in core

* expose auditTraik service to plugins

* add auditTrail x-pack plugin

* fix type errors

* update mocks

* expose asScoped interface via start. auditor via  request context

* use type from audit trail service

* wrap getActiveSpace in safeCall only. it throws exception for non-authz

* pass message to log explicitly

* update docs

* create one auditor per request

* wire es client up to auditor

* update docs

* withScope accepts only one scope

* use scoped client in context for callAsInternalUser

* use auditor in scoped cluster client

* adopt auditTrail plugin to new interface. configure log from config

* do not log audit events in console by default

* add audit trail functional tests

* cleanup

* add example

* add mocks for spaces plugin

* add unit tests

* update docs

* test description

* Apply suggestions from code review

apply @jportner suggestions

Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>

* add unit tests

* more robust tests

* make spaces optional

* address comments

* update docs

* fix WebStorm refactoring

Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
2020-07-07 21:16:39 +02:00
Steph Milovic 06bc389189
[Security Solution] [Timeline] Bugfix for timeline row actions disappear sometimes (#70958) 2020-07-07 12:50:47 -06:00
Brian Seeders 465ed21194
[CI] Add pipeline task queue framework and merge workers into one (#64011) 2020-07-07 14:14:30 -04:00
Chris Cowan 6e357988a2
[Metrics UI] Performance improvements for Observability Homepage (#70869) 2020-07-07 11:07:52 -07:00
Nicolas Chaulet b1ec391d86
[Ingest Manager] Rate limit agent config update (#70871) 2020-07-07 13:51:55 -04:00
Jean-Louis Leysens 50a2991312
[Ingest Pipeline] Processor Editor Item Styling tweak (#70786)
* Small styling tweaks to processor items

- Moved the move button to the before the processor name
- Cancel button is still after description if there is one
- Made inline text description a bit taller and changed border
  style

* Commit code that moves the cancel move button 🤦🏼‍♂️

* Do not completely hide the move button, prevent ui from jumping

* Update styling and UX of move button; EuiToggleButton

- Bring the styling of the button more in line with this comment
  https://github.com/elastic/kibana/pull/70786#issuecomment-654222298

* use cross icon for cancelling move

* replace hard values with EUI values in SCSS

* Address rerendering triggered by context

- also prevent re-renders basded on contstructing objects on
  each render

* Similarly move use of context to settings form container

We are only interested in the es docs path string in the settings
form component so no need to render for other updates.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 19:22:58 +02:00
Chris Cowan f946e8e2a9
[Metrics UI] Set includeTimeseries on Observability fetchData request (#70735)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 09:47:45 -07:00
Nathan L Smith 735d3bae8c
[APM] Improvements to data telemetry (#70524)
Make some changes to how we deal with data telemetry in APM and reduce the number of fields we're storing in Saved Objects in the .kibana index.

Add a telemetry doc in dev_docs explaining how telemetry is collected and how to make updates. (In this PR the docs only cover data telemetry, but there's a space for the behavioral telemetry docs.)

Stop storing the mapping for the data telemetry in the Saved Object but instead use `{ dynamic: false }`.

This reduces the number of fields used by APM in the .kibana index (as requested in #43673.)

Before:

```bash
> curl -s -X GET "admin:changeme@localhost:9200/.kibana/_field_caps?fields=*&pretty=true" |  jq '.fields|length'
653
```

After:

```bash
> curl -s -X GET "admin:changeme@localhost:9200/.kibana/_field_caps?fields=*&pretty=true" |  jq '.fields|length'
415
```

We don't need the mapping anymore for storing the saved object, but we still do need to update the telemetry repository when the mapping changes, and the `upload-telemetry-data` script uses that mapping when generating data.

For these purposes the mapping in now defined in TypeScript in a function in common/apm_telemetry.ts.

It's broken down into some variables that and put together as the same mapping object that was there before, but having it in this form should make it easier to update.

A new script, `merge-telemetry-mapping`, takes the telemetry repository's xpack-phone-home.json mapping, merges in the result of our mapping and replaces the file. The result can be committed to the telemetry repo, making it easier to make changes to the mapping.

References #61583
Fixes #67032
2020-07-07 11:19:59 -05:00
Alejandro Fernández e41692e0e7
[Logs UI] Process long running requests in logs overview (#70791) 2020-07-07 17:42:45 +02:00
Mathis 49da11f244
[APM] Quote trace id to ensure a word is searched (#69500) (#69504)
* [APM] Quote trace id to ensure a word is searched (#69500)

Signed-off-by: Mathis Raguin <mathis@cri.epita.fr>

* [APM] Fix TransactionActionMenu tests (one test was not updated)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 17:41:10 +02:00
Zacqary Adam Xeper 5e869b0e77
[Metrics UI] Display Too Many Buckets error when previewing Inventory Alerts (#70508)
* [Metrics UI] Display Too Many Buckets error when previewing Inventory Alerts

* Fix typecheck
2020-07-07 10:05:50 -05:00
Pierre Gayvallet e7c54d3684
Rename HttpServerInfo.host to hostname (#70792)
* rename host to hostname

* update generated doc

* adapt tests

* fix types
2020-07-07 16:54:39 +02:00
Rudolf Meijering 2f905e7d86
CI Reporter for saved objects field count (#70580)
* CI Reporter for saved objects field count

* Metrics needs to be an array

* Fix type failures

* Link to field count issue

* Revert "Link to field count issue"

This reverts commit 8c0126b838.

* Break down field count per type

* Don't log total metric as metrics report already calculates this

* Add saved objects field count ci metrics test to codeowners

* Address review comments

* Add field count CI metrics for disabled plugins

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 16:53:47 +02:00
Alejandro Fernández 21fc56ed10
[Logs UI] Actions menu in log entry categorization page (#69567) 2020-07-07 16:52:17 +02:00
Tiago Costa 32190758fb
skip flaky suite (#70757) 2020-07-07 14:03:25 +01:00
CJ Cenizal e58cc173f1
Fix Data Streams and Rollups Jobs deep-link bugs (#70903)
* Add extractQueryParams to es_ui_shared/public/url. Update CCR, Remote Clusters, and Rollup to consume this service via shared_imports.
* Fix Data Streams bug in which clicking a data stream would apply a deep-link filter to the table.
* Fix Rollup Job deep-link bug.
2020-07-07 05:41:50 -07:00
Pierre Gayvallet 648468dae1
Rename legacy ES mock accessors (#70432)
* rename legacy client mocks

* move legacy client mocks to legacy folder

* fix usages

* adapt new usages

* adapt new usages
2020-07-07 13:38:17 +02:00
Oliver Gupte 7d44d022c9
[APM] Adds 'Anomaly detection' settings page to create ML jobs per environment (#70560)
* Adds 'Anomaly detection' settings page along with require API endpoints
to list and create the apm anomaly detection jobs per environment.
Some test data is hardcoded while the the required changes in the ML
plugin are in flight.

* Converts the environment name to a compatible ML id string and persist
in groups array. Also adds random token to the job ID to prevent
collisions for job ids where diffferent environment names convert to the
same string

* - Improve job creation with latest updates for the `apm_transaction` ML module
- Implements job list in settings by reading from `custom_settings.job_tags['service.environment']`
- Add ML module method `createModuleItem` for job configuration
- Don't allow user to type in duplicate environments

* Update x-pack/plugins/apm/public/components/app/Settings/anomaly_detection/add_environments.tsx

Co-authored-by: Casper Hübertz <casper@formgeist.com>

* Update x-pack/plugins/apm/public/components/app/Settings/anomaly_detection/index.tsx

Co-authored-by: Casper Hübertz <casper@formgeist.com>

* UX feedback, adds i18n, and handles failed state for ML jobs fetch.

* - Moves get_all_environments from agent_configuration dir to common dir
- makes the 'all' environment name ALL_OPTION_VALUE agent configuration-specific
- replace field literals with constants

* PR feedback

* Adds support to create jobs for environment which are not defined.

* Fixes description copy, rearranges settings links, and makes sure the 'Not defined' option is disabled if it already exists.

* Only show "Not defined" in environment selector if there are actually
documents without service.environment set

* get the indexPatternName for the ML job from the set of user-definned indices

* updated job_tags type definition

Co-authored-by: Casper Hübertz <casper@formgeist.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 13:30:14 +02:00
James Gowdy f18002c3cd
[ML] Adding peak_model_bytes to model size stats type (#70825)
* [ML] Adding peak_model_bytes to model size stats type

* adding formatter
2020-07-07 09:24:40 +01:00
Sébastien Loix 053b922b7c
[Composable template] Details panel + delete functionality (#70814) 2020-07-07 09:58:00 +02:00
Shahzad 77e40199b8
[Uptime] Ping list body scroll (#70781) 2020-07-07 09:22:09 +02:00
Matthias Wilhelm 4257afad1b
Adapt expected response of advanced settings feature control for cloud tests (#70793) 2020-07-07 07:27:12 +02:00
Tiago Costa f62f3e3727
skip flaky suite (#70885) 2020-07-07 05:34:02 +01:00
Tiago Costa 468201acf3
skip flaky suite (#67814) 2020-07-07 05:22:48 +01:00
Tiago Costa 79e5a07bdc
skip flaky suite (#70906) 2020-07-07 05:16:58 +01:00
Melissa Alvarez c5eab1021f
Revert "reenable regression and classification functional tests (#70661)" (#70908)
This reverts commit a9b543d9bc.
2020-07-06 23:09:26 -04:00
Yuliia Naumenko 438e905800
Added UI validation when creating a Webhook connector with invalid URL (#70025)
* Added UI validation when creating a Webhook connector with invalid URL

* fixed tests

* Fixed due to comments

* fixed type check and extended error message for invalid URL

* Fixed whitelisting of URL

* fixed failing tests

* fixed str
2020-07-06 17:35:47 -07:00
Robert Austin 610bff1269
[Security Solution] Change default index pattern (#70797)
* [Security Solution] Change default index pattern

Add `logs-*` to the Security Solution default index pattern. This should
allow the app to recognize events from the Elastic Endpoint.
2020-07-06 19:52:58 -04:00
Yuliia Naumenko 57915e1641
ServiceNow push to Incident generic implementation (supporting both Case specific and generic Alerts) (#68464)
* Draft ServiceNow generic implementation

* simple working servicenow incident per alert

* fixed running times

* rely on externalId for update incident on the next execution

* Added consumer to the action type to be able to split ServiceNow for Cases and Alerts

* Added subActions support for ServiceNow action form

* Basic version for Alerts part for ServiceNow

* Keep Case ServiceNow functionality working

* Revert changes on app_router

* Fixed type checks

* Fixed language check issues

* Fixed actions unit tests

* Fixed functional tests

* Fixed jest tests

* fixed tests

* Copied case mappings to alerting plugin

* made consumer optional

* Cleanup tests

* more cleanup

* Fixed jest tests and type checks

* fixed tests

* fixed servicenow validation tests

* Added ServiceNow unit tests

* Removed consumer for actions

* fixed client side isCaseOwned support

* fixed failing tests

* fixed jest tests

* Fixed URL validation

* fixed due to comments

* fixed tests

* fixed jest tests

* Fixed due to comments. Moved ServiceNow filtering in case plugin to server side

* fixed mock for ServiceNow

* fixed consumer config

* fixed test

* fixed type check

* Fixed jest test

* fixed type check
2020-07-06 16:38:45 -07:00
Candace Park 5992424312
add button link to ingest (#70142)
update security solution empty page
2020-07-06 18:52:00 -04:00
Melissa Alvarez a9b543d9bc
reenable regression and classification functional tests (#70661) 2020-07-06 18:40:12 -04:00
Alison Goryachev e35a42aa07
[Component templates] Form wizard (#69732) 2020-07-06 18:14:59 -04:00
Sandra Gonzales 2eb0896415
[Ingest Manager] Copy changes (#70828)
* update overview page

* remove streams column from config table

* fleet name chanegs

* remove unused component

* update translations
2020-07-06 18:07:29 -04:00
Bhavya RM 94a18fda5d
Adding test user to maps functional tests - PR 1 (#70649)
adding test user to pr 1 of maps functional tests.
2020-07-06 17:51:27 -04:00
Jen Huang 7debf4dd9f
[Ingest Manager] Support limiting integrations on an agent config (#70542)
* Add API endpoint and hook for retrieving restricted packages

* Filter out restricted packages already in use from list of integrations available for an agent config

* Allow list agent configs to optionally return expanded package configs, re

* Filter out agent configs which already use the restricted package already from list of agent configs available for an integration

* Allow more than 20 agent configs to be shown

* Rename restricted to limited; add some common methods to DRY

* Add limited package check on server side

* Adjust copy wording

* Fix typings

* Add some package config api integration tests, update es archive mappings

* Move test to dockerized integation tests directory; move existing epm tests to their own directory

* Remove extra assignPackageConfigs() - already handled in packageConfigService.create()

* Review fixes

* Fix type, reenabled skipped test

* Move new EPM integration test file
2020-07-06 14:12:15 -07:00
Tiago Costa c5dd942b72
Merge remote-tracking branch 'upstream/master' 2020-07-06 22:09:54 +01:00
Tiago Costa ad20a17bc6
skip flaky suite (#70880) 2020-07-06 22:09:19 +01:00
Zacqary Adam Xeper 11cfe80020
[Metrics UI] Fix a bug in Metric Threshold query filter construction (#70672)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-06 15:33:27 -05:00
John Schulz ee0653658d
Remove the legacy Ingest Manager plugin. (#65534)
The last thing we were using from it was configuring a static assets directory (which is only use for the EPM Integrations header graphic).

This is now provided by platform and is not configurable da28df5b15/src/core/MIGRATION.md (L1344)

Moved the header assets to the new directory & updated the `toAssets` helper
2020-07-06 15:49:14 -04:00
Sandra Gonzales 984ea0700e
[Ingest Manager ] prepend kibana asset ids with package name (#70502)
* prepend asset ids with package name

* fix type

* cleanup

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-06 15:46:30 -04:00
Sandra Gonzales b8591bc948
change user facing text Data streams to datasets (#70840) 2020-07-06 15:38:21 -04:00
MadameSheema cd50899493
fixes and unskips 'export rule' test (#70699)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-06 21:26:34 +02:00
MadameSheema 7b0e9dfe9a
[SIEM] Unskips and fixes 'Detection rules, custom' test (#70693)
* unskips and fixes 'Detection rules, custom' test

* deletes comment

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-06 21:25:52 +02:00
Melissa Alvarez a4340f0ece
[ML] DF Analytics: add ability to edit job for fields supported by API (#70489)
* wip: add edit action to dfanalytics table

* add update endpoint and edit flyout

* show success and error toasts. close flyout and refresh on success

* show permission message in edit action

* update types

* disable update button if mml not valid

* show error in toast, init values are config values

* fix undefined check for allow lazy start

* prevent update if mml is empty
2020-07-06 15:10:01 -04:00
Jonathan Buttner 321fb871cc
[EPM][Security Solution] Implementing dataset component templates (#70517)
* Implementing dataset component templates

* Fixing test

* Temporary fix to include timestamp with any component template created

* Update package registry docker image for CI.

* Adapt to new registry filesystem layout.

* Adjust tests to changed registry behavior.

* Adding a test for mappings and settings overrides

* Wrap all the tests in the docker check

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Sonja Krause-Harder <sonja.krause-harder@elastic.co>
2020-07-06 15:09:35 -04:00