Commit graph

1972 commits

Author SHA1 Message Date
Shahzad
949941758f
[RUM Dashboard] New rum services api to replace usage of get services API (#70746) 2020-07-08 11:26:34 +02:00
Madison Caldwell
f5b77e1026
[Security Solution][Endpoint] Fix base64 download bug and adopt new user artifact/manifest format (#70998)
* Fix base64 download bug

* Add test for artifact download

* Add more tests to ensure cached versions of artifacts are correct

* Convert to new format

* missed some refs

* partial fix to wrapper format

* update fixtures and integration test

* Fixing unit tests

Co-authored-by: Alex Kahan <alexander.kahan@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 21:52:08 -04:00
Frank Hassanabad
531cac058f
[SIEM][Detection Engine][Lists] Removes feature flag for lists
## Summary

* Removes the feature flag and turns on lists by default
* Applies to both exception lists and value lists
* Removes all scary messages about having it enabled
* Updates the unit tests to work with it on 

- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
2020-07-07 19:13:53 -06:00
Paul Tavares
e99974d8e4
[Endpoint] Support redirect from Policy Details to Ingest when user initiates Edit Policy from Datasource Edit page (#70874)
* allow Policy Details to support route state
* Functional Tests cases that cover ingest navigation
2020-07-07 16:55:40 -04:00
Søren Louv-Jansen
cb6fbc0e1b
[APM] Add API tests (#70740) 2020-07-07 22:40:28 +02:00
Mikhail Shustov
aeff8c154b
[Audit Logging] Add AuditTrail service (#69278)
* add generic audit_trail service in core

* expose auditTraik service to plugins

* add auditTrail x-pack plugin

* fix type errors

* update mocks

* expose asScoped interface via start. auditor via  request context

* use type from audit trail service

* wrap getActiveSpace in safeCall only. it throws exception for non-authz

* pass message to log explicitly

* update docs

* create one auditor per request

* wire es client up to auditor

* update docs

* withScope accepts only one scope

* use scoped client in context for callAsInternalUser

* use auditor in scoped cluster client

* adopt auditTrail plugin to new interface. configure log from config

* do not log audit events in console by default

* add audit trail functional tests

* cleanup

* add example

* add mocks for spaces plugin

* add unit tests

* update docs

* test description

* Apply suggestions from code review

apply @jportner suggestions

Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>

* add unit tests

* more robust tests

* make spaces optional

* address comments

* update docs

* fix WebStorm refactoring

Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
2020-07-07 21:16:39 +02:00
Chris Cowan
6e357988a2
[Metrics UI] Performance improvements for Observability Homepage (#70869) 2020-07-07 11:07:52 -07:00
Pierre Gayvallet
e7c54d3684
Rename HttpServerInfo.host to hostname (#70792)
* rename host to hostname

* update generated doc

* adapt tests

* fix types
2020-07-07 16:54:39 +02:00
Rudolf Meijering
2f905e7d86
CI Reporter for saved objects field count (#70580)
* CI Reporter for saved objects field count

* Metrics needs to be an array

* Fix type failures

* Link to field count issue

* Revert "Link to field count issue"

This reverts commit 8c0126b838.

* Break down field count per type

* Don't log total metric as metrics report already calculates this

* Add saved objects field count ci metrics test to codeowners

* Address review comments

* Add field count CI metrics for disabled plugins

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 16:53:47 +02:00
Sébastien Loix
053b922b7c
[Composable template] Details panel + delete functionality (#70814) 2020-07-07 09:58:00 +02:00
Matthias Wilhelm
4257afad1b
Adapt expected response of advanced settings feature control for cloud tests (#70793) 2020-07-07 07:27:12 +02:00
Tiago Costa
f62f3e3727
skip flaky suite (#70885) 2020-07-07 05:34:02 +01:00
Tiago Costa
79e5a07bdc
skip flaky suite (#70906) 2020-07-07 05:16:58 +01:00
Melissa Alvarez
c5eab1021f
Revert "reenable regression and classification functional tests (#70661)" (#70908)
This reverts commit a9b543d9bc.
2020-07-06 23:09:26 -04:00
Yuliia Naumenko
57915e1641
ServiceNow push to Incident generic implementation (supporting both Case specific and generic Alerts) (#68464)
* Draft ServiceNow generic implementation

* simple working servicenow incident per alert

* fixed running times

* rely on externalId for update incident on the next execution

* Added consumer to the action type to be able to split ServiceNow for Cases and Alerts

* Added subActions support for ServiceNow action form

* Basic version for Alerts part for ServiceNow

* Keep Case ServiceNow functionality working

* Revert changes on app_router

* Fixed type checks

* Fixed language check issues

* Fixed actions unit tests

* Fixed functional tests

* Fixed jest tests

* fixed tests

* Copied case mappings to alerting plugin

* made consumer optional

* Cleanup tests

* more cleanup

* Fixed jest tests and type checks

* fixed tests

* fixed servicenow validation tests

* Added ServiceNow unit tests

* Removed consumer for actions

* fixed client side isCaseOwned support

* fixed failing tests

* fixed jest tests

* Fixed URL validation

* fixed due to comments

* fixed tests

* fixed jest tests

* Fixed due to comments. Moved ServiceNow filtering in case plugin to server side

* fixed mock for ServiceNow

* fixed consumer config

* fixed test

* fixed type check

* Fixed jest test

* fixed type check
2020-07-06 16:38:45 -07:00
Melissa Alvarez
a9b543d9bc
reenable regression and classification functional tests (#70661) 2020-07-06 18:40:12 -04:00
Alison Goryachev
e35a42aa07
[Component templates] Form wizard (#69732) 2020-07-06 18:14:59 -04:00
Bhavya RM
94a18fda5d
Adding test user to maps functional tests - PR 1 (#70649)
adding test user to pr 1 of maps functional tests.
2020-07-06 17:51:27 -04:00
Jen Huang
7debf4dd9f
[Ingest Manager] Support limiting integrations on an agent config (#70542)
* Add API endpoint and hook for retrieving restricted packages

* Filter out restricted packages already in use from list of integrations available for an agent config

* Allow list agent configs to optionally return expanded package configs, re

* Filter out agent configs which already use the restricted package already from list of agent configs available for an integration

* Allow more than 20 agent configs to be shown

* Rename restricted to limited; add some common methods to DRY

* Add limited package check on server side

* Adjust copy wording

* Fix typings

* Add some package config api integration tests, update es archive mappings

* Move test to dockerized integation tests directory; move existing epm tests to their own directory

* Remove extra assignPackageConfigs() - already handled in packageConfigService.create()

* Review fixes

* Fix type, reenabled skipped test

* Move new EPM integration test file
2020-07-06 14:12:15 -07:00
Tiago Costa
c5dd942b72
Merge remote-tracking branch 'upstream/master' 2020-07-06 22:09:54 +01:00
Tiago Costa
ad20a17bc6
skip flaky suite (#70880) 2020-07-06 22:09:19 +01:00
MadameSheema
cd50899493
fixes and unskips 'export rule' test (#70699)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-06 21:26:34 +02:00
MadameSheema
7b0e9dfe9a
[SIEM] Unskips and fixes 'Detection rules, custom' test (#70693)
* unskips and fixes 'Detection rules, custom' test

* deletes comment

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-06 21:25:52 +02:00
Jonathan Buttner
321fb871cc
[EPM][Security Solution] Implementing dataset component templates (#70517)
* Implementing dataset component templates

* Fixing test

* Temporary fix to include timestamp with any component template created

* Update package registry docker image for CI.

* Adapt to new registry filesystem layout.

* Adjust tests to changed registry behavior.

* Adding a test for mappings and settings overrides

* Wrap all the tests in the docker check

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Sonja Krause-Harder <sonja.krause-harder@elastic.co>
2020-07-06 15:09:35 -04:00
Melissa Alvarez
31abd6dc28
[ML] DF Analytics creation: switch to includes table (#70009)
* update modelMemoryLimit when hyperParams change

* update functional clone tests

* switch excludes table to includes table

* Job configuration details update

* fix jest tests and types

* fix translations and validate includes fields

* fix functional test

* handle empty includes selection

* switch filter to field_value_toggle_group

* update clone functional test
2020-07-06 13:10:01 -04:00
Sonja Krause-Harder
89dcdbbbee
[Ingest Manager] Update package registry docker image for CI. (#70716)
* Update package registry docker image for CI.

* Adapt to new registry filesystem layout.

* Adjust tests to changed registry behavior.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-06 18:57:18 +02:00
Felix Stürmer
2399780d99
[Logs + Metrics UI] Add index names for the new indexing strategy (#70245)
This add support for the new index name patterns `logs-*` and `metrics-*` of the new indexing strategy to the Logs and Metrics UI source configurations in the form of a migration and changed defaults.
2020-07-06 17:37:28 +02:00
Tiago Costa
3250816d02
skip flaky suite (#70818) 2020-07-06 16:13:37 +01:00
Pierre Gayvallet
04aaba80f7
[GS] add savedObjects result provider (#68619)
* create server-side skeleton

* add base implementation & tests

* add unit test for provider

* remove useless contracts

* add preference search option

* implement score from find results

* fix types

* add FTR test

* fix test plugin types

* address ome review comments

* add multi results test

* use `getVisibleTypes`
2020-07-06 13:04:43 +02:00
Madison Caldwell
e429670c35
[Security Solution][Endpoint] Update to new manifest format (without compression) (#70752)
* Stateless exception list translation with improved runtime checks

* use flatMap and reduce to simplify logic

* Update to new manifest format

* Fix test fixture SO data type

* Fix another test fixture data type

* Fix sha256 reference in artifact_client
2020-07-04 19:23:07 -04:00
Lee Drengenberg
c3cacba7e2
logout from transform_poweruser user in after method of transform tests (#70644)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-03 13:14:18 -05:00
Nicolas Chaulet
72b300424b
[Ingest Manager] Improve agent unenrollment with unenroll action (#70031) 2020-07-03 08:23:12 -04:00
Sébastien Loix
bc1599ef6b
[Composable template] Create / Edit wizard (#70220) 2020-07-03 11:56:17 +02:00
Jen Huang
54348a761e
[Ingest Manager] Add ability to sort to agent configs and package configs (#70676)
* Add sorting params to list endpoints; allow sorting on agent config and package config tables; normalize casing of 'desc' and 'asc'

* Fix es archiver data

* Fix tests
2020-07-02 23:26:56 -07:00
Yuliia Naumenko
21efd235f7
Fixed adding an extra space character on selecting alert variable in action text fields (#70028)
* Fixed adding an extra space character on selecting alert variable in action text fields.

* Made components for variables to be able to insert the variable by the cursor position

* cleanup

* Added variables support for all components

* update on handle selections for text

* Fixed functional tests
2020-07-02 18:08:14 -07:00
Tiago Costa
6c62c686cf
chore(NA): upgrade to lodash@4 (#69868)
* chore(NA): upgrade oss to lodash4

chore(NA): migrate cli, cli_plugin, cli_keystore,  dev, test_utils and apm src script to lodash4

chore(NA): missing file for cli plugin

chore(NA): add src core

chore(NA): es archiver and fixtures

chore(NA): try to fix functional test failure

chore(NA): migrate src/legacy entirely to lodash4 except src/legacy/core_plugins

chore(NA): move legacy core plugins to lodash4

chore(NA): upgrade optimize to lodash4

chore(NA): upgrade to lodash4 on advanced_settings, charts, console and dashboard

chore(NA): migrate to lodash4 on dev_tools, discover, embeddable, es_ui)shared, expressions, home plugins

chore(NA): upgrade data plugin to lodash4

chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4

chore(NA): missing data upgrades to lodash4

Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4"

This reverts commit 137055c5fed2fc52bb26547e0bc1ad2e3d4fe309.

Revert "Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4""

This reverts commit f7e73688782998513d9fb6d7e8f0765e9beb28d1.

Revert "chore(NA): missing data upgrades to lodash4"

This reverts commit 92b85bf947a89bfc70cc4052738a6b2128ffb076.

Revert "chore(NA): upgrade data plugin to lodash4"

This reverts commit 88fdb075ee1e26c4ac979b6681d8a2b002df74c6.

chore(NA): upgrade idx_pattern_mgt, input_control_vis, inspector, kbn_legacy, kbn_react, kbn_usage_collections, kbn_utils, management and maps_legacy to lodash4

chore(NA): map src plugin data to lodash3

chore(NA): missing lodash.clonedeep dep

chore(NA): change packages kbn-config-schema deps

chore(NA): update renovate config

chore(NA): upgrade vis_type plugins to lodash4

chore(NA): move vis_type_vislib to lodash3

chore(NA): update visualizations and visualize to lodash4

chore(NA): remove lodash 3 types from src and move test to lodash4

chore(NA): move home, usage_collection and management to lodash 3

Revert "chore(NA): move home, usage_collection and management to lodash 3"

This reverts commit f86e8585f02d21550746569af54215b076a79a3d.

chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3

chore(NA): update x-pack test to mock lodash4

Revert "chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3"

This reverts commit 2d10fe450533e1b36db21d99cfae3ce996a244e0.

* chore(NA): move x-pack and packages to lodash 4

* chore(NA): remove mention to lodash from main package.json

* chore(NA): remove helper alias for lodash4 and make it the default lodash

* chore(NA): fix last failing types in the repo

* chore(NA): fix public api

* chore(NA): fix types for agg_row.tsx

* chore(NA): fix increment of optimizer modules in the rollup plugin

* chore(NA): migrate `src/core/public/http/fetch.ts` (#5)

* omit undefined query props

* just remove merge usage

* fix types

* chore(NA): fixes for feedback from apm team

* chore(NA): recover old behaviour on apm LoadingIndeicatorContext.tsx

* chore(NA): fixes for feedback from watson

* Platform lodash4 tweaks (#6)

* chore(NA): fix types and behaviour on src/core/server/elasticsearch/errors.ts

* Canvas fixes for lodash upgrade

* [APM] Adds unit test for APM service maps transform (#7)

* Adds a snapshot unit test for getConnections and rearranges some code to make testing easier

* reverts `ArrayList` back to `String[]` in the painless script within `fetch_service_paths_from_trace_ids.ts`

* chore(NA): update yarn.lock

* chore(NA): remove any and use a real type for alerts task runner

Co-authored-by: Gidi Meir Morris <github@gidi.io>

* chore(NA): used named import for triggers_actions_ui file

* chore(NA): fix eslint

* chore(NA): fix types

* Delete most uptime lodash references.

* Simplify. Clean up types.

* [Uptime] Delete most uptime lodash references (#8)

* Delete most uptime lodash references.

* Simplify. Clean up types.

* chore(NA): add eslint rule to avoid using lodash3

* chore(NA): apply changes on feedback from es-ui team

* fix some types (#9)

* Clean up some expressions types.

* chore(NA): missing ts-expect-error statements

* Upgrade lodash 4 vislib (#11)

* replace lodash 3 with lodash 4 on vislib plugin

* Further changes

* further replacement of lodash3 to 4

* further work on upgrading to lodash 4

* final changes to update lodash

* chore(NA): upgrade data plugin to lodash4

chore(NA): upgrade data plugin public to lodash4

chore(NA): fix typecheck task

chore(NA): fix agg_config with hasIn

chore(NA): assign to assignIn and has to hasIn

chore(NA): upgrade data plugin server to lodash4

chore(NA): new signature for core api

fix(NA): match behaviour between lodash3 and lodash4 for set in search_source

* chore(NA): remove lodash3 completely from the repo

* chore(NA): fix x-pack/test/api_integration/apis/metrics_ui/snapshot.ts missing content

* chore(NA): fix lodash usage on apm

* chore(NA): fix typecheck for maps

* Patch lodash template (#12)

* Applying changes from https://github.com/elastic/kibana/pull/64985

* Using isIterateeCall, because it seems less brittle

* Also patching `lodash/template` and `lodash/fp/template`

* Reorganizing some files...

* Revising comment

* Ends up `_` is a function also... I hate JavaScript

Co-authored-by: Pierre Gayvallet <pierre.gayvallet@gmail.com>
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
Co-authored-by: Oliver Gupte <ogupte@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Justin Kambic <justin.kambic@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
Co-authored-by: Luke Elmers <luke.elmers@elastic.co>
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
Co-authored-by: kobelb <brandon.kobel@elastic.co>
2020-07-03 01:30:13 +01:00
Kevin Logan
a921bbf4c2
[SECURITY SOLUTION] Remove unneeded options from default policy, remove Notify user option from UI (#70546) 2020-07-02 16:18:52 -04:00
Jonathan Budzenski
0e008e30e9 skip fleet agent flow tests 2020-07-02 12:46:40 -05:00
Michael Hirsch
8a09f247e3
[ML] Updates APM Module to Work with Service Maps (#70361)
* updates apm integration job to work with service maps

* rename apm job in setup_module test

* modifies detector description

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-02 11:35:40 -04:00
CJ Cenizal
59ece7992b
Make Index Management functional and API integration tests robust against side effects introduced by Ingest Manager. (#70533) 2020-07-02 08:24:42 -07:00
Dima Arnautov
854e7a5204
[ML] Anomaly Explorer swim lane pagination (#70063)
* [ML] use explorer service

* [ML] WIP pagination

* [ML] add to dashboard without the limit

* [ML] WIP

* [ML] loading states

* [ML] viewBySwimlaneDataLoading on field change

* [ML] fix dashboard control

* [ML] universal swim lane container, embeddable pagination

* [ML] fix css issue

* [ML] rename anomalyTimelineService

* [ML] rename callback

* [ML] rename container component

* [ML] empty state, increase pagination margin

* [ML] check for loading

* [ML] fix i18n

* [ML] fix unit test

* [ML] improve selected cells

* [ML] fix overall selection with changing job selection

* [ML] required props for pagination component

* [ML] move RESIZE_IGNORED_DIFF_PX

* [ML] jest tests

* [ML] add test subject

* [ML] SWIM_LANE_DEFAULT_PAGE_SIZE

* [ML] change empty state styling

* [ML] fix agg size for influencer filters

* [ML] remove debounce

* [ML] SCSS variables, rename swim lane class

* [ML] job selector using context

* [ML] set padding for embeddable panel

* [ML] adjust pagination styles

* [ML] replace custom time range subject with timefilter

* [ML] change loading indicator to mono

* [ML] use swim lane type constant

* [ML] change context naming

* [ML] update jest snapshot

* [ML] fix tests
2020-07-02 16:30:18 +02:00
Thomas Neirynck
9c76f19186
[Maps] Add styling and tooltip support to mapbox mvt vector tile sources (#64488)
* tmp commit

* rename

* more boilerpalte

* more boiler

* more boilerpalte

* typing

* fix import

* boilerplate

* more boiler

* enable custom palettes

* fix label text and orientation

* fix merge errors

* remove dupe import

* stash commit

* tmp commit

* debounce settings

* return null

* slight rearrangement

* tooltip guard

* minor tweaks

* feedback

* ts fixes

* ts fixes

* more ts fixes

* ts fixes

* jest test

* fix typo

* spacing

* fix typing

* add unit test

* add more tests

* add snapshot test

* add snapshot

* add field editor snapshot test

* fix snapshot

* add snapshot

* remove unused import

* test stub for mvt layer

fix optional param

more checks

* add snapshot test

more unit tests

more unit tests

ts fixes

* add data syncing unit test

* fix autorefactor

* fix merge and replace snapshots

* field editor changes

* field editor changes

* ts fixes

* update snapshots

* fix things

* fix names

* fix tooltip

* add more error handling

* improve copy

* styling changes

* style option box a little better

* ts fixes

* fix console error

* remove mbProperties from interface

* remove unused method

* remove cruft

* rename for consistency

* remove unused param

* feedback

* feedback

* ensure properties are always present

* handle possible null values

* feedback

* typo

* update SIEM

* feedback

* remove cruft

* remove unused translations

* feedback

* improve readability

* fix brittle test

* fix snapshot after master merge

* remove unused method

* feedback

* revert some feedback

* remove micro-optimization

* initialize in constructor

* simplify wording

* add snapshot

* naming

* add clarifying comment

* remove unused import

* sanitize tooltips

* remove cruft

* feedback

* fix typo

* remove export

* Design fixes

* clean up supportsAutoDomain

* remove patch.txt

* cleanup

* clean-up

* Merge in styling changes

* Tweak message format

* fix broken import

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: miukimiu <elizabet.oliveira@elastic.co>
Co-authored-by: Nathan Reese <reese.nathan@gmail.com>
2020-07-02 09:24:56 -04:00
Jonathan Buttner
c081caa634
[Security_Solution][Endpoint] Leveraging msearch and ancestry array for resolver (#70134)
* Refactor generator for ancestry support

* Adding optional ancestry array

* Refactor the pagination since the totals are not used anymore

* Updating the queries to not use aggregations for determining the totals

* Refactoring the children helper to handle pagination without totals

* Pinning the seed for the resolver tree generator service

* Splitting the fetcher into multiple classes for msearch

* Updating tests and api for ancestry array and msearch

* Adding more comments and fixing type errors

* Fixing resolver test import

* Fixing tests and type errors

* Fixing type errors and tests

* Removing useAncestry field

* Fixing test

* Removing useAncestry field from tests

* An empty array will be returned because that's how ES will do it too
2020-07-02 08:47:37 -04:00
Alejandro Fernández Haro
6607bf7b49
[Telemetry] Report data shippers (#64935)
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-02 08:08:35 +01:00
Spencer
eca4cc5d3e
Skip failing endgame tests (#70548)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2020-07-01 23:01:21 -07:00
Madison Caldwell
0f7afd4402
[SIEM][Security Solution][Endpoint] Endpoint Artifact Manifest Management + Artifact Download and Distribution (#67707)
* stub out task for the exceptions list packager

* Hits list code and pages

* refactor

* Begin adding saved object and type definitions

* Transforms to endpoint exceptions

* Get internal SO client

* update messaging

* cleanup

* Integrating with task manager

* Integrated with task manager properly

* Begin adding schemas

* Add multiple OS and schema version support

* filter by OS

* Fixing sort

* Move to security_solutions

* siem -> securitySolution

* Progress on downloads, cleanup

* Add config, update artifact creation, add TODOs

* Fixing buffer serialization problem

* Adding cleanup to task

* Handle HEAD req

* proper header

* More robust task management

* single -> agnostic

* Fix OS filtering

* Scaffolding digital signatures / tests

* Adds rotue for creating endpoint user

* Cleanup

* persisting user

* Adding route to fetch created user

* Addings tests for translating exceptions

* Adding test for download API

* Download tweaks + artifact generation fixes

* reorganize

* fix imports

* Fixing test

* Changes id of SO

* integration tests setup

* Add first integration tests

* Cache layer

* more schema validation

* Set up for manifest update

* minor change

* remove setup code

* add manifest schema

* refactoring

* manifest rewrite (partial)

* finish scaffolding new manifest logic

* syntax errors

* more refactoring

* Move to endpoint directory

* minor cleanup

* clean up old artifacts

* Use diff appropriately

* Fix download

* schedule task on interval

* Split up into client/manager

* more mocks

* config interval

* Fixing download tests and adding cache tests

* lint

* mo money, mo progress

* Converting to io-ts

* More tests and mocks

* even more tests and mocks

* Merging both refactors

* Adding more tests for the convertion layer

* fix conflicts

* Adding lzma types

* Bug fixes

* lint

* resolve some type errors

* Adding back in cache

* Fixing download test

* Changing cache to be sized

* Fix manifest manager initialization

* Hook up datasource service

* Fix download tests

* Incremental progress

* Adds integration with ingest manager for auth

* Update test fixture

* Add manifest dispatch

* Refactoring to use the same SO Client from ingest

* bug fixes

* build renovate config

* Fix endpoint_app_context_services tests

* Only index the fields that are necessary for searching

* Integ test progress

* mock and test city

* Add task tests

* Tests for artifact_client and manifest_client

* Add manifest_manager tests

* minor refactor

* Finish manifest_manager tests

* Type errors

* Update integ test

* Type errors, final cleanup

* Fix integration test and add test for invalid api key

* minor fixup

* Remove compression

* Update task interval

* Removing .text suffix from translated list

* Fixes hashes for unit tests

* clean up yarn.lock

* Remove lzma-native from package.json

* missed updating one of the tests

Co-authored-by: Alex Kahan <alexander.kahan@elastic.co>
2020-07-02 01:00:27 -04:00
Garrett Spong
591e10355a
[Security] Adds field mapping support to rule creation (#70288)
## Summary

Resolves: https://github.com/elastic/kibana/issues/65941, https://github.com/elastic/kibana/issues/66317, and `Add support for "building block" alerts`

This PR is `Part I` and adds additional fields to the `rules schema` in supporting the ability to map and override fields when generating alerts. A few bookkeeping fields like `license` and `author` have been added as well. The new fields are as follows:

``` ts
export interface TheseAreTheNewFields {
  author: string[];
  building_block_type: string; // 'default'
  license: string;
  risk_score_mapping: Array<
    {
      field: string;
      operator: string; // 'equals'
      value: string;
    }
  >;
  rule_name_override: string;
  severity_mapping: Array<
    {
      field: string;
      operator: string; // 'equals'
      value: string;
      severity: string; // 'low' | 'medium' | 'high' | 'critical'
    }
  >;
  timestamp_override: string;
}
```

These new fields are exposed as additional settings on the `About rule` section of the Rule Creation UI.

##### Default collapsed view, no severity or risk score override specified:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86090417-49c0ee80-ba67-11ea-898f-a43af6d9383f.png" />
</p>

##### Severity & risk score override specified:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86091165-a8d33300-ba68-11ea-86ac-89393a7ca3f5.png" />
</p>

##### Additional fields in Advanced settings:
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/86091256-cbfde280-ba68-11ea-9b63-acf2524039bd.png" />
</p>


Note: This PR adds the fields to the `Rules Schema`, the `signals index mapping`,  and creates the UI for adding these fields during Rule Creation/Editing. The follow-up `Part II` will add the business logic for mapping fields during `rule execution`, and also add UI validation/additional tests.

### Checklist

Delete any items that are not applicable to this PR.

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials
  - Syncing w/ @benskelker 
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
- [x] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)

### For maintainers

- [x] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
2020-07-01 22:49:30 -06:00
John Dorlus
b21e43d848
Fixed assertion in hybrid index pattern test to iterate through indices (#70130)
* Fixed assertion to check through all indices before making assertion.

* Restored yarn.lock file.

* Removed only from describe.

* Fixed linting issue.

* Fixed nits that were in the PR conversation.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-01 21:04:03 -04:00
Jen Huang
4f7da59a51
[Ingest Manager] Rename data sources to package configs (#70259)
* Rename `datasource` saved object to `package_config` (SO type `ingest-datasource` to `ingest-package-config`) and adjust mappings:

- Remove unused `processors` field on input & stream levels
- Remove unnecessary `enabled` field on package config & input levels
- Rename `agent_stream` field to `compiled_stream`
- Reorder other fields so that important fields are closer to top & similar fields are grouped together

Also,
- Remove 7.9.0 migrations as we are not supporting an upgrade path from experimental to beta release

* Pluralize `ingest-package-configs` for consistency

* Rename `Datasource`-related types to `PackageConfig`, update all references (does not include actual type definition changes yet)

* Rename `Datasource` schemas and rest spec typings to `PackageConfig` (does not include actual schema changes yet)

* Change `datasources` on agent config typings and schemas to `package_configs` and update all references

* Add back `enabled` field on package config and input levels. They are needed for current & future UI features. Also:

- Match types and schemas with saved object mappings (`agent_stream` to `compiled_stream`, removal of `processors`)
- Set `namespace` to be a required property on agent config and package config types, add validation support for it on UI

* Rename server-side datasource references in file names, variable names, and routes

* Update spec file and schema file

* Update doc wording

* Rename all instances of datasource in file paths and variable names on client sides, and for Endpoint too

* Minor copy adjustments, fix i18n check

* Replace datasource references in tests and fixtures; remove unused `ingest/policies` es archiver data

* Fix tests

* Fix test field name

* Fix test fixtures fields again

* Fix i18n
2020-07-01 16:14:21 -07:00
spalger
0f418bb7f0 skip suites blocking es snapshot promomotion (#70532) 2020-07-01 16:10:01 -07:00