Commit graph

40318 commits

Author SHA1 Message Date
Rudolf Meijering
ce441bdc32
RFC Improve saved object migrations algorithm (#84333)
* Instead of cloning, reindex legacy index

* Reindex for every v2 migration

* Use _reindex?require_alias=true and a write block toggle to prevent lost deletes

* Use a ..._reindex_in_progress alias so that waiting for and preventing other reindex operations is idempotent

The first version of the reindex block had only the instance which was able to mark the migration as complete
set and remove the write block. This means other instances couldn't know if any reindex operaitons were in
progress if the migration was already marked as complete. It also meant that a failure in this critical step
could result in a permanent write block.

* Revert "Use a ..._reindex_in_progress alias so that waiting for and preventing other reindex operations is idempotent"

This reverts commit 8baf9b13db.

* Revert "Use _reindex?require_alias=true and a write block toggle to prevent lost deletes"

This reverts commit d7237ca42c.

* Use reindex + clone as a way to prevent lost deletes

* Fix numbering and ignore index_not_found_exceptionfor temporary index

* Apply suggestions from code review

Co-authored-by: Josh Dover <me@joshdover.com>

Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-10 13:54:52 +01:00
Marta Bondyra
df3e14dbbd
[Lens] (Accessibility) Fix focus on drag and drop actions (#90561) 2021-02-10 13:50:41 +01:00
Sonja Krause-Harder
4ee9603801
Use new shortcut links to Fleet discuss forums. (#90786)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-10 11:33:17 +01:00
Aleh Zasypkin
03a53b9f39
Do not generate an ephemeral encryption key in production. (#81511) 2021-02-10 11:27:31 +01:00
Nicolas Ruflin
634c0b3424
[Fleet] Use staging registry for snapshot builds (#90327)
The staging registry is used in Kibana builds which are not built of the master branch or release version. This means, any build ending with `-SNAPSHOT` not the master branch will use the staging registry.

Closes https://github.com/elastic/kibana/issues/90131

Co-authored-by: Jen Huang <its.jenetic@gmail.com>
2021-02-10 09:57:09 +01:00
Brandon Kobel
240da2bf2a
Actually deleting x-pack/tsconfig.refs.json (#90898) 2021-02-10 09:38:37 +01:00
Sonja Krause-Harder
af277b8396
Add deprecation warning to all Beats CM pages. (#90741) 2021-02-10 09:16:19 +01:00
Tyler Smalley
ab401c9857 skip flaky suite (#90136) 2021-02-09 23:29:52 -08:00
Zacqary Adam Xeper
38a6475396
Revert "Revert "[Metrics UI] Add Metrics Anomaly Alert Type (#89244)"" (#90889)
* Revert "Revert "[Metrics UI] Add Metrics Anomaly Alert Type (#89244)""

This reverts commit 8166becc55.

* Fix type error
2021-02-09 21:30:25 -05:00
spalger
59cc39434e remove ref to removed tsconfig file 2021-02-09 17:57:05 -07:00
Christiane (Tina) Heiligers
646732f96d
[core.logging] Uses host timezone as default (#90368)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 19:49:53 -05:00
Nathan Reese
1ba7d7af9a
[Maps] remove maps_file_upload plugin and fold public folder into file_upload plugin (#90292)
* get geojson working with api/file_upload/import

* remove maps_file_upload server code

* remove common folder

* remove maps_file_upload plugin

* fix tsconfig paths

* rename file_upload plugin in maps tsconfig

* fix file path

* node scripts/build_plugin_list_docs

* fix webpack compile errors

* telemetry schema cleanup, i18n cleanup, limits cleanup

* remove mapsFileUpload from limits.yml

* remove index pattern link test case

* update telemetry/v2/clusters/_stats for new file_upload path

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 17:31:57 -07:00
spalger
8166becc55 Revert "[Metrics UI] Add Metrics Anomaly Alert Type (#89244)"
This reverts commit 0d94968df1.
2021-02-09 17:15:12 -07:00
Spencer
4bab95229d
[dev-utils/ci-stats] support disabling ship errors (#90851)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-02-09 19:09:02 -05:00
Chris Roberson
82f0328232
Prefix with / (#90836) 2021-02-09 18:59:47 -05:00
Zacqary Adam Xeper
0d94968df1
[Metrics UI] Add Metrics Anomaly Alert Type (#89244)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 18:40:18 -05:00
Brandon Kobel
7627d35443
Getting rid of the x-pack tsconfig refs (#90537) 2021-02-09 15:34:42 -08:00
Ryland Herrick
8e4f899380
Ignore intermediate records in ML Rule anomalies query (#90316)
We were incorrectly including records with is_interim: true in our
query, which lead to false positive signals if the rule executed while
an anomaly's score was (temporarily) above the specified threshold, but
then dipped below after it was finalized.

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 17:21:44 -06:00
Nick Peihl
d14e37fd68
Bump @elastic/ems-client@7.12.0 (#90841) 2021-02-09 15:00:37 -08:00
Nathan Reese
a9ee83c37c
[Maps] convert HeatmapLayer to TS (#90666)
* [Maps] convert HeatmapLayer to TS

* make VectorLayer implement IVectorLayer

* fix jest test and functional test

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 15:49:44 -07:00
Rashmi Kulkarni
1a9f8cab17
[uptime] unskip previously skipped functional tests (#90702)
* fixes https://github.com/elastic/kibana/issues/74449

* unskipping uptime -functional test

* unskipping location.ts script
2021-02-09 14:12:46 -08:00
Shahzad
08111e40d3
[Uptime-UX] Added nav search keywords for uptime and user experience app (#90616) 2021-02-09 15:41:47 -06:00
Nathan Reese
6c224e5a10
Switch import route tag from access:ml:canFindFileStructure to access:fileUpload:import (#90677) 2021-02-09 14:34:34 -07:00
Stacey Gammon
10adb72ddb
Plugin introduction doc improvements (#90502)
* More plugin docs improvements

* remove speculation about the future

* remove registries section

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 16:27:36 -05:00
Scotty Bollinger
ad2f5d7918
[Workplace Search] Fix paths to nest correctly (#90831)
We have serveral places that I went up one level too many when getting shared components. This refactors that to align with other usages.
2021-02-09 16:19:08 -05:00
Zacqary Adam Xeper
95948fd5eb
[Metrics UI] Add warning severity to Metric Alerts (#90070) 2021-02-09 15:12:13 -06:00
Frank Hassanabad
5a27e69c6b
[Security Solutions][Detection Engine] Unskips tests after ES promotion and adds deleteUserRole utility (#90533)
## Summary

Unskips tests after a ES promotion and adds a delete user role utility.

Ref:
https://github.com/elastic/kibana/issues/90229
https://github.com/elastic/kibana/issues/88302

Removes one `any` from the utils by switching to using `ProvidedType`

Before:
<img width="558" alt="Screen Shot 2021-02-05 at 2 45 37 PM" src="https://user-images.githubusercontent.com/1151048/107098890-8dce5b80-67cd-11eb-8f6e-51f83eef4647.png">

After:
<img width="513" alt="Screen Shot 2021-02-05 at 4 13 23 PM" src="https://user-images.githubusercontent.com/1151048/107098898-9161e280-67cd-11eb-8085-a5220938834e.png">

Turns out that return types on overloaded functions aren't easy fwiw and will fall on the bottom one which in this case looked to be `any` which we don't want:
https://github.com/Microsoft/TypeScript/issues/24275#issuecomment-390701982


### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-02-09 13:35:37 -07:00
Spencer
de3f4aa419
[docs/ci-stats] describe new metrics files (#90711)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-02-09 13:21:29 -07:00
Devon Thomson
128488c6d1
[Time to Visualize] Clear Unsaved Changes When Dashboard Fails to Load (#90527)
* added error handling to dashboard_unsaved_listing. It should now remove all unsaved changes from dashboards which error on load
2021-02-09 15:20:58 -05:00
Melissa Alvarez
f6b6a8219b
[ML] Data Frame Analytics: Support early stopping data frame analytics job parameter (#90695)
* add support for early_stopping_enabled in cloning

* remove obsolete comment
2021-02-09 14:52:41 -05:00
John Dorlus
f939334b6b
[Index Management][Datastreams] Fixed datastreams functional app load test to check for visibility of element to prevent false positives (#85708) 2021-02-09 14:38:13 -05:00
Christiane (Tina) Heiligers
6dde543b6c
TS project refs: Converts rollup, remoteClusters, crossClusterReplication, indexLifecycleManagement to a TS project refs (#90713)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 14:26:28 -05:00
ymao1
5f8de693b9
[Alerting] Configurable number of hits for ES query alert (#90089)
* Adding size parameter to ES query alert

* Can't use const inside validation

* Updating docs

* Fixing functional test

* License

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 14:07:53 -05:00
Sandra Gonzales
1f5d52ea2e
[Metrics UI] Add ability for user to set anomaly threshold (#90313)
* add anomaly threshold configuration to settings

* hide panel if not ml capable

* send threshold value to query

* update license

* update api integration to expect anomalyThreshold to exist

* add some tests to the anomaly queries

* change type to number

* api validate in source update and config for anamolyThreshold and tests

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 13:28:46 -05:00
Tyler Smalley
08c08e88d9
[CI] Combines Jest unit tests (#89948)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2021-02-09 10:08:46 -08:00
Constance
f3399620cf
[Enterprise Search] Add eslint import/order rules (#90530)
* Add import rules

- newlines between each group
- mocks in test files before everything else
- React before all other externals

* Run --fix on public/applications/workplace_search

* Manually fix errors still present in WS public files

- these appear to be mostly due to jest.mock() or const mixed in with imports, which confuses the autofixer

* Run --fix on public/applications/app_search

+ some manual fixes/tweaks along the way

* Run --fix on public/applications/shared

- some opinionated changes, particularly around IFlashMessages and grouping together types coming from Kibana (src->../../src)

* Run --fix on public/applications/enterprise_search

- mostly straightforward

* Run --fix on public/applications/__mocks__

* Fix remaining top-level public files

- Some opinionated changes (src/core -> ../../src/core) to keep types/mocks together

* Run --fix on server/ files

- same opinionated src->../src changes to keep deps grouped together

- opinionated require->import fetch change in enterprise_esarch_config_api.test.ts

- opinionated [] inclusion of builtins & external imports together (mostly for enterprise_search_request_handler.ts)
2021-02-09 09:33:39 -08:00
Tim Sullivan
4b3d1bf83c
[Tabify] Add meta option to include top-level underscored field values (#90535)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 10:01:40 -07:00
Tyler Smalley
d43547af82 skip flaky suite (#86546) 2021-02-09 07:53:00 -08:00
Kerry Gallagher
3ad86f1328
[Logs UI] Add setup error telemetry for ML functionality (#90298)
* Add logs ML setup error telemetry
2021-02-09 15:50:00 +00:00
Wylie Conlon
3cb04fc6d0
Expand Tinymath grammar, including named arguments (#89795)
* Expand Tinymath grammar, including named arguments

* Add tsconfig project

* Fix tests

* Allow named arguments with numeric types

* Remove dashes from named argument validation

* Fix license header

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 10:46:07 -05:00
Jean-Louis Leysens
5099eab19f
[ILM] Simplify timeline and rollover info (#90004)
* remove rollover indicator on timeline and relative timing text

* simplify rollover description text

* remove non-existent export

* incorporate copy recommendations

* slight visual adjustment to timeline, infinity icon more subdued, moved icon to left

* update visual appearance of delete icon on timeline, grey circle and trash can

* remove tooltip next to recommended defaults

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 16:29:16 +01:00
Kerry Gallagher
32ddd5e795
[Logs UI] Show anomalies across both the log rate and categorization ML jobs in a swimlane visualization. (#89589)
* Allow anomalies visualisation to represent anomalies from all jobs

Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
2021-02-09 15:28:59 +00:00
James Gowdy
3d068c5db9
[ML] Lazy ml node UI improvements (#90455)
* [ML] Lazy ml node UI improvements

* fixing test

* adding awaitingMlNodeAllocation to default datafeed response

* changing datafeed icon when node is not assigned

* updating text

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 15:03:23 +00:00
Nathan Reese
9314b8e2fa
[Maps] use chart pallete registry to support sync colors in dashboard (#88099)
* [Maps] use chart pallete registry to support sync colors in dashboard

* pass getColor to createLayerInstance

* use chartsPaletteServiceGetColor to get categorical color

* revert changes to layer_actions

* tslint and jest test updates

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 07:57:00 -07:00
Mikhail Shustov
82d1672e79
[Core] add timeout for "stop" lifecycle (#90432)
* add timeout for stop lifecycle

* add timeout for stop lifecycle

* update message

* cleanup timeout to remove async tasks
2021-02-09 15:38:35 +01:00
John Schulz
810e4ab8e8
[Fleet] Prevent agents from enrolling in a managed policy (#90458)
## Summary
Add guard to `/agents/enroll` API preventing agents from enrolling in managed policies

closes #90435
- [x] No Agents can be enrolled into this policy by the user.
- [x] The install & enroll commands should print an error to the console if the enroll command fails (due to being a managed policy or any other reason)



#### So how do you associate an agent with a managed policy?
Enroll in an unmanaged policy then set that policy to managed. 

We don't restrict the agent policy, only what other things (agents, integrations) can do if they're associated with a managed policy.

A _force flag_ has been mentioned for some other actions. It might work here as well, but I'd like to handle discussion & implementation of those later.

### Manual testing
<details><summary>Prevent enroll for managed policies</summary>

1. Created a managed agent policy
    ```
    curl --user elastic:changeme -X POST localhost:5601/api/fleet/agent_policies -H 'Content-Type: application/json' -d'{ "name": "User created MANAGED", "namespace": "default", "is_managed": true}' -H 'kbn-xsrf: true'
    {"item":{"id":"3bd07db0-67d0-11eb-b656-21ad68ebfa8a","name":"User created MANAGED","namespace":"default","is_managed":true,"revision":1,"updated_at":"2021-02-05T16:36:01.931Z","updated_by":"elastic"}}
    ```
2. Try `install` command show in the UI
    ```
    sudo ./elastic-agent install -f --kibana-url=http://localhost:5601 --enrollment-token=WmcwTWMzY0IzWlBUUWJJUjZqRDA6UGRZelVlaS1STml1cVdjSUVwSkJRQQ== --insecure
    Password:
    The Elastic Agent is currently in BETA and should not be used in production

    Error: fail to enroll: fail to execute request to Kibana: Status code: 400, Kibana returned an error: Bad Request, message: Cannot enroll in managed policy 3bd07db0-67d0-11eb-b656-21ad68ebfa8a
    Error: enroll command failed with exit code: 1
    ```
3. Observe `Cannot enroll in managed policy 3bd07db0-67d0-11eb-b656-21ad68ebfa8a` error
4. Try `enroll` instead:
    ```
    sudo ./elastic-agent enroll http://localhost:5601 WmcwTWMzY0IzWlBUUWJJUjZqRDA6UGRZelVlaS1STml1cVdjSUVwSkJRQQ== --insecure
    The Elastic Agent is currently in BETA and should not be used in production

    This will replace your current settings. Do you want to continue? [Y/n]:
    Error: fail to enroll: fail to execute request to Kibana: Status code: 400, Kibana returned an error: Bad Request, message: Cannot enroll in managed policy 3bd07db0-67d0-11eb-b656-21ad68ebfa8a
    ```
5. Observe same `Cannot enroll in managed policy 3bd07db0-67d0-11eb-b656-21ad68ebfa8a` error

</details>

<details><summary>Enroll in unmanaged policy, then update it to managed</summary>

Agent policies are `is_managed: false` by default, or we can update the policy to `is_managed: false`, like:

```
curl --user elastic:changeme -X PUT localhost:5601/api/fleet/agent_policies/3bd07db0-67d0-11eb-b656-21ad68ebfa8a -H 'Content-Type: application/json' -d'{ "is_managed": false, "name": "xyz", "namespace": "default" }' -H 'kbn-xsrf: true'
{"item":{"id":"3bd07db0-67d0-11eb-b656-21ad68ebfa8a","name":"xyz","namespace":"default","is_managed":false,"revision":4,"updated_at":"2021-02-05T17:42:05.610Z","updated_by":"elastic","package_policies":[]}}
```

then enroll

```
 sudo ./elastic-agent install -f --kibana-url=http://localhost:5601 --enrollment-token=WmcwTWMzY0IzWlBUUWJJUjZqRDA6UGRZelVlaS1STml1cVdjSUVwSkJRQQ== --insecure
The Elastic Agent is currently in BETA and should not be used in production

Successfully enrolled the Elastic Agent.
Installation was successful and Elastic Agent is running.
```

and set the policy back to managed

```
curl --user elastic:changeme -X PUT localhost:5601/api/fleet/agent_policies/3bd07db0-67d0-11eb-b656-21ad68ebfa8a -H 'Content-Type: application/json' -d'{ "is_managed": true, "name": "xyz", "namespace": "default" }' -H 'kbn-xsrf: true'
{"item":{"id":"3bd07db0-67d0-11eb-b656-21ad68ebfa8a","name":"xyz","namespace":"default","is_managed":true,"revision":5,"updated_at":"2021-02-05T17:44:18.757Z","updated_by":"elastic","package_policies":[]}}
```

with all the restrictions that entails (cannot unenroll, reassign, etc)

```
curl --user elastic:changeme -X PUT 'http://localhost:5601/api/fleet/agents/8169f0a0-67d9-11eb-80f2-73dd45e7318e/reassign'   -X 'PUT'  -H 'kbn-xsrf: abc'  -H 'Content-Type: application/json'   --data-raw '{"policy_id":"729f8440-67cf-11eb-b656-21ad68ebfa8a"}'   
{
  "statusCode": 400,
  "error": "Bad Request",
  "message": "Cannot reassign an agent from managed agent policy 3bd07db0-67d0-11eb-b656-21ad68ebfa8a"
}
```

</details>

### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-02-09 08:54:51 -05:00
Liza Katz
731e333078
[Search Sessions] Apply awaits to avoid unhandled errors (#90593)
* Apply awaits to avoid unhandled errors

* catch and ignore tracking error

* added reject test for search service

* Improve search service api test coverage

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-09 15:43:40 +02:00
Tiago Costa
b78f9f9c46
skip flaky suite (#50448) 2021-02-09 13:42:48 +00:00
Tiago Costa
3456766c39
skip flaky suite (#90576) 2021-02-09 13:40:12 +00:00
Dario Gieselaar
ac18273df5
[APM] Higher timeout for flaky abort test (#90728) 2021-02-09 14:33:46 +01:00