Commit graph

38724 commits

Author SHA1 Message Date
Dario Gieselaar
213b934386
[APM] Make sure jest script can be run from anywhere (#85111) 2020-12-08 07:55:24 +01:00
Scotty Bollinger
62e79eec70
Add EuiButtonEmptyTo components (#85213)
This PR adds another wrapper component for internal routing, the EuiButtonEmptyTo component.
2020-12-07 21:03:10 -06:00
Tiago Costa
568ce65039
skip flaky suite (#85216) 2020-12-08 02:29:41 +00:00
Tiago Costa
eb1c29e656
skip flaky suite (#83775) 2020-12-08 02:27:07 +00:00
Tiago Costa
cd1be91343
skip flaky suite (#83774) 2020-12-08 02:24:26 +00:00
Tiago Costa
eeb68132b3
skip flaky suite (#83773) 2020-12-08 02:22:15 +00:00
Tiago Costa
f667b12cec
skip flaky suite (#83793) 2020-12-08 02:16:52 +00:00
Tiago Costa
8dcbac5518
skip flaky suite (#85215) 2020-12-08 02:12:55 +00:00
Tiago Costa
1a6b88c02c
skip flaky suite (#85217) 2020-12-08 02:10:59 +00:00
Catherine Liu
99ae137f63
[Usage Collection] Kibana Overview Page UI Metrics (#81937)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 18:52:52 -07:00
Yuliia Naumenko
6757b95b1e
[Alerting UI] Replaced AlertsContextProvider with KibanaContextProvider and exposed components in API (#84604)
* [Alerting UI] Replaced AlertsContextProvider with KibanaContextProvider and exposed components in API

* removed AlertContextProvider

* exposed AlertAdd and EditAlert flyouts with triggers_actions_ui plugin start

* fixed type check

* fixed tests

* fixed typechecks

* fixed wrong consumer

* fixed monitoring flyout flickering

* fixed due to comments

* fixed typechecks

* fixed typechecks

* fixed typechecks

* fixed typechecks

* fixed due to comments
2020-12-07 16:44:40 -08:00
Tiago Costa
a6a8cc2175
skip flaky suite (#85208) 2020-12-07 23:42:11 +00:00
Frank Hassanabad
1bc42b9171
[Security Solutions][Detection Engine] Fixes cypress errors by using latest signals mapping (#84600)
## Summary

Fixes issues within Cypress whenever developers would re-run tests they could become blocked as some of the tests cause a migration/roll over of the signals.
 
To reproduce the error off of master:

Start Cypress 
```ts
cd ./kibana/x-pack/plugins/security_solution
yarn cypress:open-as-ci
```

Then run the export test twice. The first time it will run ok. The second time the test will refuse to run.

<img width="585" alt="Screen Shot 2020-11-30 at 1 57 24 PM" src="https://user-images.githubusercontent.com/1151048/100683706-2233c200-3336-11eb-949f-48f86e884d8b.png">

The second time that you run the test you will get these errors from Cypress and will be blocked until you do workarounds such as logging into `https://localhost:5620` and manually changing the indexes manually or restarting Cypress altogether.
<img width="1067" alt="Screen Shot 2020-11-30 at 6 05 07 PM" src="https://user-images.githubusercontent.com/1151048/100683945-9a9a8300-3336-11eb-9d49-187dee4fc1dc.png">

What is going on with the errors is that a migration is occurring since the existing signals mapping for the tests are not the newer migrated version and these are creating a new index named `.siem-signals-default-0000002` like so:
<img width="1242" alt="Screen Shot 2020-11-30 at 1 57 12 PM" src="https://user-images.githubusercontent.com/1151048/100684145-fcf38380-3336-11eb-8fe1-b29d3462a1ae.png">

This index is not being cleaned up and when the next time we do an es_archive load we are marking two indexes as being writable


file: x-pack/test/security_solution_cypress/es_archives/alerts/mappings.json
```ts
".siem-signals-default": {
  "is_write_index": true
  }
},
"index": ".siem-signals-default-000001",
```

which leads to the stack trace and the Cypress errors on the front end:

```ts
MacBook-Pro.local] path: /.siem-signals-default-000001, params: {index=.siem-signals-default-000001}
     │      java.lang.IllegalStateException: alias [.siem-signals-default] has more than one write index [.siem-signals-default-000002,.siem-signals-default-000001]
     │          at org.elasticsearch.cluster.metadata.IndexAbstraction$Alias.computeAndValidateAliasProperties(IndexAbstraction.java:276) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
```

This fixes that by updating the mapping and migration number. Each time we migrate/change the signals mapping we will have to perform a PR like this to update each location.

At the moment this is 5 different locations we have to update with the latest mappings.

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2020-12-07 15:51:12 -07:00
Dave Snider
1f75d0b79f
Small fixes to Kibana search bar (#85079)
* Small fixes to Kibana search bar

* Animate width

* tweak for ipad breakpoint

Co-authored-by: Ryan Keairns <contactryank@gmail.com>
2020-12-07 12:59:45 -08:00
DeDe Morton
28ea225bef
Change link text to say Fleet (#85083) 2020-12-07 12:54:41 -08:00
Zacqary Adam Xeper
de289de6c1
[Metrics UI] Refactor Process List API to fetch only top processes (#84716)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 14:46:26 -06:00
Spencer
a7c5b49343
[esArchiver] support kibana and es ssl from cli (#85073)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 13:20:15 -07:00
John Schulz
81a340e681
[Fleet][EPM] Save installed package assets in ES (#83391)
## Summary
Store package assets (from Registry or local upload) in Elasticsearch. Related to proposal [issue](https://github.com/elastic/kibana/issues/83426) & [document](https://docs.google.com/document/d/18XoS6CSl9UxxPPBt9LXuJngf1Jv-4tl3jY6l19U1yH8)

 * New `epm-packages-assets` saved objects are stored on `.kibana` index, like our existing saved object `epm-packages`
 * Asset id is uuid v5 based on the package name, package version & file path. See 1974324
 * Add a list of IDs of all the installed assets, to `epm-packages` saved object. Like the existing `installed_` properties.  [Example](https://github.com/elastic/kibana/pull/83391/files#diff-fa07cac51b6a49bf1e4824bc2250c9a77dac6c7d6b0a56020f559ef1ff9be25fR491-R512) from a test

<details><summary>Mapping for new Saved Object</summary>

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fsaved_objects%2Findex.ts (L329-L339)
</details>

<details><summary>Additional property on existing <code>epm-packages</code> Saved Object</summary>

c4f27ab257/x-pack/plugins/fleet/server/saved_objects/index.ts (L306-L312)

 I don't think the saved object changes are strictly required. It can be removed without changing much about how things work

- Pros: 
      - Preserves accurate record of the assets added at installation time. Separates what assets are currently available for package-version from what was installed. They _should_ be the same, but things happen.
      - Avoids a query to get the installed assets before operating on them
- Cons:
      - size/noise? Could be tens or hundreds of ids
      - migration?
</details>

### More details

**When are saved objects added?**
During installation, after all other actions have succeeded, just before marking the save object as installed, we commit all the files from the package to ES

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2F_install_package.ts (L193-L198)

**When are documents removed from the index?**

In the `removeInstallation` function which is called in response to a `DELETE /api/fleet/epm/packages/pkgkey`

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Fremove.ts (L72)

or a failed package (re-)installation

bf068739ac/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Finstall.ts (L145)




**How are we using these assets?**
We're not, currently. Here's an example showing how we could update [`getFileHandler`](514b50e4c2/x-pack%2Fplugins%2Ffleet%2Fserver%2Froutes%2Fepm%2Fhandlers.ts (L101)) to check for local assets before reaching out to the Registry if we wished. It's not DRY, but it does work

```typescript
const esDocRoot = `http://elastic:changeme@localhost:9200/${PACKAGE_ASSETS_INDEX_NAME}/_doc`;
const escapedDocId = encodeURIComponent(`${pkgName}-${pkgVersion}/${filePath}`);
const esRes = await fetch(`${esDocRoot}/${escapedDocId}`);
const esJson = await esRes.json();
if (esJson.found) {
  const asset: PackageAsset = esJson._source;
  const body = asset.data_utf8 || Buffer.from(asset.data_base64, 'base64');
  return response.ok({
    body,
    headers: {
      'content-type': asset.media_type,
      // should add our own `cache-control` header here
      // kibana default is prevents caching: `private, no-cache, no-store, must-revalidate`
      // https://github.com/elastic/kibana/issues/83631
    },
  });
}
```

### Checklist
_updated tests to include new saved object output, no tests added yet_
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2020-12-07 15:11:09 -05:00
Tiago Costa
f961e90ea7
chore(NA): remove scripts on plugins to find circular deps (#84852)
* chore(NA): remove extra scripts on plugins to find circular deps

* chore(NA): remove madge as dependency

* Move cyclic dep jobs

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

Co-authored-by: Tyler Smalley <tyler.smalley@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 19:41:38 +00:00
Christiane (Tina) Heiligers
3604995ac2
[Docs] Adds docs on how to run Kibana with the APM agent locally (#84700)
Co-authored-by: Vignesh Shanmugam <vignesh.shanmugam22@gmail.com>
2020-12-07 12:39:48 -07:00
Spencer
b01f33eeb0
[cli/dev] make optimizer delays more obvious and hide proxy target url (#84835)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 11:34:59 -07:00
Davis Plumlee
3d3eb40964
[Security Solution][Detection Rules] Mitre subtechniques (#83511) 2020-12-07 13:34:15 -05:00
Spencer
8aec85cf3d
[dev/cli] don't filter inspect flags (#85156)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2020-12-07 11:17:48 -07:00
KOTungseth
78538a0c7a Merge branch 'master' of github.com:elastic/kibana 2020-12-07 12:01:22 -06:00
KOTungseth
a1116af74b Merge branch 'KOTungseth-docs/dashboard' 2020-12-07 12:00:42 -06:00
KOTungseth
0668571182 Merge branch 'docs/dashboard' of https://github.com/KOTungseth/kibana into KOTungseth-docs/dashboard 2020-12-07 12:00:21 -06:00
Alison Goryachev
079a7e82ba
Integrate painless autocomplete in runtime fields editor (#84943) 2020-12-07 12:55:53 -05:00
KOTungseth
cd050f37da Fixes conflicts 2020-12-07 11:42:44 -06:00
KOTungseth
0c1effe7bf Fixes conflict 2020-12-07 11:29:02 -06:00
igoristic
d19b558c88
[Monitoring][Alerting] Added core features to Kibana services UPDATED (#85074)
* added core features to kibana services

* Added test for alert form

* Added mocking of legacy shims and actions

* Fixed typing

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 12:25:28 -05:00
KOTungseth
1cc2986353 Fixes conflict 2020-12-07 11:11:53 -06:00
Thomas Neirynck
4a44804041
[Maps] Disallow on-prem EMS when proxyElasticMapsServiceInMaps is turned on (#85049)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 12:06:46 -05:00
ymao1
46e8c540aa
[Alerting] Fixes Failing test: X-Pack Alerting API Integration Tests.x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting - alerting api integration security and spaces enabled Alerts do stuff when AAD is broken (#84707)
* Adding delay between creating and updating alert to avoid 409 conflicts

* Unskipping update test

* Using retry.try instead of delay

* PR fixes

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 11:52:37 -05:00
igoristic
008c6a0e19
Safegaurding alerts in case they come back undefined (#85108) 2020-12-07 11:24:44 -05:00
Wylie Conlon
2466dd634a
[Lens] Fix functional test failures (#84979) 2020-12-07 17:10:04 +01:00
Lukas Olson
e48e7446c0
[data.search] Move background session service to data enhanced plugin (#84837)
* [data.search] Move search method inside session service and add tests

* Move background session service to data_enhanced plugin

* Fix types

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 08:51:19 -07:00
Robert Oskamp
0366a892b3
[ML] Initial API integration tests for ML jobs in spaces (#84789)
This PR adds initial API integration tests for the endpoints related to ML jobs in spaces.
2020-12-07 16:26:57 +01:00
Søren Louv-Jansen
cee681afb3
[APM] Improve pointer event hook (#85117) 2020-12-07 15:43:58 +01:00
Søren Louv-Jansen
e0d07c227f
[APM] Add ignore_unavailable to avoid querying closed indices (#84813) 2020-12-07 15:43:33 +01:00
John Schulz
ef7367ddb3
[Fleet][EPM] Add TS type for package-spec. Clarify EPR relationship with spec (#84946)
## Summary

 * Create a [TS type for `package-spec`](09cce235de/x-pack/plugins/fleet/common/types/models/package_spec.ts) to differentiate Registry shape from spec shape
 * Clarify EPR types relationship with package spec. i.e. what required properties it makes optional, what properties it adds, which property types it changes, etc.
5f2c4a5547/x-pack/plugins/fleet/common/types/models/epm.ts (L71-L80)
  * Updated `manifest.yml` in `apache_invalid_manifest_missing_field_0.1.4.zip`. Previously it was missing the `type` property, but that's optional [according to package-spec](3b4d820755/versions/1/manifest.spec.yml (L241-L248)). Removed the required `format_version` and re-zipped.

<details><summary>screenshots show errors if catches in a mock PackageInfo object</summary>
using `requirement`; not `conditions`
<img width="1084" alt="Screen Shot 2020-12-02 at 12 07 15 PM" src="https://user-images.githubusercontent.com/57655/101083986-4b915f80-357b-11eb-84c6-14605e1ea3d8.png">
using `versions`; not `version`
<img width="1395" alt="Screen Shot 2020-12-02 at 12 08 13 PM" src="https://user-images.githubusercontent.com/57655/101083988-4c29f600-357b-11eb-9ff2-2b73e313c130.png">
including `elasticsearch` when spec says it' not valid yet
<img width="1589" alt="Screen Shot 2020-12-02 at 12 08 36 PM" src="https://user-images.githubusercontent.com/57655/101083990-4c29f600-357b-11eb-8250-8926f7189af8.png">
</details>

<details><summary>screenshots showing editor autocomplete for registry response values like <code>categories</code></summary>

<img width="422" alt="Screen Shot 2020-12-02 at 12 25 11 PM" src="https://user-images.githubusercontent.com/57655/101083994-4cc28c80-357b-11eb-9013-ae208f7c311a.png">

<img width="345" alt="Screen Shot 2020-12-02 at 12 25 01 PM" src="https://user-images.githubusercontent.com/57655/101083991-4c29f600-357b-11eb-9468-bbb9d27513b1.png">
</details>

<details><summary>screenshot showing type check preventing adding properties which aren't explicitly listed as "additions"</summary>

<img width="1295" alt="Screen Shot 2020-12-03 at 11 38 43 AM" src="https://user-images.githubusercontent.com/57655/101083997-4cc28c80-357b-11eb-83b9-206b85521f11.png">
</details>


### Checklist


- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2020-12-07 09:30:27 -05:00
Shahzad
e476baf276
[Uptime] Details page refactor for browser monitor (#84425)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 14:35:19 +01:00
Tiago Costa
08680882ce
skip flaky suite (#85085) 2020-12-07 13:22:20 +00:00
Pierre Gayvallet
446390d86a
Add bulk assign action to tag management (#84177)
* initial draft

* move components to their own files

* create services folder and move tags package

* add assignment service

* fix some types

* prepare assign tag route

* move server-side tag client under the `services` folder

* add security check, move a lot of stuff.

* design improvements

* display tags in flyout

* improve button and add notification on save

* add action on tag rows

* fix types

* fix mock import paths

* add lens to the list of assignable types

* update generated doc

* add base functional tests

* move api to internal

* add api/security test suites

* add / use get_assignable_types API

* fix feature control tests

* fix assignable types propagation

* rename actions folder to bulk_actions

* extract actions to their own module

* add common / server unit tests

* add client-side assign tests

* add some tests and tsdoc

* typo

* add getActions test

* revert width change

* fix typo in API

* various minor improvements

* typo

* tsdoc on flyout page object

* close flyout when leaving the page

* fix bug when redirecting to SO management with a tag having whitespaces in its name

* check for dupes in toAdd and toRemove

* add lazy load to assign modal opener

* add lazy load to edit/create modals

* check if at least one assign or unassign tag id is specified

* grammar

* add explicit type existence check
2020-12-07 11:18:43 +01:00
Jean-Louis Leysens
34be1e724d
[ILM] Add searchable snapshot field (#83783)
* Added server-side endpoint for getting list of repos

* part one of client side changes: added searchable snapshot field

- added to both hot and cold
- serializing into JSON payload

* make searchable snapshot field toggleable and require value when it is being used

* fix typo in file name and remove whitespace

* added searchable snapshot state context, wip

* finished updating fields to show and hide based on searchable snapshot in the hot phase

* hiding fields when searchable snapshots is enabled in hot

- removed nested EuiFieldRow components in data allocation field
- added SCSS files for data allocation field and searchable
  snapshot field

* added translations and a first hot phase serialization test

* appease type check and i18n

* added cloud-specific behaviour for searchable snapshot default inlc. test

* refactor snapshot state -> configuration issues as this a can be re-used for other issues

* added configuration context file

* hide replicas in cold if snapshotting

* updated new field copy

* update test coverage, test for hiding certain fields too

* added license check to client side!

* moved warning to below field again and moved hot phase searchable snapshot notice to right

* make described form field lazy if needed

* render field even when license requirement is not met

* update serializer for removing searchable_snapshot field

* handle 404 from ES when looking up * snapshot repos - we return an empty array

* address license TODO

* added tests for license check and removed license check HoC

* fix legacy jest tests

* added readme about legacy tests

* updated jest tests and fix type issues

* remove unused import

* refactor component names to have "Field" at the end

* refactor searchable snapshot to single interface def and add comment about force_merge_index option

* address stakeholder feebdack and pr comments

* update tests after latest changes

* link to force merge

* Revert "link to force merge"

This reverts commit 6c714fbbac.

* introduce advanced section to hot, warm and cold

* added test for correctly deserializing delete phase, and added fix

* remove unused variable

* moved fields into advanced settings

* move learn more copy below enable toggle

* fix warm phase on rollover default

* remove label space above rollover toggle

* remove unused import

* update test after fixing warm on rollover default

* removed icons in callouts in searchable snapshot field, added ability to control field toggles

* move callouts to description text

* readd warning callouts to the searchable snapshot field

* slight i18n adjustment

* made callout for actions disabled a bit smaller

* fix i18n

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 10:52:44 +01:00
Matthew Kime
2c7a85dd38
Query string input - load index patterns instead of saved objects (#84457)
* load index patterns instead of saved objects
* remove getFromSavedObject
* add test
2020-12-06 11:24:11 -06:00
Tiago Costa
058f28ab23
skip flaky suite (#62060) 2020-12-05 22:21:12 +00:00
Patryk Kopyciński
2f386e8961
[Security Solution] 7.11 Timeline EVOLUTION (#83378) 2020-12-05 16:04:52 +01:00
Cauê Marcondes
d4370ff14e
[APM] APIs refactoring (#85010)
* renaiming /transaction_groups to /transactions/groups

* renaiming /transaction_groups to /transactions/groups

* renaiming /transaction_groups/charts to /transactions/charts

* renaiming /transaction_groups/distribution to transactions/charts/distribution

* renaiming /transaction_groups/breakdown to transactions/charts/breakdown

* removing /api/apm/transaction_sample. Unused

* renaiming /transaction_groups/error_rate to transactions/charts/error_rate

* removing transaction_groups

* removing /api/apm/transaction_sample. Unused

* renaiming /overview_transaction_groups to transactions/groups/overview

* refactoring error_groups

* addressing pr comments

* fixing test
2020-12-05 11:41:13 +01:00
Tim Roes
2f327543c2
Remove left over "refresh index pattern" message from Discover (#85018)
* Remove the unmapped field refresh warning

* Remove unneeded translations

* Remove old imports
2020-12-05 10:43:09 +01:00
Brian Seeders
aa2525c842
[CI] Bump intake instance size (#85082) 2020-12-04 20:50:55 -05:00