Commit graph

38651 commits

Author SHA1 Message Date
Spencer a7c5b49343
[esArchiver] support kibana and es ssl from cli (#85073)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 13:20:15 -07:00
John Schulz 81a340e681
[Fleet][EPM] Save installed package assets in ES (#83391)
## Summary
Store package assets (from Registry or local upload) in Elasticsearch. Related to proposal [issue](https://github.com/elastic/kibana/issues/83426) & [document](https://docs.google.com/document/d/18XoS6CSl9UxxPPBt9LXuJngf1Jv-4tl3jY6l19U1yH8)

 * New `epm-packages-assets` saved objects are stored on `.kibana` index, like our existing saved object `epm-packages`
 * Asset id is uuid v5 based on the package name, package version & file path. See 1974324
 * Add a list of IDs of all the installed assets, to `epm-packages` saved object. Like the existing `installed_` properties.  [Example](https://github.com/elastic/kibana/pull/83391/files#diff-fa07cac51b6a49bf1e4824bc2250c9a77dac6c7d6b0a56020f559ef1ff9be25fR491-R512) from a test

<details><summary>Mapping for new Saved Object</summary>

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fsaved_objects%2Findex.ts (L329-L339)
</details>

<details><summary>Additional property on existing <code>epm-packages</code> Saved Object</summary>

c4f27ab257/x-pack/plugins/fleet/server/saved_objects/index.ts (L306-L312)

 I don't think the saved object changes are strictly required. It can be removed without changing much about how things work

- Pros: 
      - Preserves accurate record of the assets added at installation time. Separates what assets are currently available for package-version from what was installed. They _should_ be the same, but things happen.
      - Avoids a query to get the installed assets before operating on them
- Cons:
      - size/noise? Could be tens or hundreds of ids
      - migration?
</details>

### More details

**When are saved objects added?**
During installation, after all other actions have succeeded, just before marking the save object as installed, we commit all the files from the package to ES

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2F_install_package.ts (L193-L198)

**When are documents removed from the index?**

In the `removeInstallation` function which is called in response to a `DELETE /api/fleet/epm/packages/pkgkey`

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Fremove.ts (L72)

or a failed package (re-)installation

bf068739ac/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Finstall.ts (L145)




**How are we using these assets?**
We're not, currently. Here's an example showing how we could update [`getFileHandler`](514b50e4c2/x-pack%2Fplugins%2Ffleet%2Fserver%2Froutes%2Fepm%2Fhandlers.ts (L101)) to check for local assets before reaching out to the Registry if we wished. It's not DRY, but it does work

```typescript
const esDocRoot = `http://elastic:changeme@localhost:9200/${PACKAGE_ASSETS_INDEX_NAME}/_doc`;
const escapedDocId = encodeURIComponent(`${pkgName}-${pkgVersion}/${filePath}`);
const esRes = await fetch(`${esDocRoot}/${escapedDocId}`);
const esJson = await esRes.json();
if (esJson.found) {
  const asset: PackageAsset = esJson._source;
  const body = asset.data_utf8 || Buffer.from(asset.data_base64, 'base64');
  return response.ok({
    body,
    headers: {
      'content-type': asset.media_type,
      // should add our own `cache-control` header here
      // kibana default is prevents caching: `private, no-cache, no-store, must-revalidate`
      // https://github.com/elastic/kibana/issues/83631
    },
  });
}
```

### Checklist
_updated tests to include new saved object output, no tests added yet_
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2020-12-07 15:11:09 -05:00
Tiago Costa f961e90ea7
chore(NA): remove scripts on plugins to find circular deps (#84852)
* chore(NA): remove extra scripts on plugins to find circular deps

* chore(NA): remove madge as dependency

* Move cyclic dep jobs

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

Co-authored-by: Tyler Smalley <tyler.smalley@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 19:41:38 +00:00
Christiane (Tina) Heiligers 3604995ac2
[Docs] Adds docs on how to run Kibana with the APM agent locally (#84700)
Co-authored-by: Vignesh Shanmugam <vignesh.shanmugam22@gmail.com>
2020-12-07 12:39:48 -07:00
Spencer b01f33eeb0
[cli/dev] make optimizer delays more obvious and hide proxy target url (#84835)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 11:34:59 -07:00
Davis Plumlee 3d3eb40964
[Security Solution][Detection Rules] Mitre subtechniques (#83511) 2020-12-07 13:34:15 -05:00
Spencer 8aec85cf3d
[dev/cli] don't filter inspect flags (#85156)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2020-12-07 11:17:48 -07:00
Alison Goryachev 079a7e82ba
Integrate painless autocomplete in runtime fields editor (#84943) 2020-12-07 12:55:53 -05:00
igoristic d19b558c88
[Monitoring][Alerting] Added core features to Kibana services UPDATED (#85074)
* added core features to kibana services

* Added test for alert form

* Added mocking of legacy shims and actions

* Fixed typing

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 12:25:28 -05:00
Thomas Neirynck 4a44804041
[Maps] Disallow on-prem EMS when proxyElasticMapsServiceInMaps is turned on (#85049)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 12:06:46 -05:00
ymao1 46e8c540aa
[Alerting] Fixes Failing test: X-Pack Alerting API Integration Tests.x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting - alerting api integration security and spaces enabled Alerts do stuff when AAD is broken (#84707)
* Adding delay between creating and updating alert to avoid 409 conflicts

* Unskipping update test

* Using retry.try instead of delay

* PR fixes

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 11:52:37 -05:00
igoristic 008c6a0e19
Safegaurding alerts in case they come back undefined (#85108) 2020-12-07 11:24:44 -05:00
Wylie Conlon 2466dd634a
[Lens] Fix functional test failures (#84979) 2020-12-07 17:10:04 +01:00
Lukas Olson e48e7446c0
[data.search] Move background session service to data enhanced plugin (#84837)
* [data.search] Move search method inside session service and add tests

* Move background session service to data_enhanced plugin

* Fix types

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 08:51:19 -07:00
Robert Oskamp 0366a892b3
[ML] Initial API integration tests for ML jobs in spaces (#84789)
This PR adds initial API integration tests for the endpoints related to ML jobs in spaces.
2020-12-07 16:26:57 +01:00
Søren Louv-Jansen cee681afb3
[APM] Improve pointer event hook (#85117) 2020-12-07 15:43:58 +01:00
Søren Louv-Jansen e0d07c227f
[APM] Add ignore_unavailable to avoid querying closed indices (#84813) 2020-12-07 15:43:33 +01:00
John Schulz ef7367ddb3
[Fleet][EPM] Add TS type for package-spec. Clarify EPR relationship with spec (#84946)
## Summary

 * Create a [TS type for `package-spec`](09cce235de/x-pack/plugins/fleet/common/types/models/package_spec.ts) to differentiate Registry shape from spec shape
 * Clarify EPR types relationship with package spec. i.e. what required properties it makes optional, what properties it adds, which property types it changes, etc.
5f2c4a5547/x-pack/plugins/fleet/common/types/models/epm.ts (L71-L80)
  * Updated `manifest.yml` in `apache_invalid_manifest_missing_field_0.1.4.zip`. Previously it was missing the `type` property, but that's optional [according to package-spec](3b4d820755/versions/1/manifest.spec.yml (L241-L248)). Removed the required `format_version` and re-zipped.

<details><summary>screenshots show errors if catches in a mock PackageInfo object</summary>
using `requirement`; not `conditions`
<img width="1084" alt="Screen Shot 2020-12-02 at 12 07 15 PM" src="https://user-images.githubusercontent.com/57655/101083986-4b915f80-357b-11eb-84c6-14605e1ea3d8.png">
using `versions`; not `version`
<img width="1395" alt="Screen Shot 2020-12-02 at 12 08 13 PM" src="https://user-images.githubusercontent.com/57655/101083988-4c29f600-357b-11eb-9ff2-2b73e313c130.png">
including `elasticsearch` when spec says it' not valid yet
<img width="1589" alt="Screen Shot 2020-12-02 at 12 08 36 PM" src="https://user-images.githubusercontent.com/57655/101083990-4c29f600-357b-11eb-8250-8926f7189af8.png">
</details>

<details><summary>screenshots showing editor autocomplete for registry response values like <code>categories</code></summary>

<img width="422" alt="Screen Shot 2020-12-02 at 12 25 11 PM" src="https://user-images.githubusercontent.com/57655/101083994-4cc28c80-357b-11eb-9013-ae208f7c311a.png">

<img width="345" alt="Screen Shot 2020-12-02 at 12 25 01 PM" src="https://user-images.githubusercontent.com/57655/101083991-4c29f600-357b-11eb-9468-bbb9d27513b1.png">
</details>

<details><summary>screenshot showing type check preventing adding properties which aren't explicitly listed as "additions"</summary>

<img width="1295" alt="Screen Shot 2020-12-03 at 11 38 43 AM" src="https://user-images.githubusercontent.com/57655/101083997-4cc28c80-357b-11eb-83b9-206b85521f11.png">
</details>


### Checklist


- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2020-12-07 09:30:27 -05:00
Shahzad e476baf276
[Uptime] Details page refactor for browser monitor (#84425)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 14:35:19 +01:00
Tiago Costa 08680882ce
skip flaky suite (#85085) 2020-12-07 13:22:20 +00:00
Pierre Gayvallet 446390d86a
Add bulk assign action to tag management (#84177)
* initial draft

* move components to their own files

* create services folder and move tags package

* add assignment service

* fix some types

* prepare assign tag route

* move server-side tag client under the `services` folder

* add security check, move a lot of stuff.

* design improvements

* display tags in flyout

* improve button and add notification on save

* add action on tag rows

* fix types

* fix mock import paths

* add lens to the list of assignable types

* update generated doc

* add base functional tests

* move api to internal

* add api/security test suites

* add / use get_assignable_types API

* fix feature control tests

* fix assignable types propagation

* rename actions folder to bulk_actions

* extract actions to their own module

* add common / server unit tests

* add client-side assign tests

* add some tests and tsdoc

* typo

* add getActions test

* revert width change

* fix typo in API

* various minor improvements

* typo

* tsdoc on flyout page object

* close flyout when leaving the page

* fix bug when redirecting to SO management with a tag having whitespaces in its name

* check for dupes in toAdd and toRemove

* add lazy load to assign modal opener

* add lazy load to edit/create modals

* check if at least one assign or unassign tag id is specified

* grammar

* add explicit type existence check
2020-12-07 11:18:43 +01:00
Jean-Louis Leysens 34be1e724d
[ILM] Add searchable snapshot field (#83783)
* Added server-side endpoint for getting list of repos

* part one of client side changes: added searchable snapshot field

- added to both hot and cold
- serializing into JSON payload

* make searchable snapshot field toggleable and require value when it is being used

* fix typo in file name and remove whitespace

* added searchable snapshot state context, wip

* finished updating fields to show and hide based on searchable snapshot in the hot phase

* hiding fields when searchable snapshots is enabled in hot

- removed nested EuiFieldRow components in data allocation field
- added SCSS files for data allocation field and searchable
  snapshot field

* added translations and a first hot phase serialization test

* appease type check and i18n

* added cloud-specific behaviour for searchable snapshot default inlc. test

* refactor snapshot state -> configuration issues as this a can be re-used for other issues

* added configuration context file

* hide replicas in cold if snapshotting

* updated new field copy

* update test coverage, test for hiding certain fields too

* added license check to client side!

* moved warning to below field again and moved hot phase searchable snapshot notice to right

* make described form field lazy if needed

* render field even when license requirement is not met

* update serializer for removing searchable_snapshot field

* handle 404 from ES when looking up * snapshot repos - we return an empty array

* address license TODO

* added tests for license check and removed license check HoC

* fix legacy jest tests

* added readme about legacy tests

* updated jest tests and fix type issues

* remove unused import

* refactor component names to have "Field" at the end

* refactor searchable snapshot to single interface def and add comment about force_merge_index option

* address stakeholder feebdack and pr comments

* update tests after latest changes

* link to force merge

* Revert "link to force merge"

This reverts commit 6c714fbbac.

* introduce advanced section to hot, warm and cold

* added test for correctly deserializing delete phase, and added fix

* remove unused variable

* moved fields into advanced settings

* move learn more copy below enable toggle

* fix warm phase on rollover default

* remove label space above rollover toggle

* remove unused import

* update test after fixing warm on rollover default

* removed icons in callouts in searchable snapshot field, added ability to control field toggles

* move callouts to description text

* readd warning callouts to the searchable snapshot field

* slight i18n adjustment

* made callout for actions disabled a bit smaller

* fix i18n

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-07 10:52:44 +01:00
Matthew Kime 2c7a85dd38
Query string input - load index patterns instead of saved objects (#84457)
* load index patterns instead of saved objects
* remove getFromSavedObject
* add test
2020-12-06 11:24:11 -06:00
Tiago Costa 058f28ab23
skip flaky suite (#62060) 2020-12-05 22:21:12 +00:00
Patryk Kopyciński 2f386e8961
[Security Solution] 7.11 Timeline EVOLUTION (#83378) 2020-12-05 16:04:52 +01:00
Cauê Marcondes d4370ff14e
[APM] APIs refactoring (#85010)
* renaiming /transaction_groups to /transactions/groups

* renaiming /transaction_groups to /transactions/groups

* renaiming /transaction_groups/charts to /transactions/charts

* renaiming /transaction_groups/distribution to transactions/charts/distribution

* renaiming /transaction_groups/breakdown to transactions/charts/breakdown

* removing /api/apm/transaction_sample. Unused

* renaiming /transaction_groups/error_rate to transactions/charts/error_rate

* removing transaction_groups

* removing /api/apm/transaction_sample. Unused

* renaiming /overview_transaction_groups to transactions/groups/overview

* refactoring error_groups

* addressing pr comments

* fixing test
2020-12-05 11:41:13 +01:00
Tim Roes 2f327543c2
Remove left over "refresh index pattern" message from Discover (#85018)
* Remove the unmapped field refresh warning

* Remove unneeded translations

* Remove old imports
2020-12-05 10:43:09 +01:00
Brian Seeders aa2525c842
[CI] Bump intake instance size (#85082) 2020-12-04 20:50:55 -05:00
Joel Griffith 9073aec955
[Reporting] Bump puppeteer 5.4.1 + roll chromium rev (#85066)
* Update puppeteer + new headless shell bin

* Bump types for pptr

* Fix broken mock for pptr
2020-12-04 14:50:07 -08:00
Thomas Neirynck d41fbf948e
[Maps] Update style when metrics change (#83586) 2020-12-04 17:24:21 -05:00
Ryland Herrick 918dbb17de
[Security Solution] Fixes some misconfigured settings (#85056)
* Remove nonexistent paths from our API tests config

These paths (or rather their absence) breaks the functional_tests_server script.

An analogous problem for cases was fixed in #79127.

* Remove duplicated context from logger

This context is already provided to the logger by kibana; adding it a
second time leads to duplicated log tags, e.g.:

```
server    log   [12:36:07.138] [debug][plugins][plugins][securitySolution][securitySolution] ...
```

This is now fixed:

```
server    log   [12:41:52.112] [debug][plugins][securitySolution] ...
```
2020-12-04 15:57:12 -06:00
Tre b6a6a30f26
[QA][Stack Integration Tests] Support defining the tests list in the Integration Test Repo (#83363)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 14:34:03 -07:00
Spencer fd1328f6f8
[cli/dev] remove cluster module, modernize, test (#84726)
Co-authored-by: Alejandro Fernández Haro <afharo@gmail.com>
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 14:28:06 -07:00
Henry Harding c6ef1ae30b
Design cleanup details panel (#85044)
* cleanup overlay panel. fixed sizes, variable panel height, responsive breakpoint

* cleanup metrics tab. use EuiFlexGrid + ChartSizeArray for chart sizing + fixed responsive issues on flex items

* cleanup metadata tab. disabled responsive table and fixed alignment of filter icon + value

* cleanup logs search. adjusted button size + spacing

* fix responsivness on array values

* remove responsive behavior on search + view-in-logs button

* fix typecheck
2020-12-04 15:05:13 -05:00
Gidi Meir Morris c85f2545da
[Actions] Fixes issue which causes PagerDuty Params to rerender continuously. (#85050)
* prevent aciton form from rerendering constantly

* fixed typing
2020-12-04 14:50:50 -05:00
Christos Nasikas fcccb016f4
[Security Solution][Case] Add in-progress status to case (#84321) 2020-12-04 21:36:23 +02:00
Brian Seeders 554ee9ebf9
Revert "[Monitoring][Alerting] Added core features to Kibana services (#84486)"
This reverts commit 4545f56a46.
2020-12-04 14:24:49 -05:00
spalger b507dbf8a1 [renovate] update label config 2020-12-04 12:23:47 -07:00
Thom Heymann f413957827
ECS audit events for alerting (#84113)
* ECS audit events for alerts plugin

* added api changes

* fixed linting and testing errors

* fix test

* Fixed linting errors after prettier update

* Revert "Allow predefined ids for encrypted saved objects (#83482)"

This reverts commit 7d929fe903.

* Added suggestions from code review

* Fixed unit tests

* Added suggestions from code review

* Changed names of alert events

* Changed naming as suggested in code review

* Added suggestions from PR

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 19:13:30 +00:00
Tiago Costa 6be3cf4f9f
chore(NA): check git version on pre-commit hook install (#84811)
* chore(NA): checks installed git version when installing pre-commit hook

* chore(NA): throw an error instead of log a warning

* chore(NA): use createFailError instead

* fix(NA): apply feedback from pr review for isCorrectGitVersionInstalled

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 19:10:33 +00:00
renovate[bot] 81b45ee9d0
Update dependency @elastic/charts to v24.3.0 (#85039) 2020-12-04 13:05:07 -06:00
Henry Harding ba428fc2d9
Fleet agent details design review (#84939)
* small design fixs for integrations list

* use tooltip for upgrade available

* remove enrollment token info

* remove border-bottom from last table row

* Fix type issue

Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 13:47:45 -05:00
Chris Roberson 58f90280cb
[Monitoring] Convert APM-related server files that read from _source to typescript (#84829)
* get_apms converted

* More APM ones

* get_beat_summary

* Fix test

* This is optional

* Fix tests

* Be more safe
2020-12-04 13:36:09 -05:00
igoristic 4545f56a46
[Monitoring][Alerting] Added core features to Kibana services (#84486)
* added core features to kibana services

* Added test for alert form

* Added mocking of legacy shims and actions
2020-12-04 13:33:47 -05:00
Marta Bondyra 2cf4e72394
[Lens] (Accessibility) Added focus state and accessible name to suggestions (#84653)
* [Lens] (Accessibility) Added focus state and accessible name to suggestions

* Apply suggestions from code review

* Update x-pack/plugins/lens/public/editor_frame_service/editor_frame/suggestion_panel.tsx

padding oops

* cr
2020-12-04 19:28:33 +01:00
Scotty Bollinger 7310ea7f3c
[Workplace Search] Cleanup a couple of minor sources issues. (#84961)
* Set queued message instead of immediate message

After being redirected from an oauth configuration, a redirect occurs to show the flash message. A queued message is needed here because the message is lost before the redirect happens otherwise

* Don’t pass empty query params

The kibana server didn’t like the empty param so this commit removes it

Before:
/status?

After
/status

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 11:38:03 -06:00
Tim Sullivan 880c8d35e8
[Send To Background UI] Isolate functional test for wip feature (#84833)
* [Send To Background UI] Isolate functional test for wip feature

* add tests to cigroup 3
2020-12-04 10:22:26 -07:00
Jonathan Budzenski 0dbe1cd006
[deb/rpm] Build aarch64 distributions (#84364)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 10:59:49 -06:00
Chandler Prall 2f72c8ad10
Upgrade EUI to v30.5.1 (#84677)
* Updated to eui@30.4.1, fixed types and unit tests

* Cleanup some imports

* Fix a text color swap, now back to danger text

* Bump EUI to v30.4.2

* Revert snapshot changes from ownFocus modification

* Clean up alert flyout test actions to better represent user actions

* Upgrade EUI to 30.5.1

* More accurate test interaction

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 09:39:03 -07:00
Jason Stoltzfus 40e206e587
[App Search] Added the Documents View (#83947) 2020-12-04 11:17:10 -05:00