Commit graph

67 commits

Author SHA1 Message Date
John Schulz
ec4fbf2901
[Fleet][EPM] Revert es-storage-related commits (#85942)
* Revert "[Fleet][EPM] Save installed package assets in ES (#83391)"

This reverts commit 81a340e681.

* Revert 00c2e96

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-15 15:11:46 -05:00
Nicolas Chaulet
724b90e01b
[Fleet] Fix agent details logs on safari (#85884) 2020-12-15 09:10:17 -05:00
Silvia Mitter
98308f8680
[Fleet] Change permissions for Fleet enroll role (#85802)
* Add APM traces index names to Fleet enroll role

* Removes fleet permissions for `events-*` as they became obsolete

fixes #85761
2020-12-15 09:31:39 +01:00
Paul Tavares
f6cd2648af
[Security_Solution][Endpoint] Register Custom tab into Fleet Endpoint Integration Detail (#85643)
* Fleet: add component props to the Package Custom UI extension
* Endpoint: Register UI Extension with fleet for endpoint custom content
* Endpoint: UI for Trusted Apps custom entry
2020-12-14 17:09:34 -05:00
nnamdifrankie
23c5daa622
[Fleet] add ilm policy per data stream (#85492)
Co-authored-by: kevinlog <kevin.logan@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-14 16:15:55 -05:00
Kevin Logan
06993c469b
[Fleet] Installation of hidden field (#85703)
Co-authored-by: nnamdifrankie <franklin.ejoh@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-14 13:45:56 -05:00
Sonja Krause-Harder
1b3a1bb385
Don't rollback on saved objects conflict errors. (#85131) 2020-12-14 17:36:25 +01:00
Larry Gregory
96bb72f68d
Fix fleet route protections (#85626)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-13 12:40:50 -05:00
Candace Park
4e4e55029c
[Security Solution][Endpoint][Admin] Api guard for paid policy features (#84667) 2020-12-10 18:53:55 -05:00
John Schulz
1c36ab9236
[Fleet][EPM] Move SO work from getFileHandler to service method (#85594)
## Summary

 - Move registry URL details from handler to Registry service 
 - rename `archive/save_to_es` to `archive/storage` to better reflect its role
 - Move SO handling from http into new  `getAsset` method in `archive/storage`
 - Rearrange code in HTTP handler to be more easily moved to one or two service method(s) which deal with the logic for getting the asset.


### Checklist

- [ ] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
2020-12-10 13:28:41 -05:00
John Schulz
00c2e960b6
[Fleet][EPM] - Do cache || saved_object || network in http handler (#85331)
## Summary

1. cf0d567 - Export function which maps file path to saved object id
2. e33d7d4 - "make it work" ™️ quality version of "use cached or saved object or registry" file in HTTP handler. It's doing too much and should eventually move the logic to service method(s).
2020-12-10 11:23:03 -05:00
Nicolas Chaulet
c9b5ec7303
[Fleet] Update agent listing for better status reporting (#84798) 2020-12-09 21:07:31 -05:00
Paul Tavares
6c52ac84c6
[FLEET] New Integration Policy Details page for use in Integrations section (#85355)
* new UI route to show Edit Package Policy page
* Package policy List items point to new Integration Policy details page
* Refactor to use common service to generate pkgKey
* add breadcrumb for edit policy under integrations
2020-12-09 17:50:33 -05:00
Nicolas Chaulet
2355dde1e9
[Fleet] Support editing bool variable in agent policy (#85070) 2020-12-09 14:50:51 -05:00
Aleh Zasypkin
88e61a6651
Migrate API keys functionality to a new Elasticsearch client. (#85029) 2020-12-09 20:43:24 +01:00
Nicolas Chaulet
a264038b36
[Fleet] Allow to edit package policy with input level template (#85277) 2020-12-09 09:10:07 -05:00
Nicolas Chaulet
943bce1512
[Fleet] Enforce superuser role for all fleet APIs (#85136) 2020-12-08 17:21:45 -05:00
Paul Tavares
e74cb409c8
[Fleet] Return to Integration (package) details after create integration policy (#85177)
* useIntraAppState() now also supports state set via Fleet's HashRouter
* Remove use of `<Router>` from inside EPM pages
* Enable round-trip navigation for Integrations add package
2020-12-08 15:08:54 -05:00
John Schulz
81a340e681
[Fleet][EPM] Save installed package assets in ES (#83391)
## Summary
Store package assets (from Registry or local upload) in Elasticsearch. Related to proposal [issue](https://github.com/elastic/kibana/issues/83426) & [document](https://docs.google.com/document/d/18XoS6CSl9UxxPPBt9LXuJngf1Jv-4tl3jY6l19U1yH8)

 * New `epm-packages-assets` saved objects are stored on `.kibana` index, like our existing saved object `epm-packages`
 * Asset id is uuid v5 based on the package name, package version & file path. See 1974324
 * Add a list of IDs of all the installed assets, to `epm-packages` saved object. Like the existing `installed_` properties.  [Example](https://github.com/elastic/kibana/pull/83391/files#diff-fa07cac51b6a49bf1e4824bc2250c9a77dac6c7d6b0a56020f559ef1ff9be25fR491-R512) from a test

<details><summary>Mapping for new Saved Object</summary>

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fsaved_objects%2Findex.ts (L329-L339)
</details>

<details><summary>Additional property on existing <code>epm-packages</code> Saved Object</summary>

c4f27ab257/x-pack/plugins/fleet/server/saved_objects/index.ts (L306-L312)

 I don't think the saved object changes are strictly required. It can be removed without changing much about how things work

- Pros: 
      - Preserves accurate record of the assets added at installation time. Separates what assets are currently available for package-version from what was installed. They _should_ be the same, but things happen.
      - Avoids a query to get the installed assets before operating on them
- Cons:
      - size/noise? Could be tens or hundreds of ids
      - migration?
</details>

### More details

**When are saved objects added?**
During installation, after all other actions have succeeded, just before marking the save object as installed, we commit all the files from the package to ES

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2F_install_package.ts (L193-L198)

**When are documents removed from the index?**

In the `removeInstallation` function which is called in response to a `DELETE /api/fleet/epm/packages/pkgkey`

37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Fremove.ts (L72)

or a failed package (re-)installation

bf068739ac/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Finstall.ts (L145)




**How are we using these assets?**
We're not, currently. Here's an example showing how we could update [`getFileHandler`](514b50e4c2/x-pack%2Fplugins%2Ffleet%2Fserver%2Froutes%2Fepm%2Fhandlers.ts (L101)) to check for local assets before reaching out to the Registry if we wished. It's not DRY, but it does work

```typescript
const esDocRoot = `http://elastic:changeme@localhost:9200/${PACKAGE_ASSETS_INDEX_NAME}/_doc`;
const escapedDocId = encodeURIComponent(`${pkgName}-${pkgVersion}/${filePath}`);
const esRes = await fetch(`${esDocRoot}/${escapedDocId}`);
const esJson = await esRes.json();
if (esJson.found) {
  const asset: PackageAsset = esJson._source;
  const body = asset.data_utf8 || Buffer.from(asset.data_base64, 'base64');
  return response.ok({
    body,
    headers: {
      'content-type': asset.media_type,
      // should add our own `cache-control` header here
      // kibana default is prevents caching: `private, no-cache, no-store, must-revalidate`
      // https://github.com/elastic/kibana/issues/83631
    },
  });
}
```

### Checklist
_updated tests to include new saved object output, no tests added yet_
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2020-12-07 15:11:09 -05:00
John Schulz
ef7367ddb3
[Fleet][EPM] Add TS type for package-spec. Clarify EPR relationship with spec (#84946)
## Summary

 * Create a [TS type for `package-spec`](09cce235de/x-pack/plugins/fleet/common/types/models/package_spec.ts) to differentiate Registry shape from spec shape
 * Clarify EPR types relationship with package spec. i.e. what required properties it makes optional, what properties it adds, which property types it changes, etc.
5f2c4a5547/x-pack/plugins/fleet/common/types/models/epm.ts (L71-L80)
  * Updated `manifest.yml` in `apache_invalid_manifest_missing_field_0.1.4.zip`. Previously it was missing the `type` property, but that's optional [according to package-spec](3b4d820755/versions/1/manifest.spec.yml (L241-L248)). Removed the required `format_version` and re-zipped.

<details><summary>screenshots show errors if catches in a mock PackageInfo object</summary>
using `requirement`; not `conditions`
<img width="1084" alt="Screen Shot 2020-12-02 at 12 07 15 PM" src="https://user-images.githubusercontent.com/57655/101083986-4b915f80-357b-11eb-84c6-14605e1ea3d8.png">
using `versions`; not `version`
<img width="1395" alt="Screen Shot 2020-12-02 at 12 08 13 PM" src="https://user-images.githubusercontent.com/57655/101083988-4c29f600-357b-11eb-9ff2-2b73e313c130.png">
including `elasticsearch` when spec says it' not valid yet
<img width="1589" alt="Screen Shot 2020-12-02 at 12 08 36 PM" src="https://user-images.githubusercontent.com/57655/101083990-4c29f600-357b-11eb-8250-8926f7189af8.png">
</details>

<details><summary>screenshots showing editor autocomplete for registry response values like <code>categories</code></summary>

<img width="422" alt="Screen Shot 2020-12-02 at 12 25 11 PM" src="https://user-images.githubusercontent.com/57655/101083994-4cc28c80-357b-11eb-9013-ae208f7c311a.png">

<img width="345" alt="Screen Shot 2020-12-02 at 12 25 01 PM" src="https://user-images.githubusercontent.com/57655/101083991-4c29f600-357b-11eb-9468-bbb9d27513b1.png">
</details>

<details><summary>screenshot showing type check preventing adding properties which aren't explicitly listed as "additions"</summary>

<img width="1295" alt="Screen Shot 2020-12-03 at 11 38 43 AM" src="https://user-images.githubusercontent.com/57655/101083997-4cc28c80-357b-11eb-83b9-206b85521f11.png">
</details>


### Checklist


- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2020-12-07 09:30:27 -05:00
Henry Harding
ba428fc2d9
Fleet agent details design review (#84939)
* small design fixs for integrations list

* use tooltip for upgrade available

* remove enrollment token info

* remove border-bottom from last table row

* Fix type issue

Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 13:47:45 -05:00
Chandler Prall
2f72c8ad10
Upgrade EUI to v30.5.1 (#84677)
* Updated to eui@30.4.1, fixed types and unit tests

* Cleanup some imports

* Fix a text color swap, now back to danger text

* Bump EUI to v30.4.2

* Revert snapshot changes from ownFocus modification

* Clean up alert flyout test actions to better represent user actions

* Upgrade EUI to 30.5.1

* More accurate test interaction

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-04 09:39:03 -07:00
Sandra Gonzales
db70286bca
[Fleet] add readme to uploaded package info and fix images (#84944)
* use the relative src instead of path for package images

* add readme to package info for uploaded packages

* let toPackageImage take the image object and check path if no src

* use variable
2020-12-04 10:58:04 -05:00
Sandra Gonzales
a5dd5b6998
[Fleet] EPM support to handle uploaded file paths (#84708)
* modify file route to handle uploaded packge file paths

* update messaging

* improve tests

* fix bug and add test to check the version of the uploaded package before failing

* fix similar bug for getting package info from registry when a different version is uploaded
2020-12-02 18:33:27 -05:00
Tyler Smalley
b593781009
Jest multi-project configuration (#77894)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2020-12-02 11:42:23 -08:00
Paul Tavares
2ffdf75b6e
[SECURITY_SOLUTION] Enable usage of the Endpoint Policy form from Fleet (#84684)
* Endpoint: add `withSecurityContext` HOC + refactor endpoint policy edit lazy component to use it
* Endpoint: refactor Policy Details to separate form from view
* Endpoint: Enable the Redux store for the Policy form when displayed via Fleet
* Fleet: Allow partial package policy updates to be sent via `onChange()`
2020-12-02 14:37:36 -05:00
John Schulz
90a18cc15d
[Fleet][EPM] Pass through valid manifest values from upload (#84703)
* Add missing properties & improve type safety

* Break up types for better readability
2020-12-02 09:49:21 -05:00
Nicolas Chaulet
9b74fe6d39
[Fleet] Handler api key creation errors when Fleet Admin is invalid (#84576) 2020-12-01 12:55:15 -05:00
Sonja Krause-Harder
c9fc876da1
Return early when parallel install process detected (#84190) 2020-12-01 15:00:15 +01:00
Nicolas Chaulet
0ba7d9ca1a
[Fleet] Update agent details page (#84434) 2020-12-01 08:46:57 -05:00
Paul Tavares
39ceadd96d
[Fleet] Fix initialization of history instance provided to react-router (#84585) 2020-12-01 07:55:48 -05:00
Paul Tavares
7ba6c73a71
[Fleet] Default Integration Policy Configuration section is only shown if no UI extension is registered (#84534)
* Do not render out-of-box integration policy configuration step if a custom UI extension is registered
* Remove endpoint specific logic from fleet and move it to UI extension
2020-11-30 14:17:19 -05:00
Paul Tavares
707dbcd2b1
[Fleet] Support for showing an Integration Detail Custom (UI Extension) tab (#83805)
* Support for rendering a custom component in Integration Details
* Refactor Fleet app initialization contexts in order to support testing setup
* New test rendering helper tool
* refactor Endpoint to use mock builder from Fleet
2020-11-30 11:12:39 -05:00
Jen Huang
459263fd59
[Fleet] Support URL query state in agent logs UI (#84298)
* Initial attempt at URL state

* Break into smaller files

* Handle invalid date range expressions
2020-11-25 13:29:57 -08:00
Jen Huang
284b1046df
[Fleet] Support input-level vars & templates (#83878)
* Fix bug creating new policy on the fly

* Adjust UI for input with vars but no streams

* Revert "Fix bug creating new policy on the fly"

This reverts commit 34f7014d69.

* Add `compiled_input` field and compile input template, if any. Make compilation method names more generic (instead of only for streams). Add testts

* Add compiled input to generated agent yaml

* Don't return empty streams in agent yaml when there aren't any

* Update missed assertion

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-25 10:13:36 -08:00
Nicolas Chaulet
5f53d856c0
[Fleet] Only display log level in Filter that are present in the logs (#84277) 2020-11-24 17:54:26 -05:00
Mikhail Shustov
5ec6fe315f
[DX] Bump TS version to v4.1 (#83397)
* bump version to 4.1.1-rc

* fix code to run kbn bootstrap

* fix errors

* DO NOT MERGE. mute errors and ping teams to fix them

* Address EuiSelectableProps configuration in discover sidebar

* use explicit type for EuiSelectable

* update to ts v4.1.2

* fix ts error in EuiSelectable

* update docs

* update prettier with ts version support

* Revert "update prettier with ts version support"

This reverts commit 3de48db3ec.

* address another new problem

Co-authored-by: Chandler Prall <chandler.prall@gmail.com>
2020-11-24 16:04:33 +01:00
Paul Tavares
9521ac4fed
[Fleet] Integration Policies List view (#83634)
* Show table with list of Integration polices
* hook to support list pagination that uses/persist to the URL
2020-11-24 09:43:21 -05:00
Nicolas Chaulet
e4ff981b94
[Ingest Manager] UI to configure agent log level (#84112) 2020-11-24 08:41:13 -05:00
Sonja Krause-Harder
aa07f5c1d0
[Fleet] Rename usage collection object to 'fleet'. (#83407)
* Rename usage collection object to 'fleet'.

* Update telemetry mapping.

* Adjust naming.

* Rename ingestManager -> fleet in telemetry collector
2020-11-24 10:11:58 +01:00
Jen Huang
cf255e478f
Fix error happening on inline create agent policy (#83993)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-23 12:53:02 -08:00
Jen Huang
3d5d99e037
Use new query bar props (#84108) 2020-11-23 10:14:57 -08:00
Jen Huang
97c9fc2699
Disallow dashes in namespace strings (#83996)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-23 09:51:50 -08:00
Sandra Gonzales
4e9afeebd5
[Fleet] index patterns to handle uploaded packages (#83994)
* fix index patterns, remove install_source from cache key, add missing test file

* remove unused function

* fix types

* fix comment

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-23 08:29:13 -05:00
Mikhail Shustov
95861a0fb0
[DX] Prettier v2.2 (#83899)
* update prettier with ts version support

* mute type-error

* run prettier on codebase

* fix examples

* fix errors after master merged
2020-11-23 13:17:05 +01:00
Dan Panzarella
2cd2528ac8
[Security Solution] Give notice when endpoint policy is out of date (#83469) 2020-11-20 15:21:23 -05:00
Jen Huang
a8913d30c1
[Fleet] Agent logs UI (#83356)
* Initial agent log table

* Add data and storage services to Kibana context, rename useCore to useStartServices and replace usage of useStartDeps

* Initial attempt at adding query bar for log stream

* Adjust app layout to allow page content to be full height

* Dataset and log level filters, split into smaller files

* Use data plugin's `QueryStringInput` component for query bar

* Add open in Logs UI link

* Clean up unused storage dependency

* Remove agent activity log and related dead code

* Clean up i18n

* Clean up plugin deps, fix routing

* Add back storage dependency that data plugin components need

* Remove dependency on infra for logs UI link

* Change default date range to last one day

* Adjust panel padding

* Add comment

* Move module declarations to top-level x-pack/typings

* Fix missed renaming

* Remove unused path

* Move building of log stream query into separate service with unit tests

* Adjust test conditions
2020-11-20 10:49:02 -08:00
Sandra Gonzales
39291e16b5
[Fleet] update getPackageInfo to handle uploaded packages (#83854)
* update getPackgeInfo handler to fetch from install source

* add tests and modify fixtures  to distinguish between registry and uploaded package

* improve error handling

* fix type

* fix test

* remove try/catch

* fix zip file test to have the right number of assets

* fix compressed files
2020-11-20 11:40:45 -05:00
Nicolas Chaulet
c1a263cff4
[Fleet] Allow to send SETTINGS action (#83707) 2020-11-20 09:21:24 -05:00
Jonathan Budzenski
6c23302b36
[cli] Add bin/kibana-encryption-keys (#82838)
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Tyler Smalley <tylersmalley@me.com>
2020-11-19 12:41:48 -06:00