Spencer
e3856a9e45
[security/apiKeys] migrate from id
to ids
in ES request ( #87053 )
...
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-01-04 15:46:37 -07:00
Kevin Logan
5e5a410204
[Fleet] add leading dot to hidden indices ( #86277 )
...
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: John Schulz <github.com@jfsiii.org>
2021-01-04 10:50:54 -05:00
Tiago Costa
4c3c2ac156
skip failing es promotion suite ( #87166 )
2021-01-04 15:23:18 +00:00
Matthew Kime
c129f93083
[index patterns] Fleep app - Keep saved object field list until field caps provides fields ( #85370 )
2020-12-19 21:56:06 -06:00
Sandra Gonzales
7ae4f2d572
remove test that checks for data stream backing index names ( #86052 )
2020-12-15 20:34:55 -05:00
John Schulz
ec4fbf2901
[Fleet][EPM] Revert es-storage-related commits ( #85942 )
...
* Revert "[Fleet][EPM] Save installed package assets in ES (#83391 )"
This reverts commit 81a340e681
.
* Revert 00c2e96
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-15 15:11:46 -05:00
Silvia Mitter
98308f8680
[Fleet] Change permissions for Fleet enroll role ( #85802 )
...
* Add APM traces index names to Fleet enroll role
* Removes fleet permissions for `events-*` as they became obsolete
fixes #85761
2020-12-15 09:31:39 +01:00
nnamdifrankie
23c5daa622
[Fleet] add ilm policy per data stream ( #85492 )
...
Co-authored-by: kevinlog <kevin.logan@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-12-14 16:15:55 -05:00
Nicolas Chaulet
943bce1512
[Fleet] Enforce superuser role for all fleet APIs ( #85136 )
2020-12-08 17:21:45 -05:00
John Schulz
81a340e681
[Fleet][EPM] Save installed package assets in ES ( #83391 )
...
## Summary
Store package assets (from Registry or local upload) in Elasticsearch. Related to proposal [issue](https://github.com/elastic/kibana/issues/83426 ) & [document](https://docs.google.com/document/d/18XoS6CSl9UxxPPBt9LXuJngf1Jv-4tl3jY6l19U1yH8 )
* New `epm-packages-assets` saved objects are stored on `.kibana` index, like our existing saved object `epm-packages`
* Asset id is uuid v5 based on the package name, package version & file path. See 1974324
* Add a list of IDs of all the installed assets, to `epm-packages` saved object. Like the existing `installed_` properties. [Example](https://github.com/elastic/kibana/pull/83391/files#diff-fa07cac51b6a49bf1e4824bc2250c9a77dac6c7d6b0a56020f559ef1ff9be25fR491-R512 ) from a test
<details><summary>Mapping for new Saved Object</summary>
37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fsaved_objects%2Findex.ts (L329-L339)
</details>
<details><summary>Additional property on existing <code>epm-packages</code> Saved Object</summary>
c4f27ab257/x-pack/plugins/fleet/server/saved_objects/index.ts (L306-L312)
I don't think the saved object changes are strictly required. It can be removed without changing much about how things work
- Pros:
- Preserves accurate record of the assets added at installation time. Separates what assets are currently available for package-version from what was installed. They _should_ be the same, but things happen.
- Avoids a query to get the installed assets before operating on them
- Cons:
- size/noise? Could be tens or hundreds of ids
- migration?
</details>
### More details
**When are saved objects added?**
During installation, after all other actions have succeeded, just before marking the save object as installed, we commit all the files from the package to ES
37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2F_install_package.ts (L193-L198)
**When are documents removed from the index?**
In the `removeInstallation` function which is called in response to a `DELETE /api/fleet/epm/packages/pkgkey`
37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Fremove.ts (L72)
or a failed package (re-)installation
bf068739ac/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Finstall.ts (L145)
**How are we using these assets?**
We're not, currently. Here's an example showing how we could update [`getFileHandler`](514b50e4c2/x-pack%2Fplugins%2Ffleet%2Fserver%2Froutes%2Fepm%2Fhandlers.ts (L101)
) to check for local assets before reaching out to the Registry if we wished. It's not DRY, but it does work
```typescript
const esDocRoot = `http://elastic:changeme@localhost:9200/${PACKAGE_ASSETS_INDEX_NAME}/_doc `;
const escapedDocId = encodeURIComponent(`${pkgName}-${pkgVersion}/${filePath}`);
const esRes = await fetch(`${esDocRoot}/${escapedDocId}`);
const esJson = await esRes.json();
if (esJson.found) {
const asset: PackageAsset = esJson._source;
const body = asset.data_utf8 || Buffer.from(asset.data_base64, 'base64');
return response.ok({
body,
headers: {
'content-type': asset.media_type,
// should add our own `cache-control` header here
// kibana default is prevents caching: `private, no-cache, no-store, must-revalidate`
// https://github.com/elastic/kibana/issues/83631
},
});
}
```
### Checklist
_updated tests to include new saved object output, no tests added yet_
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html ) were updated or added to match the most common scenarios
2020-12-07 15:11:09 -05:00
John Schulz
ef7367ddb3
[Fleet][EPM] Add TS type for package-spec. Clarify EPR relationship with spec ( #84946 )
...
## Summary
* Create a [TS type for `package-spec`](09cce235de/x-pack/plugins/fleet/common/types/models/package_spec.ts
) to differentiate Registry shape from spec shape
* Clarify EPR types relationship with package spec. i.e. what required properties it makes optional, what properties it adds, which property types it changes, etc.
5f2c4a5547/x-pack/plugins/fleet/common/types/models/epm.ts (L71-L80)
* Updated `manifest.yml` in `apache_invalid_manifest_missing_field_0.1.4.zip`. Previously it was missing the `type` property, but that's optional [according to package-spec](3b4d820755/versions/1/manifest.spec.yml (L241-L248)
). Removed the required `format_version` and re-zipped.
<details><summary>screenshots show errors if catches in a mock PackageInfo object</summary>
using `requirement`; not `conditions`
<img width="1084" alt="Screen Shot 2020-12-02 at 12 07 15 PM" src="https://user-images.githubusercontent.com/57655/101083986-4b915f80-357b-11eb-84c6-14605e1ea3d8.png ">
using `versions`; not `version`
<img width="1395" alt="Screen Shot 2020-12-02 at 12 08 13 PM" src="https://user-images.githubusercontent.com/57655/101083988-4c29f600-357b-11eb-9ff2-2b73e313c130.png ">
including `elasticsearch` when spec says it' not valid yet
<img width="1589" alt="Screen Shot 2020-12-02 at 12 08 36 PM" src="https://user-images.githubusercontent.com/57655/101083990-4c29f600-357b-11eb-8250-8926f7189af8.png ">
</details>
<details><summary>screenshots showing editor autocomplete for registry response values like <code>categories</code></summary>
<img width="422" alt="Screen Shot 2020-12-02 at 12 25 11 PM" src="https://user-images.githubusercontent.com/57655/101083994-4cc28c80-357b-11eb-9013-ae208f7c311a.png ">
<img width="345" alt="Screen Shot 2020-12-02 at 12 25 01 PM" src="https://user-images.githubusercontent.com/57655/101083991-4c29f600-357b-11eb-9468-bbb9d27513b1.png ">
</details>
<details><summary>screenshot showing type check preventing adding properties which aren't explicitly listed as "additions"</summary>
<img width="1295" alt="Screen Shot 2020-12-03 at 11 38 43 AM" src="https://user-images.githubusercontent.com/57655/101083997-4cc28c80-357b-11eb-83b9-206b85521f11.png ">
</details>
### Checklist
- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html ) were updated or added to match the most common scenarios
2020-12-07 09:30:27 -05:00
Sandra Gonzales
a5dd5b6998
[Fleet] EPM support to handle uploaded file paths ( #84708 )
...
* modify file route to handle uploaded packge file paths
* update messaging
* improve tests
* fix bug and add test to check the version of the uploaded package before failing
* fix similar bug for getting package info from registry when a different version is uploaded
2020-12-02 18:33:27 -05:00
Nicolas Chaulet
9b74fe6d39
[Fleet] Handler api key creation errors when Fleet Admin is invalid ( #84576 )
2020-12-01 12:55:15 -05:00
Sandra Gonzales
4e9afeebd5
[Fleet] index patterns to handle uploaded packages ( #83994 )
...
* fix index patterns, remove install_source from cache key, add missing test file
* remove unused function
* fix types
* fix comment
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-23 08:29:13 -05:00
Sandra Gonzales
39291e16b5
[Fleet] update getPackageInfo to handle uploaded packages ( #83854 )
...
* update getPackgeInfo handler to fetch from install source
* add tests and modify fixtures to distinguish between registry and uploaded package
* improve error handling
* fix type
* fix test
* remove try/catch
* fix zip file test to have the right number of assets
* fix compressed files
2020-11-20 11:40:45 -05:00
Nicolas Chaulet
c1a263cff4
[Fleet] Allow to send SETTINGS action ( #83707 )
2020-11-20 09:21:24 -05:00
Nicolas Chaulet
1b6cfe819d
[Fleet] Rename ingestManager plugin ID fleet ( #83200 )
2020-11-19 08:43:14 -05:00
Nicolas Chaulet
fed9a4fddc
[Fleet] Rename ingest_manager_api_integration tests fleet_api_integration ( #83011 )
2020-11-12 13:50:59 -05:00