[Fleet][EPM] Save installed package assets in ES (#83391)
## Summary Store package assets (from Registry or local upload) in Elasticsearch. Related to proposal [issue](https://github.com/elastic/kibana/issues/83426) & [document](https://docs.google.com/document/d/18XoS6CSl9UxxPPBt9LXuJngf1Jv-4tl3jY6l19U1yH8) * New `epm-packages-assets` saved objects are stored on `.kibana` index, like our existing saved object `epm-packages` * Asset id is uuid v5 based on the package name, package version & file path. See1974324
* Add a list of IDs of all the installed assets, to `epm-packages` saved object. Like the existing `installed_` properties. [Example](https://github.com/elastic/kibana/pull/83391/files#diff-fa07cac51b6a49bf1e4824bc2250c9a77dac6c7d6b0a56020f559ef1ff9be25fR491-R512) from a test <details><summary>Mapping for new Saved Object</summary>37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fsaved_objects%2Findex.ts (L329-L339)
</details> <details><summary>Additional property on existing <code>epm-packages</code> Saved Object</summary>c4f27ab257/x-pack/plugins/fleet/server/saved_objects/index.ts (L306-L312)
I don't think the saved object changes are strictly required. It can be removed without changing much about how things work - Pros: - Preserves accurate record of the assets added at installation time. Separates what assets are currently available for package-version from what was installed. They _should_ be the same, but things happen. - Avoids a query to get the installed assets before operating on them - Cons: - size/noise? Could be tens or hundreds of ids - migration? </details> ### More details **When are saved objects added?** During installation, after all other actions have succeeded, just before marking the save object as installed, we commit all the files from the package to ES37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2F_install_package.ts (L193-L198)
**When are documents removed from the index?** In the `removeInstallation` function which is called in response to a `DELETE /api/fleet/epm/packages/pkgkey`37f7b6ded7/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Fremove.ts (L72)
or a failed package (re-)installationbf068739ac/x-pack%2Fplugins%2Ffleet%2Fserver%2Fservices%2Fepm%2Fpackages%2Finstall.ts (L145)
**How are we using these assets?** We're not, currently. Here's an example showing how we could update [`getFileHandler`](514b50e4c2/x-pack%2Fplugins%2Ffleet%2Fserver%2Froutes%2Fepm%2Fhandlers.ts (L101)
) to check for local assets before reaching out to the Registry if we wished. It's not DRY, but it does work ```typescript const esDocRoot = `http://elastic:changeme@localhost:9200/${PACKAGE_ASSETS_INDEX_NAME}/_doc`; const escapedDocId = encodeURIComponent(`${pkgName}-${pkgVersion}/${filePath}`); const esRes = await fetch(`${esDocRoot}/${escapedDocId}`); const esJson = await esRes.json(); if (esJson.found) { const asset: PackageAsset = esJson._source; const body = asset.data_utf8 || Buffer.from(asset.data_base64, 'base64'); return response.ok({ body, headers: { 'content-type': asset.media_type, // should add our own `cache-control` header here // kibana default is prevents caching: `private, no-cache, no-store, must-revalidate` // https://github.com/elastic/kibana/issues/83631 }, }); } ``` ### Checklist _updated tests to include new saved object output, no tests added yet_ - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
This commit is contained in:
parent
f961e90ea7
commit
81a340e681
|
@ -3,8 +3,8 @@
|
||||||
* or more contributor license agreements. Licensed under the Elastic License;
|
* or more contributor license agreements. Licensed under the Elastic License;
|
||||||
* you may not use this file except in compliance with the Elastic License.
|
* you may not use this file except in compliance with the Elastic License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
export const PACKAGES_SAVED_OBJECT_TYPE = 'epm-packages';
|
export const PACKAGES_SAVED_OBJECT_TYPE = 'epm-packages';
|
||||||
|
export const ASSETS_SAVED_OBJECT_TYPE = 'epm-packages-assets';
|
||||||
export const INDEX_PATTERN_SAVED_OBJECT_TYPE = 'index-pattern';
|
export const INDEX_PATTERN_SAVED_OBJECT_TYPE = 'index-pattern';
|
||||||
export const INDEX_PATTERN_PLACEHOLDER_SUFFIX = '-index_pattern_placeholder';
|
export const INDEX_PATTERN_PLACEHOLDER_SUFFIX = '-index_pattern_placeholder';
|
||||||
export const MAX_TIME_COMPLETE_INSTALL = 60000;
|
export const MAX_TIME_COMPLETE_INSTALL = 60000;
|
||||||
|
|
|
@ -8,6 +8,7 @@
|
||||||
// TODO: Update when https://github.com/elastic/kibana/issues/53021 is closed
|
// TODO: Update when https://github.com/elastic/kibana/issues/53021 is closed
|
||||||
import { SavedObject, SavedObjectAttributes, SavedObjectReference } from 'src/core/public';
|
import { SavedObject, SavedObjectAttributes, SavedObjectReference } from 'src/core/public';
|
||||||
import {
|
import {
|
||||||
|
ASSETS_SAVED_OBJECT_TYPE,
|
||||||
agentAssetTypes,
|
agentAssetTypes,
|
||||||
dataTypes,
|
dataTypes,
|
||||||
defaultPackages,
|
defaultPackages,
|
||||||
|
@ -264,6 +265,7 @@ export type PackageInfo =
|
||||||
export interface Installation extends SavedObjectAttributes {
|
export interface Installation extends SavedObjectAttributes {
|
||||||
installed_kibana: KibanaAssetReference[];
|
installed_kibana: KibanaAssetReference[];
|
||||||
installed_es: EsAssetReference[];
|
installed_es: EsAssetReference[];
|
||||||
|
package_assets: PackageAssetReference[];
|
||||||
es_index_patterns: Record<string, string>;
|
es_index_patterns: Record<string, string>;
|
||||||
name: string;
|
name: string;
|
||||||
version: string;
|
version: string;
|
||||||
|
@ -293,6 +295,10 @@ export type EsAssetReference = Pick<SavedObjectReference, 'id'> & {
|
||||||
type: ElasticsearchAssetType;
|
type: ElasticsearchAssetType;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
export type PackageAssetReference = Pick<SavedObjectReference, 'id'> & {
|
||||||
|
type: typeof ASSETS_SAVED_OBJECT_TYPE;
|
||||||
|
};
|
||||||
|
|
||||||
export type RequiredPackage = typeof requiredPackages;
|
export type RequiredPackage = typeof requiredPackages;
|
||||||
|
|
||||||
export type DefaultPackages = typeof defaultPackages;
|
export type DefaultPackages = typeof defaultPackages;
|
||||||
|
|
|
@ -41,6 +41,7 @@ export {
|
||||||
PACKAGE_POLICY_SAVED_OBJECT_TYPE,
|
PACKAGE_POLICY_SAVED_OBJECT_TYPE,
|
||||||
OUTPUT_SAVED_OBJECT_TYPE,
|
OUTPUT_SAVED_OBJECT_TYPE,
|
||||||
PACKAGES_SAVED_OBJECT_TYPE,
|
PACKAGES_SAVED_OBJECT_TYPE,
|
||||||
|
ASSETS_SAVED_OBJECT_TYPE,
|
||||||
INDEX_PATTERN_SAVED_OBJECT_TYPE,
|
INDEX_PATTERN_SAVED_OBJECT_TYPE,
|
||||||
ENROLLMENT_API_KEYS_SAVED_OBJECT_TYPE,
|
ENROLLMENT_API_KEYS_SAVED_OBJECT_TYPE,
|
||||||
GLOBAL_SETTINGS_SAVED_OBJECT_TYPE,
|
GLOBAL_SETTINGS_SAVED_OBJECT_TYPE,
|
||||||
|
|
|
@ -12,6 +12,7 @@ import {
|
||||||
AGENT_POLICY_SAVED_OBJECT_TYPE,
|
AGENT_POLICY_SAVED_OBJECT_TYPE,
|
||||||
PACKAGE_POLICY_SAVED_OBJECT_TYPE,
|
PACKAGE_POLICY_SAVED_OBJECT_TYPE,
|
||||||
PACKAGES_SAVED_OBJECT_TYPE,
|
PACKAGES_SAVED_OBJECT_TYPE,
|
||||||
|
ASSETS_SAVED_OBJECT_TYPE,
|
||||||
AGENT_SAVED_OBJECT_TYPE,
|
AGENT_SAVED_OBJECT_TYPE,
|
||||||
AGENT_EVENT_SAVED_OBJECT_TYPE,
|
AGENT_EVENT_SAVED_OBJECT_TYPE,
|
||||||
AGENT_ACTION_SAVED_OBJECT_TYPE,
|
AGENT_ACTION_SAVED_OBJECT_TYPE,
|
||||||
|
@ -304,6 +305,13 @@ const getSavedObjectTypes = (
|
||||||
type: { type: 'keyword' },
|
type: { type: 'keyword' },
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
package_assets: {
|
||||||
|
type: 'nested',
|
||||||
|
properties: {
|
||||||
|
id: { type: 'keyword' },
|
||||||
|
type: { type: 'keyword' },
|
||||||
|
},
|
||||||
|
},
|
||||||
install_started_at: { type: 'date' },
|
install_started_at: { type: 'date' },
|
||||||
install_version: { type: 'keyword' },
|
install_version: { type: 'keyword' },
|
||||||
install_status: { type: 'keyword' },
|
install_status: { type: 'keyword' },
|
||||||
|
@ -311,6 +319,25 @@ const getSavedObjectTypes = (
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
[ASSETS_SAVED_OBJECT_TYPE]: {
|
||||||
|
name: ASSETS_SAVED_OBJECT_TYPE,
|
||||||
|
hidden: false,
|
||||||
|
namespaceType: 'agnostic',
|
||||||
|
management: {
|
||||||
|
importableAndExportable: false,
|
||||||
|
},
|
||||||
|
mappings: {
|
||||||
|
properties: {
|
||||||
|
package_name: { type: 'keyword' },
|
||||||
|
package_version: { type: 'keyword' },
|
||||||
|
install_source: { type: 'keyword' },
|
||||||
|
asset_path: { type: 'keyword' },
|
||||||
|
media_type: { type: 'keyword' },
|
||||||
|
data_utf8: { type: 'text', index: false },
|
||||||
|
data_base64: { type: 'binary' },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
export function registerSavedObjects(
|
export function registerSavedObjects(
|
||||||
|
|
121
x-pack/plugins/fleet/server/services/epm/archive/save_to_es.ts
Normal file
121
x-pack/plugins/fleet/server/services/epm/archive/save_to_es.ts
Normal file
|
@ -0,0 +1,121 @@
|
||||||
|
/*
|
||||||
|
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||||
|
* or more contributor license agreements. Licensed under the Elastic License;
|
||||||
|
* you may not use this file except in compliance with the Elastic License.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { extname } from 'path';
|
||||||
|
import { isBinaryFile } from 'isbinaryfile';
|
||||||
|
import mime from 'mime-types';
|
||||||
|
import uuidv5 from 'uuid/v5';
|
||||||
|
import { SavedObjectsClientContract, SavedObjectsBulkCreateObject } from 'src/core/server';
|
||||||
|
import {
|
||||||
|
ASSETS_SAVED_OBJECT_TYPE,
|
||||||
|
InstallablePackage,
|
||||||
|
InstallSource,
|
||||||
|
PackageAssetReference,
|
||||||
|
} from '../../../../common';
|
||||||
|
import { getArchiveEntry } from './index';
|
||||||
|
|
||||||
|
// uuid v5 requires a SHA-1 UUID as a namespace
|
||||||
|
// used to ensure same input produces the same id
|
||||||
|
const ID_NAMESPACE = '71403015-cdd5-404b-a5da-6c43f35cad84';
|
||||||
|
|
||||||
|
// could be anything, picked this from https://github.com/elastic/elastic-agent-client/issues/17
|
||||||
|
const MAX_ES_ASSET_BYTES = 4 * 1024 * 1024;
|
||||||
|
|
||||||
|
export interface PackageAsset {
|
||||||
|
package_name: string;
|
||||||
|
package_version: string;
|
||||||
|
install_source: string;
|
||||||
|
asset_path: string;
|
||||||
|
media_type: string;
|
||||||
|
data_utf8: string;
|
||||||
|
data_base64: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function archiveEntryToESDocument(opts: {
|
||||||
|
path: string;
|
||||||
|
buffer: Buffer;
|
||||||
|
name: string;
|
||||||
|
version: string;
|
||||||
|
installSource: InstallSource;
|
||||||
|
}): Promise<PackageAsset> {
|
||||||
|
const { path, buffer, name, version, installSource } = opts;
|
||||||
|
const fileExt = extname(path);
|
||||||
|
const contentType = mime.lookup(fileExt);
|
||||||
|
const mediaType = mime.contentType(contentType || fileExt);
|
||||||
|
// can use to create a data URL like `data:${mediaType};base64,${base64Data}`
|
||||||
|
|
||||||
|
const bufferIsBinary = await isBinaryFile(buffer);
|
||||||
|
const dataUtf8 = bufferIsBinary ? '' : buffer.toString('utf8');
|
||||||
|
const dataBase64 = bufferIsBinary ? buffer.toString('base64') : '';
|
||||||
|
|
||||||
|
// validation: filesize? asset type? anything else
|
||||||
|
if (dataUtf8.length > MAX_ES_ASSET_BYTES) {
|
||||||
|
throw new Error(`File at ${path} is larger than maximum allowed size of ${MAX_ES_ASSET_BYTES}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (dataBase64.length > MAX_ES_ASSET_BYTES) {
|
||||||
|
throw new Error(
|
||||||
|
`After base64 encoding file at ${path} is larger than maximum allowed size of ${MAX_ES_ASSET_BYTES}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
package_name: name,
|
||||||
|
package_version: version,
|
||||||
|
install_source: installSource,
|
||||||
|
asset_path: path,
|
||||||
|
media_type: mediaType || '',
|
||||||
|
data_utf8: dataUtf8,
|
||||||
|
data_base64: dataBase64,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function removeArchiveEntries(opts: {
|
||||||
|
savedObjectsClient: SavedObjectsClientContract;
|
||||||
|
refs: PackageAssetReference[];
|
||||||
|
}) {
|
||||||
|
const { savedObjectsClient, refs } = opts;
|
||||||
|
const results = await Promise.all(
|
||||||
|
refs.map((ref) => savedObjectsClient.delete(ASSETS_SAVED_OBJECT_TYPE, ref.id))
|
||||||
|
);
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function saveArchiveEntries(opts: {
|
||||||
|
savedObjectsClient: SavedObjectsClientContract;
|
||||||
|
paths: string[];
|
||||||
|
packageInfo: InstallablePackage;
|
||||||
|
installSource: InstallSource;
|
||||||
|
}) {
|
||||||
|
const { savedObjectsClient, paths, packageInfo, installSource } = opts;
|
||||||
|
const bulkBody = await Promise.all(
|
||||||
|
paths.map((path) => {
|
||||||
|
const buffer = getArchiveEntry(path);
|
||||||
|
if (!buffer) throw new Error(`Could not find ArchiveEntry at ${path}`);
|
||||||
|
const { name, version } = packageInfo;
|
||||||
|
return archiveEntryToBulkCreateObject({ path, buffer, name, version, installSource });
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
const results = await savedObjectsClient.bulkCreate<PackageAsset>(bulkBody);
|
||||||
|
return results;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function archiveEntryToBulkCreateObject(opts: {
|
||||||
|
path: string;
|
||||||
|
buffer: Buffer;
|
||||||
|
name: string;
|
||||||
|
version: string;
|
||||||
|
installSource: InstallSource;
|
||||||
|
}): Promise<SavedObjectsBulkCreateObject<PackageAsset>> {
|
||||||
|
const { path, buffer, name, version, installSource } = opts;
|
||||||
|
const doc = await archiveEntryToESDocument({ path, buffer, name, version, installSource });
|
||||||
|
return {
|
||||||
|
id: uuidv5(doc.asset_path, ID_NAMESPACE),
|
||||||
|
type: ASSETS_SAVED_OBJECT_TYPE,
|
||||||
|
attributes: doc,
|
||||||
|
};
|
||||||
|
}
|
|
@ -5,7 +5,13 @@
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { SavedObject, SavedObjectsClientContract } from 'src/core/server';
|
import { SavedObject, SavedObjectsClientContract } from 'src/core/server';
|
||||||
import { InstallablePackage, InstallSource, MAX_TIME_COMPLETE_INSTALL } from '../../../../common';
|
import {
|
||||||
|
InstallablePackage,
|
||||||
|
InstallSource,
|
||||||
|
PackageAssetReference,
|
||||||
|
MAX_TIME_COMPLETE_INSTALL,
|
||||||
|
ASSETS_SAVED_OBJECT_TYPE,
|
||||||
|
} from '../../../../common';
|
||||||
import { PACKAGES_SAVED_OBJECT_TYPE } from '../../../constants';
|
import { PACKAGES_SAVED_OBJECT_TYPE } from '../../../constants';
|
||||||
import {
|
import {
|
||||||
AssetReference,
|
AssetReference,
|
||||||
|
@ -23,6 +29,7 @@ import { updateCurrentWriteIndices } from '../elasticsearch/template/template';
|
||||||
import { deleteKibanaSavedObjectsAssets } from './remove';
|
import { deleteKibanaSavedObjectsAssets } from './remove';
|
||||||
import { installTransform } from '../elasticsearch/transform/install';
|
import { installTransform } from '../elasticsearch/transform/install';
|
||||||
import { createInstallation, saveKibanaAssetsRefs, updateVersion } from './install';
|
import { createInstallation, saveKibanaAssetsRefs, updateVersion } from './install';
|
||||||
|
import { saveArchiveEntries } from '../archive/save_to_es';
|
||||||
|
|
||||||
// this is only exported for testing
|
// this is only exported for testing
|
||||||
// use a leading underscore to indicate it's not the supported path
|
// use a leading underscore to indicate it's not the supported path
|
||||||
|
@ -177,12 +184,28 @@ export async function _installPackage({
|
||||||
if (installKibanaAssetsError) throw installKibanaAssetsError;
|
if (installKibanaAssetsError) throw installKibanaAssetsError;
|
||||||
await Promise.all([installKibanaAssetsPromise, installIndexPatternPromise]);
|
await Promise.all([installKibanaAssetsPromise, installIndexPatternPromise]);
|
||||||
|
|
||||||
|
const packageAssetResults = await saveArchiveEntries({
|
||||||
|
savedObjectsClient,
|
||||||
|
paths,
|
||||||
|
packageInfo,
|
||||||
|
installSource,
|
||||||
|
});
|
||||||
|
const packageAssetRefs: PackageAssetReference[] = packageAssetResults.saved_objects.map(
|
||||||
|
(result) => ({
|
||||||
|
id: result.id,
|
||||||
|
type: ASSETS_SAVED_OBJECT_TYPE,
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
// update to newly installed version when all assets are successfully installed
|
// update to newly installed version when all assets are successfully installed
|
||||||
if (installedPkg) await updateVersion(savedObjectsClient, pkgName, pkgVersion);
|
if (installedPkg) await updateVersion(savedObjectsClient, pkgName, pkgVersion);
|
||||||
|
|
||||||
await savedObjectsClient.update(PACKAGES_SAVED_OBJECT_TYPE, pkgName, {
|
await savedObjectsClient.update(PACKAGES_SAVED_OBJECT_TYPE, pkgName, {
|
||||||
install_version: pkgVersion,
|
install_version: pkgVersion,
|
||||||
install_status: 'installed',
|
install_status: 'installed',
|
||||||
|
package_assets: packageAssetRefs,
|
||||||
});
|
});
|
||||||
|
|
||||||
return [
|
return [
|
||||||
...installedKibanaAssetsRefs,
|
...installedKibanaAssetsRefs,
|
||||||
...installedPipelines,
|
...installedPipelines,
|
||||||
|
|
|
@ -43,6 +43,7 @@ const mockInstallation: SavedObject<Installation> = {
|
||||||
id: 'test-pkg',
|
id: 'test-pkg',
|
||||||
installed_kibana: [{ type: KibanaSavedObjectType.dashboard, id: 'dashboard-1' }],
|
installed_kibana: [{ type: KibanaSavedObjectType.dashboard, id: 'dashboard-1' }],
|
||||||
installed_es: [{ type: ElasticsearchAssetType.ingestPipeline, id: 'pipeline' }],
|
installed_es: [{ type: ElasticsearchAssetType.ingestPipeline, id: 'pipeline' }],
|
||||||
|
package_assets: [],
|
||||||
es_index_patterns: { pattern: 'pattern-name' },
|
es_index_patterns: { pattern: 'pattern-name' },
|
||||||
name: 'test package',
|
name: 'test package',
|
||||||
version: '1.0.0',
|
version: '1.0.0',
|
||||||
|
|
|
@ -15,6 +15,7 @@ const mockInstallation: SavedObject<Installation> = {
|
||||||
id: 'test-pkg',
|
id: 'test-pkg',
|
||||||
installed_kibana: [{ type: KibanaSavedObjectType.dashboard, id: 'dashboard-1' }],
|
installed_kibana: [{ type: KibanaSavedObjectType.dashboard, id: 'dashboard-1' }],
|
||||||
installed_es: [{ type: ElasticsearchAssetType.ingestPipeline, id: 'pipeline' }],
|
installed_es: [{ type: ElasticsearchAssetType.ingestPipeline, id: 'pipeline' }],
|
||||||
|
package_assets: [],
|
||||||
es_index_patterns: { pattern: 'pattern-name' },
|
es_index_patterns: { pattern: 'pattern-name' },
|
||||||
name: 'test packagek',
|
name: 'test packagek',
|
||||||
version: '1.0.0',
|
version: '1.0.0',
|
||||||
|
@ -32,6 +33,7 @@ const mockInstallationUpdateFail: SavedObject<Installation> = {
|
||||||
id: 'test-pkg',
|
id: 'test-pkg',
|
||||||
installed_kibana: [{ type: KibanaSavedObjectType.dashboard, id: 'dashboard-1' }],
|
installed_kibana: [{ type: KibanaSavedObjectType.dashboard, id: 'dashboard-1' }],
|
||||||
installed_es: [{ type: ElasticsearchAssetType.ingestPipeline, id: 'pipeline' }],
|
installed_es: [{ type: ElasticsearchAssetType.ingestPipeline, id: 'pipeline' }],
|
||||||
|
package_assets: [],
|
||||||
es_index_patterns: { pattern: 'pattern-name' },
|
es_index_patterns: { pattern: 'pattern-name' },
|
||||||
name: 'test packagek',
|
name: 'test packagek',
|
||||||
version: '1.0.0',
|
version: '1.0.0',
|
||||||
|
|
|
@ -379,6 +379,7 @@ export async function createInstallation(options: {
|
||||||
{
|
{
|
||||||
installed_kibana: [],
|
installed_kibana: [],
|
||||||
installed_es: [],
|
installed_es: [],
|
||||||
|
package_assets: [],
|
||||||
es_index_patterns: toSaveESIndexPatterns,
|
es_index_patterns: toSaveESIndexPatterns,
|
||||||
name: pkgName,
|
name: pkgName,
|
||||||
version: pkgVersion,
|
version: pkgVersion,
|
||||||
|
|
|
@ -23,6 +23,7 @@ import { deleteTransforms } from '../elasticsearch/transform/remove';
|
||||||
import { packagePolicyService, appContextService } from '../..';
|
import { packagePolicyService, appContextService } from '../..';
|
||||||
import { splitPkgKey } from '../registry';
|
import { splitPkgKey } from '../registry';
|
||||||
import { deletePackageCache } from '../archive';
|
import { deletePackageCache } from '../archive';
|
||||||
|
import { removeArchiveEntries } from '../archive/save_to_es';
|
||||||
|
|
||||||
export async function removeInstallation(options: {
|
export async function removeInstallation(options: {
|
||||||
savedObjectsClient: SavedObjectsClientContract;
|
savedObjectsClient: SavedObjectsClientContract;
|
||||||
|
@ -48,7 +49,7 @@ export async function removeInstallation(options: {
|
||||||
`unable to remove package with existing package policy(s) in use by agent(s)`
|
`unable to remove package with existing package policy(s) in use by agent(s)`
|
||||||
);
|
);
|
||||||
|
|
||||||
// Delete the installed assets
|
// Delete the installed assets. Don't include installation.package_assets. Those are irrelevant to users
|
||||||
const installedAssets = [...installation.installed_kibana, ...installation.installed_es];
|
const installedAssets = [...installation.installed_kibana, ...installation.installed_es];
|
||||||
await deleteAssets(installation, savedObjectsClient, callCluster);
|
await deleteAssets(installation, savedObjectsClient, callCluster);
|
||||||
|
|
||||||
|
@ -68,6 +69,8 @@ export async function removeInstallation(options: {
|
||||||
version: pkgVersion,
|
version: pkgVersion,
|
||||||
});
|
});
|
||||||
|
|
||||||
|
await removeArchiveEntries({ savedObjectsClient, refs: installation.package_assets });
|
||||||
|
|
||||||
// successful delete's in SO client return {}. return something more useful
|
// successful delete's in SO client return {}. return something more useful
|
||||||
return installedAssets;
|
return installedAssets;
|
||||||
}
|
}
|
||||||
|
|
|
@ -1337,6 +1337,7 @@ export class EndpointDocGenerator {
|
||||||
{ id: 'logs-endpoint.events.security', type: 'index_template' },
|
{ id: 'logs-endpoint.events.security', type: 'index_template' },
|
||||||
{ id: 'metrics-endpoint.telemetry', type: 'index_template' },
|
{ id: 'metrics-endpoint.telemetry', type: 'index_template' },
|
||||||
] as EsAssetReference[],
|
] as EsAssetReference[],
|
||||||
|
package_assets: [],
|
||||||
es_index_patterns: {
|
es_index_patterns: {
|
||||||
alerts: 'logs-endpoint.alerts-*',
|
alerts: 'logs-endpoint.alerts-*',
|
||||||
events: 'events-endpoint-*',
|
events: 'events-endpoint-*',
|
||||||
|
|
|
@ -433,6 +433,7 @@ const expectAssetsInstalled = ({
|
||||||
...res.attributes,
|
...res.attributes,
|
||||||
installed_kibana: sortBy(res.attributes.installed_kibana, (o: AssetReference) => o.type),
|
installed_kibana: sortBy(res.attributes.installed_kibana, (o: AssetReference) => o.type),
|
||||||
installed_es: sortBy(res.attributes.installed_es, (o: AssetReference) => o.type),
|
installed_es: sortBy(res.attributes.installed_es, (o: AssetReference) => o.type),
|
||||||
|
package_assets: sortBy(res.attributes.package_assets, (o: AssetReference) => o.type),
|
||||||
};
|
};
|
||||||
expect(sortedRes).eql({
|
expect(sortedRes).eql({
|
||||||
installed_kibana: [
|
installed_kibana: [
|
||||||
|
@ -487,6 +488,28 @@ const expectAssetsInstalled = ({
|
||||||
test_logs: 'logs-all_assets.test_logs-*',
|
test_logs: 'logs-all_assets.test_logs-*',
|
||||||
test_metrics: 'metrics-all_assets.test_metrics-*',
|
test_metrics: 'metrics-all_assets.test_metrics-*',
|
||||||
},
|
},
|
||||||
|
package_assets: [
|
||||||
|
{ id: '333a22a1-e639-5af5-ae62-907ffc83d603', type: 'epm-packages-assets' },
|
||||||
|
{ id: '256f3dad-6870-56c3-80a1-8dfa11e2d568', type: 'epm-packages-assets' },
|
||||||
|
{ id: '3fa0512f-bc01-5c2e-9df1-bc2f2a8259c8', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'ea334ad8-80c2-5acd-934b-2a377290bf97', type: 'epm-packages-assets' },
|
||||||
|
{ id: '96c6eb85-fe2e-56c6-84be-5fda976796db', type: 'epm-packages-assets' },
|
||||||
|
{ id: '2d73a161-fa69-52d0-aa09-1bdc691b95bb', type: 'epm-packages-assets' },
|
||||||
|
{ id: '0a00c2d2-ce63-5b9c-9aa0-0cf1938f7362', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'b36e6dd0-58f7-5dd0-a286-8187e4019274', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'f839c76e-d194-555a-90a1-3265a45789e4', type: 'epm-packages-assets' },
|
||||||
|
{ id: '9af7bbb3-7d8a-50fa-acc9-9dde6f5efca2', type: 'epm-packages-assets' },
|
||||||
|
{ id: '1e97a20f-9d1c-529b-8ff2-da4e8ba8bb71', type: 'epm-packages-assets' },
|
||||||
|
{ id: '8cfe0a2b-7016-5522-87e4-6d352360d1fc', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'bd5ff3c5-655e-5385-9918-b60ff3040aad', type: 'epm-packages-assets' },
|
||||||
|
{ id: '0954ce3b-3165-5c1f-a4c0-56eb5f2fa487', type: 'epm-packages-assets' },
|
||||||
|
{ id: '60d6d054-57e4-590f-a580-52bf3f5e7cca', type: 'epm-packages-assets' },
|
||||||
|
{ id: '47758dc2-979d-5fbe-a2bd-9eded68a5a43', type: 'epm-packages-assets' },
|
||||||
|
{ id: '318959c9-997b-5a14-b328-9fc7355b4b74', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'e786cbd9-0f3b-5a0b-82a6-db25145ebf58', type: 'epm-packages-assets' },
|
||||||
|
{ id: '53c94591-aa33-591d-8200-cd524c2a0561', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'b658d2d4-752e-54b8-afc2-4c76155c1466', type: 'epm-packages-assets' },
|
||||||
|
],
|
||||||
name: 'all_assets',
|
name: 'all_assets',
|
||||||
version: '0.1.0',
|
version: '0.1.0',
|
||||||
internal: false,
|
internal: false,
|
||||||
|
|
|
@ -318,6 +318,26 @@ export default function (providerContext: FtrProviderContext) {
|
||||||
test_logs: 'logs-all_assets.test_logs-*',
|
test_logs: 'logs-all_assets.test_logs-*',
|
||||||
test_metrics: 'metrics-all_assets.test_metrics-*',
|
test_metrics: 'metrics-all_assets.test_metrics-*',
|
||||||
},
|
},
|
||||||
|
package_assets: [
|
||||||
|
{ id: '3eb4c54a-638f-51b6-84e2-d53f5a666e37', type: 'epm-packages-assets' },
|
||||||
|
{ id: '4acfbf69-7a27-5c58-9c99-7c86843d958f', type: 'epm-packages-assets' },
|
||||||
|
{ id: '938655df-b339-523c-a9e4-123c89c0e1e1', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'eec4606c-dbfa-565b-8e9c-fce1e641f3fc', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'ef67e7e0-dca3-5a62-a42a-745db5ad7c1f', type: 'epm-packages-assets' },
|
||||||
|
{ id: '64239d25-be40-5e10-94b5-f6b74b8c5474', type: 'epm-packages-assets' },
|
||||||
|
{ id: '071b5113-4c9f-5ee9-aafe-d098a4c066f6', type: 'epm-packages-assets' },
|
||||||
|
{ id: '498d8215-2613-5399-9a13-fa4f0bf513e2', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'd2f87071-c866-503a-8fcb-7b23a8c7afbf', type: 'epm-packages-assets' },
|
||||||
|
{ id: '5a080eba-f482-545c-8695-6ccbd426b2a2', type: 'epm-packages-assets' },
|
||||||
|
{ id: '28523a82-1328-578d-84cb-800970560200', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'cc1e3e1d-f27b-5d05-86f6-6e4b9a47c7dc', type: 'epm-packages-assets' },
|
||||||
|
{ id: '5c3aa147-089c-5084-beca-53c00e72ac80', type: 'epm-packages-assets' },
|
||||||
|
{ id: '48e582df-b1d2-5f88-b6ea-ba1fafd3a569', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'bf3b0b65-9fdc-53c6-a9ca-e76140e56490', type: 'epm-packages-assets' },
|
||||||
|
{ id: '2e56f08b-1d06-55ed-abee-4708e1ccf0aa', type: 'epm-packages-assets' },
|
||||||
|
{ id: 'c7bf1a39-e057-58a0-afde-fb4b48751d8c', type: 'epm-packages-assets' },
|
||||||
|
{ id: '8c665f28-a439-5f43-b5fd-8fda7b576735', type: 'epm-packages-assets' },
|
||||||
|
],
|
||||||
name: 'all_assets',
|
name: 'all_assets',
|
||||||
version: '0.2.0',
|
version: '0.2.0',
|
||||||
internal: false,
|
internal: false,
|
||||||
|
|
Loading…
Reference in a new issue