* Use client from branch * Get type checking working in core * Fix types in other plugins * Update client types + remove type errors from core * migrate Task Manager Elasticsearch typing from legacy library to client library * use SortOrder instead o string in alerts * Update client types + fix core type issues * fix maps ts errors * Update Lens types * Convert Search Profiler body from a string to an object to conform to SearchRequest type. * Fix SOT types * Fix/mute Security/Spaces plugins type errors. * Fix bootstrap types * Fix painless_lab * corrected es typing in Event Log * Use new types from client for inferred search responses * Latest type defs * Integrate latest type defs for APM/UX * fix core errors * fix telemetry errors * fix canvas errors * fix data_enhanced errors * fix event_log errors * mute lens errors * fix or mute maps errors * fix reporting errors * fix security errors * mute errors in task_manager * fix errors in telemetry_collection_xpack * fix errors in data plugins * fix errors in alerts * mute errors in index_management * fix task_manager errors * mute or fix lens errors * fix upgrade_assistant errors * fix or mute errors in index_lifecycle_management * fix discover errors * fix core tests * ML changes * fix core type errors * mute error in kbn-es-archiver * fix error in data plugin * fix error in telemetry plugin * fix error in discover * fix discover errors * fix errors in task_manager * fix security errors * fix wrong conflict resolution * address errors with upstream code * update deps to the last commit * remove outdated comments * fix core errors * fix errors after update * adding more expect errors to ML * pull the lastest changes * fix core errors * fix errors in infra plugin * fix errors in uptime plugin * fix errors in ml * fix errors in xpack telemetry * fix or mute errors in transform * fix errors in upgrade assistant * fix or mute fleet errors * start fixing apm errors * fix errors in osquery * fix telemetry tests * core cleanup * fix asMutableArray imports * cleanup * data_enhanced cleanup * cleanup events_log * cleaup * fix error in kbn-es-archiver * fix errors in kbn-es-archiver * fix errors in kbn-es-archiver * fix ES typings for Hit * fix SO * fix actions plugin * fix fleet * fix maps * fix stack_alerts * fix eslint problems * fix event_log unit tests * fix failures in data_enhanced tests * fix test failure in kbn-es-archiver * fix test failures in index_pattern_management * fixing ML test * remove outdated comment in kbn-es-archiver * fix error type in ml * fix eslint errors in osquery plugin * fix runtime error in infra plugin * revert changes to event_log cluser exist check * fix eslint error in osquery * fixing ML endpoint argument types * fx types * Update api-extractor docs * attempt fix for ese test * Fix lint error * Fix types for ts refs * Fix data_enhanced unit test * fix lens types * generate docs * Fix a number of type issues in monitoring and ml * fix triggers_actions_ui * Fix ILM functional test * Put search.d.ts typings back * fix data plugin * Update typings in typings/elasticsearch * Update snapshots * mute errors in task_manager * mute fleet errors * lens. remove unnecessary ts-expect-errors * fix errors in stack_alerts * mute errors in osquery * fix errors in security_solution * fix errors in lists * fix errors in cases * mute errors in search_examples * use KibanaClient to enforce promise-based API * fix errors in test/ folder * update comment * fix errors in x-pack/test folder * fix errors in ml plugin * fix optional fields in ml api_integartoon tests * fix another casting problem in ml tests * fix another ml test failure * fix fleet problem after conflict resolution * rollback changes in security_solution. trying to fix test * Update type for discover rows * uncomment runtime_mappings as its outdated * address comments from Wylie * remove eslint error due to any * mute error due to incompatibility * Apply suggestions from code review Co-authored-by: John Schulz <github.com@jfsiii.org> * fix type error in lens tests * Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com> * Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com> * update deps * fix errors in core types * fix errors for the new elastic/elasticsearch version * remove unused type * remove unnecessary manual type cast and put optional chaining back * ML: mute Datafeed is missing indices_options * Apply suggestions from code review Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com> * use canary pacakge instead of git commit Co-authored-by: Josh Dover <me@joshdover.com> Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com> Co-authored-by: Gidi Meir Morris <github@gidi.io> Co-authored-by: Nathan Reese <reese.nathan@gmail.com> Co-authored-by: Wylie Conlon <wylieconlon@gmail.com> Co-authored-by: CJ Cenizal <cj@cenizal.com> Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com> Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co> Co-authored-by: restrry <restrry@gmail.com> Co-authored-by: James Gowdy <jgowdy@elastic.co> Co-authored-by: John Schulz <github.com@jfsiii.org> Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
116 lines
3.8 KiB
TypeScript
116 lines
3.8 KiB
TypeScript
/*
|
|
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
|
* or more contributor license agreements. Licensed under the Elastic License
|
|
* 2.0; you may not use this file except in compliance with the Elastic License
|
|
* 2.0.
|
|
*/
|
|
|
|
import {
|
|
TreeOptions,
|
|
Tree,
|
|
EndpointDocGenerator,
|
|
Event,
|
|
} from '../../../plugins/security_solution/common/endpoint/generate_data';
|
|
import { firstNonNullValue } from '../../../plugins/security_solution/common/endpoint/models/ecs_safety_helpers';
|
|
import { FtrProviderContext } from '../ftr_provider_context';
|
|
|
|
export const processEventsIndex = 'logs-endpoint.events.process-default';
|
|
|
|
/**
|
|
* Options for build a resolver tree
|
|
*/
|
|
export interface Options extends TreeOptions {
|
|
/**
|
|
* Number of trees to generate.
|
|
*/
|
|
numTrees?: number;
|
|
seed?: string;
|
|
}
|
|
|
|
/**
|
|
* Structure containing the generated trees and the ES index they live in
|
|
*/
|
|
export interface GeneratedTrees {
|
|
trees: Tree[];
|
|
indices: string[];
|
|
}
|
|
|
|
/**
|
|
* Structure containing the events inserted into ES and the index they live in
|
|
*/
|
|
export interface InsertedEvents {
|
|
eventsInfo: Array<{ _id: string; event: Event }>;
|
|
indices: string[];
|
|
}
|
|
|
|
interface BulkCreateHeader {
|
|
create: {
|
|
_index: string;
|
|
};
|
|
}
|
|
|
|
export function ResolverGeneratorProvider({ getService }: FtrProviderContext) {
|
|
const client = getService('es');
|
|
|
|
return {
|
|
async insertEvents(
|
|
events: Event[],
|
|
eventsIndex: string = processEventsIndex
|
|
): Promise<InsertedEvents> {
|
|
const body = events.reduce((array: Array<BulkCreateHeader | Event>, doc) => {
|
|
array.push({ create: { _index: eventsIndex } }, doc);
|
|
return array;
|
|
}, []);
|
|
const bulkResp = await client.bulk({ body, refresh: true });
|
|
|
|
const eventsInfo = events.map((event: Event, i: number) => {
|
|
return { event, _id: bulkResp.body.items[i].create?._id };
|
|
});
|
|
|
|
// @ts-expect-error @elastic/elasticsearch expected BulkResponseItemBase._id: string
|
|
return { eventsInfo, indices: [eventsIndex] };
|
|
},
|
|
async createTrees(
|
|
options: Options,
|
|
eventsIndex: string = processEventsIndex,
|
|
alertsIndex: string = 'logs-endpoint.alerts-default'
|
|
): Promise<GeneratedTrees> {
|
|
const seed = options.seed || 'resolver-seed';
|
|
const allTrees: Tree[] = [];
|
|
const generator = new EndpointDocGenerator(seed);
|
|
const numTrees = options.numTrees ?? 1;
|
|
for (let j = 0; j < numTrees; j++) {
|
|
const tree = generator.generateTree(options);
|
|
const body = tree.allEvents.reduce((array: Array<BulkCreateHeader | Event>, doc) => {
|
|
let index = eventsIndex;
|
|
if (firstNonNullValue(doc.event?.kind) === 'alert') {
|
|
index = alertsIndex;
|
|
}
|
|
/**
|
|
* We're using data streams which require that a bulk use `create` instead of `index`.
|
|
*/
|
|
array.push({ create: { _index: index } }, doc);
|
|
return array;
|
|
}, []);
|
|
// force a refresh here otherwise the documents might not be available when the tests search for them
|
|
await client.bulk({ body, refresh: true });
|
|
allTrees.push(tree);
|
|
}
|
|
return { trees: allTrees, indices: [eventsIndex, alertsIndex] };
|
|
},
|
|
async deleteData(genData: { indices: string[] }) {
|
|
for (const index of genData.indices) {
|
|
/**
|
|
* The ingest manager handles creating the template for the endpoint's indices. It is using a V2 template
|
|
* with data streams. Data streams aren't included in the javascript elasticsearch client in kibana yet so we
|
|
* need to do raw requests here. Delete a data stream is slightly different than that of a regular index which
|
|
* is why we're using _data_stream here.
|
|
*/
|
|
await client.transport.request({
|
|
method: 'DELETE',
|
|
path: `_data_stream/${index}`,
|
|
});
|
|
}
|
|
},
|
|
};
|
|
}
|