[SavedObjects] Add aggregations support (#96292)
* step 1 to add aggs in the find function of saved object * setp 2 - add specific unit test to aggs + fix bug found during integrations * step 3 - add security api_integration arounds aggs * fix types * unit test added for aggs_utils * add documentation * fix docs * review I * doc * try to fix test * add the new property to the saved object globaltype * fix types * delete old files * fix types + test api integration * type fix + test * Update src/core/server/saved_objects/types.ts Co-authored-by: Rudolf Meijering <skaapgif@gmail.com> * review I * change our validation to match discussion with Pierre and Rudolph * Validate multiple items nested filter query through KueryNode * remove unused import * review + put back test * migrate added tests to new TS file * fix documentation * fix license header * move stuff * duplicating test mappings * rename some stuff * move ALL the things * cast to aggregation container * update generated doc * add deep nested validation * rewrite the whole validation mechanism * some cleanup * minor cleanup * update generated doc * adapt telemetry client * fix API integ tests * fix doc * TOTO-less * remove xpack tests * list supported / unsupported aggregations * typo fix * extract some validation function * fix indent * add some unit tests * adapt FTR assertions * update doc * fix doc * doc again * cleanup test names * improve tsdoc on validation functions * perf nit Co-authored-by: Xavier Mouligneau <189600+XavierM@users.noreply.github.com> Co-authored-by: Rudolf Meijering <skaapgif@gmail.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Pierre Gayvallet
GitHub
parent
15e8ca1161
commit
106afd41b6
|
|
@ -53,9 +53,14 @@ experimental[] Retrieve a paginated set of {kib} saved objects by various condit
|
|||
(Optional, object) Filters to objects that have a relationship with the type and ID combination.
|
||||
|
||||
`filter`::
|
||||
(Optional, string) The filter is a KQL string with the caveat that if you filter with an attribute from your type saved object.
|
||||
It should look like that savedObjectType.attributes.title: "myTitle". However, If you used a direct attribute of a saved object like `updatedAt`,
|
||||
you will have to define your filter like that savedObjectType.updatedAt > 2018-12-22.
|
||||
(Optional, string) The filter is a KQL string with the caveat that if you filter with an attribute from your saved object type,
|
||||
it should look like that: `savedObjectType.attributes.title: "myTitle"`. However, If you use a root attribute of a saved
|
||||
object such as `updated_at`, you will have to define your filter like that: `savedObjectType.updated_at > 2018-12-22`.
|
||||
|
||||
`aggs`::
|
||||
(Optional, string) **experimental** An aggregation structure, serialized as a string. The field format is similar to `filter`, meaning
|
||||
that to use a saved object type attribute in the aggregation, the `savedObjectType.attributes.title`: "myTitle"` format
|
||||
must be used. For root fields, the syntax is `savedObjectType.rootField`
|
||||
|
||||
NOTE: As objects change in {kib}, the results on each page of the response also
|
||||
change. Use the find API for traditional paginated results, but avoid using it to export large amounts of data.
|
||||
|
|
|
|||
|
|
@ -9,5 +9,5 @@ Search for objects
|
|||
<b>Signature:</b>
|
||||
|
||||
```typescript
|
||||
find: <T = unknown>(options: SavedObjectsFindOptions) => Promise<SavedObjectsFindResponsePublic<T>>;
|
||||
find: <T = unknown, A = unknown>(options: SavedObjectsFindOptions) => Promise<SavedObjectsFindResponsePublic<T, unknown>>;
|
||||
```
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ The constructor for this class is marked as internal. Third-party code should no
|
|||
| [bulkGet](./kibana-plugin-core-public.savedobjectsclient.bulkget.md) | | <code>(objects?: Array<{</code><br/><code> id: string;</code><br/><code> type: string;</code><br/><code> }>) => Promise<SavedObjectsBatchResponse<unknown>></code> | Returns an array of objects by id |
|
||||
| [create](./kibana-plugin-core-public.savedobjectsclient.create.md) | | <code><T = unknown>(type: string, attributes: T, options?: SavedObjectsCreateOptions) => Promise<SimpleSavedObject<T>></code> | Persists an object |
|
||||
| [delete](./kibana-plugin-core-public.savedobjectsclient.delete.md) | | <code>(type: string, id: string, options?: SavedObjectsDeleteOptions | undefined) => ReturnType<SavedObjectsApi['delete']></code> | Deletes an object |
|
||||
| [find](./kibana-plugin-core-public.savedobjectsclient.find.md) | | <code><T = unknown>(options: SavedObjectsFindOptions) => Promise<SavedObjectsFindResponsePublic<T>></code> | Search for objects |
|
||||
| [find](./kibana-plugin-core-public.savedobjectsclient.find.md) | | <code><T = unknown, A = unknown>(options: SavedObjectsFindOptions) => Promise<SavedObjectsFindResponsePublic<T, unknown>></code> | Search for objects |
|
||||
| [get](./kibana-plugin-core-public.savedobjectsclient.get.md) | | <code><T = unknown>(type: string, id: string) => Promise<SimpleSavedObject<T>></code> | Fetches a single object |
|
||||
|
||||
## Methods
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
<!-- Do not edit this file. It is automatically generated by API Documenter. -->
|
||||
|
||||
[Home](./index.md) > [kibana-plugin-core-public](./kibana-plugin-core-public.md) > [SavedObjectsFindResponsePublic](./kibana-plugin-core-public.savedobjectsfindresponsepublic.md) > [aggregations](./kibana-plugin-core-public.savedobjectsfindresponsepublic.aggregations.md)
|
||||
|
||||
## SavedObjectsFindResponsePublic.aggregations property
|
||||
|
||||
<b>Signature:</b>
|
||||
|
||||
```typescript
|
||||
aggregations?: A;
|
||||
```
|
||||
|
|
@ -11,13 +11,14 @@ Return type of the Saved Objects `find()` method.
|
|||
<b>Signature:</b>
|
||||
|
||||
```typescript
|
||||
export interface SavedObjectsFindResponsePublic<T = unknown> extends SavedObjectsBatchResponse<T>
|
||||
export interface SavedObjectsFindResponsePublic<T = unknown, A = unknown> extends SavedObjectsBatchResponse<T>
|
||||
```
|
||||
|
||||
## Properties
|
||||
|
||||
| Property | Type | Description |
|
||||
| --- | --- | --- |
|
||||
| [aggregations](./kibana-plugin-core-public.savedobjectsfindresponsepublic.aggregations.md) | <code>A</code> | |
|
||||
| [page](./kibana-plugin-core-public.savedobjectsfindresponsepublic.page.md) | <code>number</code> | |
|
||||
| [perPage](./kibana-plugin-core-public.savedobjectsfindresponsepublic.perpage.md) | <code>number</code> | |
|
||||
| [total](./kibana-plugin-core-public.savedobjectsfindresponsepublic.total.md) | <code>number</code> | |
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ Find all SavedObjects matching the search query
|
|||
<b>Signature:</b>
|
||||
|
||||
```typescript
|
||||
find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T>>;
|
||||
find<T = unknown, A = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T, A>>;
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
|
@ -20,5 +20,5 @@ find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindRes
|
|||
|
||||
<b>Returns:</b>
|
||||
|
||||
`Promise<SavedObjectsFindResponse<T>>`
|
||||
`Promise<SavedObjectsFindResponse<T, A>>`
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,11 @@
|
|||
<!-- Do not edit this file. It is automatically generated by API Documenter. -->
|
||||
|
||||
[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [SavedObjectsFindResponse](./kibana-plugin-core-server.savedobjectsfindresponse.md) > [aggregations](./kibana-plugin-core-server.savedobjectsfindresponse.aggregations.md)
|
||||
|
||||
## SavedObjectsFindResponse.aggregations property
|
||||
|
||||
<b>Signature:</b>
|
||||
|
||||
```typescript
|
||||
aggregations?: A;
|
||||
```
|
||||
|
|
@ -11,13 +11,14 @@ Return type of the Saved Objects `find()` method.
|
|||
<b>Signature:</b>
|
||||
|
||||
```typescript
|
||||
export interface SavedObjectsFindResponse<T = unknown>
|
||||
export interface SavedObjectsFindResponse<T = unknown, A = unknown>
|
||||
```
|
||||
|
||||
## Properties
|
||||
|
||||
| Property | Type | Description |
|
||||
| --- | --- | --- |
|
||||
| [aggregations](./kibana-plugin-core-server.savedobjectsfindresponse.aggregations.md) | <code>A</code> | |
|
||||
| [page](./kibana-plugin-core-server.savedobjectsfindresponse.page.md) | <code>number</code> | |
|
||||
| [per\_page](./kibana-plugin-core-server.savedobjectsfindresponse.per_page.md) | <code>number</code> | |
|
||||
| [pit\_id](./kibana-plugin-core-server.savedobjectsfindresponse.pit_id.md) | <code>string</code> | |
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
<b>Signature:</b>
|
||||
|
||||
```typescript
|
||||
find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T>>;
|
||||
find<T = unknown, A = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T, A>>;
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
|
@ -18,7 +18,7 @@ find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindRes
|
|||
|
||||
<b>Returns:</b>
|
||||
|
||||
`Promise<SavedObjectsFindResponse<T>>`
|
||||
`Promise<SavedObjectsFindResponse<T, A>>`
|
||||
|
||||
{<!-- -->promise<!-- -->} - { saved\_objects: \[{ id, type, version, attributes }<!-- -->\], total, per\_page, page }
|
||||
|
||||
|
|
|
|||
|
|
@ -9,5 +9,5 @@ Creates an empty response for a find operation. This is only intended to be used
|
|||
<b>Signature:</b>
|
||||
|
||||
```typescript
|
||||
static createEmptyFindResponse: <T>({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse<T>;
|
||||
static createEmptyFindResponse: <T, A>({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse<T, A>;
|
||||
```
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ export declare class SavedObjectsUtils
|
|||
|
||||
| Property | Modifiers | Type | Description |
|
||||
| --- | --- | --- | --- |
|
||||
| [createEmptyFindResponse](./kibana-plugin-core-server.savedobjectsutils.createemptyfindresponse.md) | <code>static</code> | <code><T>({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse<T></code> | Creates an empty response for a find operation. This is only intended to be used by saved objects client wrappers. |
|
||||
| [createEmptyFindResponse](./kibana-plugin-core-server.savedobjectsutils.createemptyfindresponse.md) | <code>static</code> | <code><T, A>({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse<T, A></code> | Creates an empty response for a find operation. This is only intended to be used by saved objects client wrappers. |
|
||||
| [namespaceIdToString](./kibana-plugin-core-server.savedobjectsutils.namespaceidtostring.md) | <code>static</code> | <code>(namespace?: string | undefined) => string</code> | Converts a given saved object namespace ID to its string representation. All namespace IDs have an identical string representation, with the exception of the <code>undefined</code> namespace ID (which has a namespace string of <code>'default'</code>). |
|
||||
| [namespaceStringToId](./kibana-plugin-core-server.savedobjectsutils.namespacestringtoid.md) | <code>static</code> | <code>(namespace: string) => string | undefined</code> | Converts a given saved object namespace string to its ID representation. All namespace strings have an identical ID representation, with the exception of the <code>'default'</code> namespace string (which has a namespace ID of <code>undefined</code>). |
|
||||
|
||||
|
|
|
|||
|
|
@ -1224,7 +1224,7 @@ export class SavedObjectsClient {
|
|||
// Warning: (ae-forgotten-export) The symbol "SavedObjectsClientContract" needs to be exported by the entry point index.d.ts
|
||||
delete: (type: string, id: string, options?: SavedObjectsDeleteOptions | undefined) => ReturnType<SavedObjectsClientContract_2['delete']>;
|
||||
// Warning: (ae-forgotten-export) The symbol "SavedObjectsFindOptions" needs to be exported by the entry point index.d.ts
|
||||
find: <T = unknown>(options: SavedObjectsFindOptions_2) => Promise<SavedObjectsFindResponsePublic<T>>;
|
||||
find: <T = unknown, A = unknown>(options: SavedObjectsFindOptions_2) => Promise<SavedObjectsFindResponsePublic<T, unknown>>;
|
||||
get: <T = unknown>(type: string, id: string) => Promise<SimpleSavedObject<T>>;
|
||||
update<T = unknown>(type: string, id: string, attributes: T, { version, migrationVersion, references }?: SavedObjectsUpdateOptions): Promise<SimpleSavedObject<T>>;
|
||||
}
|
||||
|
|
@ -1244,6 +1244,8 @@ export interface SavedObjectsCreateOptions {
|
|||
|
||||
// @public (undocumented)
|
||||
export interface SavedObjectsFindOptions {
|
||||
// @alpha
|
||||
aggs?: Record<string, estypes.AggregationContainer>;
|
||||
defaultSearchOperator?: 'AND' | 'OR';
|
||||
fields?: string[];
|
||||
// Warning: (ae-forgotten-export) The symbol "KueryNode" needs to be exported by the entry point index.d.ts
|
||||
|
|
@ -1284,7 +1286,9 @@ export interface SavedObjectsFindOptionsReference {
|
|||
}
|
||||
|
||||
// @public
|
||||
export interface SavedObjectsFindResponsePublic<T = unknown> extends SavedObjectsBatchResponse<T> {
|
||||
export interface SavedObjectsFindResponsePublic<T = unknown, A = unknown> extends SavedObjectsBatchResponse<T> {
|
||||
// (undocumented)
|
||||
aggregations?: A;
|
||||
// (undocumented)
|
||||
page: number;
|
||||
// (undocumented)
|
||||
|
|
|
|||
|
|
@ -103,7 +103,9 @@ export interface SavedObjectsDeleteOptions {
|
|||
*
|
||||
* @public
|
||||
*/
|
||||
export interface SavedObjectsFindResponsePublic<T = unknown> extends SavedObjectsBatchResponse<T> {
|
||||
export interface SavedObjectsFindResponsePublic<T = unknown, A = unknown>
|
||||
extends SavedObjectsBatchResponse<T> {
|
||||
aggregations?: A;
|
||||
total: number;
|
||||
perPage: number;
|
||||
page: number;
|
||||
|
|
@ -310,7 +312,7 @@ export class SavedObjectsClient {
|
|||
* @property {object} [options.hasReference] - { type, id }
|
||||
* @returns A find result with objects matching the specified search.
|
||||
*/
|
||||
public find = <T = unknown>(
|
||||
public find = <T = unknown, A = unknown>(
|
||||
options: SavedObjectsFindOptions
|
||||
): Promise<SavedObjectsFindResponsePublic<T>> => {
|
||||
const path = this.getPath(['_find']);
|
||||
|
|
@ -326,6 +328,7 @@ export class SavedObjectsClient {
|
|||
sortField: 'sort_field',
|
||||
type: 'type',
|
||||
filter: 'filter',
|
||||
aggs: 'aggs',
|
||||
namespaces: 'namespaces',
|
||||
preference: 'preference',
|
||||
};
|
||||
|
|
@ -342,6 +345,12 @@ export class SavedObjectsClient {
|
|||
query.has_reference = JSON.stringify(query.has_reference);
|
||||
}
|
||||
|
||||
// `aggs` is a structured object. we need to stringify it before sending it, as `fetch`
|
||||
// is not doing it implicitly.
|
||||
if (query.aggs) {
|
||||
query.aggs = JSON.stringify(query.aggs);
|
||||
}
|
||||
|
||||
const request: ReturnType<SavedObjectsApi['find']> = this.savedObjectsFetch(path, {
|
||||
method: 'GET',
|
||||
query,
|
||||
|
|
@ -349,6 +358,7 @@ export class SavedObjectsClient {
|
|||
return request.then((resp) => {
|
||||
return renameKeys<SavedObjectsFindResponse, SavedObjectsFindResponsePublic>(
|
||||
{
|
||||
aggregations: 'aggregations',
|
||||
saved_objects: 'savedObjects',
|
||||
total: 'total',
|
||||
per_page: 'perPage',
|
||||
|
|
|
|||
|
|
@ -44,6 +44,7 @@ export const registerFindRoute = (router: IRouter, { coreUsageData }: RouteDepen
|
|||
has_reference_operator: searchOperatorSchema,
|
||||
fields: schema.maybe(schema.oneOf([schema.string(), schema.arrayOf(schema.string())])),
|
||||
filter: schema.maybe(schema.string()),
|
||||
aggs: schema.maybe(schema.string()),
|
||||
namespaces: schema.maybe(
|
||||
schema.oneOf([schema.string(), schema.arrayOf(schema.string())])
|
||||
),
|
||||
|
|
@ -59,6 +60,20 @@ export const registerFindRoute = (router: IRouter, { coreUsageData }: RouteDepen
|
|||
const usageStatsClient = coreUsageData.getClient();
|
||||
usageStatsClient.incrementSavedObjectsFind({ request: req }).catch(() => {});
|
||||
|
||||
// manually validation to avoid using JSON.parse twice
|
||||
let aggs;
|
||||
if (query.aggs) {
|
||||
try {
|
||||
aggs = JSON.parse(query.aggs);
|
||||
} catch (e) {
|
||||
return res.badRequest({
|
||||
body: {
|
||||
message: 'invalid aggs value',
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const result = await context.core.savedObjects.client.find({
|
||||
perPage: query.per_page,
|
||||
page: query.page,
|
||||
|
|
@ -72,6 +87,7 @@ export const registerFindRoute = (router: IRouter, { coreUsageData }: RouteDepen
|
|||
hasReferenceOperator: query.has_reference_operator,
|
||||
fields: typeof query.fields === 'string' ? [query.fields] : query.fields,
|
||||
filter: query.filter,
|
||||
aggs,
|
||||
namespaces,
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,86 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0 and the Server Side Public License, v 1; you may not use this file except
|
||||
* in compliance with, at your election, the Elastic License 2.0 or the Server
|
||||
* Side Public License, v 1.
|
||||
*/
|
||||
|
||||
import { schema as s, ObjectType } from '@kbn/config-schema';
|
||||
|
||||
/**
|
||||
* Schemas for the Bucket aggregations.
|
||||
*
|
||||
* Currently supported:
|
||||
* - filter
|
||||
* - histogram
|
||||
* - terms
|
||||
*
|
||||
* Not implemented:
|
||||
* - adjacency_matrix
|
||||
* - auto_date_histogram
|
||||
* - children
|
||||
* - composite
|
||||
* - date_histogram
|
||||
* - date_range
|
||||
* - diversified_sampler
|
||||
* - filters
|
||||
* - geo_distance
|
||||
* - geohash_grid
|
||||
* - geotile_grid
|
||||
* - global
|
||||
* - ip_range
|
||||
* - missing
|
||||
* - multi_terms
|
||||
* - nested
|
||||
* - parent
|
||||
* - range
|
||||
* - rare_terms
|
||||
* - reverse_nested
|
||||
* - sampler
|
||||
* - significant_terms
|
||||
* - significant_text
|
||||
* - variable_width_histogram
|
||||
*/
|
||||
export const bucketAggsSchemas: Record<string, ObjectType> = {
|
||||
filter: s.object({
|
||||
term: s.recordOf(s.string(), s.oneOf([s.string(), s.boolean(), s.number()])),
|
||||
}),
|
||||
histogram: s.object({
|
||||
field: s.maybe(s.string()),
|
||||
interval: s.maybe(s.number()),
|
||||
min_doc_count: s.maybe(s.number()),
|
||||
extended_bounds: s.maybe(
|
||||
s.object({
|
||||
min: s.number(),
|
||||
max: s.number(),
|
||||
})
|
||||
),
|
||||
hard_bounds: s.maybe(
|
||||
s.object({
|
||||
min: s.number(),
|
||||
max: s.number(),
|
||||
})
|
||||
),
|
||||
missing: s.maybe(s.number()),
|
||||
keyed: s.maybe(s.boolean()),
|
||||
order: s.maybe(
|
||||
s.object({
|
||||
_count: s.string(),
|
||||
_key: s.string(),
|
||||
})
|
||||
),
|
||||
}),
|
||||
terms: s.object({
|
||||
field: s.maybe(s.string()),
|
||||
collect_mode: s.maybe(s.string()),
|
||||
exclude: s.maybe(s.oneOf([s.string(), s.arrayOf(s.string())])),
|
||||
include: s.maybe(s.oneOf([s.string(), s.arrayOf(s.string())])),
|
||||
execution_hint: s.maybe(s.string()),
|
||||
missing: s.maybe(s.number()),
|
||||
min_doc_count: s.maybe(s.number()),
|
||||
size: s.maybe(s.number()),
|
||||
show_term_doc_count_error: s.maybe(s.boolean()),
|
||||
order: s.maybe(s.oneOf([s.literal('asc'), s.literal('desc')])),
|
||||
}),
|
||||
};
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0 and the Server Side Public License, v 1; you may not use this file except
|
||||
* in compliance with, at your election, the Elastic License 2.0 or the Server
|
||||
* Side Public License, v 1.
|
||||
*/
|
||||
|
||||
import { bucketAggsSchemas } from './bucket_aggs';
|
||||
import { metricsAggsSchemas } from './metrics_aggs';
|
||||
|
||||
export const aggregationSchemas = {
|
||||
...metricsAggsSchemas,
|
||||
...bucketAggsSchemas,
|
||||
};
|
||||
|
|
@ -0,0 +1,94 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0 and the Server Side Public License, v 1; you may not use this file except
|
||||
* in compliance with, at your election, the Elastic License 2.0 or the Server
|
||||
* Side Public License, v 1.
|
||||
*/
|
||||
|
||||
import { schema as s, ObjectType } from '@kbn/config-schema';
|
||||
|
||||
/**
|
||||
* Schemas for the metrics Aggregations
|
||||
*
|
||||
* Currently supported:
|
||||
* - avg
|
||||
* - cardinality
|
||||
* - min
|
||||
* - max
|
||||
* - sum
|
||||
* - top_hits
|
||||
* - weighted_avg
|
||||
*
|
||||
* Not implemented:
|
||||
* - boxplot
|
||||
* - extended_stats
|
||||
* - geo_bounds
|
||||
* - geo_centroid
|
||||
* - geo_line
|
||||
* - matrix_stats
|
||||
* - median_absolute_deviation
|
||||
* - percentile_ranks
|
||||
* - percentiles
|
||||
* - rate
|
||||
* - scripted_metric
|
||||
* - stats
|
||||
* - string_stats
|
||||
* - t_test
|
||||
* - value_count
|
||||
*/
|
||||
export const metricsAggsSchemas: Record<string, ObjectType> = {
|
||||
avg: s.object({
|
||||
field: s.maybe(s.string()),
|
||||
missing: s.maybe(s.oneOf([s.string(), s.number(), s.boolean()])),
|
||||
}),
|
||||
cardinality: s.object({
|
||||
field: s.maybe(s.string()),
|
||||
precision_threshold: s.maybe(s.number()),
|
||||
rehash: s.maybe(s.boolean()),
|
||||
missing: s.maybe(s.oneOf([s.string(), s.number(), s.boolean()])),
|
||||
}),
|
||||
min: s.object({
|
||||
field: s.maybe(s.string()),
|
||||
missing: s.maybe(s.oneOf([s.string(), s.number(), s.boolean()])),
|
||||
format: s.maybe(s.string()),
|
||||
}),
|
||||
max: s.object({
|
||||
field: s.maybe(s.string()),
|
||||
missing: s.maybe(s.oneOf([s.string(), s.number(), s.boolean()])),
|
||||
format: s.maybe(s.string()),
|
||||
}),
|
||||
sum: s.object({
|
||||
field: s.maybe(s.string()),
|
||||
missing: s.maybe(s.oneOf([s.string(), s.number(), s.boolean()])),
|
||||
}),
|
||||
top_hits: s.object({
|
||||
explain: s.maybe(s.boolean()),
|
||||
docvalue_fields: s.maybe(s.oneOf([s.string(), s.arrayOf(s.string())])),
|
||||
stored_fields: s.maybe(s.oneOf([s.string(), s.arrayOf(s.string())])),
|
||||
from: s.maybe(s.number()),
|
||||
size: s.maybe(s.number()),
|
||||
sort: s.maybe(s.oneOf([s.literal('asc'), s.literal('desc')])),
|
||||
seq_no_primary_term: s.maybe(s.boolean()),
|
||||
version: s.maybe(s.boolean()),
|
||||
track_scores: s.maybe(s.boolean()),
|
||||
highlight: s.maybe(s.any()),
|
||||
_source: s.maybe(s.oneOf([s.boolean(), s.string(), s.arrayOf(s.string())])),
|
||||
}),
|
||||
weighted_avg: s.object({
|
||||
format: s.maybe(s.string()),
|
||||
value_type: s.maybe(s.string()),
|
||||
value: s.maybe(
|
||||
s.object({
|
||||
field: s.maybe(s.string()),
|
||||
missing: s.maybe(s.number()),
|
||||
})
|
||||
),
|
||||
weight: s.maybe(
|
||||
s.object({
|
||||
field: s.maybe(s.string()),
|
||||
missing: s.maybe(s.number()),
|
||||
})
|
||||
),
|
||||
}),
|
||||
};
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0 and the Server Side Public License, v 1; you may not use this file except
|
||||
* in compliance with, at your election, the Elastic License 2.0 or the Server
|
||||
* Side Public License, v 1.
|
||||
*/
|
||||
|
||||
export { validateAndConvertAggregations } from './validation';
|
||||
|
|
@ -0,0 +1,431 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0 and the Server Side Public License, v 1; you may not use this file except
|
||||
* in compliance with, at your election, the Elastic License 2.0 or the Server
|
||||
* Side Public License, v 1.
|
||||
*/
|
||||
|
||||
import type { estypes } from '@elastic/elasticsearch';
|
||||
import { validateAndConvertAggregations } from './validation';
|
||||
|
||||
type AggsMap = Record<string, estypes.AggregationContainer>;
|
||||
|
||||
const mockMappings = {
|
||||
properties: {
|
||||
updated_at: {
|
||||
type: 'date',
|
||||
},
|
||||
foo: {
|
||||
properties: {
|
||||
title: {
|
||||
type: 'text',
|
||||
},
|
||||
description: {
|
||||
type: 'text',
|
||||
},
|
||||
bytes: {
|
||||
type: 'number',
|
||||
},
|
||||
},
|
||||
},
|
||||
bean: {
|
||||
properties: {
|
||||
canned: {
|
||||
fields: {
|
||||
text: {
|
||||
type: 'text',
|
||||
},
|
||||
},
|
||||
type: 'keyword',
|
||||
},
|
||||
},
|
||||
},
|
||||
alert: {
|
||||
properties: {
|
||||
actions: {
|
||||
type: 'nested',
|
||||
properties: {
|
||||
group: {
|
||||
type: 'keyword',
|
||||
},
|
||||
actionRef: {
|
||||
type: 'keyword',
|
||||
},
|
||||
actionTypeId: {
|
||||
type: 'keyword',
|
||||
},
|
||||
params: {
|
||||
enabled: false,
|
||||
type: 'object',
|
||||
},
|
||||
},
|
||||
},
|
||||
params: {
|
||||
type: 'flattened',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
describe('validateAndConvertAggregations', () => {
|
||||
it('validates a simple aggregations', () => {
|
||||
expect(
|
||||
validateAndConvertAggregations(
|
||||
['foo'],
|
||||
{ aggName: { max: { field: 'foo.attributes.bytes' } } },
|
||||
mockMappings
|
||||
)
|
||||
).toEqual({
|
||||
aggName: {
|
||||
max: {
|
||||
field: 'foo.bytes',
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('validates a nested field in simple aggregations', () => {
|
||||
expect(
|
||||
validateAndConvertAggregations(
|
||||
['alert'],
|
||||
{ aggName: { cardinality: { field: 'alert.attributes.actions.group' } } },
|
||||
mockMappings
|
||||
)
|
||||
).toEqual({
|
||||
aggName: {
|
||||
cardinality: {
|
||||
field: 'alert.actions.group',
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('validates a nested aggregations', () => {
|
||||
expect(
|
||||
validateAndConvertAggregations(
|
||||
['alert'],
|
||||
{
|
||||
aggName: {
|
||||
cardinality: {
|
||||
field: 'alert.attributes.actions.group',
|
||||
},
|
||||
aggs: {
|
||||
aggName: {
|
||||
max: { field: 'alert.attributes.actions.group' },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
mockMappings
|
||||
)
|
||||
).toEqual({
|
||||
aggName: {
|
||||
cardinality: {
|
||||
field: 'alert.actions.group',
|
||||
},
|
||||
aggs: {
|
||||
aggName: {
|
||||
max: {
|
||||
field: 'alert.actions.group',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('validates a deeply nested aggregations', () => {
|
||||
expect(
|
||||
validateAndConvertAggregations(
|
||||
['alert'],
|
||||
{
|
||||
first: {
|
||||
cardinality: {
|
||||
field: 'alert.attributes.actions.group',
|
||||
},
|
||||
aggs: {
|
||||
second: {
|
||||
max: { field: 'alert.attributes.actions.group' },
|
||||
aggs: {
|
||||
third: {
|
||||
min: {
|
||||
field: 'alert.attributes.actions.actionTypeId',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
mockMappings
|
||||
)
|
||||
).toEqual({
|
||||
first: {
|
||||
cardinality: {
|
||||
field: 'alert.actions.group',
|
||||
},
|
||||
aggs: {
|
||||
second: {
|
||||
max: { field: 'alert.actions.group' },
|
||||
aggs: {
|
||||
third: {
|
||||
min: {
|
||||
field: 'alert.actions.actionTypeId',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('rewrites type attributes when valid', () => {
|
||||
const aggregations: AggsMap = {
|
||||
average: {
|
||||
avg: {
|
||||
field: 'alert.attributes.actions.group',
|
||||
missing: 10,
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(validateAndConvertAggregations(['alert'], aggregations, mockMappings)).toEqual({
|
||||
average: {
|
||||
avg: {
|
||||
field: 'alert.actions.group',
|
||||
missing: 10,
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('rewrites root attributes when valid', () => {
|
||||
const aggregations: AggsMap = {
|
||||
average: {
|
||||
avg: {
|
||||
field: 'alert.updated_at',
|
||||
missing: 10,
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(validateAndConvertAggregations(['alert'], aggregations, mockMappings)).toEqual({
|
||||
average: {
|
||||
avg: {
|
||||
field: 'updated_at',
|
||||
missing: 10,
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('throws an error when the `field` name is not using attributes path', () => {
|
||||
const aggregations: AggsMap = {
|
||||
average: {
|
||||
avg: {
|
||||
field: 'alert.actions.group',
|
||||
missing: 10,
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(() =>
|
||||
validateAndConvertAggregations(['alert'], aggregations, mockMappings)
|
||||
).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[average.avg.field] Invalid attribute path: alert.actions.group"`
|
||||
);
|
||||
});
|
||||
|
||||
it('throws an error when the `field` name is referencing an invalid field', () => {
|
||||
const aggregations: AggsMap = {
|
||||
average: {
|
||||
avg: {
|
||||
field: 'alert.attributes.actions.non_existing',
|
||||
missing: 10,
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(() =>
|
||||
validateAndConvertAggregations(['alert'], aggregations, mockMappings)
|
||||
).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[average.avg.field] Invalid attribute path: alert.attributes.actions.non_existing"`
|
||||
);
|
||||
});
|
||||
|
||||
it('throws an error when the attribute path is referencing an invalid root field', () => {
|
||||
const aggregations: AggsMap = {
|
||||
average: {
|
||||
avg: {
|
||||
field: 'alert.bad_root',
|
||||
missing: 10,
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(() =>
|
||||
validateAndConvertAggregations(['alert'], aggregations, mockMappings)
|
||||
).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[average.avg.field] Invalid attribute path: alert.bad_root"`
|
||||
);
|
||||
});
|
||||
|
||||
it('rewrites the `field` name even when nested', () => {
|
||||
const aggregations: AggsMap = {
|
||||
average: {
|
||||
weighted_avg: {
|
||||
value: {
|
||||
field: 'alert.attributes.actions.group',
|
||||
missing: 10,
|
||||
},
|
||||
weight: {
|
||||
field: 'alert.attributes.actions.actionRef',
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(validateAndConvertAggregations(['alert'], aggregations, mockMappings)).toEqual({
|
||||
average: {
|
||||
weighted_avg: {
|
||||
value: {
|
||||
field: 'alert.actions.group',
|
||||
missing: 10,
|
||||
},
|
||||
weight: {
|
||||
field: 'alert.actions.actionRef',
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('rewrites the entries of a filter term record', () => {
|
||||
const aggregations: AggsMap = {
|
||||
myFilter: {
|
||||
filter: {
|
||||
term: {
|
||||
'foo.attributes.description': 'hello',
|
||||
'foo.attributes.bytes': 10,
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
expect(validateAndConvertAggregations(['foo'], aggregations, mockMappings)).toEqual({
|
||||
myFilter: {
|
||||
filter: {
|
||||
term: { 'foo.description': 'hello', 'foo.bytes': 10 },
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('throws an error when referencing non-allowed types', () => {
|
||||
const aggregations: AggsMap = {
|
||||
myFilter: {
|
||||
max: {
|
||||
field: 'foo.attributes.bytes',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
expect(() => {
|
||||
validateAndConvertAggregations(['alert'], aggregations, mockMappings);
|
||||
}).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[myFilter.max.field] Invalid attribute path: foo.attributes.bytes"`
|
||||
);
|
||||
});
|
||||
|
||||
it('throws an error when an attributes is not respecting its schema definition', () => {
|
||||
const aggregations: AggsMap = {
|
||||
someAgg: {
|
||||
terms: {
|
||||
missing: 'expecting a number',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
expect(() =>
|
||||
validateAndConvertAggregations(['alert'], aggregations, mockMappings)
|
||||
).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[someAgg.terms.missing]: expected value of type [number] but got [string]"`
|
||||
);
|
||||
});
|
||||
|
||||
it('throws an error when trying to validate an unknown aggregation type', () => {
|
||||
const aggregations: AggsMap = {
|
||||
someAgg: {
|
||||
auto_date_histogram: {
|
||||
field: 'foo.attributes.bytes',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
expect(() => {
|
||||
validateAndConvertAggregations(['foo'], aggregations, mockMappings);
|
||||
}).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[someAgg.auto_date_histogram] auto_date_histogram aggregation is not valid (or not registered yet)"`
|
||||
);
|
||||
});
|
||||
|
||||
it('throws an error when a child aggregation is unknown', () => {
|
||||
const aggregations: AggsMap = {
|
||||
someAgg: {
|
||||
max: {
|
||||
field: 'foo.attributes.bytes',
|
||||
},
|
||||
aggs: {
|
||||
unknownAgg: {
|
||||
cumulative_cardinality: {
|
||||
format: 'format',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
expect(() => {
|
||||
validateAndConvertAggregations(['foo'], aggregations, mockMappings);
|
||||
}).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[someAgg.aggs.unknownAgg.cumulative_cardinality] cumulative_cardinality aggregation is not valid (or not registered yet)"`
|
||||
);
|
||||
});
|
||||
|
||||
it('throws an error when using a script attribute', () => {
|
||||
const aggregations: AggsMap = {
|
||||
someAgg: {
|
||||
max: {
|
||||
field: 'foo.attributes.bytes',
|
||||
script: 'This is a bad script',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
expect(() => {
|
||||
validateAndConvertAggregations(['foo'], aggregations, mockMappings);
|
||||
}).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[someAgg.max.script]: definition for this key is missing"`
|
||||
);
|
||||
});
|
||||
|
||||
it('throws an error when using a script attribute in a nested aggregation', () => {
|
||||
const aggregations: AggsMap = {
|
||||
someAgg: {
|
||||
min: {
|
||||
field: 'foo.attributes.bytes',
|
||||
},
|
||||
aggs: {
|
||||
nested: {
|
||||
max: {
|
||||
field: 'foo.attributes.bytes',
|
||||
script: 'This is a bad script',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
expect(() => {
|
||||
validateAndConvertAggregations(['foo'], aggregations, mockMappings);
|
||||
}).toThrowErrorMatchingInlineSnapshot(
|
||||
`"[someAgg.aggs.nested.max.script]: definition for this key is missing"`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,229 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0 and the Server Side Public License, v 1; you may not use this file except
|
||||
* in compliance with, at your election, the Elastic License 2.0 or the Server
|
||||
* Side Public License, v 1.
|
||||
*/
|
||||
|
||||
import type { estypes } from '@elastic/elasticsearch';
|
||||
import { ObjectType } from '@kbn/config-schema';
|
||||
import { isPlainObject } from 'lodash';
|
||||
|
||||
import { IndexMapping } from '../../../mappings';
|
||||
import {
|
||||
isObjectTypeAttribute,
|
||||
rewriteObjectTypeAttribute,
|
||||
isRootLevelAttribute,
|
||||
rewriteRootLevelAttribute,
|
||||
} from './validation_utils';
|
||||
import { aggregationSchemas } from './aggs_types';
|
||||
|
||||
const aggregationKeys = ['aggs', 'aggregations'];
|
||||
|
||||
interface ValidationContext {
|
||||
allowedTypes: string[];
|
||||
indexMapping: IndexMapping;
|
||||
currentPath: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate an aggregation structure against the declared mappings and
|
||||
* aggregation schemas, and rewrite the attribute fields using the KQL-like syntax
|
||||
* - `{type}.attributes.{attribute}` to `{type}.{attribute}`
|
||||
* - `{type}.{rootField}` to `{rootField}`
|
||||
*
|
||||
* throws on the first validation error if any is encountered.
|
||||
*/
|
||||
export const validateAndConvertAggregations = (
|
||||
allowedTypes: string[],
|
||||
aggs: Record<string, estypes.AggregationContainer>,
|
||||
indexMapping: IndexMapping
|
||||
): Record<string, estypes.AggregationContainer> => {
|
||||
return validateAggregations(aggs, {
|
||||
allowedTypes,
|
||||
indexMapping,
|
||||
currentPath: [],
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* Validate a record of aggregation containers,
|
||||
* Which can either be the root level aggregations (`SearchRequest.body.aggs`)
|
||||
* Or a nested record of aggregation (`SearchRequest.body.aggs.myAggregation.aggs`)
|
||||
*/
|
||||
const validateAggregations = (
|
||||
aggregations: Record<string, estypes.AggregationContainer>,
|
||||
context: ValidationContext
|
||||
) => {
|
||||
return Object.entries(aggregations).reduce((memo, [aggrName, aggrContainer]) => {
|
||||
memo[aggrName] = validateAggregation(aggrContainer, childContext(context, aggrName));
|
||||
return memo;
|
||||
}, {} as Record<string, estypes.AggregationContainer>);
|
||||
};
|
||||
|
||||
/**
|
||||
* Validate an aggregation container, e.g an entry of `SearchRequest.body.aggs`, or
|
||||
* from a nested aggregation record, including its potential nested aggregations.
|
||||
*/
|
||||
const validateAggregation = (
|
||||
aggregation: estypes.AggregationContainer,
|
||||
context: ValidationContext
|
||||
) => {
|
||||
const container = validateAggregationContainer(aggregation, context);
|
||||
|
||||
if (aggregation.aggregations) {
|
||||
container.aggregations = validateAggregations(
|
||||
aggregation.aggregations,
|
||||
childContext(context, 'aggregations')
|
||||
);
|
||||
}
|
||||
if (aggregation.aggs) {
|
||||
container.aggs = validateAggregations(aggregation.aggs, childContext(context, 'aggs'));
|
||||
}
|
||||
|
||||
return container;
|
||||
};
|
||||
|
||||
/**
|
||||
* Validates root-level aggregation of given aggregation container
|
||||
* (ignoring its nested aggregations)
|
||||
*/
|
||||
const validateAggregationContainer = (
|
||||
container: estypes.AggregationContainer,
|
||||
context: ValidationContext
|
||||
) => {
|
||||
return Object.entries(container).reduce((memo, [aggName, aggregation]) => {
|
||||
if (aggregationKeys.includes(aggName)) {
|
||||
return memo;
|
||||
}
|
||||
return {
|
||||
...memo,
|
||||
[aggName]: validateAggregationType(aggName, aggregation, childContext(context, aggName)),
|
||||
};
|
||||
}, {} as estypes.AggregationContainer);
|
||||
};
|
||||
|
||||
const validateAggregationType = (
|
||||
aggregationType: string,
|
||||
aggregation: Record<string, any>,
|
||||
context: ValidationContext
|
||||
) => {
|
||||
const aggregationSchema = aggregationSchemas[aggregationType];
|
||||
if (!aggregationSchema) {
|
||||
throw new Error(
|
||||
`[${context.currentPath.join(
|
||||
'.'
|
||||
)}] ${aggregationType} aggregation is not valid (or not registered yet)`
|
||||
);
|
||||
}
|
||||
|
||||
validateAggregationStructure(aggregationSchema, aggregation, context);
|
||||
return validateAndRewriteFieldAttributes(aggregation, context);
|
||||
};
|
||||
|
||||
/**
|
||||
* Validate an aggregation structure against its declared schema.
|
||||
*/
|
||||
const validateAggregationStructure = (
|
||||
schema: ObjectType,
|
||||
aggObject: unknown,
|
||||
context: ValidationContext
|
||||
) => {
|
||||
return schema.validate(aggObject, {}, context.currentPath.join('.'));
|
||||
};
|
||||
|
||||
/**
|
||||
* List of fields that have an attribute path as value
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* avg: {
|
||||
* field: 'alert.attributes.actions.group',
|
||||
* },
|
||||
* ```
|
||||
*/
|
||||
const attributeFields = ['field'];
|
||||
/**
|
||||
* List of fields that have a Record<attribute path, value> as value
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* filter: {
|
||||
* term: {
|
||||
* 'alert.attributes.actions.group': 'value'
|
||||
* },
|
||||
* },
|
||||
* ```
|
||||
*/
|
||||
const attributeMaps = ['term'];
|
||||
|
||||
const validateAndRewriteFieldAttributes = (
|
||||
aggregation: Record<string, any>,
|
||||
context: ValidationContext
|
||||
) => {
|
||||
return recursiveRewrite(aggregation, context, []);
|
||||
};
|
||||
|
||||
const recursiveRewrite = (
|
||||
currentLevel: Record<string, any>,
|
||||
context: ValidationContext,
|
||||
parents: string[]
|
||||
): Record<string, any> => {
|
||||
return Object.entries(currentLevel).reduce((memo, [key, value]) => {
|
||||
const rewriteKey = isAttributeKey(parents);
|
||||
const rewriteValue = isAttributeValue(key, value);
|
||||
|
||||
const nestedContext = childContext(context, key);
|
||||
const newKey = rewriteKey ? validateAndRewriteAttributePath(key, nestedContext) : key;
|
||||
const newValue = rewriteValue
|
||||
? validateAndRewriteAttributePath(value, nestedContext)
|
||||
: isPlainObject(value)
|
||||
? recursiveRewrite(value, nestedContext, [...parents, key])
|
||||
: value;
|
||||
|
||||
return {
|
||||
...memo,
|
||||
[newKey]: newValue,
|
||||
};
|
||||
}, {});
|
||||
};
|
||||
|
||||
const childContext = (context: ValidationContext, path: string): ValidationContext => {
|
||||
return {
|
||||
...context,
|
||||
currentPath: [...context.currentPath, path],
|
||||
};
|
||||
};
|
||||
|
||||
const lastParent = (parents: string[]) => {
|
||||
if (parents.length) {
|
||||
return parents[parents.length - 1];
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
const isAttributeKey = (parents: string[]) => {
|
||||
const last = lastParent(parents);
|
||||
if (last) {
|
||||
return attributeMaps.includes(last);
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
const isAttributeValue = (fieldName: string, fieldValue: unknown): boolean => {
|
||||
return attributeFields.includes(fieldName) && typeof fieldValue === 'string';
|
||||
};
|
||||
|
||||
const validateAndRewriteAttributePath = (
|
||||
attributePath: string,
|
||||
{ allowedTypes, indexMapping, currentPath }: ValidationContext
|
||||
) => {
|
||||
if (isRootLevelAttribute(attributePath, indexMapping, allowedTypes)) {
|
||||
return rewriteRootLevelAttribute(attributePath);
|
||||
}
|
||||
if (isObjectTypeAttribute(attributePath, indexMapping, allowedTypes)) {
|
||||
return rewriteObjectTypeAttribute(attributePath);
|
||||
}
|
||||
throw new Error(`[${currentPath.join('.')}] Invalid attribute path: ${attributePath}`);
|
||||
};
|
||||
|
|
@ -0,0 +1,148 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0 and the Server Side Public License, v 1; you may not use this file except
|
||||
* in compliance with, at your election, the Elastic License 2.0 or the Server
|
||||
* Side Public License, v 1.
|
||||
*/
|
||||
|
||||
import { IndexMapping } from '../../../mappings';
|
||||
import {
|
||||
isRootLevelAttribute,
|
||||
rewriteRootLevelAttribute,
|
||||
isObjectTypeAttribute,
|
||||
rewriteObjectTypeAttribute,
|
||||
} from './validation_utils';
|
||||
|
||||
const mockMappings: IndexMapping = {
|
||||
properties: {
|
||||
updated_at: {
|
||||
type: 'date',
|
||||
},
|
||||
foo: {
|
||||
properties: {
|
||||
title: {
|
||||
type: 'text',
|
||||
},
|
||||
description: {
|
||||
type: 'text',
|
||||
},
|
||||
bytes: {
|
||||
type: 'number',
|
||||
},
|
||||
},
|
||||
},
|
||||
bean: {
|
||||
properties: {
|
||||
canned: {
|
||||
fields: {
|
||||
text: {
|
||||
type: 'text',
|
||||
},
|
||||
},
|
||||
type: 'keyword',
|
||||
},
|
||||
},
|
||||
},
|
||||
alert: {
|
||||
properties: {
|
||||
actions: {
|
||||
type: 'nested',
|
||||
properties: {
|
||||
group: {
|
||||
type: 'keyword',
|
||||
},
|
||||
actionRef: {
|
||||
type: 'keyword',
|
||||
},
|
||||
actionTypeId: {
|
||||
type: 'keyword',
|
||||
},
|
||||
params: {
|
||||
enabled: false,
|
||||
type: 'object',
|
||||
},
|
||||
},
|
||||
},
|
||||
params: {
|
||||
type: 'flattened',
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
describe('isRootLevelAttribute', () => {
|
||||
it('returns true when referring to a path to a valid root level field', () => {
|
||||
expect(isRootLevelAttribute('foo.updated_at', mockMappings, ['foo'])).toBe(true);
|
||||
});
|
||||
it('returns false when referring to a direct path to a valid root level field', () => {
|
||||
expect(isRootLevelAttribute('updated_at', mockMappings, ['foo'])).toBe(false);
|
||||
});
|
||||
it('returns false when referring to a path to a unknown root level field', () => {
|
||||
expect(isRootLevelAttribute('foo.not_present', mockMappings, ['foo'])).toBe(false);
|
||||
});
|
||||
it('returns false when referring to a path to an existing nested field', () => {
|
||||
expect(isRootLevelAttribute('foo.properties.title', mockMappings, ['foo'])).toBe(false);
|
||||
});
|
||||
it('returns false when referring to a path to a valid root level field of an unknown type', () => {
|
||||
expect(isRootLevelAttribute('bar.updated_at', mockMappings, ['foo'])).toBe(false);
|
||||
});
|
||||
it('returns false when referring to a path to a valid root level type field', () => {
|
||||
expect(isRootLevelAttribute('foo.foo', mockMappings, ['foo'])).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('rewriteRootLevelAttribute', () => {
|
||||
it('rewrites the attribute path to strip the type', () => {
|
||||
expect(rewriteRootLevelAttribute('foo.references')).toEqual('references');
|
||||
});
|
||||
it('does not handle real root level path', () => {
|
||||
expect(rewriteRootLevelAttribute('references')).not.toEqual('references');
|
||||
});
|
||||
});
|
||||
|
||||
describe('isObjectTypeAttribute', () => {
|
||||
it('return true if attribute path is valid', () => {
|
||||
expect(isObjectTypeAttribute('foo.attributes.description', mockMappings, ['foo'])).toEqual(
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
it('return true for nested attributes', () => {
|
||||
expect(isObjectTypeAttribute('bean.attributes.canned.text', mockMappings, ['bean'])).toEqual(
|
||||
true
|
||||
);
|
||||
});
|
||||
|
||||
it('return false if attribute path points to an invalid type', () => {
|
||||
expect(isObjectTypeAttribute('foo.attributes.description', mockMappings, ['bean'])).toEqual(
|
||||
false
|
||||
);
|
||||
});
|
||||
|
||||
it('returns false if attribute path refers to a type', () => {
|
||||
expect(isObjectTypeAttribute('bean', mockMappings, ['bean'])).toEqual(false);
|
||||
});
|
||||
|
||||
it('Return error if key does not match SO attribute structure', () => {
|
||||
expect(isObjectTypeAttribute('bean.canned.text', mockMappings, ['bean'])).toEqual(false);
|
||||
});
|
||||
|
||||
it('Return false if key matches nested type attribute parent', () => {
|
||||
expect(isObjectTypeAttribute('alert.actions', mockMappings, ['alert'])).toEqual(false);
|
||||
});
|
||||
|
||||
it('returns false if path refers to a non-existent attribute', () => {
|
||||
expect(isObjectTypeAttribute('bean.attributes.red', mockMappings, ['bean'])).toEqual(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('rewriteObjectTypeAttribute', () => {
|
||||
it('rewrites the attribute path to strip the type', () => {
|
||||
expect(rewriteObjectTypeAttribute('foo.attributes.prop')).toEqual('foo.prop');
|
||||
});
|
||||
it('returns invalid input unchanged', () => {
|
||||
expect(rewriteObjectTypeAttribute('foo.references')).toEqual('foo.references');
|
||||
});
|
||||
});
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
/*
|
||||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
||||
* or more contributor license agreements. Licensed under the Elastic License
|
||||
* 2.0 and the Server Side Public License, v 1; you may not use this file except
|
||||
* in compliance with, at your election, the Elastic License 2.0 or the Server
|
||||
* Side Public License, v 1.
|
||||
*/
|
||||
|
||||
import { IndexMapping } from '../../../mappings';
|
||||
import { fieldDefined, hasFilterKeyError } from '../filter_utils';
|
||||
|
||||
/**
|
||||
* Returns true if the given attribute path is a valid root level SO attribute path
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* isRootLevelAttribute('myType.updated_at', indexMapping, ['myType']})
|
||||
* // => true
|
||||
* ```
|
||||
*/
|
||||
export const isRootLevelAttribute = (
|
||||
attributePath: string,
|
||||
indexMapping: IndexMapping,
|
||||
allowedTypes: string[]
|
||||
): boolean => {
|
||||
const splits = attributePath.split('.');
|
||||
if (splits.length !== 2) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const [type, fieldName] = splits;
|
||||
if (allowedTypes.includes(fieldName)) {
|
||||
return false;
|
||||
}
|
||||
return allowedTypes.includes(type) && fieldDefined(indexMapping, fieldName);
|
||||
};
|
||||
|
||||
/**
|
||||
* Rewrites a root level attribute path to strip the type
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* rewriteRootLevelAttribute('myType.updated_at')
|
||||
* // => 'updated_at'
|
||||
* ```
|
||||
*/
|
||||
export const rewriteRootLevelAttribute = (attributePath: string) => {
|
||||
return attributePath.split('.')[1];
|
||||
};
|
||||
|
||||
/**
|
||||
* Returns true if the given attribute path is a valid object type level SO attribute path
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* isObjectTypeAttribute('myType.attributes.someField', indexMapping, ['myType']})
|
||||
* // => true
|
||||
* ```
|
||||
*/
|
||||
export const isObjectTypeAttribute = (
|
||||
attributePath: string,
|
||||
indexMapping: IndexMapping,
|
||||
allowedTypes: string[]
|
||||
): boolean => {
|
||||
const error = hasFilterKeyError(attributePath, allowedTypes, indexMapping);
|
||||
return error == null;
|
||||
};
|
||||
|
||||
/**
|
||||
* Rewrites a object type attribute path to strip the type
|
||||
*
|
||||
* @example
|
||||
* ```ts
|
||||
* rewriteObjectTypeAttribute('myType.attributes.foo')
|
||||
* // => 'myType.foo'
|
||||
* ```
|
||||
*/
|
||||
export const rewriteObjectTypeAttribute = (attributePath: string) => {
|
||||
return attributePath.replace('.attributes', '');
|
||||
};
|
||||
|
|
@ -18,7 +18,7 @@ import {
|
|||
|
||||
const mockMappings = {
|
||||
properties: {
|
||||
updatedAt: {
|
||||
updated_at: {
|
||||
type: 'date',
|
||||
},
|
||||
foo: {
|
||||
|
|
@ -123,12 +123,12 @@ describe('Filter Utils', () => {
|
|||
expect(
|
||||
validateConvertFilterToKueryNode(
|
||||
['foo'],
|
||||
'foo.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)',
|
||||
'foo.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)',
|
||||
mockMappings
|
||||
)
|
||||
).toEqual(
|
||||
esKuery.fromKueryExpression(
|
||||
'(type: foo and updatedAt: 5678654567) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or foo.description :*)'
|
||||
'(type: foo and updated_at: 5678654567) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or foo.description :*)'
|
||||
)
|
||||
);
|
||||
});
|
||||
|
|
@ -137,12 +137,12 @@ describe('Filter Utils', () => {
|
|||
expect(
|
||||
validateConvertFilterToKueryNode(
|
||||
['foo', 'bar'],
|
||||
'foo.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)',
|
||||
'foo.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)',
|
||||
mockMappings
|
||||
)
|
||||
).toEqual(
|
||||
esKuery.fromKueryExpression(
|
||||
'(type: foo and updatedAt: 5678654567) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or foo.description :*)'
|
||||
'(type: foo and updated_at: 5678654567) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or foo.description :*)'
|
||||
)
|
||||
);
|
||||
});
|
||||
|
|
@ -151,12 +151,12 @@ describe('Filter Utils', () => {
|
|||
expect(
|
||||
validateConvertFilterToKueryNode(
|
||||
['foo', 'bar'],
|
||||
'(bar.updatedAt: 5678654567 OR foo.updatedAt: 5678654567) and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or bar.attributes.description :*)',
|
||||
'(bar.updated_at: 5678654567 OR foo.updated_at: 5678654567) and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or bar.attributes.description :*)',
|
||||
mockMappings
|
||||
)
|
||||
).toEqual(
|
||||
esKuery.fromKueryExpression(
|
||||
'((type: bar and updatedAt: 5678654567) or (type: foo and updatedAt: 5678654567)) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or bar.description :*)'
|
||||
'((type: bar and updated_at: 5678654567) or (type: foo and updated_at: 5678654567)) and foo.bytes > 1000 and foo.bytes < 8000 and foo.title: "best" and (foo.description: t* or bar.description :*)'
|
||||
)
|
||||
);
|
||||
});
|
||||
|
|
@ -181,11 +181,11 @@ describe('Filter Utils', () => {
|
|||
expect(() => {
|
||||
validateConvertFilterToKueryNode(
|
||||
['foo', 'bar'],
|
||||
'updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)',
|
||||
'updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)',
|
||||
mockMappings
|
||||
);
|
||||
}).toThrowErrorMatchingInlineSnapshot(
|
||||
`"This key 'updatedAt' need to be wrapped by a saved object type like foo,bar: Bad Request"`
|
||||
`"This key 'updated_at' need to be wrapped by a saved object type like foo,bar: Bad Request"`
|
||||
);
|
||||
});
|
||||
|
||||
|
|
@ -200,7 +200,7 @@ describe('Filter Utils', () => {
|
|||
test('Validate filter query through KueryNode - happy path', () => {
|
||||
const validationObject = validateFilterKueryNode({
|
||||
astFilter: esKuery.fromKueryExpression(
|
||||
'foo.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)'
|
||||
'foo.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)'
|
||||
),
|
||||
types: ['foo'],
|
||||
indexMapping: mockMappings,
|
||||
|
|
@ -211,7 +211,7 @@ describe('Filter Utils', () => {
|
|||
astPath: 'arguments.0',
|
||||
error: null,
|
||||
isSavedObjectAttr: true,
|
||||
key: 'foo.updatedAt',
|
||||
key: 'foo.updated_at',
|
||||
type: 'foo',
|
||||
},
|
||||
{
|
||||
|
|
@ -275,7 +275,7 @@ describe('Filter Utils', () => {
|
|||
test('Return Error if key is not wrapper by a saved object type', () => {
|
||||
const validationObject = validateFilterKueryNode({
|
||||
astFilter: esKuery.fromKueryExpression(
|
||||
'updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)'
|
||||
'updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)'
|
||||
),
|
||||
types: ['foo'],
|
||||
indexMapping: mockMappings,
|
||||
|
|
@ -284,9 +284,9 @@ describe('Filter Utils', () => {
|
|||
expect(validationObject).toEqual([
|
||||
{
|
||||
astPath: 'arguments.0',
|
||||
error: "This key 'updatedAt' need to be wrapped by a saved object type like foo",
|
||||
error: "This key 'updated_at' need to be wrapped by a saved object type like foo",
|
||||
isSavedObjectAttr: true,
|
||||
key: 'updatedAt',
|
||||
key: 'updated_at',
|
||||
type: null,
|
||||
},
|
||||
{
|
||||
|
|
@ -330,7 +330,7 @@ describe('Filter Utils', () => {
|
|||
test('Return Error if key of a saved object type is not wrapped with attributes', () => {
|
||||
const validationObject = validateFilterKueryNode({
|
||||
astFilter: esKuery.fromKueryExpression(
|
||||
'foo.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.description :*)'
|
||||
'foo.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.description :*)'
|
||||
),
|
||||
types: ['foo'],
|
||||
indexMapping: mockMappings,
|
||||
|
|
@ -341,7 +341,7 @@ describe('Filter Utils', () => {
|
|||
astPath: 'arguments.0',
|
||||
error: null,
|
||||
isSavedObjectAttr: true,
|
||||
key: 'foo.updatedAt',
|
||||
key: 'foo.updated_at',
|
||||
type: 'foo',
|
||||
},
|
||||
{
|
||||
|
|
@ -387,7 +387,7 @@ describe('Filter Utils', () => {
|
|||
test('Return Error if filter is not using an allowed type', () => {
|
||||
const validationObject = validateFilterKueryNode({
|
||||
astFilter: esKuery.fromKueryExpression(
|
||||
'bar.updatedAt: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)'
|
||||
'bar.updated_at: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.title: "best" and (foo.attributes.description: t* or foo.attributes.description :*)'
|
||||
),
|
||||
types: ['foo'],
|
||||
indexMapping: mockMappings,
|
||||
|
|
@ -398,7 +398,7 @@ describe('Filter Utils', () => {
|
|||
astPath: 'arguments.0',
|
||||
error: 'This type bar is not allowed',
|
||||
isSavedObjectAttr: true,
|
||||
key: 'bar.updatedAt',
|
||||
key: 'bar.updated_at',
|
||||
type: 'bar',
|
||||
},
|
||||
{
|
||||
|
|
@ -442,7 +442,7 @@ describe('Filter Utils', () => {
|
|||
test('Return Error if filter is using an non-existing key in the index patterns of the saved object type', () => {
|
||||
const validationObject = validateFilterKueryNode({
|
||||
astFilter: esKuery.fromKueryExpression(
|
||||
'foo.updatedAt33: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.header: "best" and (foo.attributes.description: t* or foo.attributes.description :*)'
|
||||
'foo.updated_at33: 5678654567 and foo.attributes.bytes > 1000 and foo.attributes.bytes < 8000 and foo.attributes.header: "best" and (foo.attributes.description: t* or foo.attributes.description :*)'
|
||||
),
|
||||
types: ['foo'],
|
||||
indexMapping: mockMappings,
|
||||
|
|
@ -451,9 +451,9 @@ describe('Filter Utils', () => {
|
|||
expect(validationObject).toEqual([
|
||||
{
|
||||
astPath: 'arguments.0',
|
||||
error: "This key 'foo.updatedAt33' does NOT exist in foo saved object index patterns",
|
||||
error: "This key 'foo.updated_at33' does NOT exist in foo saved object index patterns",
|
||||
isSavedObjectAttr: false,
|
||||
key: 'foo.updatedAt33',
|
||||
key: 'foo.updated_at33',
|
||||
type: 'foo',
|
||||
},
|
||||
{
|
||||
|
|
@ -519,6 +519,33 @@ describe('Filter Utils', () => {
|
|||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test('Validate multiple items nested filter query through KueryNode', () => {
|
||||
const validationObject = validateFilterKueryNode({
|
||||
astFilter: esKuery.fromKueryExpression(
|
||||
'alert.attributes.actions:{ actionTypeId: ".server-log" AND actionRef: "foo" }'
|
||||
),
|
||||
types: ['alert'],
|
||||
indexMapping: mockMappings,
|
||||
});
|
||||
|
||||
expect(validationObject).toEqual([
|
||||
{
|
||||
astPath: 'arguments.1.arguments.0',
|
||||
error: null,
|
||||
isSavedObjectAttr: false,
|
||||
key: 'alert.attributes.actions.actionTypeId',
|
||||
type: 'alert',
|
||||
},
|
||||
{
|
||||
astPath: 'arguments.1.arguments.1',
|
||||
error: null,
|
||||
isSavedObjectAttr: false,
|
||||
key: 'alert.attributes.actions.actionRef',
|
||||
type: 'alert',
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('#hasFilterKeyError', () => {
|
||||
|
|
|
|||
|
|
@ -109,7 +109,15 @@ export const validateFilterKueryNode = ({
|
|||
return astFilter.arguments.reduce((kueryNode: string[], ast: KueryNode, index: number) => {
|
||||
if (hasNestedKey && ast.type === 'literal' && ast.value != null) {
|
||||
localNestedKeys = ast.value;
|
||||
} else if (ast.type === 'literal' && ast.value && typeof ast.value === 'string') {
|
||||
const key = ast.value.replace('.attributes', '');
|
||||
const mappingKey = 'properties.' + key.split('.').join('.properties.');
|
||||
const field = get(indexMapping, mappingKey);
|
||||
if (field != null && field.type === 'nested') {
|
||||
localNestedKeys = ast.value;
|
||||
}
|
||||
}
|
||||
|
||||
if (ast.arguments) {
|
||||
const myPath = `${path}.${index}`;
|
||||
return [
|
||||
|
|
@ -121,7 +129,7 @@ export const validateFilterKueryNode = ({
|
|||
storeValue: ast.type === 'function' && astFunctionType.includes(ast.function),
|
||||
path: `${myPath}.arguments`,
|
||||
hasNestedKey: ast.type === 'function' && ast.function === 'nested',
|
||||
nestedKeys: localNestedKeys,
|
||||
nestedKeys: localNestedKeys || nestedKeys,
|
||||
}),
|
||||
];
|
||||
}
|
||||
|
|
@ -226,7 +234,7 @@ export const fieldDefined = (indexMappings: IndexMapping, key: string): boolean
|
|||
return true;
|
||||
}
|
||||
|
||||
// If the path is for a flattned type field, we'll assume the mappings are defined.
|
||||
// If the path is for a flattened type field, we'll assume the mappings are defined.
|
||||
const keys = key.split('.');
|
||||
for (let i = 0; i < keys.length; i++) {
|
||||
const path = `properties.${keys.slice(0, i + 1).join('.properties.')}`;
|
||||
|
|
|
|||
|
|
@ -66,6 +66,7 @@ import {
|
|||
import { LegacyUrlAlias, LEGACY_URL_ALIAS_TYPE } from '../../object_types';
|
||||
import { SavedObjectTypeRegistry } from '../../saved_objects_type_registry';
|
||||
import { validateConvertFilterToKueryNode } from './filter_utils';
|
||||
import { validateAndConvertAggregations } from './aggregations';
|
||||
import {
|
||||
ALL_NAMESPACES_STRING,
|
||||
FIND_DEFAULT_PAGE,
|
||||
|
|
@ -748,7 +749,9 @@ export class SavedObjectsRepository {
|
|||
* @property {string} [options.preference]
|
||||
* @returns {promise} - { saved_objects: [{ id, type, version, attributes }], total, per_page, page }
|
||||
*/
|
||||
async find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T>> {
|
||||
async find<T = unknown, A = unknown>(
|
||||
options: SavedObjectsFindOptions
|
||||
): Promise<SavedObjectsFindResponse<T, A>> {
|
||||
const {
|
||||
search,
|
||||
defaultSearchOperator = 'OR',
|
||||
|
|
@ -768,6 +771,7 @@ export class SavedObjectsRepository {
|
|||
typeToNamespacesMap,
|
||||
filter,
|
||||
preference,
|
||||
aggs,
|
||||
} = options;
|
||||
|
||||
if (!type && !typeToNamespacesMap) {
|
||||
|
|
@ -799,7 +803,7 @@ export class SavedObjectsRepository {
|
|||
: Array.from(typeToNamespacesMap!.keys());
|
||||
const allowedTypes = types.filter((t) => this._allowedTypes.includes(t));
|
||||
if (allowedTypes.length === 0) {
|
||||
return SavedObjectsUtils.createEmptyFindResponse<T>(options);
|
||||
return SavedObjectsUtils.createEmptyFindResponse<T, A>(options);
|
||||
}
|
||||
|
||||
if (searchFields && !Array.isArray(searchFields)) {
|
||||
|
|
@ -811,16 +815,24 @@ export class SavedObjectsRepository {
|
|||
}
|
||||
|
||||
let kueryNode;
|
||||
|
||||
try {
|
||||
if (filter) {
|
||||
if (filter) {
|
||||
try {
|
||||
kueryNode = validateConvertFilterToKueryNode(allowedTypes, filter, this._mappings);
|
||||
} catch (e) {
|
||||
if (e.name === 'KQLSyntaxError') {
|
||||
throw SavedObjectsErrorHelpers.createBadRequestError(`KQLSyntaxError: ${e.message}`);
|
||||
} else {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
if (e.name === 'KQLSyntaxError') {
|
||||
throw SavedObjectsErrorHelpers.createBadRequestError('KQLSyntaxError: ' + e.message);
|
||||
} else {
|
||||
throw e;
|
||||
}
|
||||
|
||||
let aggsObject;
|
||||
if (aggs) {
|
||||
try {
|
||||
aggsObject = validateAndConvertAggregations(allowedTypes, aggs, this._mappings);
|
||||
} catch (e) {
|
||||
throw SavedObjectsErrorHelpers.createBadRequestError(`Invalid aggregation: ${e.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -838,6 +850,7 @@ export class SavedObjectsRepository {
|
|||
seq_no_primary_term: true,
|
||||
from: perPage * (page - 1),
|
||||
_source: includedFields(type, fields),
|
||||
...(aggsObject ? { aggs: aggsObject } : {}),
|
||||
...getSearchDsl(this._mappings, this._registry, {
|
||||
search,
|
||||
defaultSearchOperator,
|
||||
|
|
@ -872,6 +885,7 @@ export class SavedObjectsRepository {
|
|||
}
|
||||
|
||||
return {
|
||||
...(body.aggregations ? { aggregations: (body.aggregations as unknown) as A } : {}),
|
||||
page,
|
||||
per_page: perPage,
|
||||
total: body.hits.total,
|
||||
|
|
@ -885,7 +899,7 @@ export class SavedObjectsRepository {
|
|||
})
|
||||
),
|
||||
pit_id: body.pit_id,
|
||||
} as SavedObjectsFindResponse<T>;
|
||||
} as SavedObjectsFindResponse<T, A>;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -51,10 +51,10 @@ export class SavedObjectsUtils {
|
|||
/**
|
||||
* Creates an empty response for a find operation. This is only intended to be used by saved objects client wrappers.
|
||||
*/
|
||||
public static createEmptyFindResponse = <T>({
|
||||
public static createEmptyFindResponse = <T, A>({
|
||||
page = FIND_DEFAULT_PAGE,
|
||||
perPage = FIND_DEFAULT_PER_PAGE,
|
||||
}: SavedObjectsFindOptions): SavedObjectsFindResponse<T> => ({
|
||||
}: SavedObjectsFindOptions): SavedObjectsFindResponse<T, A> => ({
|
||||
page,
|
||||
per_page: perPage,
|
||||
total: 0,
|
||||
|
|
|
|||
|
|
@ -173,7 +173,8 @@ export interface SavedObjectsFindResult<T = unknown> extends SavedObject<T> {
|
|||
*
|
||||
* @public
|
||||
*/
|
||||
export interface SavedObjectsFindResponse<T = unknown> {
|
||||
export interface SavedObjectsFindResponse<T = unknown, A = unknown> {
|
||||
aggregations?: A;
|
||||
saved_objects: Array<SavedObjectsFindResult<T>>;
|
||||
total: number;
|
||||
per_page: number;
|
||||
|
|
@ -463,7 +464,9 @@ export class SavedObjectsClient {
|
|||
*
|
||||
* @param options
|
||||
*/
|
||||
async find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T>> {
|
||||
async find<T = unknown, A = unknown>(
|
||||
options: SavedObjectsFindOptions
|
||||
): Promise<SavedObjectsFindResponse<T, A>> {
|
||||
return await this._repository.find(options);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -116,6 +116,28 @@ export interface SavedObjectsFindOptions {
|
|||
*/
|
||||
defaultSearchOperator?: 'AND' | 'OR';
|
||||
filter?: string | KueryNode;
|
||||
/**
|
||||
* A record of aggregations to perform.
|
||||
* The API currently only supports a limited set of metrics and bucket aggregation types.
|
||||
* Additional aggregation types can be contributed to Core.
|
||||
*
|
||||
* @example
|
||||
* Aggregating on SO attribute field
|
||||
* ```ts
|
||||
* const aggs = { latest_version: { max: { field: 'dashboard.attributes.version' } } };
|
||||
* return client.find({ type: 'dashboard', aggs })
|
||||
* ```
|
||||
*
|
||||
* @example
|
||||
* Aggregating on SO root field
|
||||
* ```ts
|
||||
* const aggs = { latest_update: { max: { field: 'dashboard.updated_at' } } };
|
||||
* return client.find({ type: 'dashboard', aggs })
|
||||
* ```
|
||||
*
|
||||
* @alpha
|
||||
*/
|
||||
aggs?: Record<string, estypes.AggregationContainer>;
|
||||
namespaces?: string[];
|
||||
/**
|
||||
* This map defines each type to search for, and the namespace(s) to search for the type in; this is only intended to be used by a saved
|
||||
|
|
|
|||
|
|
@ -2244,7 +2244,7 @@ export class SavedObjectsClient {
|
|||
static errors: typeof SavedObjectsErrorHelpers;
|
||||
// (undocumented)
|
||||
errors: typeof SavedObjectsErrorHelpers;
|
||||
find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T>>;
|
||||
find<T = unknown, A = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T, A>>;
|
||||
get<T = unknown>(type: string, id: string, options?: SavedObjectsBaseOptions): Promise<SavedObject<T>>;
|
||||
openPointInTimeForType(type: string | string[], options?: SavedObjectsOpenPointInTimeOptions): Promise<SavedObjectsOpenPointInTimeResponse>;
|
||||
removeReferencesTo(type: string, id: string, options?: SavedObjectsRemoveReferencesToOptions): Promise<SavedObjectsRemoveReferencesToResponse>;
|
||||
|
|
@ -2501,6 +2501,8 @@ export type SavedObjectsFieldMapping = SavedObjectsCoreFieldMapping | SavedObjec
|
|||
|
||||
// @public (undocumented)
|
||||
export interface SavedObjectsFindOptions {
|
||||
// @alpha
|
||||
aggs?: Record<string, estypes.AggregationContainer>;
|
||||
defaultSearchOperator?: 'AND' | 'OR';
|
||||
fields?: string[];
|
||||
// Warning: (ae-forgotten-export) The symbol "KueryNode" needs to be exported by the entry point index.d.ts
|
||||
|
|
@ -2539,7 +2541,9 @@ export interface SavedObjectsFindOptionsReference {
|
|||
}
|
||||
|
||||
// @public
|
||||
export interface SavedObjectsFindResponse<T = unknown> {
|
||||
export interface SavedObjectsFindResponse<T = unknown, A = unknown> {
|
||||
// (undocumented)
|
||||
aggregations?: A;
|
||||
// (undocumented)
|
||||
page: number;
|
||||
// (undocumented)
|
||||
|
|
@ -2849,7 +2853,7 @@ export class SavedObjectsRepository {
|
|||
deleteByNamespace(namespace: string, options?: SavedObjectsDeleteByNamespaceOptions): Promise<any>;
|
||||
deleteFromNamespaces(type: string, id: string, namespaces: string[], options?: SavedObjectsDeleteFromNamespacesOptions): Promise<SavedObjectsDeleteFromNamespacesResponse>;
|
||||
// (undocumented)
|
||||
find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T>>;
|
||||
find<T = unknown, A = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T, A>>;
|
||||
get<T = unknown>(type: string, id: string, options?: SavedObjectsBaseOptions): Promise<SavedObject<T>>;
|
||||
incrementCounter<T = unknown>(type: string, id: string, counterFields: Array<string | SavedObjectsIncrementCounterField>, options?: SavedObjectsIncrementCounterOptions<T>): Promise<SavedObject<T>>;
|
||||
openPointInTimeForType(type: string | string[], { keepAlive, preference }?: SavedObjectsOpenPointInTimeOptions): Promise<SavedObjectsOpenPointInTimeResponse>;
|
||||
|
|
@ -2970,7 +2974,7 @@ export interface SavedObjectsUpdateResponse<T = unknown> extends Omit<SavedObjec
|
|||
|
||||
// @public (undocumented)
|
||||
export class SavedObjectsUtils {
|
||||
static createEmptyFindResponse: <T>({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse<T>;
|
||||
static createEmptyFindResponse: <T, A>({ page, perPage, }: SavedObjectsFindOptions) => SavedObjectsFindResponse<T, A>;
|
||||
static generateId(): string;
|
||||
static isRandomId(id: string | undefined): boolean;
|
||||
static namespaceIdToString: (namespace?: string | undefined) => string;
|
||||
|
|
|
|||
|
|
@ -17,7 +17,9 @@ export class TelemetrySavedObjectsClient extends SavedObjectsClient {
|
|||
* Find the SavedObjects matching the search query in all the Spaces by default
|
||||
* @param options
|
||||
*/
|
||||
async find<T = unknown>(options: SavedObjectsFindOptions): Promise<SavedObjectsFindResponse<T>> {
|
||||
async find<T = unknown, A = unknown>(
|
||||
options: SavedObjectsFindOptions
|
||||
): Promise<SavedObjectsFindResponse<T, A>> {
|
||||
return super.find({ namespaces: ['*'], ...options });
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -293,6 +293,75 @@ export default function ({ getService }: FtrProviderContext) {
|
|||
}));
|
||||
});
|
||||
|
||||
describe('using aggregations', () => {
|
||||
it('should return 200 with valid response for a valid aggregation', async () =>
|
||||
await supertest
|
||||
.get(
|
||||
`/api/saved_objects/_find?type=visualization&per_page=0&aggs=${encodeURIComponent(
|
||||
JSON.stringify({
|
||||
type_count: { max: { field: 'visualization.attributes.version' } },
|
||||
})
|
||||
)}`
|
||||
)
|
||||
.expect(200)
|
||||
.then((resp) => {
|
||||
expect(resp.body).to.eql({
|
||||
aggregations: {
|
||||
type_count: {
|
||||
value: 1,
|
||||
},
|
||||
},
|
||||
page: 1,
|
||||
per_page: 0,
|
||||
saved_objects: [],
|
||||
total: 1,
|
||||
});
|
||||
}));
|
||||
|
||||
it('should return a 400 when referencing an invalid SO attribute', async () =>
|
||||
await supertest
|
||||
.get(
|
||||
`/api/saved_objects/_find?type=visualization&per_page=0&aggs=${encodeURIComponent(
|
||||
JSON.stringify({
|
||||
type_count: { max: { field: 'dashboard.attributes.version' } },
|
||||
})
|
||||
)}`
|
||||
)
|
||||
.expect(400)
|
||||
.then((resp) => {
|
||||
expect(resp.body).to.eql({
|
||||
error: 'Bad Request',
|
||||
message:
|
||||
'Invalid aggregation: [type_count.max.field] Invalid attribute path: dashboard.attributes.version: Bad Request',
|
||||
statusCode: 400,
|
||||
});
|
||||
}));
|
||||
|
||||
it('should return a 400 when using a forbidden aggregation option', async () =>
|
||||
await supertest
|
||||
.get(
|
||||
`/api/saved_objects/_find?type=visualization&per_page=0&aggs=${encodeURIComponent(
|
||||
JSON.stringify({
|
||||
type_count: {
|
||||
max: {
|
||||
field: 'visualization.attributes.version',
|
||||
script: 'Bad script is bad',
|
||||
},
|
||||
},
|
||||
})
|
||||
)}`
|
||||
)
|
||||
.expect(400)
|
||||
.then((resp) => {
|
||||
expect(resp.body).to.eql({
|
||||
error: 'Bad Request',
|
||||
message:
|
||||
'Invalid aggregation: [type_count.max.script]: definition for this key is missing: Bad Request',
|
||||
statusCode: 400,
|
||||
});
|
||||
}));
|
||||
});
|
||||
|
||||
describe('`has_reference` and `has_reference_operator` parameters', () => {
|
||||
before(() => esArchiver.load('saved_objects/references'));
|
||||
after(() => esArchiver.unload('saved_objects/references'));
|
||||
|
|
|
|||
|
|
@ -162,9 +162,9 @@ export class EncryptedSavedObjectsClientWrapper implements SavedObjectsClientCon
|
|||
return await this.options.baseClient.delete(type, id, options);
|
||||
}
|
||||
|
||||
public async find<T>(options: SavedObjectsFindOptions) {
|
||||
public async find<T, A>(options: SavedObjectsFindOptions) {
|
||||
return await this.handleEncryptedAttributesInBulkResponse(
|
||||
await this.options.baseClient.find<T>(options),
|
||||
await this.options.baseClient.find<T, A>(options),
|
||||
undefined
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -213,7 +213,7 @@ export class SecureSavedObjectsClientWrapper implements SavedObjectsClientContra
|
|||
return await this.baseClient.delete(type, id, options);
|
||||
}
|
||||
|
||||
public async find<T = unknown>(options: SavedObjectsFindOptions) {
|
||||
public async find<T = unknown, A = unknown>(options: SavedObjectsFindOptions) {
|
||||
if (
|
||||
this.getSpacesService() == null &&
|
||||
Array.isArray(options.namespaces) &&
|
||||
|
|
@ -245,7 +245,7 @@ export class SecureSavedObjectsClientWrapper implements SavedObjectsClientContra
|
|||
error: new Error(status),
|
||||
})
|
||||
);
|
||||
return SavedObjectsUtils.createEmptyFindResponse<T>(options);
|
||||
return SavedObjectsUtils.createEmptyFindResponse<T, A>(options);
|
||||
}
|
||||
|
||||
const typeToNamespacesMap = Array.from(typeMap).reduce<Map<string, string[] | undefined>>(
|
||||
|
|
@ -254,7 +254,7 @@ export class SecureSavedObjectsClientWrapper implements SavedObjectsClientContra
|
|||
new Map()
|
||||
);
|
||||
|
||||
const response = await this.baseClient.find<T>({
|
||||
const response = await this.baseClient.find<T, A>({
|
||||
...options,
|
||||
typeToNamespacesMap: undefined, // if the user is fully authorized, use `undefined` as the typeToNamespacesMap to prevent privilege escalation
|
||||
...(status === 'partially_authorized' && { typeToNamespacesMap, type: '', namespaces: [] }), // the repository requires that `type` and `namespaces` must be empty if `typeToNamespacesMap` is defined
|
||||
|
|
|
|||
|
|
@ -171,7 +171,7 @@ export class SpacesSavedObjectsClient implements SavedObjectsClientContract {
|
|||
* @property {object} [options.hasReference] - { type, id }
|
||||
* @returns {promise} - { saved_objects: [{ id, type, version, attributes }], total, per_page, page }
|
||||
*/
|
||||
public async find<T = unknown>(options: SavedObjectsFindOptions) {
|
||||
public async find<T = unknown, A = unknown>(options: SavedObjectsFindOptions) {
|
||||
throwErrorIfNamespaceSpecified(options);
|
||||
|
||||
let namespaces = options.namespaces;
|
||||
|
|
@ -187,12 +187,12 @@ export class SpacesSavedObjectsClient implements SavedObjectsClientContract {
|
|||
}
|
||||
if (namespaces.length === 0) {
|
||||
// return empty response, since the user is unauthorized in this space (or these spaces), but we don't return forbidden errors for `find` operations
|
||||
return SavedObjectsUtils.createEmptyFindResponse<T>(options);
|
||||
return SavedObjectsUtils.createEmptyFindResponse<T, A>(options);
|
||||
}
|
||||
} catch (err) {
|
||||
if (Boom.isBoom(err) && err.output.payload.statusCode === 403) {
|
||||
// return empty response, since the user is unauthorized in any space, but we don't return forbidden errors for `find` operations
|
||||
return SavedObjectsUtils.createEmptyFindResponse<T>(options);
|
||||
return SavedObjectsUtils.createEmptyFindResponse<T, A>(options);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
|
|
@ -200,7 +200,7 @@ export class SpacesSavedObjectsClient implements SavedObjectsClientContract {
|
|||
namespaces = [this.spaceId];
|
||||
}
|
||||
|
||||
return await this.client.find<T>({
|
||||
return await this.client.find<T, A>({
|
||||
...options,
|
||||
type: (options.type ? coerceToArray(options.type) : this.types).filter(
|
||||
(type) => type !== 'space'
|
||||
|
|
|
|||
Loading…
Reference in a new issue