Merge branch 'master' of https://github.com/elastic/kibana into alerting/default-es-index-schema

This commit is contained in:
Ying Mao 2021-03-22 14:20:56 -04:00
commit 957c333aa4
34 changed files with 637 additions and 173 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 783 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 176 KiB

View file

@ -0,0 +1,72 @@
[[search-sessions]]
=== Run a search session in the background
Sometimes you might need to search through large amounts of data no matter
how long the search takes. While this might not happen often,
there are times that long-running queries are required.
Consider a threat hunting scenario, where you need to search through years of data.
If your query is running long, you can save your search session, which
allows {kib} to continue processing your request in the
background. Save your search session from *Discover* or *Dashboard*,
and when your session is complete, view and manage it in *Stack Management*.
[role="screenshot"]
image::images/search-session.png[Search Session indicator displaying the current state of the search, which you can click to stop or save a running Search Session ]
Search sessions are <<search-session-settings-kb,enabled by default>>. Saving
a search session is only available when
<<set-time-filter,auto refresh>> is off.
[float]
==== Requirements
* To save a session, you must have permissions for *Discover* and *Dashboard*,
and the <<kibana-feature-privileges, search sessions subfeature>>.
* To view and restore a saved session, you must have access to *Stack Management*.
[float]
==== Example: Save a search session
Youre trying to understand a trend you see on a dashboard. You
need to look at several years of data, currently in
{ref}/data-tiers.html#cold-tier[cold storage],
but you dont have time to wait. You want {kib} to
continue working in the background, so tomorrow you can
open your browser and pick up where you left off.
. Load your dashboard.
+
Your search session begins automatically. The icon after the dashboard title
displays the current state of the search session. A clock indicates the search session is in progress.
A checkmark indicates that the search session is complete.
. To instruct {kib} to continue a search in the background, click the clock icon,
and then click *Save session*. Once you save a search session, you can start a new search,
navigate to a different application, or close the browser.
+
[role="screenshot"]
image::images/search-session-awhile.png[Search Session indicator displaying the current state of the search, which you can click to stop or save a running Search Session ]
. To view your saved searches, open the main menu, and then click
*Stack Management > Search Sessions*. You can also open this view from the search sessions popup for a saved or completed session.
+
[role="screenshot"]
image::images/search-sessions-menu.png[Search Sessions management view with actions for inspecting, extending, and deleting a session. ]
. Use the edit menu in *Search Sessions* to:
* *Inspect* the queries and filters that makeup the session.
* *Extend* the expiration of a completed session.
* *Delete* a session.
. To restore a search session, click its name in the *Search Sessions* view.
+
You're returned to the place from where you started the search session. The data is the same, but
behaves differently:
+
* Relative dates are converted to absolute dates.
* Panning and zooming is disabled for maps.
* Changing a filter, query, or drilldown starts a new search session, which can be slow.

View file

@ -1,13 +1,15 @@
[[search]]
== Search data
Many Kibana apps embed a query bar for real-time search, including
*Discover* and *Dashboard*.
== Search your data
You can search your data in any app that has a query bar, or by clicking on
elements in a visualization. A search matches indices in the current
<<index-patterns, index pattern>> and in the current <<set-time-filter,time frame>>.
[float]
=== Search your data
=== Search with KQL
To search the indices that match the current <<index-patterns, index pattern>>,
enter your search criteria in the query bar. By default, you'll use
By default, you search using
{kib}'s <<kuery-query, standard query language>> (KQL), which
features autocomplete and a simple, easy-to-use syntax. If you prefer to use
{kib}'s legacy query
@ -21,32 +23,17 @@ JSON-based {ref}/query-dsl.html[Elasticsearch Query DSL].
[float]
[[autorefresh]]
=== Refresh search results
As more documents are added to the indices you're searching, the search results
shown in *Discover*, and used to display visualizations, get stale. Using the
time filter, you can
As more documents are added to the indices you're searching, the search results get stale.
Using the time filter, you can
configure a refresh interval to periodically resubmit your searches to
retrieve the latest results.
[role="screenshot"]
image::images/autorefresh-interval.png[Image showing what refresh interval option looks like. The configurable time interval is located in the dropdown]
image::images/autorefresh-interval.png[Refresh interval option in time filter. The configurable time interval is located in the dropdown.]
You can also manually refresh the search results by
clicking the *Refresh* button.
[float]
=== Searching large amounts of data
Sometimes you want to search through large amounts of data no matter how long
the search takes. While this might not happen often, there are times
that long-running queries are required. Consider a threat hunting scenario
where you need to search through years of data.
If you run a query, and the run time gets close to the
timeout, you're presented the option to ignore the timeout. This enables you to
run queries with large amounts of data to completion.
By default, a query times out after 30 seconds.
The timeout is in place to avoid unintentional load on the cluster.
include::kuery.asciidoc[]
@ -211,3 +198,5 @@ To completely delete a query:
image::discover/images/saved-query-management-component-delete-query-button.png["Example of the saved query management popover when a query is hovered over and we are about to delete a query",width="80%"]
You can import, export, and delete saved queries from <<managing-saved-objects, Saved Objects in Management>>.
include::search-sessions.asciidoc[]

View file

@ -0,0 +1,25 @@
[[search-session-settings-kb]]
=== Search sessions settings in {kib}
++++
<titleabbrev>Search sessions settings</titleabbrev>
++++
Configure the search session settings in your `kibana.yml` configuration file.
[cols="2*<"]
|===
a| `xpack.data_enhanced.`
`search.sessions:enabled`
| Set to `true` (default) to enable search sessions.
a| `xpack.data.enhanced.`
`search.sessions:trackingInterval`
| The frequency for updating the state of a search session. The default is 10s.
a| `xpack.data.enhanced.`
`search.sessions:defaultExpiration`
| How long search session results are stored before they are deleted.
Extending a search session resets the expiration by the same value. The default is 7d.
|===

View file

@ -227,7 +227,7 @@ The default application to load. *Default: `"home"`*
for more details. {kib} uses an index in {es} to store saved searches, visualizations, and
dashboards. {kib} creates a new index if the index doesnt already exist.
If you configure a custom index, the name must be lowercase, and conform to the
{es} {ref}/indices-create-index.html[index name limitations].
{es} {ref}/indices-create-index.html[index name limitations].
*Default: `".kibana"`*
| `kibana.autocompleteTimeout:` {ess-icon}
@ -475,7 +475,7 @@ running behind a proxy. Use the <<server-rewriteBasePath, `server.rewriteBasePat
if it should remove the basePath from requests it receives, and to prevent a
deprecation warning at startup. This setting cannot end in a slash (`/`).
|[[server-publicBaseUrl]] `server.publicBaseUrl:`
|[[server-publicBaseUrl]] `server.publicBaseUrl:` {ess-icon}
| The publicly available URL that end-users access Kibana at. Must include the protocol, hostname, port
(if different than the defaults for `http` and `https`, 80 and 443 respectively), and the
<<server-basePath, `server.basePath`>> (if configured). This setting cannot end in a slash (`/`).
@ -696,6 +696,7 @@ include::{kib-repo-dir}/settings/ml-settings.asciidoc[]
include::{kib-repo-dir}/settings/monitoring-settings.asciidoc[]
include::{kib-repo-dir}/settings/reporting-settings.asciidoc[]
include::secure-settings.asciidoc[]
include::{kib-repo-dir}/settings/search-sessions-settings.asciidoc[]
include::{kib-repo-dir}/settings/security-settings.asciidoc[]
include::{kib-repo-dir}/settings/spaces-settings.asciidoc[]
include::{kib-repo-dir}/settings/task-manager-settings.asciidoc[]

View file

@ -141,6 +141,12 @@ These include dashboards, visualizations, maps, index patterns, Canvas workpads,
| <<managing-tags, Tags>>
|Create, manage, and assign tags to your saved objects.
| <<search-sessions, Search Sessions>>
| Manage your saved search sessions, groups of queries that run in the background.
Search sessions are useful when your queries take longer than usual to process,
for example, when you have a large volume of data or when the performance of your storage location is slow.
| <<xpack-spaces, Spaces>>
| Create spaces to organize your dashboards and other saved objects into categories.
A space is isolated from all other spaces,

View file

@ -15,14 +15,20 @@ import { getNotificationsSettings } from './notifications';
import { getThemeSettings } from './theme';
import { getStateSettings } from './state';
export const getCoreSettings = (): Record<string, UiSettingsParams> => {
interface GetCoreSettingsOptions {
isDist?: boolean;
}
export const getCoreSettings = (
options?: GetCoreSettingsOptions
): Record<string, UiSettingsParams> => {
return {
...getAccessibilitySettings(),
...getDateFormatSettings(),
...getMiscUiSettings(),
...getNavigationSettings(),
...getNotificationsSettings(),
...getThemeSettings(),
...getThemeSettings(options),
...getStateSettings(),
};
};

View file

@ -44,3 +44,57 @@ describe('theme settings', () => {
});
});
});
describe('process.env.KBN_OPTIMIZER_THEMES handling', () => {
it('provides valid options based on tags', () => {
process.env.KBN_OPTIMIZER_THEMES = 'v7light,v8dark';
let settings = getThemeSettings({ isDist: false });
expect(settings['theme:version'].options).toEqual(['v7', 'v8']);
process.env.KBN_OPTIMIZER_THEMES = 'v8dark,v7light';
settings = getThemeSettings({ isDist: false });
expect(settings['theme:version'].options).toEqual(['v7', 'v8']);
process.env.KBN_OPTIMIZER_THEMES = 'v8dark,v7light,v7dark,v8light';
settings = getThemeSettings({ isDist: false });
expect(settings['theme:version'].options).toEqual(['v7', 'v8']);
process.env.KBN_OPTIMIZER_THEMES = '*';
settings = getThemeSettings({ isDist: false });
expect(settings['theme:version'].options).toEqual(['v7', 'v8']);
process.env.KBN_OPTIMIZER_THEMES = 'v7light';
settings = getThemeSettings({ isDist: false });
expect(settings['theme:version'].options).toEqual(['v7']);
process.env.KBN_OPTIMIZER_THEMES = 'v8light';
settings = getThemeSettings({ isDist: false });
expect(settings['theme:version'].options).toEqual(['v8']);
});
it('defaults to properties of first tag', () => {
process.env.KBN_OPTIMIZER_THEMES = 'v8dark,v7light';
let settings = getThemeSettings({ isDist: false });
expect(settings['theme:darkMode'].value).toBe(true);
expect(settings['theme:version'].value).toBe('v8');
process.env.KBN_OPTIMIZER_THEMES = 'v7light,v8dark';
settings = getThemeSettings({ isDist: false });
expect(settings['theme:darkMode'].value).toBe(false);
expect(settings['theme:version'].value).toBe('v7');
});
it('ignores the value when isDist is undefined', () => {
process.env.KBN_OPTIMIZER_THEMES = 'v7light';
const settings = getThemeSettings({ isDist: undefined });
expect(settings['theme:darkMode'].value).toBe(false);
expect(settings['theme:version'].options).toEqual(['v7', 'v8']);
});
it('ignores the value when isDist is true', () => {
process.env.KBN_OPTIMIZER_THEMES = 'v7light';
const settings = getThemeSettings({ isDist: true });
expect(settings['theme:darkMode'].value).toBe(false);
expect(settings['theme:version'].options).toEqual(['v7', 'v8']);
});
});

View file

@ -6,17 +6,54 @@
* Side Public License, v 1.
*/
import { schema } from '@kbn/config-schema';
import { schema, Type } from '@kbn/config-schema';
import { i18n } from '@kbn/i18n';
import { UiSettingsParams } from '../../../types';
export const getThemeSettings = (): Record<string, UiSettingsParams> => {
function parseThemeTags() {
if (!process.env.KBN_OPTIMIZER_THEMES) {
return ['v8light', 'v8dark'];
}
if (process.env.KBN_OPTIMIZER_THEMES === '*') {
return ['v8light', 'v8dark', 'v7light', 'v7dark'];
}
return process.env.KBN_OPTIMIZER_THEMES.split(',').map((t) => t.trim());
}
function getThemeInfo(options: GetThemeSettingsOptions) {
if (options?.isDist ?? true) {
return {
defaultDarkMode: false,
defaultVersion: 'v8',
availableVersions: ['v7', 'v8'],
};
}
const themeTags = parseThemeTags();
return {
defaultDarkMode: themeTags[0].endsWith('dark'),
defaultVersion: themeTags[0].slice(0, 2),
availableVersions: ['v7', 'v8'].filter((v) => themeTags.some((t) => t.startsWith(v))),
};
}
interface GetThemeSettingsOptions {
isDist?: boolean;
}
export const getThemeSettings = (
options: GetThemeSettingsOptions = {}
): Record<string, UiSettingsParams> => {
const { availableVersions, defaultDarkMode, defaultVersion } = getThemeInfo(options);
return {
'theme:darkMode': {
name: i18n.translate('core.ui_settings.params.darkModeTitle', {
defaultMessage: 'Dark mode',
}),
value: false,
value: defaultDarkMode,
description: i18n.translate('core.ui_settings.params.darkModeText', {
defaultMessage: `Enable a dark mode for the Kibana UI. A page refresh is required for the setting to be applied.`,
}),
@ -27,14 +64,14 @@ export const getThemeSettings = (): Record<string, UiSettingsParams> => {
name: i18n.translate('core.ui_settings.params.themeVersionTitle', {
defaultMessage: 'Theme version',
}),
value: 'v8',
value: defaultVersion,
type: 'select',
options: ['v7', 'v8'],
options: availableVersions,
description: i18n.translate('core.ui_settings.params.themeVersionText', {
defaultMessage: `Switch between the theme used for the current and next version of Kibana. A page refresh is required for the setting to be applied.`,
}),
requiresPageReload: true,
schema: schema.oneOf([schema.literal('v7'), schema.literal('v8')]),
schema: schema.oneOf(availableVersions.map((v) => schema.literal(v)) as [Type<string>]),
},
};
};

View file

@ -37,11 +37,13 @@ export class UiSettingsService
implements CoreService<InternalUiSettingsServiceSetup, InternalUiSettingsServiceStart> {
private readonly log: Logger;
private readonly config$: Observable<UiSettingsConfigType>;
private readonly isDist: boolean;
private readonly uiSettingsDefaults = new Map<string, UiSettingsParams>();
private overrides: Record<string, any> = {};
constructor(private readonly coreContext: CoreContext) {
this.log = coreContext.logger.get('ui-settings-service');
this.isDist = coreContext.env.packageInfo.dist;
this.config$ = coreContext.configService.atPath<UiSettingsConfigType>(uiConfigDefinition.path);
}
@ -50,7 +52,11 @@ export class UiSettingsService
savedObjects.registerType(uiSettingsType);
registerRoutes(http.createRouter(''));
this.register(getCoreSettings());
this.register(
getCoreSettings({
isDist: this.isDist,
})
);
const config = await this.config$.pipe(first()).toPromise();
this.overrides = config.overrides;

View file

@ -1,16 +1,50 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
exports[`IndexPatterns correctly composes runtime field 1`] = `
FldList [
Object {
"aggregatable": true,
"conflictDescriptions": undefined,
"count": 5,
"customLabel": "A Runtime Field",
"esTypes": Array [
"keyword",
],
"lang": undefined,
"name": "aRuntimeField",
"readFromDocValues": false,
"script": undefined,
"scripted": false,
"searchable": true,
"subType": undefined,
"type": "string",
},
]
`;
exports[`IndexPatterns savedObjectToSpec 1`] = `
Object {
"allowNoIndex": undefined,
"fieldAttrs": Object {},
"fieldAttrs": Object {
"aRuntimeField": Object {
"count": 5,
"customLabel": "A Runtime Field",
},
},
"fieldFormats": Object {
"field": Object {},
},
"fields": Object {},
"id": "id",
"intervalName": undefined,
"runtimeFieldMap": Object {},
"runtimeFieldMap": Object {
"aRuntimeField": Object {
"script": Object {
"source": "emit('hello')",
},
"type": "keyword",
},
},
"sourceFilters": Array [
Object {
"value": "item1",

View file

@ -26,6 +26,25 @@ function setDocsourcePayload(id: string | null, providedPayload: any) {
object = defaults(providedPayload || {}, stubbedSavedObjectIndexPattern(id));
}
const savedObject = {
id: 'id',
version: 'version',
attributes: {
title: 'kibana-*',
timeFieldName: '@timestamp',
fields: '[]',
sourceFilters: '[{"value":"item1"},{"value":"item2"}]',
fieldFormatMap: '{"field":{}}',
typeMeta: '{}',
type: '',
runtimeFieldMap:
'{"aRuntimeField": { "type": "keyword", "script": {"source": "emit(\'hello\')"}}}',
fieldAttrs: '{"aRuntimeField": { "count": 5, "customLabel": "A Runtime Field"}}',
},
type: 'index-pattern',
references: [],
};
describe('IndexPatterns', () => {
let indexPatterns: IndexPatternsService;
let savedObjectsClient: SavedObjectsClientCommon;
@ -219,23 +238,14 @@ describe('IndexPatterns', () => {
});
test('savedObjectToSpec', () => {
const savedObject = {
id: 'id',
version: 'version',
attributes: {
title: 'kibana-*',
timeFieldName: '@timestamp',
fields: '[]',
sourceFilters: '[{"value":"item1"},{"value":"item2"}]',
fieldFormatMap: '{"field":{}}',
typeMeta: '{}',
type: '',
},
type: 'index-pattern',
references: [],
};
const spec = indexPatterns.savedObjectToSpec(savedObject);
expect(spec).toMatchSnapshot();
});
expect(indexPatterns.savedObjectToSpec(savedObject)).toMatchSnapshot();
test('correctly composes runtime field', async () => {
setDocsourcePayload('id', savedObject);
const indexPattern = await indexPatterns.get('id');
expect(indexPattern.fields).toMatchSnapshot();
});
test('failed requests are not cached', async () => {

View file

@ -425,8 +425,9 @@ export class IndexPatternsService {
runtimeField: value,
aggregatable: true,
searchable: true,
count: 0,
readFromDocValues: false,
customLabel: spec.fieldAttrs?.[key]?.customLabel,
count: spec.fieldAttrs?.[key]?.count,
};
}
}

View file

@ -260,7 +260,8 @@ export default function ({ getService }: FtrProviderContext) {
});
});
it("should only use the first 10k docs for the application_usage data (they'll be rolled up in a later process)", async () => {
// flaky https://github.com/elastic/kibana/issues/94513
it.skip("should only use the first 10k docs for the application_usage data (they'll be rolled up in a later process)", async () => {
const { body } = await supertest
.post('/api/telemetry/v2/clusters/_stats')
.set('kbn-xsrf', 'xxx')

View file

@ -81,6 +81,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await pieChart.expectPieSliceCount(0);
await dashboardExpect.panelCount(2);
await PageObjects.dashboard.waitForRenderComplete();
});
});
@ -96,6 +97,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await pieChart.expectPieSliceCount(5);
await dashboardExpect.panelCount(2);
await PageObjects.dashboard.waitForRenderComplete();
await dashboardExpect.selectedLegendColorCount('#F9D9F9', 5);
});
@ -115,6 +117,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await pieChart.expectPieSliceCount(5);
await dashboardExpect.panelCount(2);
await PageObjects.dashboard.waitForRenderComplete();
await dashboardExpect.selectedLegendColorCount('#F9D9F9', 5);
});

View file

@ -250,7 +250,9 @@ describe('When invoking Trusted Apps Schema', () => {
const bodyMsg = createNewTrustedApp({
entries: [createConditionEntry(), createConditionEntry()],
});
expect(() => body.validate(bodyMsg)).toThrow('[Path] field can only be used once');
expect(() => body.validate(bodyMsg)).toThrow(
'[entries]: duplicatedEntry.process.executable.caseless'
);
});
it('should validate that `entry.field` hash field value can only be used once', () => {
@ -266,7 +268,7 @@ describe('When invoking Trusted Apps Schema', () => {
}),
],
});
expect(() => body.validate(bodyMsg)).toThrow('[Hash] field can only be used once');
expect(() => body.validate(bodyMsg)).toThrow('[entries]: duplicatedEntry.process.hash.*');
});
it('should validate that `entry.field` signer field value can only be used once', () => {
@ -282,7 +284,9 @@ describe('When invoking Trusted Apps Schema', () => {
}),
],
});
expect(() => body.validate(bodyMsg)).toThrow('[Signer] field can only be used once');
expect(() => body.validate(bodyMsg)).toThrow(
'[entries]: duplicatedEntry.process.Ext.code_signature'
);
});
it('should validate Hash field valid value', () => {

View file

@ -5,16 +5,10 @@
* 2.0.
*/
import { schema, Type } from '@kbn/config-schema';
import { ConditionEntry, ConditionEntryField, OperatingSystem } from '../types';
import { schema } from '@kbn/config-schema';
import { ConditionEntryField, OperatingSystem } from '../types';
import { getDuplicateFields, isValidHash } from '../validation/trusted_apps';
const entryFieldLabels: { [k in ConditionEntryField]: string } = {
[ConditionEntryField.HASH]: 'Hash',
[ConditionEntryField.PATH]: 'Path',
[ConditionEntryField.SIGNER]: 'Signer',
};
export const DeleteTrustedAppsRequestSchema = {
params: schema.object({
id: schema.string(),
@ -30,56 +24,99 @@ export const GetTrustedAppsRequestSchema = {
const ConditionEntryTypeSchema = schema.literal('match');
const ConditionEntryOperatorSchema = schema.literal('included');
const HashConditionEntrySchema = schema.object({
field: schema.literal(ConditionEntryField.HASH),
/*
* A generic Entry schema to be used for a specific entry schema depending on the OS
*/
const CommonEntrySchema = {
field: schema.oneOf([
schema.literal(ConditionEntryField.HASH),
schema.literal(ConditionEntryField.PATH),
]),
type: ConditionEntryTypeSchema,
operator: ConditionEntryOperatorSchema,
value: schema.string({
validate: (hash) => (isValidHash(hash) ? undefined : `Invalid hash value [${hash}]`),
}),
});
const PathConditionEntrySchema = schema.object({
field: schema.literal(ConditionEntryField.PATH),
type: ConditionEntryTypeSchema,
operator: ConditionEntryOperatorSchema,
value: schema.string({ minLength: 1 }),
});
const SignerConditionEntrySchema = schema.object({
field: schema.literal(ConditionEntryField.SIGNER),
type: ConditionEntryTypeSchema,
operator: ConditionEntryOperatorSchema,
value: schema.string({ minLength: 1 }),
// If field === HASH then validate hash with custom method, else validate string with minLength = 1
value: schema.conditional(
schema.siblingRef('field'),
ConditionEntryField.HASH,
schema.string({
validate: (hash) =>
isValidHash(hash) ? undefined : `invalidField.${ConditionEntryField.HASH}`,
}),
schema.conditional(
schema.siblingRef('field'),
ConditionEntryField.PATH,
schema.string({
validate: (field) =>
field.length > 0 ? undefined : `invalidField.${ConditionEntryField.PATH}`,
}),
schema.string({
validate: (field) =>
field.length > 0 ? undefined : `invalidField.${ConditionEntryField.SIGNER}`,
})
)
),
};
const WindowsEntrySchema = schema.object({
...CommonEntrySchema,
field: schema.oneOf([
schema.literal(ConditionEntryField.HASH),
schema.literal(ConditionEntryField.PATH),
schema.literal(ConditionEntryField.SIGNER),
]),
});
const createNewTrustedAppForOsScheme = <O extends OperatingSystem, E extends ConditionEntry>(
osSchema: Type<O>,
entriesSchema: Type<E>
) =>
schema.object({
name: schema.string({ minLength: 1, maxLength: 256 }),
description: schema.maybe(schema.string({ minLength: 0, maxLength: 256, defaultValue: '' })),
os: osSchema,
entries: schema.arrayOf(entriesSchema, {
minSize: 1,
validate(entries) {
return (
getDuplicateFields(entries)
.map((field) => `[${entryFieldLabels[field]}] field can only be used once`)
.join(', ') || undefined
);
},
}),
});
const LinuxEntrySchema = schema.object({
...CommonEntrySchema,
});
const MacEntrySchema = schema.object({
...CommonEntrySchema,
});
/*
* Entry Schema depending on Os type using schema.conditional.
* If OS === WINDOWS then use Windows schema,
* else if OS === LINUX then use Linux schema,
* else use Mac schema
*/
const EntrySchemaDependingOnOS = schema.conditional(
schema.siblingRef('os'),
OperatingSystem.WINDOWS,
WindowsEntrySchema,
schema.conditional(
schema.siblingRef('os'),
OperatingSystem.LINUX,
LinuxEntrySchema,
MacEntrySchema
)
);
/*
* Entities array schema.
* The validate function checks there is no duplicated entry inside the array
*/
const EntriesSchema = schema.arrayOf(EntrySchemaDependingOnOS, {
minSize: 1,
validate(entries) {
return (
getDuplicateFields(entries)
.map((field) => `duplicatedEntry.${field}`)
.join(', ') || undefined
);
},
});
export const PostTrustedAppCreateRequestSchema = {
body: schema.oneOf([
createNewTrustedAppForOsScheme(
schema.oneOf([schema.literal(OperatingSystem.LINUX), schema.literal(OperatingSystem.MAC)]),
schema.oneOf([HashConditionEntrySchema, PathConditionEntrySchema])
),
createNewTrustedAppForOsScheme(
body: schema.object({
name: schema.string({ minLength: 1, maxLength: 256 }),
description: schema.maybe(schema.string({ minLength: 0, maxLength: 256, defaultValue: '' })),
os: schema.oneOf([
schema.literal(OperatingSystem.WINDOWS),
schema.oneOf([HashConditionEntrySchema, PathConditionEntrySchema, SignerConditionEntrySchema])
),
]),
schema.literal(OperatingSystem.LINUX),
schema.literal(OperatingSystem.MAC),
]),
entries: EntriesSchema,
}),
};

View file

@ -27,8 +27,13 @@ export interface GetTrustedListAppsResponse {
data: TrustedApp[];
}
/** API Request body for creating a new Trusted App entry */
export type PostTrustedAppCreateRequest = TypeOf<typeof PostTrustedAppCreateRequestSchema.body>;
/*
* API Request body for creating a new Trusted App entry
* As this is an inferred type and the schema type doesn't match at all with the
* NewTrustedApp type it needs and overwrite from the MacosLinux/Windows custom types
*/
export type PostTrustedAppCreateRequest = TypeOf<typeof PostTrustedAppCreateRequestSchema.body> &
(MacosLinuxConditionEntries | WindowsConditionEntries);
export interface PostTrustedAppCreateResponse {
data: TrustedApp;

View file

@ -0,0 +1,52 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export type ExperimentalFeatures = typeof allowedExperimentalValues;
/**
* A list of allowed values that can be used in `xpack.securitySolution.enableExperimental`.
* This object is then used to validate and parse the value entered.
*/
const allowedExperimentalValues = Object.freeze({
fleetServerEnabled: false,
});
type ExperimentalConfigKeys = Array<keyof ExperimentalFeatures>;
type Mutable<T> = { -readonly [P in keyof T]: T[P] };
const SecuritySolutionInvalidExperimentalValue = class extends Error {};
const allowedKeys = Object.keys(allowedExperimentalValues) as Readonly<ExperimentalConfigKeys>;
/**
* Parses the string value used in `xpack.securitySolution.enableExperimental` kibana configuration,
* which should be a string of values delimited by a comma (`,`)
*
* @param configValue
* @throws SecuritySolutionInvalidExperimentalValue
*/
export const parseExperimentalConfigValue = (configValue: string[]): ExperimentalFeatures => {
const enabledFeatures: Mutable<Partial<ExperimentalFeatures>> = {};
for (const value of configValue) {
if (!isValidExperimentalValue(value)) {
throw new SecuritySolutionInvalidExperimentalValue(`[${value}] is not valid.`);
}
enabledFeatures[value as keyof ExperimentalFeatures] = true;
}
return {
...allowedExperimentalValues,
...enabledFeatures,
};
};
export const isValidExperimentalValue = (value: string): boolean => {
return allowedKeys.includes(value as keyof ExperimentalFeatures);
};
export const getExperimentalAllowedValues = (): string[] => [...allowedKeys];

View file

@ -10,6 +10,7 @@ import {
TIMELINE_DATA_PROVIDERS_EMPTY,
TIMELINE_DROPPED_DATA_PROVIDERS,
TIMELINE_DATA_PROVIDERS_ACTION_MENU,
IS_DRAGGING_DATA_PROVIDERS,
TIMELINE_FLYOUT_HEADER,
} from '../../screens/timeline';
import { HOSTS_NAMES_DRAGGABLE } from '../../screens/hosts/all_hosts';
@ -17,6 +18,7 @@ import { HOSTS_NAMES_DRAGGABLE } from '../../screens/hosts/all_hosts';
import {
dragAndDropFirstHostToTimeline,
dragFirstHostToEmptyTimelineDataProviders,
unDragFirstHostToEmptyTimelineDataProviders,
dragFirstHostToTimeline,
waitForAllHostsToBeLoaded,
} from '../../tasks/hosts/all_hosts';
@ -26,13 +28,14 @@ import { openTimelineUsingToggle } from '../../tasks/security_main';
import { addDataProvider, closeTimeline, createNewTimeline } from '../../tasks/timeline';
import { HOSTS_URL } from '../../urls/navigation';
import { cleanKibana } from '../../tasks/common';
import { cleanKibana, scrollToBottom } from '../../tasks/common';
describe('timeline data providers', () => {
before(() => {
cleanKibana();
loginAndWaitForPage(HOSTS_URL);
waitForAllHostsToBeLoaded();
scrollToBottom();
});
afterEach(() => {
@ -74,44 +77,24 @@ describe('timeline data providers', () => {
});
});
it.skip('sets the background to euiColorSuccess with a 10% alpha channel when the user starts dragging a host, but is not hovering over the data providers', () => {
it('sets correct classes when the user starts dragging a host, but is not hovering over the data providers', () => {
dragFirstHostToTimeline();
if (Cypress.browser.name === 'firefox') {
cy.get(TIMELINE_DATA_PROVIDERS)
.filter(':visible')
.should('have.css', 'background-color', 'rgba(1, 125, 115, 0.1)');
} else {
cy.get(TIMELINE_DATA_PROVIDERS)
.filter(':visible')
.should(
'have.css',
'background',
'rgba(1, 125, 115, 0.1) none repeat scroll 0% 0% / auto padding-box border-box'
);
}
cy.get(IS_DRAGGING_DATA_PROVIDERS)
.find(TIMELINE_DATA_PROVIDERS)
.filter(':visible')
.should('have.class', 'drop-target-data-providers');
});
// https://github.com/elastic/kibana/issues/94576
it.skip('sets the background to euiColorSuccess with a 20% alpha channel and renders the dashed border color as euiColorSuccess when the user starts dragging a host AND is hovering over the data providers', () => {
it('render an extra highlighted area in dataProvider when the user starts dragging a host AND is hovering over the data providers', () => {
dragFirstHostToEmptyTimelineDataProviders();
if (Cypress.browser.name === 'firefox') {
cy.get(TIMELINE_DATA_PROVIDERS_EMPTY)
.filter(':visible')
.should('have.css', 'background-color', 'rgba(1, 125, 115, 0.2)');
} else {
cy.get(TIMELINE_DATA_PROVIDERS_EMPTY)
.filter(':visible')
.should(
'have.css',
'background',
'rgba(1, 125, 115, 0.2) none repeat scroll 0% 0% / auto padding-box border-box'
);
cy.get(IS_DRAGGING_DATA_PROVIDERS)
.find(TIMELINE_DATA_PROVIDERS_EMPTY)
.children()
.should('exist');
cy.get(TIMELINE_DATA_PROVIDERS)
.filter(':visible')
.should('have.css', 'border', '3.1875px dashed rgb(1, 125, 115)');
}
// Release the dragging item so the cursor can peform other action
unDragFirstHostToEmptyTimelineDataProviders();
});
});

View file

@ -8,6 +8,7 @@
import { TIMELINE_BOTTOM_BAR_TOGGLE_BUTTON } from '../../screens/security_main';
import {
CREATE_NEW_TIMELINE,
IS_DRAGGING_DATA_PROVIDERS,
TIMELINE_DATA_PROVIDERS,
TIMELINE_FLYOUT_HEADER,
TIMELINE_SETTINGS_ICON,
@ -76,21 +77,12 @@ describe('timeline flyout button', () => {
closeTimelineUsingCloseButton();
});
it.skip('sets the data providers background to euiColorSuccess with a 10% alpha channel when the user starts dragging a host, but is not hovering over the data providers area', () => {
it('sets correct classes when the user starts dragging a host, but is not hovering over the data providers', () => {
dragFirstHostToTimeline();
if (Cypress.browser.name === 'firefox') {
cy.get(TIMELINE_DATA_PROVIDERS)
.filter(':visible')
.should('have.css', 'background-color', 'rgba(1, 125, 115, 0.1)');
} else {
cy.get(TIMELINE_DATA_PROVIDERS)
.filter(':visible')
.should(
'have.css',
'background',
'rgba(1, 125, 115, 0.1) none repeat scroll 0% 0% / auto padding-box border-box'
);
}
cy.get(IS_DRAGGING_DATA_PROVIDERS)
.find(TIMELINE_DATA_PROVIDERS)
.filter(':visible')
.should('have.class', 'drop-target-data-providers');
});
});

View file

@ -114,6 +114,8 @@ export const TIMELINE_CHANGES_IN_PROGRESS = '[data-test-subj="timeline"] .euiPro
export const TIMELINE_COLUMN_SPINNER = '[data-test-subj="timeline-loading-spinner"]';
export const IS_DRAGGING_DATA_PROVIDERS = '.is-dragging';
export const TIMELINE_DATA_PROVIDERS = '[data-test-subj="dataProviders"]';
export const TIMELINE_DATA_PROVIDERS_ACTION_MENU = '[data-test-subj="providerActions"]';

View file

@ -148,3 +148,5 @@ export const cleanKibana = () => {
esArchiverResetKibana();
};
export const scrollToBottom = () => cy.scrollTo('bottom');

View file

@ -29,6 +29,23 @@ export const dragFirstHostToEmptyTimelineDataProviders = () => {
.then((dataProvidersDropArea) => dragWithoutDrop(dataProvidersDropArea));
};
export const unDragFirstHostToEmptyTimelineDataProviders = () => {
cy.get(HOSTS_NAMES_DRAGGABLE)
.first()
.then((host) => {
cy.wrap(host)
.trigger('mousemove', {
button: 0,
clientX: host[0].getBoundingClientRect().left,
clientY: host[0].getBoundingClientRect().top,
force: true,
})
.wait(300)
.trigger('mouseup', { force: true })
.wait(300);
});
};
export const dragFirstHostToTimeline = () => {
cy.get(HOSTS_NAMES_DRAGGABLE)
.first()

View file

@ -18,7 +18,7 @@ import {
EuiText,
EuiTitle,
} from '@elastic/eui';
import React, { memo, useCallback, useEffect } from 'react';
import React, { memo, useCallback, useEffect, useMemo } from 'react';
import { EuiFlyoutProps } from '@elastic/eui/src/components/flyout/flyout';
import { FormattedMessage } from '@kbn/i18n/react';
import { useDispatch } from 'react-redux';
@ -31,7 +31,7 @@ import {
} from '../../store/selectors';
import { AppAction } from '../../../../../common/store/actions';
import { useTrustedAppsSelector } from '../hooks';
import { ABOUT_TRUSTED_APPS } from '../translations';
import { ABOUT_TRUSTED_APPS, CREATE_TRUSTED_APP_ERROR } from '../translations';
type CreateTrustedAppFlyoutProps = Omit<EuiFlyoutProps, 'hideCloseButton'>;
export const CreateTrustedAppFlyout = memo<CreateTrustedAppFlyoutProps>(
@ -45,6 +45,15 @@ export const CreateTrustedAppFlyout = memo<CreateTrustedAppFlyoutProps>(
const dataTestSubj = flyoutProps['data-test-subj'];
const creationErrorsMessage = useMemo<string | undefined>(
() =>
creationErrors
? CREATE_TRUSTED_APP_ERROR[creationErrors.message.replace(/(\[(.*)\]\: )/, '')] ||
creationErrors.message
: undefined,
[creationErrors]
);
const getTestId = useCallback(
(suffix: string): string | undefined => {
if (dataTestSubj) {
@ -102,7 +111,7 @@ export const CreateTrustedAppFlyout = memo<CreateTrustedAppFlyoutProps>(
fullWidth
onChange={handleFormOnChange}
isInvalid={!!creationErrors}
error={creationErrors?.message}
error={creationErrorsMessage}
data-test-subj={getTestId('createForm')}
/>
</EuiFlyoutBody>

View file

@ -267,6 +267,11 @@ describe('When showing the Trusted App Create Form', () => {
expect(renderResult.getByText('Name is required'));
});
it('should validate invalid Hash value', () => {
setTextFieldValue(getConditionValue(getCondition(renderResult)), 'someHASH');
expect(renderResult.getByText('[1] Invalid hash value'));
});
it('should validate that a condition value has a non empty space value', () => {
setTextFieldValue(getConditionValue(getCondition(renderResult)), ' ');
expect(renderResult.getByText('[1] Field entry must have a value'));
@ -281,13 +286,27 @@ describe('When showing the Trusted App Create Form', () => {
setTextFieldValue(getConditionValue(getCondition(renderResult)), 'someHASH');
expect(renderResult.getByText('[2] Field entry must have a value'));
});
it('should validate multiple errors in form', () => {
const andButton = getConditionBuilderAndButton(renderResult);
reactTestingLibrary.act(() => {
fireEvent.click(andButton, { button: 1 });
});
setTextFieldValue(getConditionValue(getCondition(renderResult)), 'someHASH');
expect(renderResult.getByText('[1] Invalid hash value'));
expect(renderResult.getByText('[2] Field entry must have a value'));
});
});
describe('and all required data passes validation', () => {
it('should call change callback with isValid set to true and contain the new item', () => {
const renderResult = render();
setTextFieldValue(getNameField(renderResult), 'Some Process');
setTextFieldValue(getConditionValue(getCondition(renderResult)), 'someHASH');
setTextFieldValue(
getConditionValue(getCondition(renderResult)),
'e50fb1a0e5fff590ece385082edc6c41'
);
setTextFieldValue(getDescriptionField(renderResult), 'some description');
expect(getAllValidationErrors(renderResult)).toHaveLength(0);
@ -300,7 +319,7 @@ describe('When showing the Trusted App Create Form', () => {
field: ConditionEntryField.HASH,
operator: 'included',
type: 'match',
value: 'someHASH',
value: 'e50fb1a0e5fff590ece385082edc6c41',
},
],
name: 'Some Process',

View file

@ -17,10 +17,13 @@ import {
import { i18n } from '@kbn/i18n';
import { EuiFormProps } from '@elastic/eui/src/components/form/form';
import {
ConditionEntryField,
MacosLinuxConditionEntry,
NewTrustedApp,
OperatingSystem,
} from '../../../../../../common/endpoint/types';
import { isValidHash } from '../../../../../../common/endpoint/validation/trusted_apps';
import {
isMacosLinuxTrustedAppCondition,
isWindowsTrustedAppCondition,
@ -113,7 +116,7 @@ const validateFormValues = (values: NewTrustedApp): ValidationResult => {
})
);
} else {
values.entries.some((entry, index) => {
values.entries.forEach((entry, index) => {
if (!entry.field || !entry.value.trim()) {
isValid = false;
addResultToValidation(
@ -128,9 +131,18 @@ const validateFormValues = (values: NewTrustedApp): ValidationResult => {
}
)
);
return true;
} else if (entry.field === ConditionEntryField.HASH && !isValidHash(entry.value)) {
isValid = false;
addResultToValidation(
validation,
'entries',
'errors',
i18n.translate('xpack.securitySolution.trustedapps.create.conditionFieldInvalidHashMsg', {
defaultMessage: '[{row}] Invalid hash value',
values: { row: index + 1 },
})
);
}
return false;
});
}

View file

@ -137,3 +137,36 @@ export const LIST_VIEW_TOGGLE_LABEL = i18n.translate(
export const NO_RESULTS_MESSAGE = i18n.translate('xpack.securitySolution.trustedapps.noResults', {
defaultMessage: 'No items found',
});
export const CREATE_TRUSTED_APP_ERROR: { [K in string]: string } = {
[`duplicatedEntry.${ConditionEntryField.HASH}`]: i18n.translate(
'xpack.securitySolution.trustedapps.logicalConditionBuilder.entry.field.error.duplicated.hash',
{ defaultMessage: 'Hash value can only be used once. Please enter a single valid hash.' }
),
[`duplicatedEntry.${ConditionEntryField.PATH}`]: i18n.translate(
'xpack.securitySolution.trustedapps.logicalConditionBuilder.entry.field.error.duplicated.path',
{ defaultMessage: 'Path value can only be used once. Please enter a single valid path.' }
),
[`duplicatedEntry.${ConditionEntryField.SIGNER}`]: i18n.translate(
'xpack.securitySolution.trustedapps.logicalConditionBuilder.entry.field.error.duplicated.signature',
{
defaultMessage:
'Signature value can only be used once. Please enter a single valid signature.',
}
),
[`invalidField.${ConditionEntryField.HASH}`]: i18n.translate(
'xpack.securitySolution.trustedapps.logicalConditionBuilder.entry.field.error.invalid.hash',
{
defaultMessage:
'An invalid Hash was entered. Please enter in a valid Hash (md5, sha1, or sha256).',
}
),
[`invalidField.${ConditionEntryField.PATH}`]: i18n.translate(
'xpack.securitySolution.trustedapps.logicalConditionBuilder.entry.field.error.invalid.path',
{ defaultMessage: 'An invalid Path was entered. Please enter in a valid Path.' }
),
[`invalidField.${ConditionEntryField.SIGNER}`]: i18n.translate(
'xpack.securitySolution.trustedapps.logicalConditionBuilder.entry.field.error.invalid.signature',
{ defaultMessage: 'An invalid Signature was entered. Please enter in a valid Signature.' }
),
};

View file

@ -201,7 +201,7 @@ describe('When on the Trusted Apps Page', () => {
fireEvent.change(
getByTestId('addTrustedAppFlyout-createForm-conditionsBuilder-group1-entry0-value'),
{ target: { value: 'SOME$HASH#HERE' } }
{ target: { value: '44ed10b389dbcd1cf16cec79d16d7378' } }
);
fireEvent.change(getByTestId('addTrustedAppFlyout-createForm-descriptionField'), {
@ -363,6 +363,29 @@ describe('When on the Trusted Apps Page', () => {
});
});
});
describe('and when the form data is not valid', () => {
it('should not enable the Flyout Add button with an invalid hash', async () => {
const renderResult = await renderAndClickAddButton();
const { getByTestId } = renderResult;
reactTestingLibrary.act(() => {
fireEvent.change(getByTestId('addTrustedAppFlyout-createForm-nameTextField'), {
target: { value: 'trusted app A' },
});
fireEvent.change(
getByTestId('addTrustedAppFlyout-createForm-conditionsBuilder-group1-entry0-value'),
{ target: { value: 'invalid hash' } }
);
});
const flyoutAddButton = getByTestId(
'addTrustedAppFlyout-createButton'
) as HTMLButtonElement;
expect(flyoutAddButton.disabled).toBe(true);
});
});
});
describe('and there are no trusted apps', () => {

View file

@ -8,6 +8,12 @@
import { schema, TypeOf } from '@kbn/config-schema';
import { PluginInitializerContext } from '../../../../src/core/server';
import { SIGNALS_INDEX_KEY, DEFAULT_SIGNALS_INDEX } from '../common/constants';
import {
getExperimentalAllowedValues,
isValidExperimentalValue,
} from '../common/experimental_features';
const allowedExperimentalValues = getExperimentalAllowedValues();
export const configSchema = schema.object({
enabled: schema.boolean({ defaultValue: true }),
@ -17,8 +23,30 @@ export const configSchema = schema.object({
maxTimelineImportPayloadBytes: schema.number({ defaultValue: 10485760 }),
[SIGNALS_INDEX_KEY]: schema.string({ defaultValue: DEFAULT_SIGNALS_INDEX }),
/** Fleet server integration */
fleetServerEnabled: schema.boolean({ defaultValue: false }),
/**
* For internal use. A list of string values (comma delimited) that will enable experimental
* type of functionality that is not yet released. Valid values for this settings need to
* be defined in:
* `x-pack/plugins/security_solution/common/experimental_features.ts`
* under the `allowedExperimentalValues` object
*
* @example
* xpack.securitySolution.enableExperimental:
* - fleetServerEnabled
* - trustedAppsByPolicyEnabled
*/
enableExperimental: schema.arrayOf(schema.string(), {
defaultValue: () => [],
validate(list) {
for (const key of list) {
if (!isValidExperimentalValue(key)) {
return `[${key}] is not allowed. Allowed values are: ${allowedExperimentalValues.join(
', '
)}`;
}
}
},
}),
/**
* Host Endpoint Configuration

View file

@ -21,7 +21,7 @@ export const createMockConfig = (): ConfigType => ({
maxRuleImportPayloadBytes: 10485760,
maxTimelineImportExportSize: 10000,
maxTimelineImportPayloadBytes: 10485760,
fleetServerEnabled: true,
enableExperimental: [],
endpointResultListDefaultFirstPageIndex: 0,
endpointResultListDefaultPageSize: 10,
alertResultListDefaultDateRange: {

View file

@ -77,6 +77,7 @@ import {
import { licenseService } from './lib/license/license';
import { PolicyWatcher } from './endpoint/lib/policy/license_watch';
import { securitySolutionTimelineEqlSearchStrategyProvider } from './search_strategy/timeline/eql';
import { parseExperimentalConfigValue } from '../common/experimental_features';
export interface SetupPlugins {
alerting: AlertingSetup;
@ -357,7 +358,7 @@ export class Plugin implements IPlugin<PluginSetup, PluginStart, SetupPlugins, S
logger: this.logger,
cache: this.artifactsCache,
},
this.config.fleetServerEnabled
parseExperimentalConfigValue(this.config.enableExperimental).fleetServerEnabled
);
if (this.manifestTask) {