maxmustermann/kibana
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[Lens] Formula editor (#99297) (#101900)

Browse source 
* 💄 Hack to fix suggestion box

* 🐛 Fix validation messages

* 🐛 Relax operations check for managedReferences

* Change completion params

* 🏷️ Fix missing arg issue

* ✨ Add more tinymath fns

* 🐛 Improved validation around math operations + multiple named arguments

* 🐛 Use new onError feature in math expression

* ♻️ Refactor namedArguments validation

* 🐛 Fix circular dependency issue in tests + minor fixes

* Move formula into a tab

* 🔥 Leftovers from previous merge

* ✨ Move over namedArgs from previous function

* ✅ Add tests for transferable scenarios

* ✅ Fixed broken test

* ✨ Use custom label for axis

* Allow switching back and forth to formula tab

* Add a section for the function reference

* Add modal editor and markdown docs

* Change the way math nodes are validated

* Use custom portal to fix monaco positioning

* Fix model sharing issues

* Provide signature help

* 🐛 Fix small test issue

* 🐛 Mark pow arguments as required

* 🐛 validate on first render only if a formula is present

* 🔥 Remove log10 fn for now

* ✨ Improved math validation + add tests for math functions

* Fix mount/unmount issues with Monaco

* [Lens] Fully unmount React when flyout closes

* Fix bug with editor frame unmounting

* Fix type

* Add tests for monaco providers, add hover provider

* Add test for last_value

* Usability improvements

* Add KQL and Lucene named parameters

* Add kql, lucene completion and validation

* Fix autocomplete on weird characters and properly connect KQL

* Highlight functions that have additional requirements after validating

* Fix type error and move help text to popover

* Fix escape characters inside KQL

* 🐛 Fix dataType issue when moving over to Formula

* Automatically insert single quotes on every named param

* Only insert single quotes when typing kql= or lucene=

* Reorganize help popover

* Fix merge issues

* Update grammar for formulas

* Fix bad merge

* Rough fullscreen mode

* Type updates

* Pass through fullscreen state

* Remove more chrome from full screen mode

* Fix minor bugs in formula typing

* 🐛 Decouple column order of references and output

* 🔧 Fix tests and types

* ✅ Add first functional test

* Fix copying formulas and empty formula

* Trigger suggestion prompt when hitting enter on function or typing kql=

* 🐛 Prevent flyout from closing while interacting with monaco

* refactoring

* move main column generation into parse module

* fix tests

* refactor small formula styles and markup

* documentation

* adjustments in formula footer

* Formula refactoring (#12)

* refactoring

* move main column generation into parse module

* fix tests

* more style and markup tweak for custom formula

* Fix tests

* [Expressions] Use table column ID instead of name when set

* [Lens] Create managedReference type for formulas

* Fix test failures

* Fix i18n types

* fix fullscreen flex issues

* Delete managedReference when replacing

* refactor css and markup; add button placeholders

* [Lens] Formulas

* Tests for formula

Co-authored-by: Marco Liberati <marco.liberati@elastic.co>

* added error count placeholder

* Add tooltips

* Refactoring from code review

* Fix some editor issues

* Update ID matching to match by name sometimes

* Improve performance of Monaco, fix formulas with 0, update labels

* Improve performance of full screen toggle

* Fix formula tests

* fix stuff

* Add an extra case to prevent insertion of duplicate column

* Simplify logic and add test for output ID

* add telemetry for Lens formula (#15)

* Respond to review comments

* ✨ Improve the signatures with better documentation and examples

* adjust border styles to account for docs collapse

* refactor docs markup; restructure docs obj; styles

* Fix formula auto reordering (#18)

* fix formula auto reordering

* add unit test

* Fix and improve suggestion experience in Formula (#19)

* ✨ Revisit documentation and suggestions

* 👌 Integrated feedback

* ✨ Add query validation for quotes

* Usability updates & type fixes

* add search to formula

* fix form styles to match designs

* fix text styles; revert to Markdown for control

* 👌 Integrated more feedback

* improve search

* improve suggestions

* improve suggestions even more

* 🐛 Fix i18n issues (#22)

* Persist formula on leave, fix fullscreen and popovers

* Fix documentation tests

* 🏷️ fix type issue

* 🐛 Remove hidden operations from valid functions list

* 🐛 Fix empty string query edge case

* 🐛 Enable more suggestions + extends validation

* Fix tests that depended on setState being called without function

* Error state and text wrapping updates

* ✨ Add new module to CodeEditor for brackets matching (#25)

* Fix type

* show warning

* keep current quick function

* ✨ Improve suggestions within kql query

* 📷 Fix snapshot editor test

* 🐛 Improved suggestion for single quote and refactored debounce

* Fix lodash usage

* Fix tests

* Revert "keep current quick function"

This reverts commit ed477054c5.

* Improve performance of dispatch by using timeout

* Improve memoization of datapanel

* Fix escape characters

* fix reduced suggestions

* fix responsiveness

* fix unit test

* Fix autocomplete on nested math

* Show errors and warnings on first render

* fix transposing column crash

* Update comment

* 🐛 Fix field error message

* fix test types

* 📝 Fix i18n name

* 💄 Manage wordwrap via react component

* Fix selector for palettes that interferes with quick functions

* Use word wrapping by default

* Errors for managed references are handled at the top level

* 🐛 Move the cursor just next to new inserted text

* ⚗️ First pass for performance

* 🐛 Fix unwanted change

* ⚡ Memoize as many combobox props as possible

* ⚡ More memoization

* Show errors in hover

* Use temporary invalid state when moving away from formula

* Remove setActiveDimension and shouldClose, fixed by async setters

* Fix test dependency

* do not show quick functions tab

* increase documentation popover width

* fix functional test

* Call setActiveDimension when updating visualization

* Simplify handling of flyout with incomplete columns

* Fix test issues

* add description to formula telemetry

* fix schema

* Update from design feedback

* More review comments

* Hide callout border from v7 theme

Co-authored-by: dej611 <dej611@gmail.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: Michael Marcialis <michael.marcialis@elastic.co>
Co-authored-by: Joe Reuter <email@johannes-reuter.de>
Co-authored-by: Marco Liberati <marco.liberati@elastic.co>
Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>

Co-authored-by: Wylie Conlon <william.conlon@elastic.co>
Co-authored-by: dej611 <dej611@gmail.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: Michael Marcialis <michael.marcialis@elastic.co>
Co-authored-by: Joe Reuter <email@johannes-reuter.de>
Co-authored-by: Marco Liberati <marco.liberati@elastic.co>
Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
This commit is contained in:
Kibana Machine 2021-06-10 12:14:20 -04:00 committed by GitHub
parent 50a75c5522
commit b32927aa67
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
84 changed files with 5116 additions and 658 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
packages/kbn-monaco/src/monaco_imports.ts
View file

@ -21,5 +21,6 @@ import 'monaco-editor/esm/vs/editor/contrib/folding/folding.js'; // Needed for f
import 'monaco-editor/esm/vs/editor/contrib/suggest/suggestController.js'; // Needed for suggestions
import 'monaco-editor/esm/vs/editor/contrib/hover/hover.js'; // Needed for hover
import 'monaco-editor/esm/vs/editor/contrib/parameterHints/parameterHints.js'; // Needed for signature
import 'monaco-editor/esm/vs/editor/contrib/bracketMatching/bracketMatching.js'; // Needed for brackets matching highlight
export { monaco };

44
packages/kbn-tinymath/grammar/grammar.peggy
View file

@ -1,16 +1,16 @@
// tinymath parsing grammar
{
function simpleLocation (location) {
// Returns an object representing the position of the function within the expression,
// demarcated by the position of its first character and last character. We calculate these values
// using the offset because the expression could span multiple lines, and we don't want to deal
// with column and line values.
return {
min: location.start.offset,
max: location.end.offset
function simpleLocation (location) {
// Returns an object representing the position of the function within the expression,
// demarcated by the position of its first character and last character. We calculate these values
// using the offset because the expression could span multiple lines, and we don't want to deal
// with column and line values.
return {
min: location.start.offset,
max: location.end.offset
}
}
}
}
start
@ -74,26 +74,34 @@ Expression
= AddSubtract
AddSubtract
= _ left:MultiplyDivide rest:(('+' / '-') MultiplyDivide)* _ {
return rest.reduce((acc, curr) => ({
= _ left:MultiplyDivide rest:(('+' / '-') MultiplyDivide)+ _ {
const topLevel = rest.reduce((acc, curr) => ({
type: 'function',
name: curr[0] === '+' ? 'add' : 'subtract',
args: [acc, curr[1]],
location: simpleLocation(location()),
text: text()
}), left)
}), left);
if (typeof topLevel === 'object') {
topLevel.location = simpleLocation(location());
topLevel.text = text();
}
return topLevel;
}
/ MultiplyDivide
MultiplyDivide
= _ left:Factor rest:(('*' / '/') Factor)* _ {
return rest.reduce((acc, curr) => ({
const topLevel = rest.reduce((acc, curr) => ({
type: 'function',
name: curr[0] === '*' ? 'multiply' : 'divide',
args: [acc, curr[1]],
location: simpleLocation(location()),
text: text()
}), left)
}), left);
if (typeof topLevel === 'object') {
topLevel.location = simpleLocation(location());
topLevel.text = text();
}
return topLevel;
}
/ Factor
Factor
= Group

6
packages/kbn-tinymath/index.d.ts vendored
View file

@ -24,9 +24,11 @@ export interface TinymathLocation {
export interface TinymathFunction {
type: 'function';
name: string;
text: string;
args: TinymathAST[];
location: TinymathLocation;
// Location is not guaranteed because PEG grammars are not left-recursive
location?: TinymathLocation;
// Text is not guaranteed because PEG grammars are not left-recursive
text?: string;
}
export interface TinymathVariable {

31
packages/kbn-tinymath/test/library.test.js
View file

@ -41,6 +41,35 @@ describe('Parser', () => {
});
});
describe('Math', () => {
it('converts basic symbols into left-to-right pairs', () => {
expect(parse('a + b + c - d')).toEqual({
args: [
{
name: 'add',
type: 'function',
args: [
{
name: 'add',
type: 'function',
args: [
expect.objectContaining({ location: { min: 0, max: 2 } }),
expect.objectContaining({ location: { min: 3, max: 6 } }),
],
},
expect.objectContaining({ location: { min: 7, max: 10 } }),
],
},
expect.objectContaining({ location: { min: 11, max: 13 } }),
],
name: 'subtract',
type: 'function',
text: 'a + b + c - d',
location: { min: 0, max: 13 },
});
});
});
describe('Variables', () => {
it('strings', () => {
expect(parse('f')).toEqual(variableEqual('f'));
@ -263,6 +292,8 @@ describe('Evaluate', () => {
expect(evaluate('5/20')).toEqual(0.25);
expect(evaluate('1 + 1 + 2 + 3 + 12')).toEqual(19);
expect(evaluate('100 / 10 / 10')).toEqual(1);
expect(evaluate('0 * 1 - 100 / 10 / 10')).toEqual(-1);
expect(evaluate('100 / (10 / 10)')).toEqual(100);
});
it('equations with functions', () => {

17
src/plugins/dashboard/server/usage/dashboard_telemetry.test.ts
View file

@ -72,6 +72,22 @@ const lensXYSeriesB = ({
visualization: {
preferredSeriesType: 'seriesB',
},
datasourceStates: {
indexpattern: {
layers: {
first: {
columns: {
first: {
operationType: 'terms',
},
second: {
operationType: 'formula',
},
},
},
},
},
},
},
},
},
@ -144,6 +160,7 @@ describe('dashboard telemetry', () => {
expect(collectorData.lensByValue.a).toBe(3);
expect(collectorData.lensByValue.seriesA).toBe(2);
expect(collectorData.lensByValue.seriesB).toBe(1);
expect(collectorData.lensByValue.formula).toBe(1);
});
it('handles misshapen lens panels', () => {

23
src/plugins/dashboard/server/usage/dashboard_telemetry.ts
View file

@ -27,6 +27,16 @@ interface LensPanel extends SavedDashboardPanel730ToLatest {
visualization?: {
preferredSeriesType?: string;
};
datasourceStates?: {
indexpattern?: {
layers: Record<
string,
{
columns: Record<string, { operationType: string }>;
}
>;
};
};
};
};
};
@ -109,6 +119,19 @@ export const collectByValueLensInfo: DashboardCollectorFunction = (panels, colle
}
collectorData.lensByValue[type] = collectorData.lensByValue[type] + 1;
const hasFormula = Object.values(
lensPanel.embeddableConfig.attributes.state?.datasourceStates?.indexpattern?.layers || {}
).some((layer) =>
Object.values(layer.columns).some((column) => column.operationType === 'formula')
);
if (hasFormula && !collectorData.lensByValue.formula) {
collectorData.lensByValue.formula = 0;
}
if (hasFormula) {
collectorData.lensByValue.formula++;
}
}
}
};

6
src/plugins/expressions/common/expression_functions/specs/map_column.ts
View file

@ -10,7 +10,7 @@ import { Observable } from 'rxjs';
import { take } from 'rxjs/operators';
import { i18n } from '@kbn/i18n';
import { ExpressionFunctionDefinition } from '../types';
import { Datatable, getType } from '../../expression_types';
import { Datatable, DatatableColumn, getType } from '../../expression_types';
export interface MapColumnArguments {
id?: string | null;
@ -110,10 +110,10 @@ export const mapColumn: ExpressionFunctionDefinition<
return Promise.all(rowPromises).then((rows) => {
const type = rows.length ? getType(rows[0][columnId]) : 'null';
const newColumn = {
const newColumn: DatatableColumn = {
id: columnId,
name: args.name,
meta: { type },
meta: { type, params: { id: type } },
};
if (args.copyMetaFrom) {
const metaSourceFrom = columns.find(({ id }) => id === args.copyMetaFrom);

6
src/plugins/expressions/common/expression_functions/specs/tests/map_column.test.ts
View file

@ -29,7 +29,11 @@ describe('mapColumn', () => {
expect(result.type).toBe('datatable');
expect(result.columns).toEqual([
...testTable.columns,
{ id: 'pricePlusTwo', name: 'pricePlusTwo', meta: { type: 'number' } },
{
id: 'pricePlusTwo',
name: 'pricePlusTwo',
meta: { type: 'number', params: { id: 'number' } },
},
]);
expect(result.columns[result.columns.length - 1]).toHaveProperty('name', 'pricePlusTwo');
expect(result.rows[arbitraryRowIndex]).toHaveProperty('pricePlusTwo');

2
src/plugins/kibana_react/public/code_editor/__snapshots__/code_editor.test.tsx.snap generated
View file

@ -11,6 +11,7 @@ exports[`is rendered 1`] = `
onChange={[Function]}
options={
Object {
"matchBrackets": "never",
"minimap": Object {
"enabled": false,
},
@ -39,6 +40,7 @@ exports[`is rendered 1`] = `
nodeType="div"
onResize={[Function]}
querySelector={null}
refreshMode="debounce"
refreshRate={1000}
skipOnMount={false}
targetDomEl={null}

8
src/plugins/kibana_react/public/code_editor/code_editor.tsx
View file

@ -187,10 +187,16 @@ export class CodeEditor extends React.Component<Props, {}> {
wordBasedSuggestions: false,
wordWrap: 'on',
wrappingIndent: 'indent',
matchBrackets: 'never',
...options,
}}
/>
<ReactResizeDetector handleWidth handleHeight onResize={this._updateDimensions} />
<ReactResizeDetector
handleWidth
handleHeight
onResize={this._updateDimensions}
refreshMode="debounce"
/>
</>
);
}

2
x-pack/plugins/canvas/shareable_runtime/webpack.config.js
View file

@ -38,7 +38,7 @@ module.exports = {
'src/plugins/data/public/expressions/interpreter'
),
'kbn/interpreter': path.resolve(KIBANA_ROOT, 'packages/kbn-interpreter/target/common'),
tinymath: path.resolve(KIBANA_ROOT, 'node_modules/tinymath/lib/tinymath.es5.js'),
tinymath: path.resolve(KIBANA_ROOT, 'node_modules/tinymath/lib/tinymath.min.js'),
core_app_image_assets: path.resolve(KIBANA_ROOT, 'src/core/public/core_app/images'),
},
extensions: ['.js', '.json', '.ts', '.tsx', '.scss'],

251
x-pack/plugins/lens/public/app_plugin/lens_top_nav.tsx
View file

@ -7,7 +7,7 @@
import { isEqual } from 'lodash';
import { i18n } from '@kbn/i18n';
import React from 'react';
import React, { useCallback, useMemo } from 'react';
import { TopNavMenuData } from '../../../../../src/plugins/navigation/public';
import { LensAppServices, LensTopNavActions, LensTopNavMenuProps } from './types';
import { downloadMultipleAs } from '../../../../../src/plugins/share/public';
@ -164,79 +164,152 @@ export const LensTopNavMenu = ({
const unsavedTitle = i18n.translate('xpack.lens.app.unsavedFilename', {
defaultMessage: 'unsaved',
});
const topNavConfig = getLensTopNavConfig({
showSaveAndReturn: Boolean(
isLinkedToOriginatingApp &&
// Temporarily required until the 'by value' paradigm is default.
(dashboardFeatureFlag.allowByValueEmbeddables || Boolean(initialInput))
),
enableExportToCSV: Boolean(isSaveable && activeData && Object.keys(activeData).length),
isByValueMode: getIsByValueMode(),
allowByValue: dashboardFeatureFlag.allowByValueEmbeddables,
showCancel: Boolean(isLinkedToOriginatingApp),
savingToLibraryPermitted,
savingToDashboardPermitted,
actions: {
exportToCSV: () => {
if (!activeData) {
return;
}
const datatables = Object.values(activeData);
const content = datatables.reduce<Record<string, { content: string; type: string }>>(
(memo, datatable, i) => {
// skip empty datatables
if (datatable) {
const postFix = datatables.length > 1 ? `-${i + 1}` : '';
const topNavConfig = useMemo(
() =>
getLensTopNavConfig({
showSaveAndReturn: Boolean(
isLinkedToOriginatingApp &&
// Temporarily required until the 'by value' paradigm is default.
(dashboardFeatureFlag.allowByValueEmbeddables || Boolean(initialInput))
),
enableExportToCSV: Boolean(isSaveable && activeData && Object.keys(activeData).length),
isByValueMode: getIsByValueMode(),
allowByValue: dashboardFeatureFlag.allowByValueEmbeddables,
showCancel: Boolean(isLinkedToOriginatingApp),
savingToLibraryPermitted,
savingToDashboardPermitted,
actions: {
exportToCSV: () => {
if (!activeData) {
return;
}
const datatables = Object.values(activeData);
const content = datatables.reduce<Record<string, { content: string; type: string }>>(
(memo, datatable, i) => {
// skip empty datatables
if (datatable) {
const postFix = datatables.length > 1 ? `-${i + 1}` : '';
memo[`${lastKnownDoc?.title || unsavedTitle}${postFix}.csv`] = {
content: exporters.datatableToCSV(datatable, {
csvSeparator: uiSettings.get('csv:separator', ','),
quoteValues: uiSettings.get('csv:quoteValues', true),
formatFactory: data.fieldFormats.deserialize,
}),
type: exporters.CSV_MIME_TYPE,
};
memo[`${lastKnownDoc?.title || unsavedTitle}${postFix}.csv`] = {
content: exporters.datatableToCSV(datatable, {
csvSeparator: uiSettings.get('csv:separator', ','),
quoteValues: uiSettings.get('csv:quoteValues', true),
formatFactory: data.fieldFormats.deserialize,
}),
type: exporters.CSV_MIME_TYPE,
};
}
return memo;
},
{}
);
if (content) {
downloadMultipleAs(content);
}
return memo;
},
{}
);
if (content) {
downloadMultipleAs(content);
}
},
saveAndReturn: () => {
if (savingToDashboardPermitted && lastKnownDoc) {
// disabling the validation on app leave because the document has been saved.
onAppLeave((actions) => {
return actions.default();
});
runSave(
{
newTitle: lastKnownDoc.title,
newCopyOnSave: false,
isTitleDuplicateConfirmed: false,
returnToOrigin: true,
},
{
saveToLibrary:
(initialInput && attributeService.inputIsRefType(initialInput)) ?? false,
saveAndReturn: () => {
if (savingToDashboardPermitted && lastKnownDoc) {
// disabling the validation on app leave because the document has been saved.
onAppLeave((actions) => {
return actions.default();
});
runSave(
{
newTitle: lastKnownDoc.title,
newCopyOnSave: false,
isTitleDuplicateConfirmed: false,
returnToOrigin: true,
},
{
saveToLibrary:
(initialInput && attributeService.inputIsRefType(initialInput)) ?? false,
}
);
}
);
},
showSaveModal: () => {
if (savingToDashboardPermitted || savingToLibraryPermitted) {
setIsSaveModalVisible(true);
}
},
cancel: () => {
if (redirectToOrigin) {
redirectToOrigin();
}
},
},
}),
[
activeData,
attributeService,
dashboardFeatureFlag.allowByValueEmbeddables,
data.fieldFormats.deserialize,
getIsByValueMode,
initialInput,
isLinkedToOriginatingApp,
isSaveable,
lastKnownDoc,
onAppLeave,
redirectToOrigin,
runSave,
savingToDashboardPermitted,
savingToLibraryPermitted,
setIsSaveModalVisible,
uiSettings,
unsavedTitle,
]
);
const onQuerySubmitWrapped = useCallback(
(payload) => {
const { dateRange, query: newQuery } = payload;
const currentRange = data.query.timefilter.timefilter.getTime();
if (dateRange.from !== currentRange.from || dateRange.to !== currentRange.to) {
data.query.timefilter.timefilter.setTime(dateRange);
trackUiEvent('app_date_change');
} else {
// Query has changed, renew the session id.
// Time change will be picked up by the time subscription
dispatchSetState({ searchSessionId: data.search.session.start() });
trackUiEvent('app_query_change');
}
if (newQuery) {
if (!isEqual(newQuery, query)) {
dispatchSetState({ query: newQuery });
}
},
showSaveModal: () => {
if (savingToDashboardPermitted || savingToLibraryPermitted) {
setIsSaveModalVisible(true);
}
},
cancel: () => {
if (redirectToOrigin) {
redirectToOrigin();
}
},
}
},
});
[data.query.timefilter.timefilter, data.search.session, dispatchSetState, query]
);
const onSavedWrapped = useCallback(
(newSavedQuery) => {
dispatchSetState({ savedQuery: newSavedQuery });
},
[dispatchSetState]
);
const onSavedQueryUpdatedWrapped = useCallback(
(newSavedQuery) => {
const savedQueryFilters = newSavedQuery.attributes.filters || [];
const globalFilters = data.query.filterManager.getGlobalFilters();
data.query.filterManager.setFilters([...globalFilters, ...savedQueryFilters]);
dispatchSetState({
query: newSavedQuery.attributes.query,
savedQuery: { ...newSavedQuery },
}); // Shallow query for reference issues
},
[data.query.filterManager, dispatchSetState]
);
const onClearSavedQueryWrapped = useCallback(() => {
data.query.filterManager.setFilters(data.query.filterManager.getGlobalFilters());
dispatchSetState({
filters: data.query.filterManager.getGlobalFilters(),
query: data.query.queryString.getDefaultQuery(),
savedQuery: undefined,
});
}, [data.query.filterManager, data.query.queryString, dispatchSetState]);
return (
<TopNavMenu
@ -244,44 +317,10 @@ export const LensTopNavMenu = ({
config={topNavConfig}
showSaveQuery={Boolean(application.capabilities.visualize.saveQuery)}
savedQuery={savedQuery}
onQuerySubmit={(payload) => {
const { dateRange, query: newQuery } = payload;
const currentRange = data.query.timefilter.timefilter.getTime();
if (dateRange.from !== currentRange.from || dateRange.to !== currentRange.to) {
data.query.timefilter.timefilter.setTime(dateRange);
trackUiEvent('app_date_change');
} else {
// Query has changed, renew the session id.
// Time change will be picked up by the time subscription
dispatchSetState({ searchSessionId: data.search.session.start() });
trackUiEvent('app_query_change');
}
if (newQuery) {
if (!isEqual(newQuery, query)) {
dispatchSetState({ query: newQuery });
}
}
}}
onSaved={(newSavedQuery) => {
dispatchSetState({ savedQuery: newSavedQuery });
}}
onSavedQueryUpdated={(newSavedQuery) => {
const savedQueryFilters = newSavedQuery.attributes.filters || [];
const globalFilters = data.query.filterManager.getGlobalFilters();
data.query.filterManager.setFilters([...globalFilters, ...savedQueryFilters]);
dispatchSetState({
query: newSavedQuery.attributes.query,
savedQuery: { ...newSavedQuery },
}); // Shallow query for reference issues
}}
onClearSavedQuery={() => {
data.query.filterManager.setFilters(data.query.filterManager.getGlobalFilters());
dispatchSetState({
filters: data.query.filterManager.getGlobalFilters(),
query: data.query.queryString.getDefaultQuery(),
savedQuery: undefined,
});
}}
onQuerySubmit={onQuerySubmitWrapped}
onSaved={onSavedWrapped}
onSavedQueryUpdated={onSavedQueryUpdatedWrapped}
onClearSavedQuery={onClearSavedQueryWrapped}
indexPatterns={indexPatternsForTopNav}
query={query}
dateRangeFrom={from}

2
x-pack/plugins/lens/public/datatable_visualization/components/columns.tsx
View file

@ -51,7 +51,7 @@ export const createGridColumns = (
columnId,
}: Pick<EuiDataGridColumnCellActionProps, 'rowIndex' | 'columnId'>) => {
const rowValue = table.rows[rowIndex][columnId];
const column = columnsReverseLookup[columnId];
const column = columnsReverseLookup?.[columnId];
const contentsIsDefined = rowValue != null;
const cellContent = formatFactory(column?.meta?.params).convert(rowValue);

8
x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.test.tsx
View file

@ -76,6 +76,8 @@ describe('ConfigPanel', () => {
framePublicAPI: frame,
dispatch: jest.fn(),
core: coreMock.createStart(),
isFullscreen: false,
toggleFullscreen: jest.fn(),
};
}
@ -119,19 +121,23 @@ describe('ConfigPanel', () => {
expect(component.find(LayerPanel).exists()).toBe(false);
});
it('allow datasources and visualizations to use setters', () => {
it('allow datasources and visualizations to use setters', async () => {
const props = getDefaultProps();
const component = mountWithIntl(<LayerPanels {...props} />);
const { updateDatasource, updateAll } = component.find(LayerPanel).props();
const updater = () => 'updated';
updateDatasource('ds1', updater);
// wait for one tick so async updater has a chance to trigger
await new Promise((r) => setTimeout(r, 0));
expect(props.dispatch).toHaveBeenCalledTimes(1);
expect(props.dispatch.mock.calls[0][0].updater(props.datasourceStates.ds1.state)).toEqual(
'updated'
);
updateAll('ds1', updater, props.visualizationState);
// wait for one tick so async updater has a chance to trigger
await new Promise((r) => setTimeout(r, 0));
expect(props.dispatch).toHaveBeenCalledTimes(2);
expect(props.dispatch.mock.calls[0][0].updater(props.datasourceStates.ds1.state)).toEqual(
'updated'

70
x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/config_panel.tsx
View file

@ -71,32 +71,54 @@ export function LayerPanels(
},
[dispatch]
);
const updateDatasourceAsync = useMemo(
() => (datasourceId: string, newState: unknown) => {
// React will synchronously update if this is triggered from a third party component,
// which we don't want. The timeout lets user interaction have priority, then React updates.
setTimeout(() => {
updateDatasource(datasourceId, newState);
}, 0);
},
[updateDatasource]
);
const updateAll = useMemo(
() => (datasourceId: string, newDatasourceState: unknown, newVisualizationState: unknown) => {
dispatch({
type: 'UPDATE_STATE',
subType: 'UPDATE_ALL_STATES',
updater: (prevState) => {
const updatedDatasourceState =
typeof newDatasourceState === 'function'
? newDatasourceState(prevState.datasourceStates[datasourceId].state)
: newDatasourceState;
return {
...prevState,
datasourceStates: {
...prevState.datasourceStates,
[datasourceId]: {
state: updatedDatasourceState,
isLoading: false,
// React will synchronously update if this is triggered from a third party component,
// which we don't want. The timeout lets user interaction have priority, then React updates.
setTimeout(() => {
dispatch({
type: 'UPDATE_STATE',
subType: 'UPDATE_ALL_STATES',
updater: (prevState) => {
const updatedDatasourceState =
typeof newDatasourceState === 'function'
? newDatasourceState(prevState.datasourceStates[datasourceId].state)
: newDatasourceState;
return {
...prevState,
datasourceStates: {
...prevState.datasourceStates,
[datasourceId]: {
state: updatedDatasourceState,
isLoading: false,
},
},
},
visualization: {
...prevState.visualization,
state: newVisualizationState,
},
stagedPreview: undefined,
};
},
visualization: {
...prevState.visualization,
state: newVisualizationState,
},
stagedPreview: undefined,
};
},
});
}, 0);
},
[dispatch]
);
const toggleFullscreen = useMemo(
() => () => {
dispatch({
type: 'TOGGLE_FULLSCREEN',
});
},
[dispatch]
@ -118,6 +140,7 @@ export function LayerPanels(
visualizationState={visualizationState}
updateVisualization={setVisualizationState}
updateDatasource={updateDatasource}
updateDatasourceAsync={updateDatasourceAsync}
updateAll={updateAll}
isOnlyLayer={layerIds.length === 1}
onRemoveLayer={() => {
@ -135,6 +158,7 @@ export function LayerPanels(
});
removeLayerRef(layerId);
}}
toggleFullscreen={toggleFullscreen}
/>
) : null
)}

15
x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/dimension_container.scss
View file

@ -8,21 +8,36 @@
position: absolute;
left: 0;
animation: euiFlyout $euiAnimSpeedNormal $euiAnimSlightResistance;
@include euiBreakpoint('l', 'xl') {
top: 0 !important;
height: 100% !important;
}
@include euiBreakpoint('xs', 's', 'm') {
@include euiFlyout;
}
.lnsFrameLayout__sidebar-isFullscreen & {
border-left: $euiBorderThin; // Force border regardless of theme in fullscreen
box-shadow: none;
}
}
.lnsDimensionContainer__footer {
padding: $euiSizeS;
.lnsFrameLayout__sidebar-isFullscreen & {
display: none;
}
}
.lnsDimensionContainer__header {
padding: $euiSizeS $euiSizeXS;
.lnsFrameLayout__sidebar-isFullscreen & {
display: none;
}
}
.lnsDimensionContainer__headerTitle {

29
x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/dimension_container.tsx
View file

@ -29,26 +29,33 @@ export function DimensionContainer({
groupLabel,
handleClose,
panel,
isFullscreen,
panelRef,
}: {
isOpen: boolean;
handleClose: () => void;
panel: React.ReactElement;
handleClose: () => boolean;
panel: React.ReactElement | null;
groupLabel: string;
isFullscreen: boolean;
panelRef: (el: HTMLDivElement) => void;
}) {
const [focusTrapIsEnabled, setFocusTrapIsEnabled] = useState(false);
const closeFlyout = useCallback(() => {
handleClose();
setFocusTrapIsEnabled(false);
const canClose = handleClose();
if (canClose) {
setFocusTrapIsEnabled(false);
}
return canClose;
}, [handleClose]);
const closeOnEscape = useCallback(
(event: KeyboardEvent) => {
if (event.key === keys.ESCAPE) {
event.preventDefault();
closeFlyout();
const canClose = closeFlyout();
if (canClose) {
event.preventDefault();
}
}
},
[closeFlyout]
@ -69,7 +76,15 @@ export function DimensionContainer({
<div ref={panelRef}>
<EuiFocusTrap disabled={!focusTrapIsEnabled} clickOutsideDisables={true}>
<EuiWindowEvent event="keydown" handler={closeOnEscape} />
<EuiOutsideClickDetector onOutsideClick={closeFlyout} isDisabled={!isOpen}>
<EuiOutsideClickDetector
onOutsideClick={() => {
if (isFullscreen) {
return;
}
closeFlyout();
}}
isDisabled={!isOpen}
>
<div
role="dialog"
aria-labelledby="lnsDimensionContainerTitle"

110
x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.test.tsx
View file

@ -78,6 +78,7 @@ describe('LayerPanel', () => {
visualizationState: 'state',
updateVisualization: jest.fn(),
updateDatasource: jest.fn(),
updateDatasourceAsync: jest.fn(),
updateAll: jest.fn(),
framePublicAPI: frame,
isOnlyLayer: true,
@ -86,6 +87,8 @@ describe('LayerPanel', () => {
core: coreMock.createStart(),
layerIndex: 0,
registerNewLayerRef: jest.fn(),
isFullscreen: false,
toggleFullscreen: jest.fn(),
};
}
@ -255,7 +258,7 @@ describe('LayerPanel', () => {
it('should not update the visualization if the datasource is incomplete', () => {
(generateId as jest.Mock).mockReturnValue(`newid`);
const updateAll = jest.fn();
const updateDatasource = jest.fn();
const updateDatasourceAsync = jest.fn();
mockVisualization.getConfiguration.mockReturnValue({
groups: [
@ -273,7 +276,7 @@ describe('LayerPanel', () => {
const component = mountWithIntl(
<LayerPanel
{...getDefaultProps()}
updateDatasource={updateDatasource}
updateDatasourceAsync={updateDatasourceAsync}
updateAll={updateAll}
/>
);
@ -292,15 +295,88 @@ describe('LayerPanel', () => {
mockDatasource.renderDimensionEditor.mock.calls.length - 1
][1].setState;
act(() => {
stateFn(
{
indexPatternId: '1',
columns: {},
columnOrder: [],
incompleteColumns: { newId: { operationType: 'count' } },
},
{ isDimensionComplete: false }
);
});
expect(updateAll).not.toHaveBeenCalled();
expect(updateDatasourceAsync).toHaveBeenCalled();
act(() => {
stateFn({
indexPatternId: '1',
columns: {},
columnOrder: [],
incompleteColumns: { newId: { operationType: 'count' } },
});
});
expect(updateAll).not.toHaveBeenCalled();
expect(updateAll).toHaveBeenCalled();
});
it('should remove the dimension when the datasource marks it as removed', () => {
const updateAll = jest.fn();
const updateDatasource = jest.fn();
mockVisualization.getConfiguration.mockReturnValue({
groups: [
{
groupLabel: 'A',
groupId: 'a',
accessors: [{ columnId: 'y' }],
filterOperations: () => true,
supportsMoreColumns: true,
dataTestSubj: 'lnsGroup',
},
],
});
const component = mountWithIntl(
<LayerPanel
{...getDefaultProps()}
datasourceStates={{
ds1: {
isLoading: false,
state: {
layers: [
{
indexPatternId: '1',
columns: {
y: {
operationType: 'moving_average',
references: ['ref'],
},
},
columnOrder: ['y'],
incompleteColumns: {},
},
],
},
},
}}
updateDatasource={updateDatasource}
updateAll={updateAll}
/>
);
act(() => {
component.find('[data-test-subj="lnsLayerPanel-dimensionLink"]').first().simulate('click');
});
component.update();
expect(mockDatasource.renderDimensionEditor).toHaveBeenCalledWith(
expect.any(Element),
expect.objectContaining({ columnId: 'y' })
);
const stateFn =
mockDatasource.renderDimensionEditor.mock.calls[
mockDatasource.renderDimensionEditor.mock.calls.length - 1
][1].setState;
act(() => {
stateFn(
@ -308,11 +384,19 @@ describe('LayerPanel', () => {
indexPatternId: '1',
columns: {},
columnOrder: [],
incompleteColumns: { y: { operationType: 'average' } },
},
{ shouldReplaceDimension: true }
{
isDimensionComplete: false,
}
);
});
expect(updateAll).toHaveBeenCalled();
expect(mockVisualization.removeDimension).toHaveBeenCalledWith(
expect.objectContaining({
columnId: 'y',
})
);
});
it('should keep the DimensionContainer open when configuring a new dimension', () => {
@ -331,6 +415,7 @@ describe('LayerPanel', () => {
accessors: [],
filterOperations: () => true,
supportsMoreColumns: true,
enableDimensionEditor: true,
dataTestSubj: 'lnsGroup',
},
],
@ -345,6 +430,7 @@ describe('LayerPanel', () => {
accessors: [{ columnId: 'newid' }],
filterOperations: () => true,
supportsMoreColumns: false,
enableDimensionEditor: true,
dataTestSubj: 'lnsGroup',
},
],
@ -357,6 +443,20 @@ describe('LayerPanel', () => {
component.update();
expect(component.find('EuiFlyoutHeader').exists()).toBe(true);
const lastArgs =
mockDatasource.renderDimensionEditor.mock.calls[
mockDatasource.renderDimensionEditor.mock.calls.length - 1
][1];
// Simulate what is called by the dimension editor
act(() => {
lastArgs.setState(lastArgs.state, {
isDimensionComplete: true,
});
});
expect(mockVisualization.renderDimensionEditor).toHaveBeenCalled();
});
it('should close the DimensionContainer when the active visualization changes', () => {

86
x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/layer_panel.tsx
View file

@ -42,6 +42,7 @@ export function LayerPanel(
isOnlyLayer: boolean;
updateVisualization: StateSetter<unknown>;
updateDatasource: (datasourceId: string, newState: unknown) => void;
updateDatasourceAsync: (datasourceId: string, newState: unknown) => void;
updateAll: (
datasourceId: string,
newDatasourcestate: unknown,
@ -49,6 +50,8 @@ export function LayerPanel(
) => void;
onRemoveLayer: () => void;
registerNewLayerRef: (layerId: string, instance: HTMLDivElement | null) => void;
toggleFullscreen: () => void;
isFullscreen: boolean;
}
) {
const [activeDimension, setActiveDimension] = useState<ActiveDimensionState>(
@ -65,6 +68,8 @@ export function LayerPanel(
activeVisualization,
updateVisualization,
updateDatasource,
toggleFullscreen,
isFullscreen,
} = props;
const datasourcePublicAPI = framePublicAPI.datasourceLayers[layerId];
@ -197,9 +202,16 @@ export function LayerPanel(
setNextFocusedButtonId,
]);
const isDimensionPanelOpen = Boolean(activeId);
return (
<>
<section tabIndex={-1} ref={registerLayerRef} className="lnsLayerPanel">
<section
tabIndex={-1}
ref={registerLayerRef}
className="lnsLayerPanel"
style={{ visibility: isDimensionPanelOpen ? 'hidden' : 'visible' }}
>
<EuiPanel data-test-subj={`lns-layerPanel-${layerIndex}`} paddingSize="s">
<EuiFlexGroup gutterSize="s" alignItems="flexStart" responsive={false}>
<EuiFlexItem grow={false} className="lnsLayerPanel__settingsFlexItem">
@ -407,9 +419,16 @@ export function LayerPanel(
<DimensionContainer
panelRef={(el) => (panelRef.current = el)}
isOpen={!!activeId}
isOpen={isDimensionPanelOpen}
isFullscreen={isFullscreen}
groupLabel={activeGroup?.groupLabel || ''}
handleClose={() => {
if (
layerDatasource.canCloseDimensionEditor &&
!layerDatasource.canCloseDimensionEditor(layerDatasourceState)
) {
return false;
}
if (layerDatasource.updateStateOnCloseDimension) {
const newState = layerDatasource.updateStateOnCloseDimension({
state: layerDatasourceState,
@ -421,9 +440,13 @@ export function LayerPanel(
}
}
setActiveDimension(initialActiveDimensionState);
if (isFullscreen) {
toggleFullscreen();
}
return true;
}}
panel={
<>
<div>
{activeGroup && activeId && (
<NativeRenderer
render={layerDatasource.renderDimensionEditor}
@ -435,46 +458,51 @@ export function LayerPanel(
hideGrouping: activeGroup.hideGrouping,
filterOperations: activeGroup.filterOperations,
dimensionGroups: groups,
toggleFullscreen,
isFullscreen,
setState: (
newState: unknown,
{
shouldReplaceDimension,
shouldRemoveDimension,
}: {
shouldReplaceDimension?: boolean;
shouldRemoveDimension?: boolean;
} = {}
{ isDimensionComplete = true }: { isDimensionComplete?: boolean } = {}
) => {
if (shouldReplaceDimension || shouldRemoveDimension) {
if (allAccessors.includes(activeId)) {
if (isDimensionComplete) {
props.updateDatasourceAsync(datasourceId, newState);
} else {
// The datasource can indicate that the previously-valid column is no longer
// complete, which clears the visualization. This keeps the flyout open and reuses
// the previous columnId
props.updateAll(
datasourceId,
newState,
activeVisualization.removeDimension({
layerId,
columnId: activeId,
prevState: props.visualizationState,
})
);
}
} else if (isDimensionComplete) {
props.updateAll(
datasourceId,
newState,
shouldRemoveDimension
? activeVisualization.removeDimension({
layerId,
columnId: activeId,
prevState: props.visualizationState,
})
: activeVisualization.setDimension({
layerId,
groupId: activeGroup.groupId,
columnId: activeId,
prevState: props.visualizationState,
})
activeVisualization.setDimension({
layerId,
groupId: activeGroup.groupId,
columnId: activeId,
prevState: props.visualizationState,
})
);
setActiveDimension({ ...activeDimension, isNew: false });
} else {
props.updateDatasource(datasourceId, newState);
props.updateDatasourceAsync(datasourceId, newState);
}
setActiveDimension({
...activeDimension,
isNew: false,
});
},
}}
/>
)}
{activeGroup &&
activeId &&
!isFullscreen &&
!activeDimension.isNew &&
activeVisualization.renderDimensionEditor &&
activeGroup?.enableDimensionEditor && (
@ -491,7 +519,7 @@ export function LayerPanel(
/>
</div>
)}
</>
</div>
}
/>
</>

2
x-pack/plugins/lens/public/editor_frame_service/editor_frame/config_panel/types.ts
View file

@ -29,6 +29,7 @@ export interface ConfigPanelWrapperProps {
}
>;
core: DatasourceDimensionEditorProps['core'];
isFullscreen: boolean;
}
export interface LayerPanelProps {
@ -46,6 +47,7 @@ export interface LayerPanelProps {
}
>;
core: DatasourceDimensionEditorProps['core'];
isFullscreen: boolean;
}
export interface LayerDatasourceDropProps {

60
x-pack/plugins/lens/public/editor_frame_service/editor_frame/editor_frame.tsx
View file

@ -5,7 +5,7 @@
* 2.0.
*/
import React, { useEffect, useReducer, useState, useCallback } from 'react';
import React, { useEffect, useReducer, useState, useCallback, useRef } from 'react';
import { CoreStart } from 'kibana/public';
import { isEqual } from 'lodash';
import { PaletteRegistry } from 'src/plugins/charts/public';
@ -30,6 +30,7 @@ import {
applyVisualizeFieldSuggestions,
getTopSuggestionForField,
switchToSuggestion,
Suggestion,
} from './suggestion_helpers';
import { trackUiEvent } from '../../lens_ui_telemetry';
import {
@ -327,45 +328,37 @@ export function EditorFrame(props: EditorFrameProps) {
]
);
const getSuggestionForField = React.useCallback(
(field: DragDropIdentifier) => {
const { activeDatasourceId, datasourceStates } = state;
const activeVisualizationId = state.visualization.activeId;
const visualizationState = state.visualization.state;
const { visualizationMap, datasourceMap } = props;
// Using a ref to prevent rerenders in the child components while keeping the latest state
const getSuggestionForField = useRef<(field: DragDropIdentifier) => Suggestion | undefined>();
getSuggestionForField.current = (field: DragDropIdentifier) => {
const { activeDatasourceId, datasourceStates } = state;
const activeVisualizationId = state.visualization.activeId;
const visualizationState = state.visualization.state;
const { visualizationMap, datasourceMap } = props;
if (!field || !activeDatasourceId) {
return;
}
if (!field || !activeDatasourceId) {
return;
}
return getTopSuggestionForField(
datasourceLayers,
activeVisualizationId,
visualizationMap,
visualizationState,
datasourceMap[activeDatasourceId],
datasourceStates,
field
);
},
// eslint-disable-next-line react-hooks/exhaustive-deps
[
state.visualization.state,
props.datasourceMap,
props.visualizationMap,
state.activeDatasourceId,
state.datasourceStates,
]
);
return getTopSuggestionForField(
datasourceLayers,
activeVisualizationId,
visualizationMap,
visualizationState,
datasourceMap[activeDatasourceId],
datasourceStates,
field
);
};
const hasSuggestionForField = useCallback(
(field: DragDropIdentifier) => getSuggestionForField(field) !== undefined,
(field: DragDropIdentifier) => getSuggestionForField.current!(field) !== undefined,
[getSuggestionForField]
);
const dropOntoWorkspace = useCallback(
(field) => {
const suggestion = getSuggestionForField(field);
const suggestion = getSuggestionForField.current!(field);
if (suggestion) {
trackUiEvent('drop_onto_workspace');
switchToSuggestion(dispatch, suggestion, 'SWITCH_VISUALIZATION');
@ -377,6 +370,7 @@ export function EditorFrame(props: EditorFrameProps) {
return (
<RootDragDropProvider>
<FrameLayout
isFullscreen={Boolean(state.isFullscreenDatasource)}
dataPanel={
<DataPanelWrapper
datasourceMap={props.datasourceMap}
@ -414,6 +408,7 @@ export function EditorFrame(props: EditorFrameProps) {
visualizationState={state.visualization.state}
framePublicAPI={framePublicAPI}
core={props.core}
isFullscreen={Boolean(state.isFullscreenDatasource)}
/>
)
}
@ -429,11 +424,12 @@ export function EditorFrame(props: EditorFrameProps) {
visualizationState={state.visualization.state}
visualizationMap={props.visualizationMap}
dispatch={dispatch}
isFullscreen={Boolean(state.isFullscreenDatasource)}
ExpressionRenderer={props.ExpressionRenderer}
core={props.core}
plugins={props.plugins}
visualizeTriggerFieldContext={visualizeTriggerFieldContext}
getSuggestionForField={getSuggestionForField}
getSuggestionForField={getSuggestionForField.current}
/>
)
}

19
x-pack/plugins/lens/public/editor_frame_service/editor_frame/frame_layout.scss
View file

@ -67,9 +67,16 @@ a tilemap in an iframe: https://github.com/elastic/kibana/issues/16457 */
padding: $euiSize $euiSize 0;
position: relative;
z-index: $lnsZLevel1;
&:first-child {
padding-left: $euiSize;
}
&.lnsFrameLayout__pageBody-isFullscreen {
background: $euiColorEmptyShade;
flex: 1;
padding: 0;
}
}
.lnsFrameLayout__sidebar {
@ -81,6 +88,13 @@ a tilemap in an iframe: https://github.com/elastic/kibana/issues/16457 */
position: relative;
}
.lnsFrameLayout-isFullscreen .lnsFrameLayout__sidebar--left,
.lnsFrameLayout-isFullscreen .lnsFrameLayout__suggestionPanel {
// Hide the datapanel and suggestions in fullscreen mode. Using display: none does trigger
// a rerender when the container becomes visible again, maybe pushing offscreen is better
display: none;
}
.lnsFrameLayout__sidebar--right {
flex-basis: 25%;
background-color: lightOrDarkTheme($euiColorLightestShade, $euiColorInk);
@ -106,3 +120,8 @@ a tilemap in an iframe: https://github.com/elastic/kibana/issues/16457 */
}
}
}
.lnsFrameLayout__sidebar-isFullscreen {
flex: 1;
max-width: none;
}

28
x-pack/plugins/lens/public/editor_frame_service/editor_frame/frame_layout.tsx
View file

@ -10,23 +10,32 @@ import './frame_layout.scss';
import React from 'react';
import { EuiPage, EuiPageBody, EuiScreenReaderOnly } from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import classNames from 'classnames';
export interface FrameLayoutProps {
dataPanel: React.ReactNode;
configPanel?: React.ReactNode;
suggestionsPanel?: React.ReactNode;
workspacePanel?: React.ReactNode;
isFullscreen?: boolean;
}
export function FrameLayout(props: FrameLayoutProps) {
return (
<EuiPage className="lnsFrameLayout">
<EuiPage
className={classNames('lnsFrameLayout', {
'lnsFrameLayout-isFullscreen': props.isFullscreen,
})}
>
<EuiPageBody
restrictWidth={false}
className="lnsFrameLayout__pageContent"
aria-labelledby="lns_ChartTitle"
>
<section className="lnsFrameLayout__sidebar" aria-labelledby="dataPanelId">
<section
className={classNames('lnsFrameLayout__sidebar lnsFrameLayout__sidebar--left', {})}
aria-labelledby="dataPanelId"
>
<EuiScreenReaderOnly>
<h2 id="dataPanelId">
{i18n.translate('xpack.lens.section.dataPanelLabel', {
@ -36,7 +45,13 @@ export function FrameLayout(props: FrameLayoutProps) {
</EuiScreenReaderOnly>
{props.dataPanel}
</section>
<section className="lnsFrameLayout__pageBody" aria-labelledby="workspaceId">
<section
className={classNames('lnsFrameLayout__pageBody', {
// eslint-disable-next-line @typescript-eslint/naming-convention
'lnsFrameLayout__pageBody-isFullscreen': props.isFullscreen,
})}
aria-labelledby="workspaceId"
>
<EuiScreenReaderOnly>
<h2 id="workspaceId">
{i18n.translate('xpack.lens.section.workspaceLabel', {
@ -45,10 +60,13 @@ export function FrameLayout(props: FrameLayoutProps) {
</h2>
</EuiScreenReaderOnly>
{props.workspacePanel}
{props.suggestionsPanel}
<div className="lnsFrameLayout__suggestionPanel">{props.suggestionsPanel}</div>
</section>
<section
className="lnsFrameLayout__sidebar lnsFrameLayout__sidebar--right"
className={classNames('lnsFrameLayout__sidebar lnsFrameLayout__sidebar--right', {
// eslint-disable-next-line @typescript-eslint/naming-convention
'lnsFrameLayout__sidebar-isFullscreen': props.isFullscreen,
})}
aria-labelledby="configPanel"
>
<EuiScreenReaderOnly>

6
x-pack/plugins/lens/public/editor_frame_service/editor_frame/state_management.ts
View file

@ -22,6 +22,7 @@ export interface EditorFrameState extends PreviewState {
description?: string;
stagedPreview?: PreviewState;
activeDatasourceId: string | null;
isFullscreenDatasource?: boolean;
}
export type Action =
@ -90,6 +91,9 @@ export type Action =
| {
type: 'SWITCH_DATASOURCE';
newDatasourceId: string;
}
| {
type: 'TOGGLE_FULLSCREEN';
};
export function getActiveDatasourceIdFromDoc(doc?: Document) {
@ -281,6 +285,8 @@ export const reducer = (state: EditorFrameState, action: Action): EditorFrameSta
},
stagedPreview: action.clearStagedPreview ? undefined : state.stagedPreview,
};
case 'TOGGLE_FULLSCREEN':
return { ...state, isFullscreenDatasource: !state.isFullscreenDatasource };
default:
return state;
}

3
x-pack/plugins/lens/public/editor_frame_service/editor_frame/suggestion_panel.tsx
View file

@ -200,15 +200,16 @@ export function SuggestionPanel({
visualizationState: currentVisualizationState,
activeData: frame.activeData,
})
.filter((suggestion) => !suggestion.hide)
.filter(
({
hide,
visualizationId,
visualizationState: suggestionVisualizationState,
datasourceState: suggestionDatasourceState,
datasourceId: suggetionDatasourceId,
}) => {
return (
!hide &&
validateDatasourceAndVisualization(
suggetionDatasourceId ? datasourceMap[suggetionDatasourceId] : null,
suggestionDatasourceState,

2
x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.test.tsx
View file

@ -64,6 +64,8 @@ const defaultProps = {
data: mockDataPlugin(),
},
getSuggestionForField: () => undefined,
isFullscreen: false,
toggleFullscreen: jest.fn(),
};
describe('workspace_panel', () => {

13
x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel.tsx
View file

@ -79,6 +79,7 @@ export interface WorkspacePanelProps {
title?: string;
visualizeTriggerFieldContext?: VisualizeFieldContext;
getSuggestionForField: (field: DragDropIdentifier) => Suggestion | undefined;
isFullscreen: boolean;
}
interface WorkspaceState {
@ -134,6 +135,7 @@ export const InnerWorkspacePanel = React.memo(function InnerWorkspacePanel({
title,
visualizeTriggerFieldContext,
suggestionForDraggedField,
isFullscreen,
}: Omit<WorkspacePanelProps, 'getSuggestionForField'> & {
suggestionForDraggedField: Suggestion | undefined;
}) {
@ -346,6 +348,8 @@ export const InnerWorkspacePanel = React.memo(function InnerWorkspacePanel({
);
};
const element = expression !== null ? renderVisualization() : renderEmptyWorkspace();
const dragDropContext = useContext(DragContext);
const renderDragDrop = () => {
@ -363,7 +367,10 @@ export const InnerWorkspacePanel = React.memo(function InnerWorkspacePanel({
customWorkspaceRenderer()
) : (
<DragDrop
className="lnsWorkspacePanel__dragDrop"
className={classNames('lnsWorkspacePanel__dragDrop', {
// eslint-disable-next-line @typescript-eslint/naming-convention
'lnsWorkspacePanel__dragDrop--fullscreen': isFullscreen,
})}
dataTestSubj="lnsWorkspace"
draggable={false}
dropTypes={suggestionForDraggedField ? ['field_add'] : undefined}
@ -372,8 +379,7 @@ export const InnerWorkspacePanel = React.memo(function InnerWorkspacePanel({
order={dropProps.order}
>
<EuiPageContentBody className="lnsWorkspacePanelWrapper__pageContentBody">
{renderVisualization()}
{Boolean(suggestionForDraggedField) && expression !== null && renderEmptyWorkspace()}
{element}
</EuiPageContentBody>
</DragDrop>
);
@ -389,6 +395,7 @@ export const InnerWorkspacePanel = React.memo(function InnerWorkspacePanel({
datasourceStates={datasourceStates}
datasourceMap={datasourceMap}
visualizationMap={visualizationMap}
isFullscreen={isFullscreen}
>
{renderDragDrop()}
</WorkspacePanelWrapper>

8
x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel_wrapper.scss
View file

@ -31,6 +31,10 @@
overflow: hidden;
}
}
&.lnsWorkspacePanelWrapper--fullscreen {
margin-bottom: 0;
}
}
.lnsWorkspacePanel__dragDrop {
@ -62,6 +66,10 @@
animation: lnsWorkspacePanel__illustrationPulseContinuous 1.5s ease-in-out 0s infinite normal forwards;
}
}
&.lnsWorkspacePanel__dragDrop--fullscreen {
border: none;
}
}
.lnsWorkspacePanel__emptyContent {

2
x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel_wrapper.test.tsx
View file

@ -37,6 +37,7 @@ describe('workspace_panel_wrapper', () => {
visualizationMap={{ myVis: mockVisualization }}
datasourceMap={{}}
datasourceStates={{}}
isFullscreen={false}
>
<MyChild />
</WorkspacePanelWrapper>
@ -58,6 +59,7 @@ describe('workspace_panel_wrapper', () => {
visualizationMap={{ myVis: { ...mockVisualization, renderToolbar: renderToolbarMock } }}
datasourceMap={{}}
datasourceStates={{}}
isFullscreen={false}
/>
);

73
x-pack/plugins/lens/public/editor_frame_service/editor_frame/workspace_panel/workspace_panel_wrapper.tsx
View file

@ -10,6 +10,7 @@ import './workspace_panel_wrapper.scss';
import React, { useCallback } from 'react';
import { i18n } from '@kbn/i18n';
import { EuiPageContent, EuiFlexGroup, EuiFlexItem, EuiScreenReaderOnly } from '@elastic/eui';
import classNames from 'classnames';
import { Datasource, FramePublicAPI, Visualization } from '../../../types';
import { NativeRenderer } from '../../../native_renderer';
import { Action } from '../state_management';
@ -32,6 +33,7 @@ export interface WorkspacePanelWrapperProps {
state: unknown;
}
>;
isFullscreen: boolean;
}
export function WorkspacePanelWrapper({
@ -44,6 +46,7 @@ export function WorkspacePanelWrapper({
visualizationMap,
datasourceMap,
datasourceStates,
isFullscreen,
}: WorkspacePanelWrapperProps) {
const activeVisualization = visualizationId ? visualizationMap[visualizationId] : null;
const setVisualizationState = useCallback(
@ -85,40 +88,42 @@ export function WorkspacePanelWrapper({
wrap={true}
justifyContent="spaceBetween"
>
<EuiFlexItem grow={false}>
<EuiFlexGroup
gutterSize="m"
direction="row"
responsive={false}
wrap={true}
className="lnsWorkspacePanelWrapper__toolbar"
>
<EuiFlexItem grow={false}>
<ChartSwitch
data-test-subj="lnsChartSwitcher"
visualizationMap={visualizationMap}
visualizationId={visualizationId}
visualizationState={visualizationState}
datasourceMap={datasourceMap}
datasourceStates={datasourceStates}
dispatch={dispatch}
framePublicAPI={framePublicAPI}
/>
</EuiFlexItem>
{activeVisualization && activeVisualization.renderToolbar && (
{!isFullscreen ? (
<EuiFlexItem grow={false}>
<EuiFlexGroup
gutterSize="m"
direction="row"
responsive={false}
wrap={true}
className="lnsWorkspacePanelWrapper__toolbar"
>
<EuiFlexItem grow={false}>
<NativeRenderer
render={activeVisualization.renderToolbar}
nativeProps={{
frame: framePublicAPI,
state: visualizationState,
setState: setVisualizationState,
}}
<ChartSwitch
data-test-subj="lnsChartSwitcher"
visualizationMap={visualizationMap}
visualizationId={visualizationId}
visualizationState={visualizationState}
datasourceMap={datasourceMap}
datasourceStates={datasourceStates}
dispatch={dispatch}
framePublicAPI={framePublicAPI}
/>
</EuiFlexItem>
)}
</EuiFlexGroup>
</EuiFlexItem>
{activeVisualization && activeVisualization.renderToolbar && (
<EuiFlexItem grow={false}>
<NativeRenderer
render={activeVisualization.renderToolbar}
nativeProps={{
frame: framePublicAPI,
state: visualizationState,
setState: setVisualizationState,
}}
/>
</EuiFlexItem>
)}
</EuiFlexGroup>
</EuiFlexItem>
) : null}
<EuiFlexItem grow={false}>
{warningMessages && warningMessages.length ? (
<WarningsPopover>{warningMessages}</WarningsPopover>
@ -126,7 +131,11 @@ export function WorkspacePanelWrapper({
</EuiFlexItem>
</EuiFlexGroup>
</div>
<EuiPageContent className="lnsWorkspacePanelWrapper">
<EuiPageContent
className={classNames('lnsWorkspacePanelWrapper', {
'lnsWorkspacePanelWrapper--fullscreen': isFullscreen,
})}
>
<EuiScreenReaderOnly>
<h1 id="lns_ChartTitle" data-test-subj="lns_ChartTitle">
{title ||

1
x-pack/plugins/lens/public/editor_frame_service/mocks.tsx
View file

@ -57,6 +57,7 @@ export function createMockVisualization(): jest.Mocked<Visualization> {
setDimension: jest.fn(),
removeDimension: jest.fn(),
getErrorMessages: jest.fn((_state) => undefined),
renderDimensionEditor: jest.fn(),
};
}

36
x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/dimension_editor.scss
View file

@ -2,7 +2,27 @@
height: 100%;
}
.lnsIndexPatternDimensionEditor__section {
.lnsIndexPatternDimensionEditor__header {
position: sticky;
top: 0;
background: $euiColorEmptyShade;
// Raise it above the elements that are after it in DOM order
z-index: $euiZLevel1;
}
.lnsIndexPatternDimensionEditor-isFullscreen {
position: absolute;
left: 0;
right: 0;
top: 0;
bottom: 0;
.lnsIndexPatternDimensionEditor__section {
height: 100%;
}
}
.lnsIndexPatternDimensionEditor__section--padded {
padding: $euiSizeS;
}
@ -10,6 +30,14 @@
background-color: $euiColorLightestShade;
}
.lnsIndexPatternDimensionEditor__section--top {
border-bottom: $euiBorderThin;
}
.lnsIndexPatternDimensionEditor__section--bottom {
border-top: $euiBorderThin;
}
.lnsIndexPatternDimensionEditor__columns {
column-count: 2;
column-gap: $euiSizeXL;
@ -29,3 +57,9 @@
padding-top: 0;
padding-bottom: 0;
}
.lnsIndexPatternDimensionEditor__warning {
@include kbnThemeStyle('v7') {
border: none;
}
}

270
x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/dimension_editor.tsx
View file

@ -6,7 +6,7 @@
*/
import './dimension_editor.scss';
import React, { useState, useMemo } from 'react';
import React, { useState, useMemo, useCallback } from 'react';
import { i18n } from '@kbn/i18n';
import {
EuiListGroup,
@ -17,6 +17,9 @@ import {
EuiFormLabel,
EuiToolTip,
EuiText,
EuiTabs,
EuiTab,
EuiCallOut,
} from '@elastic/eui';
import { IndexPatternDimensionEditorProps } from './dimension_panel';
import { OperationSupportMatrix } from './operation_support';
@ -91,6 +94,8 @@ export function DimensionEditor(props: DimensionEditorProps) {
hideGrouping,
dateRange,
dimensionGroups,
toggleFullscreen,
isFullscreen,
} = props;
const services = {
data: props.data,
@ -101,30 +106,34 @@ export function DimensionEditor(props: DimensionEditorProps) {
};
const { fieldByOperation, operationWithoutField } = operationSupportMatrix;
const selectedOperationDefinition =
selectedColumn && operationDefinitionMap[selectedColumn.operationType];
const setStateWrapper = (
setter: IndexPatternLayer | ((prevLayer: IndexPatternLayer) => IndexPatternLayer)
) => {
const prevOperationType =
operationDefinitionMap[state.layers[layerId].columns[columnId]?.operationType]?.input;
const hypotheticalLayer = typeof setter === 'function' ? setter(state.layers[layerId]) : setter;
const hasIncompleteColumns = Boolean(hypotheticalLayer.incompleteColumns?.[columnId]);
const prevOperationType =
operationDefinitionMap[hypotheticalLayer.columns[columnId]?.operationType]?.input;
setState(
(prevState) => {
const layer = typeof setter === 'function' ? setter(prevState.layers[layerId]) : setter;
return mergeLayer({ state: prevState, layerId, newLayer: layer });
},
{
shouldReplaceDimension: Boolean(hypotheticalLayer.columns[columnId]),
// clear the dimension if there's an incomplete column pending && previous operation was a fullReference operation
shouldRemoveDimension: Boolean(
hasIncompleteColumns && prevOperationType === 'fullReference'
),
isDimensionComplete:
prevOperationType === 'fullReference'
? !hasIncompleteColumns
: Boolean(hypotheticalLayer.columns[columnId]),
}
);
};
const selectedOperationDefinition =
selectedColumn && operationDefinitionMap[selectedColumn.operationType];
const setIsCloseable = (isCloseable: boolean) => {
setState((prevState) => ({ ...prevState, isDimensionClosePrevented: !isCloseable }));
};
const incompleteInfo = (state.layers[layerId].incompleteColumns ?? {})[columnId];
const incompleteOperation = incompleteInfo?.operationType;
@ -132,14 +141,16 @@ export function DimensionEditor(props: DimensionEditorProps) {
const ParamEditor = selectedOperationDefinition?.paramEditor;
const [temporaryQuickFunction, setQuickFunction] = useState(false);
const possibleOperations = useMemo(() => {
return Object.values(operationDefinitionMap)
.filter(({ hidden }) => !hidden)
.filter(({ type }) => fieldByOperation[type]?.size || operationWithoutField.has(type))
.sort((op1, op2) => {
return op1.displayName.localeCompare(op2.displayName);
})
.map((def) => def.type)
.filter((type) => fieldByOperation[type]?.size || operationWithoutField.has(type));
.map((def) => def.type);
}, [fieldByOperation, operationWithoutField]);
const [filterByOpenInitially, setFilterByOpenInitally] = useState(false);
@ -245,37 +256,44 @@ export function DimensionEditor(props: DimensionEditorProps) {
visualizationGroups: dimensionGroups,
targetGroup: props.groupId,
});
if (temporaryQuickFunction && newLayer.columns[columnId].operationType !== 'formula') {
// Only switch the tab once the formula is fully removed
setQuickFunction(false);
}
setStateWrapper(newLayer);
trackUiEvent(`indexpattern_dimension_operation_${operationType}`);
return;
} else if (!selectedColumn || !compatibleWithCurrentField) {
const possibleFields = fieldByOperation[operationType] || new Set();
let newLayer: IndexPatternLayer;
if (possibleFields.size === 1) {
setStateWrapper(
insertOrReplaceColumn({
layer: props.state.layers[props.layerId],
indexPattern: currentIndexPattern,
columnId,
op: operationType,
field: currentIndexPattern.getFieldByName(possibleFields.values().next().value),
visualizationGroups: dimensionGroups,
targetGroup: props.groupId,
})
);
newLayer = insertOrReplaceColumn({
layer: props.state.layers[props.layerId],
indexPattern: currentIndexPattern,
columnId,
op: operationType,
field: currentIndexPattern.getFieldByName(possibleFields.values().next().value),
visualizationGroups: dimensionGroups,
targetGroup: props.groupId,
});
} else {
setStateWrapper(
insertOrReplaceColumn({
layer: props.state.layers[props.layerId],
indexPattern: currentIndexPattern,
columnId,
op: operationType,
field: undefined,
visualizationGroups: dimensionGroups,
targetGroup: props.groupId,
})
);
newLayer = insertOrReplaceColumn({
layer: props.state.layers[props.layerId],
indexPattern: currentIndexPattern,
columnId,
op: operationType,
field: undefined,
visualizationGroups: dimensionGroups,
targetGroup: props.groupId,
});
// );
}
if (temporaryQuickFunction && newLayer.columns[columnId].operationType !== 'formula') {
// Only switch the tab once the formula is fully removed
setQuickFunction(false);
}
setStateWrapper(newLayer);
trackUiEvent(`indexpattern_dimension_operation_${operationType}`);
return;
}
@ -287,6 +305,9 @@ export function DimensionEditor(props: DimensionEditorProps) {
return;
}
if (temporaryQuickFunction) {
setQuickFunction(false);
}
const newLayer = replaceColumn({
layer: props.state.layers[props.layerId],
indexPattern: currentIndexPattern,
@ -315,9 +336,34 @@ export function DimensionEditor(props: DimensionEditorProps) {
currentFieldIsInvalid
);
return (
<div id={columnId}>
<div className="lnsIndexPatternDimensionEditor__section lnsIndexPatternDimensionEditor__section--shaded">
const shouldDisplayExtraOptions =
!currentFieldIsInvalid &&
!incompleteInfo &&
selectedColumn &&
selectedColumn.operationType !== 'formula';
const quickFunctions = (
<>
{temporaryQuickFunction && selectedColumn?.operationType === 'formula' && (
<>
<EuiCallOut
className="lnsIndexPatternDimensionEditor__warning"
size="s"
title={i18n.translate('xpack.lens.indexPattern.formulaWarning', {
defaultMessage: 'Formula currently applied',
})}
iconType="alert"
color="warning"
>
<p>
{i18n.translate('xpack.lens.indexPattern.formulaWarningText', {
defaultMessage: 'To overwrite your formula, select a quick function',
})}
</p>
</EuiCallOut>
</>
)}
<div className="lnsIndexPatternDimensionEditor__section lnsIndexPatternDimensionEditor__section--padded lnsIndexPatternDimensionEditor__section--shaded">
<EuiFormLabel>
{i18n.translate('xpack.lens.indexPattern.functionsLabel', {
defaultMessage: 'Select a function',
@ -336,7 +382,7 @@ export function DimensionEditor(props: DimensionEditorProps) {
/>
</div>
<EuiSpacer size="s" />
<div className="lnsIndexPatternDimensionEditor__section lnsIndexPatternDimensionEditor__section--shaded">
<div className="lnsIndexPatternDimensionEditor__section lnsIndexPatternDimensionEditor__section--padded lnsIndexPatternDimensionEditor__section--shaded">
{!incompleteInfo &&
selectedColumn &&
'references' in selectedColumn &&
@ -375,6 +421,9 @@ export function DimensionEditor(props: DimensionEditorProps) {
currentColumn: state.layers[layerId].columns[columnId],
})}
dimensionGroups={dimensionGroups}
isFullscreen={isFullscreen}
toggleFullscreen={toggleFullscreen}
setIsCloseable={setIsCloseable}
{...services}
/>
);
@ -385,7 +434,8 @@ export function DimensionEditor(props: DimensionEditorProps) {
{!selectedColumn ||
selectedOperationDefinition?.input === 'field' ||
(incompleteOperation && operationDefinitionMap[incompleteOperation].input === 'field') ? (
(incompleteOperation && operationDefinitionMap[incompleteOperation].input === 'field') ||
temporaryQuickFunction ? (
<EuiFormRow
data-test-subj="indexPattern-field-selection-row"
label={i18n.translate('xpack.lens.indexPattern.chooseField', {
@ -436,19 +486,20 @@ export function DimensionEditor(props: DimensionEditorProps) {
</EuiFormRow>
) : null}
{!currentFieldIsInvalid && !incompleteInfo && selectedColumn && ParamEditor && (
<>
<ParamEditor
layer={state.layers[layerId]}
updateLayer={setStateWrapper}
columnId={columnId}
currentColumn={state.layers[layerId].columns[columnId]}
dateRange={dateRange}
indexPattern={currentIndexPattern}
operationDefinitionMap={operationDefinitionMap}
{...services}
/>
</>
{shouldDisplayExtraOptions && ParamEditor && (
<ParamEditor
layer={state.layers[layerId]}
updateLayer={setStateWrapper}
columnId={columnId}
currentColumn={state.layers[layerId].columns[columnId]}
dateRange={dateRange}
indexPattern={currentIndexPattern}
operationDefinitionMap={operationDefinitionMap}
toggleFullscreen={toggleFullscreen}
isFullscreen={isFullscreen}
setIsCloseable={setIsCloseable}
{...services}
/>
)}
{!currentFieldIsInvalid && !incompleteInfo && selectedColumn && (
@ -546,9 +597,96 @@ export function DimensionEditor(props: DimensionEditorProps) {
</div>
<EuiSpacer size="s" />
</>
);
{!currentFieldIsInvalid && (
<div className="lnsIndexPatternDimensionEditor__section">
const formulaTab = ParamEditor ? (
<ParamEditor
layer={state.layers[layerId]}
updateLayer={setStateWrapper}
columnId={columnId}
currentColumn={state.layers[layerId].columns[columnId]}
dateRange={dateRange}
indexPattern={currentIndexPattern}
operationDefinitionMap={operationDefinitionMap}
toggleFullscreen={toggleFullscreen}
isFullscreen={isFullscreen}
setIsCloseable={setIsCloseable}
{...services}
/>
) : null;
const onFormatChange = useCallback(
(newFormat) => {
setState(
mergeLayer({
state,
layerId,
newLayer: updateColumnParam({
layer: state.layers[layerId],
columnId,
paramName: 'format',
value: newFormat,
}),
})
);
},
[columnId, layerId, setState, state]
);
return (
<div id={columnId}>
{!isFullscreen && operationSupportMatrix.operationWithoutField.has('formula') ? (
<EuiTabs size="s" className="lnsIndexPatternDimensionEditor__header">
<EuiTab
isSelected={temporaryQuickFunction || selectedColumn?.operationType !== 'formula'}
data-test-subj="lens-dimensionTabs-quickFunctions"
onClick={() => {
if (selectedColumn?.operationType === 'formula') {
setQuickFunction(true);
}
}}
>
{i18n.translate('xpack.lens.indexPattern.quickFunctionsLabel', {
defaultMessage: 'Quick functions',
})}
</EuiTab>
<EuiTab
isSelected={!temporaryQuickFunction && selectedColumn?.operationType === 'formula'}
data-test-subj="lens-dimensionTabs-formula"
onClick={() => {
if (selectedColumn?.operationType !== 'formula') {
setQuickFunction(false);
const newLayer = insertOrReplaceColumn({
layer: props.state.layers[props.layerId],
indexPattern: currentIndexPattern,
columnId,
op: 'formula',
visualizationGroups: dimensionGroups,
});
setStateWrapper(newLayer);
trackUiEvent(`indexpattern_dimension_operation_formula`);
return;
} else {
setQuickFunction(false);
}
}}
>
{i18n.translate('xpack.lens.indexPattern.formulaLabel', {
defaultMessage: 'Formula',
})}
</EuiTab>
</EuiTabs>
) : null}
{isFullscreen
? formulaTab
: selectedOperationDefinition?.type === 'formula' && !temporaryQuickFunction
? formulaTab
: quickFunctions}
{!isFullscreen && !currentFieldIsInvalid && !temporaryQuickFunction && (
<div className="lnsIndexPatternDimensionEditor__section lnsIndexPatternDimensionEditor__section--padded">
{!incompleteInfo && selectedColumn && (
<LabelInput
value={selectedColumn.label}
@ -578,7 +716,7 @@ export function DimensionEditor(props: DimensionEditorProps) {
/>
)}
{!incompleteInfo && !hideGrouping && (
{!isFullscreen && !incompleteInfo && !hideGrouping && (
<BucketNestingEditor
layer={state.layers[props.layerId]}
columnId={props.columnId}
@ -589,31 +727,17 @@ export function DimensionEditor(props: DimensionEditorProps) {
/>
)}
{selectedColumn &&
{!isFullscreen &&
selectedColumn &&
(selectedColumn.dataType === 'number' || selectedColumn.operationType === 'range') ? (
<FormatSelector
selectedColumn={selectedColumn}
onChange={(newFormat) => {
setState(
mergeLayer({
state,
layerId,
newLayer: updateColumnParam({
layer: state.layers[layerId],
columnId,
paramName: 'format',
value: newFormat,
}),
})
);
}}
/>
<FormatSelector selectedColumn={selectedColumn} onChange={onFormatChange} />
) : null}
</div>
)}
</div>
);
}
function getErrorMessage(
selectedColumn: IndexPatternColumn | undefined,
incompleteOperation: boolean,

111
x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/dimension_panel.test.tsx
View file

@ -208,6 +208,8 @@ describe('IndexPatternDimensionEditorPanel', () => {
core: {} as CoreSetup,
dimensionGroups: [],
groupId: 'a',
isFullscreen: false,
toggleFullscreen: jest.fn(),
};
jest.clearAllMocks();
@ -500,10 +502,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
comboBox.prop('onChange')!([option]);
});
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](defaultProps.state)).toEqual({
...initialState,
layers: {
@ -535,10 +534,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
comboBox.prop('onChange')!([option]);
});
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](defaultProps.state)).toEqual({
...state,
layers: {
@ -569,10 +565,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
wrapper.find('button[data-test-subj="lns-indexPatternDimension-min"]').simulate('click');
});
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](state)).toEqual({
...state,
layers: {
@ -643,10 +636,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
wrapper.find('button[data-test-subj="lns-indexPatternDimension-min"]').simulate('click');
});
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](state)).toEqual({
...state,
layers: {
@ -681,10 +671,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
wrapper.find('button[data-test-subj="lns-indexPatternDimension-min"]').simulate('click');
});
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](state)).toEqual({
...state,
layers: {
@ -750,10 +737,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
.simulate('click');
});
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](state)).toEqual({
...state,
layers: {
@ -879,7 +863,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: false },
{ isDimensionComplete: false },
]);
expect(setState.mock.calls[0][0](state)).toEqual({
...state,
@ -948,7 +932,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
// Now check that the dimension gets cleaned up on state update
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: false },
{ isDimensionComplete: false },
]);
expect(setState.mock.calls[0][0](state)).toEqual({
...state,
@ -1042,10 +1026,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
});
expect(setState.mock.calls.length).toEqual(2);
expect(setState.mock.calls[1]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[1]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[1][0](state)).toEqual({
...state,
layers: {
@ -1143,10 +1124,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
.find('[data-test-subj="indexPattern-time-scaling-enable"]')
.hostNodes()
.simulate('click');
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1175,10 +1153,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
wrapper
.find('button[data-test-subj="lns-indexPatternDimension-count incompatible"]')
.simulate('click');
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1205,10 +1180,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
});
wrapper = mount(<IndexPatternDimensionEditorComponent {...props} />);
wrapper.find('button[data-test-subj="lns-indexPatternDimension-average"]').simulate('click');
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1239,10 +1211,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
.prop('onChange')!(({
target: { value: 'h' },
} as unknown) as ChangeEvent<HTMLSelectElement>);
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1269,10 +1238,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
.prop('onChange')!(({
target: { value: 'h' },
} as unknown) as ChangeEvent<HTMLSelectElement>);
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1300,10 +1266,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
{} as any
);
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1593,10 +1556,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
.find('[data-test-subj="indexPattern-filter-by-enable"]')
.hostNodes()
.simulate('click');
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1627,10 +1587,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
wrapper
.find('button[data-test-subj="lns-indexPatternDimension-count incompatible"]')
.simulate('click');
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1656,10 +1613,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
language: 'kuery',
query: 'c: d',
});
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1688,10 +1642,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
{} as any
);
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](props.state)).toEqual({
...props.state,
layers: {
@ -1743,10 +1694,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
wrapper.find('button[data-test-subj="lns-indexPatternDimension-average"]').simulate('click');
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: false },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: false }]);
expect(setState.mock.calls[0][0](defaultProps.state)).toEqual({
...state,
layers: {
@ -1810,10 +1758,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
wrapper.find('button[data-test-subj="lns-indexPatternDimension-average"]').simulate('click');
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](initialState)).toEqual({
...initialState,
layers: {
@ -1838,10 +1783,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
wrapper.find('button[data-test-subj="lns-indexPatternDimension-count"]').simulate('click');
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](state)).toEqual({
...state,
layers: {
@ -1975,10 +1917,7 @@ describe('IndexPatternDimensionEditorPanel', () => {
comboBox.prop('onChange')!([option]);
});
expect(setState.mock.calls[0]).toEqual([
expect.any(Function),
{ shouldRemoveDimension: false, shouldReplaceDimension: true },
]);
expect(setState.mock.calls[0]).toEqual([expect.any(Function), { isDimensionComplete: true }]);
expect(setState.mock.calls[0][0](defaultProps.state)).toEqual({
...state,
layers: {

2
x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/droppable/droppable.test.ts
View file

@ -284,6 +284,8 @@ describe('IndexPatternDimensionEditorPanel', () => {
} as unknown) as DataPublicPluginStart,
core: {} as CoreSetup,
dimensionGroups: [],
isFullscreen: false,
toggleFullscreen: () => {},
};
jest.clearAllMocks();

96
x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/format_selector.tsx
View file

@ -5,7 +5,7 @@
* 2.0.
*/
import React, { useState } from 'react';
import React, { useCallback, useMemo, useState } from 'react';
import { i18n } from '@kbn/i18n';
import { EuiFormRow, EuiComboBox, EuiSpacer, EuiRange } from '@elastic/eui';
import { IndexPatternColumn } from '../indexpattern';
@ -28,6 +28,13 @@ const supportedFormats: Record<string, { title: string }> = {
},
};
const defaultOption = {
value: '',
label: i18n.translate('xpack.lens.indexPattern.defaultFormatLabel', {
defaultMessage: 'Default',
}),
};
interface FormatSelectorProps {
selectedColumn: IndexPatternColumn;
onChange: (newFormat?: { id: string; params?: Record<string, unknown> }) => void;
@ -37,6 +44,8 @@ interface State {
decimalPlaces: number;
}
const singleSelectionOption = { asPlainText: true };
export function FormatSelector(props: FormatSelectorProps) {
const { selectedColumn, onChange } = props;
@ -51,13 +60,6 @@ export function FormatSelector(props: FormatSelectorProps) {
const selectedFormat = currentFormat?.id ? supportedFormats[currentFormat.id] : undefined;
const defaultOption = {
value: '',
label: i18n.translate('xpack.lens.indexPattern.defaultFormatLabel', {
defaultMessage: 'Default',
}),
};
const label = i18n.translate('xpack.lens.indexPattern.columnFormatLabel', {
defaultMessage: 'Value format',
});
@ -66,6 +68,48 @@ export function FormatSelector(props: FormatSelectorProps) {
defaultMessage: 'Decimals',
});
const stableOptions = useMemo(
() => [
defaultOption,
...Object.entries(supportedFormats).map(([id, format]) => ({
value: id,
label: format.title ?? id,
})),
],
[]
);
const onChangeWrapped = useCallback(
(choices) => {
if (choices.length === 0) {
return;
}
if (!choices[0].value) {
onChange();
return;
}
onChange({
id: choices[0].value,
params: { decimals: state.decimalPlaces },
});
},
[onChange, state.decimalPlaces]
);
const currentOption = useMemo(
() =>
currentFormat
? [
{
value: currentFormat.id,
label: selectedFormat?.title ?? currentFormat.id,
},
]
: [defaultOption],
[currentFormat, selectedFormat?.title]
);
return (
<>
<EuiFormRow label={label} display="columnCompressed" fullWidth>
@ -76,38 +120,10 @@ export function FormatSelector(props: FormatSelectorProps) {
isClearable={false}
data-test-subj="indexPattern-dimension-format"
aria-label={label}
singleSelection={{ asPlainText: true }}
options={[
defaultOption,
...Object.entries(supportedFormats).map(([id, format]) => ({
value: id,
label: format.title ?? id,
})),
]}
selectedOptions={
currentFormat
? [
{
value: currentFormat.id,
label: selectedFormat?.title ?? currentFormat.id,
},
]
: [defaultOption]
}
onChange={(choices) => {
if (choices.length === 0) {
return;
}
if (!choices[0].value) {
onChange();
return;
}
onChange({
id: choices[0].value,
params: { decimals: state.decimalPlaces },
});
}}
singleSelection={singleSelectionOption}
options={stableOptions}
selectedOptions={currentOption}
onChange={onChangeWrapped}
/>
{currentFormat ? (
<>

3
x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/reference_editor.test.tsx
View file

@ -51,6 +51,9 @@ describe('reference editor', () => {
http: {} as HttpSetup,
data: {} as DataPublicPluginStart,
dimensionGroups: [],
isFullscreen: false,
toggleFullscreen: jest.fn(),
setIsCloseable: jest.fn(),
};
}

10
x-pack/plugins/lens/public/indexpattern_datasource/dimension_panel/reference_editor.tsx
View file

@ -47,10 +47,14 @@ export interface ReferenceEditorProps {
setter: IndexPatternLayer | ((prevLayer: IndexPatternLayer) => IndexPatternLayer)
) => void;
currentIndexPattern: IndexPattern;
existingFields: IndexPatternPrivateState['existingFields'];
dateRange: DateRange;
labelAppend?: EuiFormRowProps['labelAppend'];
dimensionGroups: VisualizationDimensionGroupConfig[];
isFullscreen: boolean;
toggleFullscreen: () => void;
setIsCloseable: (isCloseable: boolean) => void;
// Services
uiSettings: IUiSettingsClient;
@ -72,6 +76,9 @@ export function ReferenceEditor(props: ReferenceEditorProps) {
dateRange,
labelAppend,
dimensionGroups,
isFullscreen,
toggleFullscreen,
setIsCloseable,
...services
} = props;
@ -347,6 +354,9 @@ export function ReferenceEditor(props: ReferenceEditorProps) {
indexPattern={currentIndexPattern}
dateRange={dateRange}
operationDefinitionMap={operationDefinitionMap}
isFullscreen={isFullscreen}
toggleFullscreen={toggleFullscreen}
setIsCloseable={setIsCloseable}
{...services}
/>
</>

5
x-pack/plugins/lens/public/indexpattern_datasource/indexpattern.tsx
View file

@ -323,6 +323,11 @@ export function getIndexPatternDatasource({
domElement
);
},
canCloseDimensionEditor: (state) => {
return !state.isDimensionClosePrevented;
},
getDropProps,
onDrop,

209
x-pack/plugins/lens/public/indexpattern_datasource/indexpattern_suggestions.test.tsx
View file

@ -16,6 +16,7 @@ import {
} from './indexpattern_suggestions';
import { documentField } from './document_field';
import { getFieldByNameFactory } from './pure_helpers';
import { isEqual } from 'lodash';
jest.mock('./loader');
jest.mock('../id_generator');
@ -867,10 +868,7 @@ describe('IndexPattern Data Source suggestions', () => {
searchable: true,
});
expect(suggestions).toHaveLength(1);
// Check that the suggestion is a single metric
expect(suggestions[0].table.columns).toHaveLength(1);
expect(suggestions[0].table.columns[0].operation.isBucketed).toBeFalsy();
expect(suggestions).toHaveLength(0);
});
it('appends a terms column with default size on string field', () => {
@ -1025,6 +1023,24 @@ describe('IndexPattern Data Source suggestions', () => {
expect(suggestions).not.toContain(expect.objectContaining({ changeType: 'extended' }));
});
it('skips metric only suggestion when the field is already in use', () => {
const initialState = stateWithNonEmptyTables();
const suggestions = getDatasourceSuggestionsForField(initialState, '1', {
name: 'bytes',
displayName: 'bytes',
type: 'number',
aggregatable: true,
searchable: true,
});
expect(
suggestions.some(
(suggestion) =>
suggestion.table.changeType === 'initial' && suggestion.table.columns.length === 1
)
).toBeFalsy();
});
it('skips duplicates when the document-specific field is already in use', () => {
const initialState = stateWithNonEmptyTables();
const modifiedState: IndexPatternPrivateState = {
@ -2344,7 +2360,7 @@ describe('IndexPattern Data Source suggestions', () => {
);
});
it('will skip a reduced suggestion when handling multiple references', () => {
it('will create reduced suggestions with all referenced children when handling references', () => {
const initialState = testInitialState();
const state: IndexPatternPrivateState = {
...initialState,
@ -2352,7 +2368,17 @@ describe('IndexPattern Data Source suggestions', () => {
...initialState.layers,
first: {
...initialState.layers.first,
columnOrder: ['date', 'metric', 'metric2', 'ref', 'ref2'],
columnOrder: [
'date',
'metric',
'metric2',
'ref',
'ref2',
'ref3',
'ref4',
'metric3',
'metric4',
],
columns: {
date: {
@ -2384,6 +2410,20 @@ describe('IndexPattern Data Source suggestions', () => {
operationType: 'count',
sourceField: 'Records',
},
metric3: {
label: '',
dataType: 'number',
isBucketed: false,
operationType: 'count',
sourceField: 'Records',
},
metric4: {
label: '',
dataType: 'number',
isBucketed: false,
operationType: 'count',
sourceField: 'Records',
},
ref2: {
label: '',
dataType: 'number',
@ -2391,22 +2431,163 @@ describe('IndexPattern Data Source suggestions', () => {
operationType: 'cumulative_sum',
references: ['metric2'],
},
ref3: {
label: '',
dataType: 'number',
isBucketed: false,
operationType: 'math',
references: ['ref4', 'metric3'],
params: {
tinymathAst: '',
},
},
ref4: {
label: '',
dataType: 'number',
isBucketed: false,
operationType: 'math',
references: ['metric4'],
params: {
tinymathAst: '',
},
},
},
},
},
};
const result = getSuggestionSubset(getDatasourceSuggestionsFromCurrentState(state));
const result = getDatasourceSuggestionsFromCurrentState(state);
expect(result).not.toContainEqual(
expect.objectContaining({
table: expect.objectContaining({
changeType: 'reduced',
}),
})
);
// only generate suggestions for top level metrics
expect(
result.filter((suggestion) => suggestion.table.changeType === 'reduced').length
).toEqual(3);
// top level "ref" column
expect(
result.some(
(suggestion) =>
suggestion.table.changeType === 'reduced' &&
isEqual(suggestion.state.layers.first.columnOrder, ['ref', 'metric'])
)
).toBeTruthy();
// top level "ref2" column
expect(
result.some(
(suggestion) =>
suggestion.table.changeType === 'reduced' &&
isEqual(suggestion.state.layers.first.columnOrder, ['ref2', 'metric2'])
)
).toBeTruthy();
// top level "ref3" column
expect(
result.some(
(suggestion) =>
suggestion.table.changeType === 'reduced' &&
isEqual(suggestion.state.layers.first.columnOrder, [
'ref3',
'ref4',
'metric3',
'metric4',
])
)
).toBeTruthy();
});
});
it('will leave dangling references in place', () => {
const initialState = testInitialState();
const state: IndexPatternPrivateState = {
...initialState,
layers: {
...initialState.layers,
first: {
...initialState.layers.first,
columnOrder: ['date', 'ref'],
columns: {
date: {
label: '',
dataType: 'date',
isBucketed: true,
operationType: 'date_histogram',
sourceField: 'timestamp',
params: { interval: 'auto' },
},
ref: {
label: '',
dataType: 'number',
isBucketed: false,
operationType: 'cumulative_sum',
references: ['non_existing_metric'],
},
},
},
},
};
const result = getDatasourceSuggestionsFromCurrentState(state);
// only generate suggestions for top level metrics
expect(
result.filter((suggestion) => suggestion.table.changeType === 'reduced').length
).toEqual(1);
// top level "ref" column
expect(
result.some(
(suggestion) =>
suggestion.table.changeType === 'reduced' &&
isEqual(suggestion.state.layers.first.columnOrder, ['ref'])
)
).toBeTruthy();
});
it('will not suggest reduced tables if there is just a referenced top level metric', () => {
const initialState = testInitialState();
const state: IndexPatternPrivateState = {
...initialState,
layers: {
...initialState.layers,
first: {
...initialState.layers.first,
columnOrder: ['ref', 'metric'],
columns: {
ref: {
label: '',
dataType: 'number',
isBucketed: false,
operationType: 'math',
params: {
tinymathAst: '',
},
references: ['metric'],
},
metric: {
label: '',
dataType: 'number',
isBucketed: false,
operationType: 'count',
sourceField: 'Records',
},
},
},
},
};
const result = getDatasourceSuggestionsFromCurrentState(state);
expect(
result.filter((suggestion) => suggestion.table.changeType === 'unchanged').length
).toEqual(1);
expect(
result.filter((suggestion) => suggestion.table.changeType === 'reduced').length
).toEqual(0);
});
});
});

74
x-pack/plugins/lens/public/indexpattern_datasource/indexpattern_suggestions.ts
View file

@ -5,7 +5,7 @@
* 2.0.
*/
import { flatten, minBy, pick, mapValues } from 'lodash';
import { flatten, minBy, pick, mapValues, partition } from 'lodash';
import { i18n } from '@kbn/i18n';
import { generateId } from '../id_generator';
import { DatasourceSuggestion, TableChangeType } from '../types';
@ -20,6 +20,7 @@ import {
OperationType,
getExistingColumnGroups,
isReferenced,
getReferencedColumnIds,
} from './operations';
import { hasField } from './utils';
import {
@ -254,9 +255,11 @@ function getExistingLayerSuggestionsForField(
}
}
const metricSuggestion = createMetricSuggestion(indexPattern, layerId, state, field);
if (metricSuggestion) {
suggestions.push(metricSuggestion);
if (!fieldInUse) {
const metricSuggestion = createMetricSuggestion(indexPattern, layerId, state, field);
if (metricSuggestion) {
suggestions.push(metricSuggestion);
}
}
return suggestions;
@ -514,8 +517,11 @@ function createAlternativeMetricSuggestions(
) {
const layer = state.layers[layerId];
const suggestions: Array<DatasourceSuggestion<IndexPatternPrivateState>> = [];
const topLevelMetricColumns = layer.columnOrder.filter(
(columnId) => !isReferenced(layer, columnId)
);
layer.columnOrder.forEach((columnId) => {
topLevelMetricColumns.forEach((columnId) => {
const column = layer.columns[columnId];
if (!hasField(column)) {
return;
@ -580,11 +586,13 @@ function createSuggestionWithDefaultDateHistogram(
function createSimplifiedTableSuggestions(state: IndexPatternPrivateState, layerId: string) {
const layer = state.layers[layerId];
const [
availableBucketedColumns,
availableMetricColumns,
availableReferenceColumns,
] = getExistingColumnGroups(layer);
const [availableBucketedColumns, availableMetricColumns] = partition(
layer.columnOrder,
(colId) => layer.columns[colId].isBucketed
);
const topLevelMetricColumns = availableMetricColumns.filter(
(columnId) => !isReferenced(layer, columnId)
);
return flatten(
availableBucketedColumns.map((_col, index) => {
@ -593,46 +601,60 @@ function createSimplifiedTableSuggestions(state: IndexPatternPrivateState, layer
const allMetricsSuggestion = {
...layer,
columnOrder: [...bucketedColumns, ...availableMetricColumns],
noBuckets: false,
};
if (availableBucketedColumns.length <= 1 || availableReferenceColumns.length) {
// Don't simplify when dealing with single-bucket table. Also don't break
// reference-based columns by removing buckets.
if (availableBucketedColumns.length <= 1) {
// Don't simplify when dealing with single-bucket table.
return [];
} else if (availableMetricColumns.length > 1) {
return [{ ...layer, columnOrder: [...bucketedColumns, availableMetricColumns[0]] }];
} else if (topLevelMetricColumns.length > 1) {
return [
{
...layer,
columnOrder: [
...bucketedColumns,
topLevelMetricColumns[0],
...getReferencedColumnIds(layer, topLevelMetricColumns[0]),
],
noBuckets: false,
},
];
} else {
return allMetricsSuggestion;
}
})
)
.concat(
availableReferenceColumns.length
? []
: availableMetricColumns.map((columnId) => {
return { ...layer, columnOrder: [columnId] };
// if there is just a single top level metric, the unchanged suggestion will take care of this case - only split up if there are multiple metrics or at least one bucket
availableBucketedColumns.length > 0 || topLevelMetricColumns.length > 1
? topLevelMetricColumns.map((columnId) => {
return {
...layer,
columnOrder: [columnId, ...getReferencedColumnIds(layer, columnId)],
noBuckets: true,
};
})
: []
)
.map((updatedLayer) => {
.map(({ noBuckets, ...updatedLayer }) => {
return buildSuggestion({
state,
layerId,
updatedLayer,
changeType: 'reduced',
label:
updatedLayer.columnOrder.length === 1
? getMetricSuggestionTitle(updatedLayer, availableMetricColumns.length === 1)
: undefined,
label: noBuckets
? getMetricSuggestionTitle(updatedLayer, availableMetricColumns.length === 1)
: undefined,
});
});
}
function getMetricSuggestionTitle(layer: IndexPatternLayer, onlyMetric: boolean) {
function getMetricSuggestionTitle(layer: IndexPatternLayer, onlySimpleMetric: boolean) {
const { operationType, label } = layer.columns[layer.columnOrder[0]];
return i18n.translate('xpack.lens.indexpattern.suggestions.overallLabel', {
defaultMessage: '{operation} overall',
values: {
operation: onlyMetric ? operationDefinitionMap[operationType].displayName : label,
operation: onlySimpleMetric ? operationDefinitionMap[operationType].displayName : label,
},
description:
'Title of a suggested chart containing only a single numerical metric calculated over all available data',

3
x-pack/plugins/lens/public/indexpattern_datasource/operations/__mocks__/index.ts
View file

@ -17,6 +17,7 @@ jest.spyOn(actualHelpers, 'insertOrReplaceColumn');
jest.spyOn(actualHelpers, 'insertNewColumn');
jest.spyOn(actualHelpers, 'replaceColumn');
jest.spyOn(actualHelpers, 'getErrorMessages');
jest.spyOn(actualHelpers, 'getColumnOrder');
export const {
getAvailableOperationsByMetadata,
@ -48,6 +49,8 @@ export const {
resetIncomplete,
isOperationAllowedAsReference,
canTransition,
isColumnValidAsReference,
getManagedColumnsFrom,
} = actualHelpers;
export const { adjustTimeScaleLabelSuffix, DEFAULT_TIME_SCALE } = actualTimeScaleUtils;

18
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/calculations/counter_rate.tsx
View file

@ -121,5 +121,23 @@ export const counterRateOperation: OperationDefinition<
},
timeScalingMode: 'mandatory',
filterable: true,
documentation: {
section: 'calculation',
signature: i18n.translate('xpack.lens.indexPattern.counterRate.signature', {
defaultMessage: 'metric: number',
}),
description: i18n.translate('xpack.lens.indexPattern.counterRate.documentation', {
defaultMessage: `
Calculates the rate of an ever increasing counter. This function will only yield helpful results on counter metric fields which contain a measurement of some kind monotonically growing over time.
If the value does get smaller, it will interpret this as a counter reset. To get most precise results, \`counter_rate\` should be calculated on the \`max\` of a field.
This calculation will be done separately for separate series defined by filters or top values dimensions.
It uses the current interval when used in Formula.
Example: Visualize the rate of bytes received over time by a memcached server:
\`counter_rate(max(memcached.stats.read.bytes))\`
`,
}),
},
shiftable: true,
};

16
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/calculations/cumulative_sum.tsx
View file

@ -117,5 +117,21 @@ export const cumulativeSumOperation: OperationDefinition<
)?.join(', ');
},
filterable: true,
documentation: {
section: 'calculation',
signature: i18n.translate('xpack.lens.indexPattern.cumulative_sum.signature', {
defaultMessage: 'metric: number',
}),
description: i18n.translate('xpack.lens.indexPattern.cumulativeSum.documentation', {
defaultMessage: `
Calculates the cumulative sum of a metric over time, adding all previous values of a series to each value. To use this function, you need to configure a date histogram dimension as well.
This calculation will be done separately for separate series defined by filters or top values dimensions.
Example: Visualize the received bytes accumulated over time:
\`cumulative_sum(sum(bytes))\`
`,
}),
},
shiftable: true,
};

17
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/calculations/differences.tsx
View file

@ -109,5 +109,22 @@ export const derivativeOperation: OperationDefinition<
},
timeScalingMode: 'optional',
filterable: true,
documentation: {
section: 'calculation',
signature: i18n.translate('xpack.lens.indexPattern.differences.signature', {
defaultMessage: 'metric: number',
}),
description: i18n.translate('xpack.lens.indexPattern.differences.documentation', {
defaultMessage: `
Calculates the difference to the last value of a metric over time. To use this function, you need to configure a date histogram dimension as well.
Differences requires the data to be sequential. If your data is empty when using differences, try increasing the date histogram interval.
This calculation will be done separately for separate series defined by filters or top values dimensions.
Example: Visualize the change in bytes received over time:
\`differences(sum(bytes))\`
`,
}),
},
shiftable: true,
};

26
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/calculations/moving_average.tsx
View file

@ -65,7 +65,9 @@ export const movingAverageOperation: OperationDefinition<
validateMetadata: (meta) => meta.dataType === 'number' && !meta.isBucketed,
},
],
operationParams: [{ name: 'window', type: 'number', required: true }],
operationParams: [
{ name: 'window', type: 'number', required: false, defaultValue: WINDOW_DEFAULT_VALUE },
],
getPossibleOperation: (indexPattern) => {
if (hasDateField(indexPattern)) {
return {
@ -130,6 +132,28 @@ export const movingAverageOperation: OperationDefinition<
},
timeScalingMode: 'optional',
filterable: true,
documentation: {
section: 'calculation',
signature: i18n.translate('xpack.lens.indexPattern.moving_average.signature', {
defaultMessage: 'metric: number, [window]: number',
}),
description: i18n.translate('xpack.lens.indexPattern.movingAverage.documentation', {
defaultMessage: `
Calculates the moving average of a metric over time, averaging the last n-th values to calculate the current value. To use this function, you need to configure a date histogram dimension as well.
The default window value is {defaultValue}.
This calculation will be done separately for separate series defined by filters or top values dimensions.
Takes a named parameter \`window\` which specifies how many last values to include in the average calculation for the current value.
Example: Smooth a line of measurements:
\`moving_average(sum(bytes), window=5)\`
`,
values: {
defaultValue: WINDOW_DEFAULT_VALUE,
},
}),
},
shiftable: true,
};

17
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/cardinality.tsx
View file

@ -116,4 +116,21 @@ export const cardinalityOperation: OperationDefinition<CardinalityIndexPatternCo
sourceField: field.name,
};
},
documentation: {
section: 'elasticsearch',
signature: i18n.translate('xpack.lens.indexPattern.cardinality.signature', {
defaultMessage: 'field: string',
}),
description: i18n.translate('xpack.lens.indexPattern.cardinality.documentation', {
defaultMessage: `
Calculates the number of unique values of a specified field. Works for number, string, date and boolean values.
Example: Calculate the number of different products:
\`unique_count(product.name)\`
Example: Calculate the number of different products from the "clothes" group:
\`unique_count(product.name, kql='product.group=clothes')\`
`,
}),
},
};

15
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/count.tsx
View file

@ -111,5 +111,20 @@ export const countOperation: OperationDefinition<CountIndexPatternColumn, 'field
},
timeScalingMode: 'optional',
filterable: true,
documentation: {
section: 'elasticsearch',
signature: '',
description: i18n.translate('xpack.lens.indexPattern.count.documentation', {
defaultMessage: `
Calculates the number of documents.
Example: Calculate the number of documents:
\`count()\`
Example: Calculate the number of documents matching a certain filter:
\`count(kql='price > 500')\`
`,
}),
},
shiftable: true,
};

3
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/date_histogram.test.tsx
View file

@ -98,6 +98,9 @@ const defaultOptions = {
http: {} as HttpSetup,
indexPattern: indexPattern1,
operationDefinitionMap: {},
isFullscreen: false,
toggleFullscreen: jest.fn(),
setIsCloseable: jest.fn(),
};
describe('date_histogram', () => {

3
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/filters/filters.test.tsx
View file

@ -28,6 +28,9 @@ const defaultProps = {
http: {} as HttpSetup,
indexPattern: createMockedIndexPattern(),
operationDefinitionMap: {},
isFullscreen: false,
toggleFullscreen: jest.fn(),
setIsCloseable: jest.fn(),
};
// mocking random id generator function

167
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/editor/formula.scss Normal file
View file

@ -0,0 +1,167 @@
.lnsFormula {
display: flex;
flex-direction: column;
.lnsIndexPatternDimensionEditor-isFullscreen & {
height: 100%;
}
& > * {
flex: 1;
min-height: 0;
}
& > * + * {
border-top: $euiBorderThin;
}
}
.lnsFormula__editor {
border-bottom: $euiBorderThin;
.lnsIndexPatternDimensionEditor-isFullscreen & {
border-bottom: none;
display: flex;
flex-direction: column;
}
& > * + * {
border-top: $euiBorderThin;
}
}
.lnsFormula__editorHeader,
.lnsFormula__editorFooter {
padding: $euiSizeS;
}
.lnsFormula__editorFooter {
// make sure docs are rendered in front of monaco
z-index: 1;
background-color: $euiColorLightestShade;
}
.lnsFormula__editorHeaderGroup,
.lnsFormula__editorFooterGroup {
display: block; // Overrides EUI's styling of `display: flex` on `EuiFlexItem` components
}
.lnsFormula__editorContent {
position: relative;
height: 201px;
}
.lnsFormula__editorPlaceholder {
position: absolute;
top: 0;
left: $euiSize;
right: 0;
color: $euiTextSubduedColor;
// Matches monaco editor
font-family: Menlo, Monaco, 'Courier New', monospace;
pointer-events: none;
}
.lnsIndexPatternDimensionEditor-isFullscreen .lnsFormula__editorContent {
flex: 1;
min-height: 201px;
}
.lnsFormula__warningText + .lnsFormula__warningText {
margin-top: $euiSizeS;
border-top: $euiBorderThin;
padding-top: $euiSizeS;
}
.lnsFormula__editorHelp--inline {
align-items: center;
display: flex;
padding: $euiSizeXS;
& > * + * {
margin-left: $euiSizeXS;
}
}
.lnsFormula__editorError {
white-space: nowrap;
}
.lnsFormula__docs {
background: $euiColorEmptyShade;
}
.lnsFormula__docs--inline {
display: flex;
flex-direction: column;
// make sure docs are rendered in front of monaco
z-index: 1;
}
.lnsFormula__docsContent {
.lnsFormula__docs--overlay & {
height: 40vh;
width: #{'min(75vh, 90vw)'};
}
.lnsFormula__docs--inline & {
flex: 1;
min-height: 0;
}
& > * + * {
border-left: $euiBorderThin;
}
}
.lnsFormula__docsSidebar {
background: $euiColorLightestShade;
}
.lnsFormula__docsSidebarInner {
min-height: 0;
& > * + * {
border-top: $euiBorderThin;
}
}
.lnsFormula__docsSearch {
padding: $euiSizeS;
}
.lnsFormula__docsNav {
@include euiYScroll;
}
.lnsFormula__docsNavGroup {
padding: $euiSizeS;
& + & {
border-top: $euiBorderThin;
}
}
.lnsFormula__docsNavGroupLink {
font-weight: inherit;
}
.lnsFormula__docsText {
@include euiYScroll;
padding: $euiSize;
}
.lnsFormula__docsTextGroup,
.lnsFormula__docsTextItem {
margin-top: $euiSizeXXL;
}
.lnsFormula__docsTextGroup {
border-top: $euiBorderThin;
padding-top: $euiSizeXXL;
}
.lnsFormulaOverflow {
// Needs to be higher than the modal and all flyouts
z-index: $euiZLevel9 + 1;
}

791
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/editor/formula_editor.tsx Normal file
View file

@ -0,0 +1,791 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import React, { useCallback, useEffect, useState, useMemo } from 'react';
import { i18n } from '@kbn/i18n';
import {
EuiButtonIcon,
EuiButtonEmpty,
EuiFlexGroup,
EuiFlexItem,
EuiIcon,
EuiLink,
EuiPopover,
EuiText,
EuiToolTip,
EuiSpacer,
} from '@elastic/eui';
import useUnmount from 'react-use/lib/useUnmount';
import { monaco } from '@kbn/monaco';
import classNames from 'classnames';
import { CodeEditor } from '../../../../../../../../../src/plugins/kibana_react/public';
import type { CodeEditorProps } from '../../../../../../../../../src/plugins/kibana_react/public';
import { useDebounceWithOptions } from '../../../../../shared_components';
import { ParamEditorProps } from '../../index';
import { getManagedColumnsFrom } from '../../../layer_helpers';
import { ErrorWrapper, runASTValidation, tryToParse } from '../validation';
import {
LensMathSuggestion,
SUGGESTION_TYPE,
suggest,
getSuggestion,
getSignatureHelp,
getHover,
getTokenInfo,
offsetToRowColumn,
monacoPositionToOffset,
} from './math_completion';
import { LANGUAGE_ID } from './math_tokenization';
import { MemoizedFormulaHelp } from './formula_help';
import { trackUiEvent } from '../../../../../lens_ui_telemetry';
import './formula.scss';
import { FormulaIndexPatternColumn } from '../formula';
import { regenerateLayerFromAst } from '../parse';
import { filterByVisibleOperation } from '../util';
export const MemoizedFormulaEditor = React.memo(FormulaEditor);
export function FormulaEditor({
layer,
updateLayer,
currentColumn,
columnId,
indexPattern,
operationDefinitionMap,
data,
toggleFullscreen,
isFullscreen,
setIsCloseable,
}: ParamEditorProps<FormulaIndexPatternColumn>) {
const [text, setText] = useState(currentColumn.params.formula);
const [warnings, setWarnings] = useState<
Array<{ severity: monaco.MarkerSeverity; message: string }>
>([]);
const [isHelpOpen, setIsHelpOpen] = useState<boolean>(isFullscreen);
const [isWarningOpen, setIsWarningOpen] = useState<boolean>(false);
const [isWordWrapped, toggleWordWrap] = useState<boolean>(true);
const editorModel = React.useRef<monaco.editor.ITextModel>();
const overflowDiv1 = React.useRef<HTMLElement>();
const disposables = React.useRef<monaco.IDisposable[]>([]);
const editor1 = React.useRef<monaco.editor.IStandaloneCodeEditor>();
const visibleOperationsMap = useMemo(() => filterByVisibleOperation(operationDefinitionMap), [
operationDefinitionMap,
]);
// The Monaco editor needs to have the overflowDiv in the first render. Using an effect
// requires a second render to work, so we are using an if statement to guarantee it happens
// on first render
if (!overflowDiv1?.current) {
const node1 = (overflowDiv1.current = document.createElement('div'));
node1.setAttribute('data-test-subj', 'lnsFormulaWidget');
// Monaco CSS is targeted on the monaco-editor class
node1.classList.add('lnsFormulaOverflow', 'monaco-editor');
document.body.appendChild(node1);
}
// Clean up the monaco editor and DOM on unmount
useEffect(() => {
const model = editorModel;
const allDisposables = disposables;
const editor1ref = editor1;
return () => {
model.current?.dispose();
overflowDiv1.current?.parentNode?.removeChild(overflowDiv1.current);
editor1ref.current?.dispose();
allDisposables.current?.forEach((d) => d.dispose());
};
}, []);
useUnmount(() => {
setIsCloseable(true);
// If the text is not synced, update the column.
if (text !== currentColumn.params.formula) {
updateLayer((prevLayer) => {
return regenerateLayerFromAst(
text || '',
prevLayer,
columnId,
currentColumn,
indexPattern,
operationDefinitionMap
).newLayer;
});
}
});
useDebounceWithOptions(
() => {
if (!editorModel.current) return;
if (!text) {
setWarnings([]);
monaco.editor.setModelMarkers(editorModel.current, 'LENS', []);
if (currentColumn.params.formula) {
// Only submit if valid
const { newLayer } = regenerateLayerFromAst(
text || '',
layer,
columnId,
currentColumn,
indexPattern,
operationDefinitionMap
);
updateLayer(newLayer);
}
return;
}
let errors: ErrorWrapper[] = [];
const { root, error } = tryToParse(text, visibleOperationsMap);
if (error) {
errors = [error];
} else if (root) {
const validationErrors = runASTValidation(root, layer, indexPattern, visibleOperationsMap);
if (validationErrors.length) {
errors = validationErrors;
}
}
if (errors.length) {
if (currentColumn.params.isFormulaBroken) {
// If the formula is already broken, show the latest error message in the workspace
if (currentColumn.params.formula !== text) {
updateLayer(
regenerateLayerFromAst(
text || '',
layer,
columnId,
currentColumn,
indexPattern,
visibleOperationsMap
).newLayer
);
}
}
const markers = errors.flatMap((innerError) => {
if (innerError.locations.length) {
return innerError.locations.map((location) => {
const startPosition = offsetToRowColumn(text, location.min);
const endPosition = offsetToRowColumn(text, location.max);
return {
message: innerError.message,
startColumn: startPosition.column + 1,
startLineNumber: startPosition.lineNumber,
endColumn: endPosition.column + 1,
endLineNumber: endPosition.lineNumber,
severity:
innerError.severity === 'warning'
? monaco.MarkerSeverity.Warning
: monaco.MarkerSeverity.Error,
};
});
} else {
// Parse errors return no location info
const startPosition = offsetToRowColumn(text, 0);
const endPosition = offsetToRowColumn(text, text.length - 1);
return [
{
message: innerError.message,
startColumn: startPosition.column + 1,
startLineNumber: startPosition.lineNumber,
endColumn: endPosition.column + 1,
endLineNumber: endPosition.lineNumber,
severity:
innerError.severity === 'warning'
? monaco.MarkerSeverity.Warning
: monaco.MarkerSeverity.Error,
},
];
}
});
monaco.editor.setModelMarkers(editorModel.current, 'LENS', markers);
setWarnings(markers.map(({ severity, message }) => ({ severity, message })));
} else {
monaco.editor.setModelMarkers(editorModel.current, 'LENS', []);
// Only submit if valid
const { newLayer, locations } = regenerateLayerFromAst(
text || '',
layer,
columnId,
currentColumn,
indexPattern,
visibleOperationsMap
);
updateLayer(newLayer);
const managedColumns = getManagedColumnsFrom(columnId, newLayer.columns);
const markers: monaco.editor.IMarkerData[] = managedColumns
.flatMap(([id, column]) => {
if (locations[id]) {
const def = visibleOperationsMap[column.operationType];
if (def.getErrorMessage) {
const messages = def.getErrorMessage(
newLayer,
id,
indexPattern,
visibleOperationsMap
);
if (messages) {
const startPosition = offsetToRowColumn(text, locations[id].min);
const endPosition = offsetToRowColumn(text, locations[id].max);
return [
{
message: messages.join(', '),
startColumn: startPosition.column + 1,
startLineNumber: startPosition.lineNumber,
endColumn: endPosition.column + 1,
endLineNumber: endPosition.lineNumber,
severity: monaco.MarkerSeverity.Warning,
},
];
}
}
}
return [];
})
.filter((marker) => marker);
setWarnings(markers.map(({ severity, message }) => ({ severity, message })));
monaco.editor.setModelMarkers(editorModel.current, 'LENS', markers);
}
},
// Make it validate on flyout open in case of a broken formula left over
// from a previous edit
{ skipFirstRender: false },
256,
[text]
);
const errorCount = warnings.filter((marker) => marker.severity === monaco.MarkerSeverity.Error)
.length;
const warningCount = warnings.filter(
(marker) => marker.severity === monaco.MarkerSeverity.Warning
).length;
/**
* The way that Monaco requests autocompletion is not intuitive, but the way we use it
* we fetch new suggestions in these scenarios:
*
* - If the user types one of the trigger characters, suggestions are always fetched
* - When the user selects the kql= suggestion, we tell Monaco to trigger new suggestions after
* - When the user types the first character into an empty text box, Monaco requests suggestions
*
* Monaco also triggers suggestions automatically when there are no suggestions being displayed
* and the user types a non-whitespace character.
*
* While suggestions are being displayed, Monaco uses an in-memory cache of the last known suggestions.
*/
const provideCompletionItems = useCallback(
async (
model: monaco.editor.ITextModel,
position: monaco.Position,
context: monaco.languages.CompletionContext
) => {
const innerText = model.getValue();
let aSuggestions: { list: LensMathSuggestion[]; type: SUGGESTION_TYPE } = {
list: [],
type: SUGGESTION_TYPE.FIELD,
};
const offset = monacoPositionToOffset(innerText, position);
if (context.triggerCharacter === '(') {
// Monaco usually inserts the end quote and reports the position is after the end quote
if (innerText.slice(offset - 1, offset + 1) === '()') {
position = position.delta(0, -1);
}
const wordUntil = model.getWordAtPosition(position.delta(0, -3));
if (wordUntil) {
// Retrieve suggestions for subexpressions
aSuggestions = await suggest({
expression: innerText,
zeroIndexedOffset: offset,
context,
indexPattern,
operationDefinitionMap: visibleOperationsMap,
data,
});
}
} else {
aSuggestions = await suggest({
expression: innerText,
zeroIndexedOffset: offset,
context,
indexPattern,
operationDefinitionMap: visibleOperationsMap,
data,
});
}
return {
suggestions: aSuggestions.list.map((s) =>
getSuggestion(s, aSuggestions.type, visibleOperationsMap, context.triggerCharacter)
),
};
},
[indexPattern, visibleOperationsMap, data]
);
const provideSignatureHelp = useCallback(
async (
model: monaco.editor.ITextModel,
position: monaco.Position,
token: monaco.CancellationToken,
context: monaco.languages.SignatureHelpContext
) => {
const innerText = model.getValue();
const textRange = model.getFullModelRange();
const lengthAfterPosition = model.getValueLengthInRange({
startLineNumber: position.lineNumber,
startColumn: position.column,
endLineNumber: textRange.endLineNumber,
endColumn: textRange.endColumn,
});
return getSignatureHelp(
model.getValue(),
innerText.length - lengthAfterPosition,
visibleOperationsMap
);
},
[visibleOperationsMap]
);
const provideHover = useCallback(
async (
model: monaco.editor.ITextModel,
position: monaco.Position,
token: monaco.CancellationToken
) => {
const innerText = model.getValue();
const textRange = model.getFullModelRange();
const lengthAfterPosition = model.getValueLengthInRange({
startLineNumber: position.lineNumber,
startColumn: position.column,
endLineNumber: textRange.endLineNumber,
endColumn: textRange.endColumn,
});
return getHover(
model.getValue(),
innerText.length - lengthAfterPosition,
visibleOperationsMap
);
},
[visibleOperationsMap]
);
const onTypeHandler = useCallback(
(e: monaco.editor.IModelContentChangedEvent, editor: monaco.editor.IStandaloneCodeEditor) => {
if (e.isFlush || e.isRedoing || e.isUndoing) {
return;
}
if (e.changes.length === 1) {
const char = e.changes[0].text;
if (char !== '=' && char !== "'") {
return;
}
const currentPosition = e.changes[0].range;
if (currentPosition) {
const currentText = editor.getValue();
const offset = monacoPositionToOffset(
currentText,
new monaco.Position(currentPosition.startLineNumber, currentPosition.startColumn)
);
let tokenInfo = getTokenInfo(currentText, offset + 1);
if (!tokenInfo && char === "'") {
// try again this time replacing the current quote with an escaped quote
const line = currentText;
const lineEscaped = line.substring(0, offset) + "\\'" + line.substring(offset + 1);
tokenInfo = getTokenInfo(lineEscaped, offset + 2);
}
const isSingleQuoteCase = /'LENS_MATH_MARKER/;
// Make sure that we are only adding kql='' or lucene='', and also
// check that the = sign isn't inside the KQL expression like kql='='
if (
!tokenInfo ||
typeof tokenInfo.ast === 'number' ||
tokenInfo.ast.type !== 'namedArgument' ||
(tokenInfo.ast.name !== 'kql' && tokenInfo.ast.name !== 'lucene') ||
(tokenInfo.ast.value !== 'LENS_MATH_MARKER' &&
!isSingleQuoteCase.test(tokenInfo.ast.value))
) {
return;
}
let editOperation: monaco.editor.IIdentifiedSingleEditOperation | null = null;
const cursorOffset = 2;
if (char === '=') {
editOperation = {
range: {
...currentPosition,
// Insert after the current char
startColumn: currentPosition.startColumn + 1,
endColumn: currentPosition.startColumn + 1,
},
text: `''`,
};
}
if (char === "'") {
editOperation = {
range: {
...currentPosition,
// Insert after the current char
startColumn: currentPosition.startColumn,
endColumn: currentPosition.startColumn + 1,
},
text: `\\'`,
};
}
if (editOperation) {
setTimeout(() => {
editor.executeEdits(
'LENS',
[editOperation!],
[
// After inserting, move the cursor in between the single quotes or after the escaped quote
new monaco.Selection(
currentPosition.startLineNumber,
currentPosition.startColumn + cursorOffset,
currentPosition.startLineNumber,
currentPosition.startColumn + cursorOffset
),
]
);
// Need to move these sync to prevent race conditions between a fast user typing a single quote
// after an = char
// Timeout is required because otherwise the cursor position is not updated.
editor.setPosition({
column: currentPosition.startColumn + cursorOffset,
lineNumber: currentPosition.startLineNumber,
});
editor.trigger('lens', 'editor.action.triggerSuggest', {});
}, 0);
}
}
}
},
[]
);
const codeEditorOptions: CodeEditorProps = {
languageId: LANGUAGE_ID,
value: text ?? '',
onChange: setText,
options: {
automaticLayout: false,
fontSize: 14,
folding: false,
lineNumbers: 'off',
scrollBeyondLastLine: false,
minimap: { enabled: false },
wordWrap: isWordWrapped ? 'on' : 'off',
// Disable suggestions that appear when we don't provide a default suggestion
wordBasedSuggestions: false,
autoIndent: 'brackets',
wrappingIndent: 'none',
dimension: { width: 320, height: 200 },
fixedOverflowWidgets: true,
matchBrackets: 'always',
},
};
useEffect(() => {
// Because the monaco model is owned by Lens, we need to manually attach and remove handlers
const { dispose: dispose1 } = monaco.languages.registerCompletionItemProvider(LANGUAGE_ID, {
triggerCharacters: ['.', '(', '=', ' ', ':', `'`],
provideCompletionItems,
});
const { dispose: dispose2 } = monaco.languages.registerSignatureHelpProvider(LANGUAGE_ID, {
signatureHelpTriggerCharacters: ['(', '='],
provideSignatureHelp,
});
const { dispose: dispose3 } = monaco.languages.registerHoverProvider(LANGUAGE_ID, {
provideHover,
});
return () => {
dispose1();
dispose2();
dispose3();
};
}, [provideCompletionItems, provideSignatureHelp, provideHover]);
// The Monaco editor will lazily load Monaco, which takes a render cycle to trigger. This can cause differences
// in the behavior of Monaco when it's first loaded and then reloaded.
return (
<div
className={classNames({
lnsIndexPatternDimensionEditor: true,
'lnsIndexPatternDimensionEditor-isFullscreen': isFullscreen,
})}
>
<div className="lnsIndexPatternDimensionEditor__section lnsIndexPatternDimensionEditor__section--shaded">
<div className="lnsFormula">
<div className="lnsFormula__editor">
<div className="lnsFormula__editorHeader">
<EuiFlexGroup alignItems="center" gutterSize="m" responsive={false}>
<EuiFlexItem className="lnsFormula__editorHeaderGroup">
{/* TODO: Replace `bolt` with `wordWrap` icon (after latest EUI is deployed) and hook up button to enable/disable word wrapping. */}
<EuiToolTip
content={
isWordWrapped
? i18n.translate('xpack.lens.formula.disableWordWrapLabel', {
defaultMessage: 'Disable word wrap',
})
: i18n.translate('xpack.lens.formulaEnableWordWrapLabel', {
defaultMessage: 'Enable word wrap',
})
}
position="top"
>
<EuiButtonIcon
iconType="bolt"
display={!isWordWrapped ? 'fill' : undefined}
color={'text'}
aria-label={
isWordWrapped
? i18n.translate('xpack.lens.formula.disableWordWrapLabel', {
defaultMessage: 'Disable word wrap',
})
: i18n.translate('xpack.lens.formulaEnableWordWrapLabel', {
defaultMessage: 'Enable word wrap',
})
}
isSelected={!isWordWrapped}
onClick={() => {
editor1.current?.updateOptions({
wordWrap: isWordWrapped ? 'off' : 'on',
});
toggleWordWrap(!isWordWrapped);
}}
/>
</EuiToolTip>
</EuiFlexItem>
<EuiFlexItem className="lnsFormula__editorHeaderGroup" grow={false}>
{/* TODO: Replace `bolt` with `fullScreenExit` icon (after latest EUI is deployed). */}
<EuiButtonEmpty
onClick={() => {
toggleFullscreen();
// Help text opens when entering full screen, and closes when leaving full screen
setIsHelpOpen(!isFullscreen);
trackUiEvent('toggle_formula_fullscreen');
}}
iconType={isFullscreen ? 'bolt' : 'fullScreen'}
size="xs"
color="text"
flush="right"
data-test-subj="lnsFormula-fullscreen"
>
{isFullscreen
? i18n.translate('xpack.lens.formula.fullScreenExitLabel', {
defaultMessage: 'Collapse',
})
: i18n.translate('xpack.lens.formula.fullScreenEnterLabel', {
defaultMessage: 'Expand',
})}
</EuiButtonEmpty>
</EuiFlexItem>
</EuiFlexGroup>
</div>
<div className="lnsFormula__editorContent">
<CodeEditor
{...codeEditorOptions}
options={{
...codeEditorOptions.options,
// Shared model and overflow node
overflowWidgetsDomNode: overflowDiv1.current,
}}
editorDidMount={(editor) => {
editor1.current = editor;
const model = editor.getModel();
if (model) {
editorModel.current = model;
}
disposables.current.push(
editor.onDidFocusEditorWidget(() => {
setTimeout(() => {
setIsCloseable(false);
});
})
);
disposables.current.push(
editor.onDidBlurEditorWidget(() => {
setTimeout(() => {
setIsCloseable(true);
});
})
);
// If we ever introduce a second Monaco editor, we need to toggle
// the typing handler to the active editor to maintain the cursor
disposables.current.push(
editor.onDidChangeModelContent((e) => {
onTypeHandler(e, editor);
})
);
}}
/>
{!text ? (
<div className="lnsFormula__editorPlaceholder">
<EuiText color="subdued" size="s">
{i18n.translate('xpack.lens.formulaPlaceholderText', {
defaultMessage: 'Type a formula by combining functions with math, like:',
})}
</EuiText>
<EuiSpacer size="s" />
<pre>count() + 1</pre>
</div>
) : null}
</div>
<div className="lnsFormula__editorFooter">
<EuiFlexGroup alignItems="center" gutterSize="m" responsive={false}>
<EuiFlexItem className="lnsFormula__editorFooterGroup">
{isFullscreen ? (
<EuiToolTip
content={
isHelpOpen
? i18n.translate('xpack.lens.formula.editorHelpInlineHideToolTip', {
defaultMessage: 'Hide function reference',
})
: i18n.translate('xpack.lens.formula.editorHelpInlineShowToolTip', {
defaultMessage: 'Show function reference',
})
}
delay="long"
position="top"
>
<EuiLink
aria-label={i18n.translate('xpack.lens.formula.editorHelpInlineHideLabel', {
defaultMessage: 'Hide function reference',
})}
className="lnsFormula__editorHelp lnsFormula__editorHelp--inline"
color="text"
onClick={() => setIsHelpOpen(!isHelpOpen)}
>
<EuiIcon type="help" />
<EuiIcon type={isHelpOpen ? 'arrowDown' : 'arrowUp'} />
</EuiLink>
</EuiToolTip>
) : (
<EuiToolTip
content={i18n.translate('xpack.lens.formula.editorHelpOverlayToolTip', {
defaultMessage: 'Function reference',
})}
position="top"
>
<EuiPopover
panelClassName="lnsFormula__docs lnsFormula__docs--overlay"
panelPaddingSize="none"
anchorPosition="leftCenter"
isOpen={isHelpOpen}
closePopover={() => setIsHelpOpen(false)}
ownFocus={false}
button={
<EuiButtonIcon
className="lnsFormula__editorHelp lnsFormula__editorHelp--overlay"
onClick={() => setIsHelpOpen(!isHelpOpen)}
iconType="help"
color="text"
size="s"
aria-label={i18n.translate(
'xpack.lens.formula.editorHelpInlineShowToolTip',
{
defaultMessage: 'Show function reference',
}
)}
/>
}
>
<MemoizedFormulaHelp
isFullscreen={isFullscreen}
indexPattern={indexPattern}
operationDefinitionMap={visibleOperationsMap}
/>
</EuiPopover>
</EuiToolTip>
)}
</EuiFlexItem>
{errorCount || warningCount ? (
<EuiFlexItem className="lnsFormula__editorFooterGroup" grow={false}>
<EuiPopover
ownFocus={false}
isOpen={isWarningOpen}
closePopover={() => setIsWarningOpen(false)}
button={
<EuiButtonEmpty
color={errorCount ? 'danger' : 'warning'}
className="lnsFormula__editorError"
iconType="alert"
size="xs"
flush="right"
onClick={() => {
setIsWarningOpen(!isWarningOpen);
}}
>
{errorCount
? i18n.translate('xpack.lens.formulaErrorCount', {
defaultMessage:
'{count} {count, plural, one {error} other {errors}}',
values: { count: errorCount },
})
: null}
{warningCount
? i18n.translate('xpack.lens.formulaWarningCount', {
defaultMessage:
'{count} {count, plural, one {warning} other {warnings}}',
values: { count: warningCount },
})
: null}
</EuiButtonEmpty>
}
>
{warnings.map(({ message, severity }, index) => (
<div key={index} className="lnsFormula__warningText">
<EuiText
size="s"
color={
severity === monaco.MarkerSeverity.Warning ? 'warning' : 'danger'
}
>
{message}
</EuiText>
</div>
))}
</EuiPopover>
</EuiFlexItem>
) : null}
</EuiFlexGroup>
</div>
</div>
{isFullscreen && isHelpOpen ? (
<div className="lnsFormula__docs lnsFormula__docs--inline">
<MemoizedFormulaHelp
isFullscreen={isFullscreen}
indexPattern={indexPattern}
operationDefinitionMap={visibleOperationsMap}
/>
</div>
) : null}
</div>
</div>
</div>
);
}

469
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/editor/formula_help.tsx Normal file
View file

@ -0,0 +1,469 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import React, { useEffect, useRef, useState, useMemo } from 'react';
import { i18n } from '@kbn/i18n';
import {
EuiFlexGroup,
EuiFlexItem,
EuiLink,
EuiPopoverTitle,
EuiText,
EuiListGroupItem,
EuiListGroup,
EuiTitle,
EuiFieldSearch,
EuiHighlight,
} from '@elastic/eui';
import { Markdown } from '../../../../../../../../../src/plugins/kibana_react/public';
import { IndexPattern } from '../../../../types';
import { tinymathFunctions } from '../util';
import { getPossibleFunctions } from './math_completion';
import { hasFunctionFieldArgument } from '../validation';
import type {
GenericOperationDefinition,
IndexPatternColumn,
OperationDefinition,
ParamEditorProps,
} from '../../index';
import type { FormulaIndexPatternColumn } from '../formula';
function FormulaHelp({
indexPattern,
operationDefinitionMap,
isFullscreen,
}: {
indexPattern: IndexPattern;
operationDefinitionMap: Record<string, GenericOperationDefinition>;
isFullscreen: boolean;
}) {
const [selectedFunction, setSelectedFunction] = useState<string | undefined>();
const scrollTargets = useRef<Record<string, HTMLElement>>({});
useEffect(() => {
if (selectedFunction && scrollTargets.current[selectedFunction]) {
scrollTargets.current[selectedFunction].scrollIntoView();
}
}, [selectedFunction]);
const helpGroups: Array<{
label: string;
description?: string;
items: Array<{ label: string; description?: JSX.Element }>;
}> = [];
helpGroups.push({
label: i18n.translate('xpack.lens.formulaDocumentationHeading', {
defaultMessage: 'How it works',
}),
items: [],
});
helpGroups.push({
label: i18n.translate('xpack.lens.formulaDocumentation.elasticsearchSection', {
defaultMessage: 'Elasticsearch',
}),
description: i18n.translate('xpack.lens.formulaDocumentation.elasticsearchSectionDescription', {
defaultMessage:
'These functions will be executed on the raw documents for each row of the resulting table, aggregating all documents matching the break down dimensions into a single value.',
}),
items: [],
});
const availableFunctions = getPossibleFunctions(indexPattern);
// Es aggs
helpGroups[1].items.push(
...availableFunctions
.filter(
(key) =>
key in operationDefinitionMap &&
operationDefinitionMap[key].documentation?.section === 'elasticsearch'
)
.sort()
.map((key) => ({
label: key,
description: (
<>
<h3>
{key}({operationDefinitionMap[key].documentation?.signature})
</h3>
{operationDefinitionMap[key].documentation?.description ? (
<Markdown markdown={operationDefinitionMap[key].documentation!.description} />
) : null}
</>
),
}))
);
helpGroups.push({
label: i18n.translate('xpack.lens.formulaDocumentation.columnCalculationSection', {
defaultMessage: 'Column-wise calculation',
}),
description: i18n.translate(
'xpack.lens.formulaDocumentation.columnCalculationSectionDescription',
{
defaultMessage:
'These functions will be executed for reach row of the resulting table, using data from cells from other rows as well as the current value.',
}
),
items: [],
});
// Calculations aggs
helpGroups[2].items.push(
...availableFunctions
.filter(
(key) =>
key in operationDefinitionMap &&
operationDefinitionMap[key].documentation?.section === 'calculation'
)
.sort()
.map((key) => ({
label: key,
description: (
<>
<h3>
{key}({operationDefinitionMap[key].documentation?.signature})
</h3>
{operationDefinitionMap[key].documentation?.description ? (
<Markdown markdown={operationDefinitionMap[key].documentation!.description} />
) : null}
</>
),
checked:
selectedFunction === `${key}: ${operationDefinitionMap[key].displayName}`
? ('on' as const)
: undefined,
}))
);
helpGroups.push({
label: i18n.translate('xpack.lens.formulaDocumentation.mathSection', {
defaultMessage: 'Math',
}),
description: i18n.translate('xpack.lens.formulaDocumentation.mathSectionDescription', {
defaultMessage:
'These functions will be executed for reach row of the resulting table using single values from the same row calculated using other functions.',
}),
items: [],
});
const tinymathFns = useMemo(() => {
return getPossibleFunctions(indexPattern)
.filter((key) => key in tinymathFunctions)
.sort()
.map((key) => {
const [description, examples] = tinymathFunctions[key].help.split(`\`\`\``);
return {
label: key,
description: description.replace(/\n/g, '\n\n'),
examples: examples ? `\`\`\`${examples}\`\`\`` : '',
};
});
}, [indexPattern]);
helpGroups[3].items.push(
...tinymathFns.map(({ label, description, examples }) => {
return {
label,
description: (
<>
<h3>{getFunctionSignatureLabel(label, operationDefinitionMap)}</h3>
<Markdown markdown={`${description}${examples}`} />
</>
),
};
})
);
const [searchText, setSearchText] = useState('');
const normalizedSearchText = searchText.trim().toLocaleLowerCase();
const filteredHelpGroups = helpGroups
.map((group) => {
const items = group.items.filter((helpItem) => {
return (
!normalizedSearchText || helpItem.label.toLocaleLowerCase().includes(normalizedSearchText)
);
});
return { ...group, items };
})
.filter((group) => {
if (group.items.length > 0 || !normalizedSearchText) {
return true;
}
return group.label.toLocaleLowerCase().includes(normalizedSearchText);
});
return (
<>
<EuiPopoverTitle className="lnsFormula__docsHeader" paddingSize="s">
{i18n.translate('xpack.lens.formulaDocumentation.header', {
defaultMessage: 'Formula reference',
})}
</EuiPopoverTitle>
<EuiFlexGroup
className="lnsFormula__docsContent"
gutterSize="none"
responsive={false}
alignItems="stretch"
>
<EuiFlexItem className="lnsFormula__docsSidebar" grow={1}>
<EuiFlexGroup
className="lnsFormula__docsSidebarInner"
direction="column"
gutterSize="none"
responsive={false}
>
<EuiFlexItem className="lnsFormula__docsSearch" grow={false}>
<EuiFieldSearch
value={searchText}
onChange={(e) => {
setSearchText(e.target.value);
}}
placeholder={i18n.translate('xpack.lens.formulaSearchPlaceholder', {
defaultMessage: 'Search functions',
})}
/>
</EuiFlexItem>
<EuiFlexItem className="lnsFormula__docsNav">
{filteredHelpGroups.map((helpGroup, index) => {
return (
<nav className="lnsFormula__docsNavGroup" key={helpGroup.label}>
<EuiTitle size="xxs">
<h6>
<EuiLink
className="lnsFormula__docsNavGroupLink"
color="text"
onClick={() => {
setSelectedFunction(helpGroup.label);
}}
>
<EuiHighlight search={searchText}>{helpGroup.label}</EuiHighlight>
</EuiLink>
</h6>
</EuiTitle>
<EuiListGroup gutterSize="none">
{helpGroup.items.map((helpItem) => {
return (
<EuiListGroupItem
key={helpItem.label}
label={
<EuiHighlight search={searchText}>{helpItem.label}</EuiHighlight>
}
size="s"
onClick={() => {
setSelectedFunction(helpItem.label);
}}
/>
);
})}
</EuiListGroup>
</nav>
);
})}
</EuiFlexItem>
</EuiFlexGroup>
</EuiFlexItem>
<EuiFlexItem className="lnsFormula__docsText" grow={2}>
<EuiText size="s">
<section
className="lnsFormula__docsTextIntro"
ref={(el) => {
if (el) {
scrollTargets.current[helpGroups[0].label] = el;
}
}}
>
<Markdown
markdown={i18n.translate('xpack.lens.formulaDocumentation', {
defaultMessage: `
## How it works
Lens formulas let you do math using a combination of Elasticsearch aggregations and
math functions. There are three main types of functions:
* Elasticsearch metrics, like \`sum(bytes)\`
* Time series functions use Elasticsearch metrics as input, like \`cumulative_sum()\`
* Math functions like \`round()\`
An example formula that uses all of these:
\`\`\`
round(100 * moving_average(
average(cpu.load.pct),
window=10,
kql='datacenter.name: east*'
))
\`\`\`
Elasticsearch functions take a field name, which can be in quotes. \`sum(bytes)\` is the same
as \`sum("bytes")\`.
Some functions take named arguments, like moving_average(count(), window=5)
Elasticsearch metrics can be filtered using KQL or Lucene syntax. To add a filter, use the named
parameter \`kql='field: value'\` or \`lucene=''\`. Always use single quotes when writing KQL or Lucene
queries. If your search has a single quote in it, use a backslash to escape, like: \`kql='Women's'\'
Math functions can take positional arguments, like pow(count(), 3) is the same as count() * count() * count()
Use the symbols +, -, /, and * to perform basic math.
`,
description:
'Text is in markdown. Do not translate function names or field names like sum(bytes)',
})}
/>
</section>
{helpGroups.slice(1).map((helpGroup, index) => {
return (
<section
className="lnsFormula__docsTextGroup"
key={helpGroup.label}
ref={(el) => {
if (el) {
scrollTargets.current[helpGroup.label] = el;
}
}}
>
<h2>{helpGroup.label}</h2>
<p>{helpGroup.description}</p>
{helpGroups[index + 1].items.map((helpItem) => {
return (
<article
className="lnsFormula__docsTextItem"
key={helpItem.label}
ref={(el) => {
if (el) {
scrollTargets.current[helpItem.label] = el;
}
}}
>
{helpItem.description}
</article>
);
})}
</section>
);
})}
</EuiText>
</EuiFlexItem>
</EuiFlexGroup>
</>
);
}
export const MemoizedFormulaHelp = React.memo(FormulaHelp);
export function getFunctionSignatureLabel(
name: string,
operationDefinitionMap: ParamEditorProps<FormulaIndexPatternColumn>['operationDefinitionMap'],
firstParam?: { label: string | [number, number] } | null
): string {
if (tinymathFunctions[name]) {
return `${name}(${tinymathFunctions[name].positionalArguments
.map(({ name: argName, optional, type }) => `[${argName}]${optional ? '?' : ''}: ${type}`)
.join(', ')})`;
}
if (operationDefinitionMap[name]) {
const def = operationDefinitionMap[name];
let extraArgs = '';
if (def.filterable) {
extraArgs += hasFunctionFieldArgument(name) || 'operationParams' in def ? ',' : '';
extraArgs += i18n.translate('xpack.lens.formula.kqlExtraArguments', {
defaultMessage: '[kql]?: string, [lucene]?: string',
});
}
return `${name}(${def.documentation?.signature}${extraArgs})`;
}
return '';
}
function getFunctionArgumentsStringified(
params: Required<
OperationDefinition<IndexPatternColumn, 'field' | 'fullReference'>
>['operationParams']
) {
return params
.map(
({ name, type: argType, defaultValue = 5 }) =>
`${name}=${argType === 'string' ? `"${defaultValue}"` : defaultValue}`
)
.join(', ');
}
/**
* Get an array of strings containing all possible information about a specific
* operation type: examples and infos.
*/
export function getHelpTextContent(
type: string,
operationDefinitionMap: ParamEditorProps<FormulaIndexPatternColumn>['operationDefinitionMap']
): { description: string; examples: string[] } {
const definition = operationDefinitionMap[type];
const description = definition.documentation?.description ?? '';
// as for the time being just add examples text.
// Later will enrich with more information taken from the operation definitions.
const examples: string[] = [];
// If the description already contain examples skip it
if (!/Example/.test(description)) {
if (!hasFunctionFieldArgument(type)) {
// ideally this should have the same example automation as the operations below
examples.push(`${type}()`);
return { description, examples };
}
if (definition.input === 'field') {
const mandatoryArgs = definition.operationParams?.filter(({ required }) => required) || [];
if (mandatoryArgs.length === 0) {
examples.push(`${type}(bytes)`);
}
if (mandatoryArgs.length) {
const additionalArgs = getFunctionArgumentsStringified(mandatoryArgs);
examples.push(`${type}(bytes, ${additionalArgs})`);
}
if (
definition.operationParams &&
mandatoryArgs.length !== definition.operationParams.length
) {
const additionalArgs = getFunctionArgumentsStringified(definition.operationParams);
examples.push(`${type}(bytes, ${additionalArgs})`);
}
}
if (definition.input === 'fullReference') {
const mandatoryArgs = definition.operationParams?.filter(({ required }) => required) || [];
if (mandatoryArgs.length === 0) {
examples.push(`${type}(sum(bytes))`);
}
if (mandatoryArgs.length) {
const additionalArgs = getFunctionArgumentsStringified(mandatoryArgs);
examples.push(`${type}(sum(bytes), ${additionalArgs})`);
}
if (
definition.operationParams &&
mandatoryArgs.length !== definition.operationParams.length
) {
const additionalArgs = getFunctionArgumentsStringified(definition.operationParams);
examples.push(`${type}(sum(bytes), ${additionalArgs})`);
}
}
}
return { description, examples };
}

8
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/editor/index.ts Normal file
View file

@ -0,0 +1,8 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export * from './formula_editor';

386
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/editor/math_completion.test.ts Normal file
View file

@ -0,0 +1,386 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { parse } from '@kbn/tinymath';
import { monaco } from '@kbn/monaco';
import { createMockedIndexPattern } from '../../../../mocks';
import { GenericOperationDefinition } from '../../index';
import type { IndexPatternField } from '../../../../types';
import type { OperationMetadata } from '../../../../../types';
import { dataPluginMock } from '../../../../../../../../../src/plugins/data/public/mocks';
import { tinymathFunctions } from '../util';
import {
getSignatureHelp,
getHover,
suggest,
monacoPositionToOffset,
getInfoAtZeroIndexedPosition,
} from './math_completion';
const buildGenericColumn = (type: string) => {
return ({ field }: { field?: IndexPatternField }) => {
return {
label: type,
dataType: 'number',
operationType: type,
sourceField: field?.name ?? undefined,
isBucketed: false,
scale: 'ratio',
timeScale: false,
};
};
};
const numericOperation = () => ({ dataType: 'number', isBucketed: false });
const stringOperation = () => ({ dataType: 'string', isBucketed: true });
// Only one of each type is needed
const operationDefinitionMap: Record<string, GenericOperationDefinition> = {
sum: ({
type: 'sum',
input: 'field',
buildColumn: buildGenericColumn('sum'),
getPossibleOperationForField: (field: IndexPatternField) =>
field.type === 'number' ? numericOperation() : null,
documentation: {
section: 'elasticsearch',
signature: 'field: string',
description: 'description',
},
} as unknown) as GenericOperationDefinition,
count: ({
type: 'count',
input: 'field',
buildColumn: buildGenericColumn('count'),
getPossibleOperationForField: (field: IndexPatternField) =>
field.name === 'Records' ? numericOperation() : null,
} as unknown) as GenericOperationDefinition,
last_value: ({
type: 'last_value',
input: 'field',
buildColumn: buildGenericColumn('last_value'),
getPossibleOperationForField: (field: IndexPatternField) => ({
dataType: field.type,
isBucketed: false,
}),
} as unknown) as GenericOperationDefinition,
moving_average: ({
type: 'moving_average',
input: 'fullReference',
requiredReferences: [
{
input: ['field', 'managedReference'],
validateMetadata: (meta: OperationMetadata) =>
meta.dataType === 'number' && !meta.isBucketed,
},
],
operationParams: [{ name: 'window', type: 'number', required: true }],
buildColumn: buildGenericColumn('moving_average'),
getPossibleOperation: numericOperation,
} as unknown) as GenericOperationDefinition,
cumulative_sum: ({
type: 'cumulative_sum',
input: 'fullReference',
buildColumn: buildGenericColumn('cumulative_sum'),
getPossibleOperation: numericOperation,
} as unknown) as GenericOperationDefinition,
terms: ({
type: 'terms',
input: 'field',
getPossibleOperationForField: stringOperation,
} as unknown) as GenericOperationDefinition,
};
describe('math completion', () => {
describe('signature help', () => {
function unwrapSignatures(signatureResult: monaco.languages.SignatureHelpResult) {
return signatureResult.value.signatures[0];
}
it('should silently handle parse errors', () => {
expect(unwrapSignatures(getSignatureHelp('sum(', 4, operationDefinitionMap))).toBeUndefined();
});
it('should return a signature for a field-based ES function', () => {
expect(unwrapSignatures(getSignatureHelp('sum()', 4, operationDefinitionMap))).toEqual({
label: 'sum(field: string)',
documentation: { value: 'description' },
parameters: [{ label: 'field' }],
});
});
it('should return a signature for count', () => {
expect(unwrapSignatures(getSignatureHelp('count()', 6, operationDefinitionMap))).toEqual({
label: 'count(undefined)',
documentation: { value: '' },
parameters: [],
});
});
it('should return a signature for a function with named parameters', () => {
expect(
unwrapSignatures(
getSignatureHelp('2 * moving_average(count(), window=)', 35, operationDefinitionMap)
)
).toEqual({
label: expect.stringContaining('moving_average('),
documentation: { value: '' },
parameters: [
{ label: 'function' },
{
label: 'window=number',
documentation: 'Required',
},
],
});
});
it('should return a signature for an inner function', () => {
expect(
unwrapSignatures(
getSignatureHelp('2 * moving_average(count())', 25, operationDefinitionMap)
)
).toEqual({
label: expect.stringContaining('count('),
parameters: [],
documentation: { value: '' },
});
});
it('should return a signature for a complex tinymath function', () => {
// 15 is the whitespace between the two arguments
expect(
unwrapSignatures(getSignatureHelp('clamp(count(), 5)', 15, operationDefinitionMap))
).toEqual({
label: expect.stringContaining('clamp('),
documentation: { value: '' },
parameters: [
{ label: 'value', documentation: '' },
{ label: 'min', documentation: '' },
{ label: 'max', documentation: '' },
],
});
});
});
describe('hover provider', () => {
it('should silently handle parse errors', () => {
expect(getHover('sum(', 2, operationDefinitionMap)).toEqual({ contents: [] });
});
it('should show signature for a field-based ES function', () => {
expect(getHover('sum()', 2, operationDefinitionMap)).toEqual({
contents: [{ value: 'sum(field: string)' }],
});
});
it('should show signature for count', () => {
expect(getHover('count()', 2, operationDefinitionMap)).toEqual({
contents: [{ value: expect.stringContaining('count(') }],
});
});
it('should show signature for a function with named parameters', () => {
expect(getHover('2 * moving_average(count())', 10, operationDefinitionMap)).toEqual({
contents: [{ value: expect.stringContaining('moving_average(') }],
});
});
it('should show signature for an inner function', () => {
expect(getHover('2 * moving_average(count())', 22, operationDefinitionMap)).toEqual({
contents: [{ value: expect.stringContaining('count(') }],
});
});
it('should show signature for a complex tinymath function', () => {
expect(getHover('clamp(count(), 5)', 2, operationDefinitionMap)).toEqual({
contents: [{ value: expect.stringContaining('clamp([value]: number') }],
});
});
});
describe('autocomplete', () => {
it('should list all valid functions at the top level (fake test)', async () => {
// This test forces an invalid scenario, since the autocomplete actually requires
// some typing
const results = await suggest({
expression: '',
zeroIndexedOffset: 1,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: '',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toHaveLength(4 + Object.keys(tinymathFunctions).length);
['sum', 'moving_average', 'cumulative_sum', 'last_value'].forEach((key) => {
expect(results.list).toEqual(expect.arrayContaining([{ label: key, type: 'operation' }]));
});
Object.keys(tinymathFunctions).forEach((key) => {
expect(results.list).toEqual(expect.arrayContaining([{ label: key, type: 'operation' }]));
});
});
it('should list all valid sub-functions for a fullReference', async () => {
const results = await suggest({
expression: 'moving_average()',
zeroIndexedOffset: 15,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: '(',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toHaveLength(2);
['sum', 'last_value'].forEach((key) => {
expect(results.list).toEqual(expect.arrayContaining([{ label: key, type: 'operation' }]));
});
});
it('should list all valid named arguments for a fullReference', async () => {
const results = await suggest({
expression: 'moving_average(count(),)',
zeroIndexedOffset: 23,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: ',',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toEqual(['window']);
});
it('should not list named arguments when they are already in use', async () => {
const results = await suggest({
expression: 'moving_average(count(), window=5, )',
zeroIndexedOffset: 34,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: ',',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toEqual([]);
});
it('should list all valid positional arguments for a tinymath function used by name', async () => {
const results = await suggest({
expression: 'divide(count(), )',
zeroIndexedOffset: 16,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: ',',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toHaveLength(4 + Object.keys(tinymathFunctions).length);
['sum', 'moving_average', 'cumulative_sum', 'last_value'].forEach((key) => {
expect(results.list).toEqual(expect.arrayContaining([{ label: key, type: 'math' }]));
});
Object.keys(tinymathFunctions).forEach((key) => {
expect(results.list).toEqual(expect.arrayContaining([{ label: key, type: 'math' }]));
});
});
it('should list all valid positional arguments for a tinymath function used with alias', async () => {
const results = await suggest({
expression: 'count() / ',
zeroIndexedOffset: 10,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: ',',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toHaveLength(4 + Object.keys(tinymathFunctions).length);
['sum', 'moving_average', 'cumulative_sum', 'last_value'].forEach((key) => {
expect(results.list).toEqual(expect.arrayContaining([{ label: key, type: 'math' }]));
});
Object.keys(tinymathFunctions).forEach((key) => {
expect(results.list).toEqual(expect.arrayContaining([{ label: key, type: 'math' }]));
});
});
it('should not autocomplete any fields for the count function', async () => {
const results = await suggest({
expression: 'count()',
zeroIndexedOffset: 6,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: '(',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toHaveLength(0);
});
it('should autocomplete and validate the right type of field', async () => {
const results = await suggest({
expression: 'sum()',
zeroIndexedOffset: 4,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: '(',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toEqual(['bytes', 'memory']);
});
it('should autocomplete only operations that provide numeric output', async () => {
const results = await suggest({
expression: 'last_value()',
zeroIndexedOffset: 11,
context: {
triggerKind: monaco.languages.CompletionTriggerKind.TriggerCharacter,
triggerCharacter: '(',
},
indexPattern: createMockedIndexPattern(),
operationDefinitionMap,
data: dataPluginMock.createStartContract(),
});
expect(results.list).toEqual(['bytes', 'memory']);
});
});
describe('monacoPositionToOffset', () => {
it('should work with multi-line strings accounting for newline characters', () => {
const input = `012
456
89')`;
expect(input[monacoPositionToOffset(input, new monaco.Position(1, 1))]).toEqual('0');
expect(input[monacoPositionToOffset(input, new monaco.Position(3, 2))]).toEqual('9');
});
});
describe('getInfoAtZeroIndexedPosition', () => {
it('should return the location for a function inside multiple levels of math', () => {
const expression = `count() + 5 + average(LENS_MATH_MARKER)`;
const ast = parse(expression);
expect(getInfoAtZeroIndexedPosition(ast, 22)).toEqual({
ast: expect.objectContaining({ value: 'LENS_MATH_MARKER' }),
parent: expect.objectContaining({ name: 'average' }),
});
});
});
});

594
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/editor/math_completion.ts Normal file
View file

@ -0,0 +1,594 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { uniq, startsWith } from 'lodash';
import { i18n } from '@kbn/i18n';
import { monaco } from '@kbn/monaco';
import {
parse,
TinymathLocation,
TinymathAST,
TinymathFunction,
TinymathNamedArgument,
} from '@kbn/tinymath';
import type {
DataPublicPluginStart,
QuerySuggestion,
} from '../../../../../../../../../src/plugins/data/public';
import { IndexPattern } from '../../../../types';
import { memoizedGetAvailableOperationsByMetadata } from '../../../operations';
import { tinymathFunctions, groupArgsByType } from '../util';
import type { GenericOperationDefinition } from '../..';
import { getFunctionSignatureLabel, getHelpTextContent } from './formula_help';
import { hasFunctionFieldArgument } from '../validation';
export enum SUGGESTION_TYPE {
FIELD = 'field',
NAMED_ARGUMENT = 'named_argument',
FUNCTIONS = 'functions',
KQL = 'kql',
}
export type LensMathSuggestion =
| string
| {
label: string;
type: 'operation' | 'math';
}
| QuerySuggestion;
export interface LensMathSuggestions {
list: LensMathSuggestion[];
type: SUGGESTION_TYPE;
}
function inLocation(cursorPosition: number, location: TinymathLocation) {
return cursorPosition >= location.min && cursorPosition < location.max;
}
const MARKER = 'LENS_MATH_MARKER';
export function getInfoAtZeroIndexedPosition(
ast: TinymathAST,
zeroIndexedPosition: number,
parent?: TinymathFunction
): undefined | { ast: TinymathAST; parent?: TinymathFunction } {
if (typeof ast === 'number') {
return;
}
// +, -, *, and / do not have location any more
if (ast.location && !inLocation(zeroIndexedPosition, ast.location)) {
return;
}
if (ast.type === 'function') {
const [match] = ast.args
.map((arg) => getInfoAtZeroIndexedPosition(arg, zeroIndexedPosition, ast))
.filter((a) => a);
if (match) {
return match;
} else if (ast.location) {
return { ast };
} else {
// None of the arguments match, but we don't know the position so it's not a match
return;
}
}
return {
ast,
parent,
};
}
export function offsetToRowColumn(expression: string, offset: number): monaco.Position {
const lines = expression.split(/\n/);
let remainingChars = offset;
let lineNumber = 1;
for (const line of lines) {
if (line.length >= remainingChars) {
return new monaco.Position(lineNumber, remainingChars);
}
remainingChars -= line.length + 1;
lineNumber++;
}
throw new Error('Algorithm failure');
}
export function monacoPositionToOffset(expression: string, position: monaco.Position): number {
const lines = expression.split(/\n/);
return lines
.slice(0, position.lineNumber)
.reduce(
(prev, current, index) =>
prev + (index === position.lineNumber - 1 ? position.column - 1 : current.length + 1),
0
);
}
export async function suggest({
expression,
zeroIndexedOffset,
context,
indexPattern,
operationDefinitionMap,
data,
}: {
expression: string;
zeroIndexedOffset: number;
context: monaco.languages.CompletionContext;
indexPattern: IndexPattern;
operationDefinitionMap: Record<string, GenericOperationDefinition>;
data: DataPublicPluginStart;
}): Promise<{ list: LensMathSuggestion[]; type: SUGGESTION_TYPE }> {
const text =
expression.substr(0, zeroIndexedOffset) + MARKER + expression.substr(zeroIndexedOffset);
try {
const ast = parse(text);
const tokenInfo = getInfoAtZeroIndexedPosition(ast, zeroIndexedOffset);
const tokenAst = tokenInfo?.ast;
const isNamedArgument =
tokenInfo?.parent &&
typeof tokenAst !== 'number' &&
tokenAst &&
'type' in tokenAst &&
tokenAst.type === 'namedArgument';
if (tokenInfo?.parent && (context.triggerCharacter === '=' || isNamedArgument)) {
return await getNamedArgumentSuggestions({
ast: tokenAst as TinymathNamedArgument,
data,
indexPattern,
});
} else if (tokenInfo?.parent) {
return getArgumentSuggestions(
tokenInfo.parent,
tokenInfo.parent.args.findIndex((a) => a === tokenAst),
indexPattern,
operationDefinitionMap
);
}
if (
typeof tokenAst === 'object' &&
Boolean(tokenAst.type === 'variable' || tokenAst.type === 'function')
) {
const nameWithMarker = tokenAst.type === 'function' ? tokenAst.name : tokenAst.value;
return getFunctionSuggestions(
nameWithMarker.split(MARKER)[0],
indexPattern,
operationDefinitionMap
);
}
} catch (e) {
// Fail silently
}
return { list: [], type: SUGGESTION_TYPE.FIELD };
}
export function getPossibleFunctions(
indexPattern: IndexPattern,
operationDefinitionMap?: Record<string, GenericOperationDefinition>
) {
const available = memoizedGetAvailableOperationsByMetadata(indexPattern, operationDefinitionMap);
const possibleOperationNames: string[] = [];
available.forEach((a) => {
if (a.operationMetaData.dataType === 'number' && !a.operationMetaData.isBucketed) {
possibleOperationNames.push(
...a.operations.filter((o) => o.type !== 'managedReference').map((o) => o.operationType)
);
}
});
return [...uniq(possibleOperationNames), ...Object.keys(tinymathFunctions)];
}
function getFunctionSuggestions(
prefix: string,
indexPattern: IndexPattern,
operationDefinitionMap: Record<string, GenericOperationDefinition>
) {
return {
list: uniq(
getPossibleFunctions(indexPattern, operationDefinitionMap).filter((func) =>
startsWith(func, prefix)
)
).map((func) => ({ label: func, type: 'operation' as const })),
type: SUGGESTION_TYPE.FUNCTIONS,
};
}
function getArgumentSuggestions(
ast: TinymathFunction,
position: number,
indexPattern: IndexPattern,
operationDefinitionMap: Record<string, GenericOperationDefinition>
) {
const { name } = ast;
const operation = operationDefinitionMap[name];
if (!operation && !tinymathFunctions[name]) {
return { list: [], type: SUGGESTION_TYPE.FIELD };
}
const tinymathFunction = tinymathFunctions[name];
if (tinymathFunction) {
if (tinymathFunction.positionalArguments[position]) {
return {
list: uniq(getPossibleFunctions(indexPattern, operationDefinitionMap)).map((f) => ({
type: 'math' as const,
label: f,
})),
type: SUGGESTION_TYPE.FUNCTIONS,
};
}
return { list: [], type: SUGGESTION_TYPE.FIELD };
}
if (position > 0 || !hasFunctionFieldArgument(operation.type)) {
const { namedArguments } = groupArgsByType(ast.args);
const list = [];
if (operation.filterable) {
if (!namedArguments.find((arg) => arg.name === 'kql')) {
list.push('kql');
}
if (!namedArguments.find((arg) => arg.name === 'lucene')) {
list.push('lucene');
}
}
if ('operationParams' in operation) {
// Exclude any previously used named args
list.push(
...operation
.operationParams!.filter(
(param) =>
// Keep the param if it's the first use
!namedArguments.find((arg) => arg.name === param.name)
)
.map((p) => p.name)
);
}
return { list, type: SUGGESTION_TYPE.NAMED_ARGUMENT };
}
if (operation.input === 'field' && position === 0) {
const available = memoizedGetAvailableOperationsByMetadata(
indexPattern,
operationDefinitionMap
);
// TODO: This only allow numeric functions, will reject last_value(string) for example.
const validOperation = available.find(
({ operationMetaData }) =>
operationMetaData.dataType === 'number' && !operationMetaData.isBucketed
);
if (validOperation) {
const fields = validOperation.operations
.filter((op) => op.operationType === operation.type)
.map((op) => ('field' in op ? op.field : undefined))
.filter((field) => field);
return { list: fields as string[], type: SUGGESTION_TYPE.FIELD };
} else {
return { list: [], type: SUGGESTION_TYPE.FIELD };
}
}
if (operation.input === 'fullReference') {
const available = memoizedGetAvailableOperationsByMetadata(
indexPattern,
operationDefinitionMap
);
const possibleOperationNames: string[] = [];
available.forEach((a) => {
if (
operation.requiredReferences.some((requirement) =>
requirement.validateMetadata(a.operationMetaData)
)
) {
possibleOperationNames.push(
...a.operations
.filter((o) =>
operation.requiredReferences.some((requirement) => requirement.input.includes(o.type))
)
.map((o) => o.operationType)
);
}
});
return {
list: uniq(possibleOperationNames).map((n) => ({ label: n, type: 'operation' as const })),
type: SUGGESTION_TYPE.FUNCTIONS,
};
}
return { list: [], type: SUGGESTION_TYPE.FIELD };
}
export async function getNamedArgumentSuggestions({
ast,
data,
indexPattern,
}: {
ast: TinymathNamedArgument;
indexPattern: IndexPattern;
data: DataPublicPluginStart;
}) {
if (ast.name !== 'kql' && ast.name !== 'lucene') {
return { list: [], type: SUGGESTION_TYPE.KQL };
}
if (!data.autocomplete.hasQuerySuggestions(ast.name === 'kql' ? 'kuery' : 'lucene')) {
return { list: [], type: SUGGESTION_TYPE.KQL };
}
const query = ast.value.split(MARKER)[0];
const position = ast.value.indexOf(MARKER) + 1;
const suggestions = await data.autocomplete.getQuerySuggestions({
language: ast.name === 'kql' ? 'kuery' : 'lucene',
query,
selectionStart: position,
selectionEnd: position,
indexPatterns: [indexPattern],
boolFilter: [],
});
return {
list: suggestions ?? [],
type: SUGGESTION_TYPE.KQL,
};
}
const TRIGGER_SUGGESTION_COMMAND = {
title: 'Trigger Suggestion Dialog',
id: 'editor.action.triggerSuggest',
};
export function getSuggestion(
suggestion: LensMathSuggestion,
type: SUGGESTION_TYPE,
operationDefinitionMap: Record<string, GenericOperationDefinition>,
triggerChar: string | undefined
): monaco.languages.CompletionItem {
let kind: monaco.languages.CompletionItemKind = monaco.languages.CompletionItemKind.Method;
let label: string =
typeof suggestion === 'string'
? suggestion
: 'label' in suggestion
? suggestion.label
: suggestion.text;
let insertText: string | undefined;
let insertTextRules: monaco.languages.CompletionItem['insertTextRules'];
let detail: string = '';
let command: monaco.languages.CompletionItem['command'];
let sortText: string = '';
const filterText: string = label;
switch (type) {
case SUGGESTION_TYPE.FIELD:
kind = monaco.languages.CompletionItemKind.Value;
break;
case SUGGESTION_TYPE.FUNCTIONS:
insertText = `${label}($0)`;
insertTextRules = monaco.languages.CompletionItemInsertTextRule.InsertAsSnippet;
if (typeof suggestion !== 'string') {
if ('text' in suggestion) break;
label = getFunctionSignatureLabel(suggestion.label, operationDefinitionMap);
const tinymathFunction = tinymathFunctions[suggestion.label];
if (tinymathFunction) {
detail = 'TinyMath';
kind = monaco.languages.CompletionItemKind.Method;
} else {
kind = monaco.languages.CompletionItemKind.Constant;
detail = 'Elasticsearch';
// Always put ES functions first
sortText = `0${label}`;
command = TRIGGER_SUGGESTION_COMMAND;
}
}
break;
case SUGGESTION_TYPE.NAMED_ARGUMENT:
kind = monaco.languages.CompletionItemKind.Keyword;
if (label === 'kql' || label === 'lucene') {
command = TRIGGER_SUGGESTION_COMMAND;
insertText = `${label}='$0'`;
insertTextRules = monaco.languages.CompletionItemInsertTextRule.InsertAsSnippet;
sortText = `zzz${label}`;
}
label = `${label}=`;
detail = '';
break;
case SUGGESTION_TYPE.KQL:
if (triggerChar === ':') {
insertText = `${triggerChar} ${label}`;
} else {
// concatenate KQL suggestion for faster query composition
command = TRIGGER_SUGGESTION_COMMAND;
}
if (label.includes(`'`)) {
insertText = (insertText || label).replaceAll(`'`, "\\'");
}
break;
}
return {
detail,
kind,
label,
insertText: insertText ?? label,
insertTextRules,
command,
additionalTextEdits: [],
// @ts-expect-error Monaco says this type is required, but provides a default value
range: undefined,
sortText,
filterText,
};
}
function getOperationTypeHelp(
name: string,
operationDefinitionMap: Record<string, GenericOperationDefinition>
) {
const { description: descriptionInMarkdown, examples } = getHelpTextContent(
name,
operationDefinitionMap
);
const examplesInMarkdown = examples.length
? `\n\n**${i18n.translate('xpack.lens.formulaExampleMarkdown', {
defaultMessage: 'Examples',
})}**
${examples.map((example) => `\`${example}\``).join('\n\n')}`
: '';
return {
value: `${descriptionInMarkdown}${examplesInMarkdown}`,
};
}
function getSignaturesForFunction(
name: string,
operationDefinitionMap: Record<string, GenericOperationDefinition>
) {
if (tinymathFunctions[name]) {
const stringify = getFunctionSignatureLabel(name, operationDefinitionMap);
const documentation = tinymathFunctions[name].help.replace(/\n/g, '\n\n');
return [
{
label: stringify,
documentation: { value: documentation },
parameters: tinymathFunctions[name].positionalArguments.map((arg) => ({
label: arg.name,
documentation: arg.optional
? i18n.translate('xpack.lens.formula.optionalArgument', {
defaultMessage: 'Optional. Default value is {defaultValue}',
values: {
defaultValue: arg.defaultValue,
},
})
: '',
})),
},
];
}
if (operationDefinitionMap[name]) {
const def = operationDefinitionMap[name];
const firstParam: monaco.languages.ParameterInformation | null = hasFunctionFieldArgument(name)
? {
label: def.input === 'field' ? 'field' : def.input === 'fullReference' ? 'function' : '',
}
: null;
const functionLabel = getFunctionSignatureLabel(name, operationDefinitionMap, firstParam);
const documentation = getOperationTypeHelp(name, operationDefinitionMap);
if ('operationParams' in def && def.operationParams) {
return [
{
label: functionLabel,
parameters: [
...(firstParam ? [firstParam] : []),
...def.operationParams.map((arg) => ({
label: `${arg.name}=${arg.type}`,
documentation: arg.required
? i18n.translate('xpack.lens.formula.requiredArgument', {
defaultMessage: 'Required',
})
: '',
})),
],
documentation,
},
];
}
return [
{
label: functionLabel,
parameters: firstParam ? [firstParam] : [],
documentation,
},
];
}
return [];
}
export function getSignatureHelp(
expression: string,
position: number,
operationDefinitionMap: Record<string, GenericOperationDefinition>
): monaco.languages.SignatureHelpResult {
const text = expression.substr(0, position) + MARKER + expression.substr(position);
try {
const ast = parse(text);
const tokenInfo = getInfoAtZeroIndexedPosition(ast, position);
let signatures: ReturnType<typeof getSignaturesForFunction> = [];
let index = 0;
if (tokenInfo?.parent) {
const name = tokenInfo.parent.name;
// reference equality is fine here because of the way the getInfo function works
index = tokenInfo.parent.args.findIndex((arg) => arg === tokenInfo.ast);
signatures = getSignaturesForFunction(name, operationDefinitionMap);
} else if (typeof tokenInfo?.ast === 'object' && tokenInfo.ast.type === 'function') {
const name = tokenInfo.ast.name;
signatures = getSignaturesForFunction(name, operationDefinitionMap);
}
if (signatures.length) {
return {
value: {
// remove the documentation
signatures: signatures.map(({ documentation, ...signature }) => ({
...signature,
// extract only the first section (usually few lines)
documentation: { value: documentation.value.split('\n\n')[0] },
})),
activeParameter: index,
activeSignature: 0,
},
dispose: () => {},
};
}
} catch (e) {
// do nothing
}
return { value: { signatures: [], activeParameter: 0, activeSignature: 0 }, dispose: () => {} };
}
export function getHover(
expression: string,
position: number,
operationDefinitionMap: Record<string, GenericOperationDefinition>
): monaco.languages.Hover {
try {
const ast = parse(expression);
const tokenInfo = getInfoAtZeroIndexedPosition(ast, position);
if (!tokenInfo || typeof tokenInfo.ast === 'number' || !('name' in tokenInfo.ast)) {
return { contents: [] };
}
const name = tokenInfo.ast.name;
const signatures = getSignaturesForFunction(name, operationDefinitionMap);
if (signatures.length) {
const { label } = signatures[0];
return {
contents: [{ value: label }],
};
}
} catch (e) {
// do nothing
}
return { contents: [] };
}
export function getTokenInfo(expression: string, position: number) {
const text = expression.substr(0, position) + MARKER + expression.substr(position);
try {
const ast = parse(text);
return getInfoAtZeroIndexedPosition(ast, position);
} catch (e) {
return;
}
}

66
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/editor/math_tokenization.tsx Normal file
View file

@ -0,0 +1,66 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { monaco } from '@kbn/monaco';
export const LANGUAGE_ID = 'lens_math';
monaco.languages.register({ id: LANGUAGE_ID });
export const languageConfiguration: monaco.languages.LanguageConfiguration = {
wordPattern: /[^()'"\s]+/g,
brackets: [['(', ')']],
autoClosingPairs: [
{ open: '(', close: ')' },
{ open: `'`, close: `'` },
{ open: '"', close: '"' },
],
surroundingPairs: [
{ open: '(', close: ')' },
{ open: `'`, close: `'` },
{ open: '"', close: '"' },
],
};
export const lexerRules = {
defaultToken: 'invalid',
tokenPostfix: '',
ignoreCase: true,
brackets: [{ open: '(', close: ')', token: 'delimiter.parenthesis' }],
escapes: /\\(?:[\\"'])/,
tokenizer: {
root: [
[/\s+/, 'whitespace'],
[/-?(\d*\.)?\d+([eE][+\-]?\d+)?/, 'number'],
[/[a-zA-Z0-9][a-zA-Z0-9_\-\.]*/, 'keyword'],
[/[,=:]/, 'delimiter'],
// strings double quoted
[/"([^"\\]|\\.)*$/, 'string.invalid'], // string without termination
[/"/, 'string', '@string_dq'],
// strings single quoted
[/'([^'\\]|\\.)*$/, 'string.invalid'], // string without termination
[/'/, 'string', '@string_sq'],
[/\+|\-|\*|\//, 'keyword.operator'],
[/[\(]/, 'delimiter'],
[/[\)]/, 'delimiter'],
],
string_dq: [
[/[^\\"]+/, 'string'],
[/@escapes/, 'string.escape'],
[/\\./, 'string.escape.invalid'],
[/"/, 'string', '@pop'],
],
string_sq: [
[/[^\\']+/, 'string'],
[/@escapes/, 'string.escape'],
[/\\./, 'string.escape.invalid'],
[/'/, 'string', '@pop'],
],
},
} as monaco.languages.IMonarchLanguage;
monaco.languages.setMonarchTokensProvider(LANGUAGE_ID, lexerRules);
monaco.languages.setLanguageConfiguration(LANGUAGE_ID, languageConfiguration);

192
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/formula.test.tsx
View file

@ -14,8 +14,10 @@ import { tinymathFunctions } from './util';
jest.mock('../../layer_helpers', () => {
return {
getColumnOrder: ({ columns }: { columns: Record<string, IndexPatternColumn> }) =>
Object.keys(columns),
getColumnOrder: jest.fn(({ columns }: { columns: Record<string, IndexPatternColumn> }) =>
Object.keys(columns)
),
getManagedColumnsFrom: jest.fn().mockReturnValue([]),
};
});
@ -142,7 +144,7 @@ describe('formula', () => {
indexPattern,
})
).toEqual({
label: 'Formula',
label: 'average(bytes)',
dataType: 'number',
operationType: 'formula',
isBucketed: false,
@ -170,7 +172,7 @@ describe('formula', () => {
indexPattern,
})
).toEqual({
label: 'Formula',
label: 'average(bytes)',
dataType: 'number',
operationType: 'formula',
isBucketed: false,
@ -204,7 +206,7 @@ describe('formula', () => {
indexPattern,
})
).toEqual({
label: 'Formula',
label: `average(bytes, kql='category.keyword: "Men\\'s Clothing" or category.keyword: "Men\\'s Shoes"')`,
dataType: 'number',
operationType: 'formula',
isBucketed: false,
@ -233,7 +235,7 @@ describe('formula', () => {
indexPattern,
})
).toEqual({
label: 'Formula',
label: `count(lucene='*')`,
dataType: 'number',
operationType: 'formula',
isBucketed: false,
@ -291,7 +293,7 @@ describe('formula', () => {
operationDefinitionMap
)
).toEqual({
label: 'Formula',
label: 'moving_average(average(bytes), window=3)',
dataType: 'number',
operationType: 'formula',
isBucketed: false,
@ -375,6 +377,7 @@ describe('formula', () => {
...layer.columns,
col1: {
...currentColumn,
label: formula,
params: {
...currentColumn.params,
formula,
@ -415,6 +418,7 @@ describe('formula', () => {
...layer.columns,
col1: {
...currentColumn,
label: 'average(bytes)',
references: ['col1X1'],
params: {
...currentColumn.params,
@ -565,7 +569,7 @@ describe('formula', () => {
).toEqual({
col1X0: { min: 15, max: 29 },
col1X2: { min: 0, max: 41 },
col1X3: { min: 43, max: 50 },
col1X3: { min: 42, max: 50 },
});
});
});
@ -787,6 +791,34 @@ invalid: "
}
});
it('returns an error if formula or math operations are used', () => {
const formulaFormulas = ['formula()', 'formula(bytes)', 'formula(formula())'];
for (const formula of formulaFormulas) {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(formula),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual(['Operation formula not found']);
}
const mathFormulas = ['math()', 'math(bytes)', 'math(math())'];
for (const formula of mathFormulas) {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(formula),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual(['Operation math not found']);
}
});
it('returns an error if field operation in formula have the wrong first argument', () => {
const formulas = [
'average(7)',
@ -897,6 +929,150 @@ invalid: "
).toEqual(undefined);
});
it('returns no error for a query edge case', () => {
const formulas = [
`count(kql='')`,
`count(lucene='')`,
`moving_average(count(kql=''), window=7)`,
`count(kql='bytes >= 4000')`,
`count(kql='bytes <= 4000')`,
`count(kql='bytes = 4000')`,
];
for (const formula of formulas) {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(formula),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual(undefined);
}
});
it('returns an error for a query not wrapped in single quotes', () => {
const formulas = [
`count(kql="")`,
`count(kql='")`,
`count(kql="')`,
`count(kql="category.keyword: *")`,
`count(kql='category.keyword: *")`,
`count(kql="category.keyword: *')`,
`count(kql='category.keyword: *)`,
`count(kql=category.keyword: *')`,
`count(kql=category.keyword: *)`,
`count(kql="category.keyword: "Men's Clothing" or category.keyword: "Men's Shoes"")`,
`count(lucene="category.keyword: *")`,
`count(lucene=category.keyword: *)`,
`count(lucene=category.keyword: *) + average(bytes)`,
`count(lucene='category.keyword: *') + count(kql=category.keyword: *)`,
`count(lucene='category.keyword: *") + count(kql='category.keyword: *")`,
`count(lucene='category.keyword: *') + count(kql=category.keyword: *, kql='category.keyword: *')`,
`count(lucene='category.keyword: *') + count(kql="category.keyword: *")`,
`moving_average(count(kql=category.keyword: *), window=7, kql=category.keywork: *)`,
`moving_average(
cumulative_sum(
7 * clamp(sum(bytes), 0, last_value(memory) + max(memory))
), window=10, kql=category.keywork: *
)`,
];
for (const formula of formulas) {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(formula),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual(expect.arrayContaining([expect.stringMatching(`Single quotes are required`)]));
}
});
it('it returns parse fail error rather than query message if the formula is only a query condition (false positive cases for query checks)', () => {
const formulas = [
`kql="category.keyword: *"`,
`kql=category.keyword: *`,
`kql='category.keyword: *'`,
`(kql="category.keyword: *")`,
`(kql=category.keyword: *)`,
`(lucene="category.keyword: *")`,
`(lucene=category.keyword: *)`,
`(lucene='category.keyword: *') + (kql=category.keyword: *)`,
`(lucene='category.keyword: *') + (kql=category.keyword: *, kql='category.keyword: *')`,
`(lucene='category.keyword: *') + (kql="category.keyword: *")`,
`((kql=category.keyword: *), window=7, kql=category.keywork: *)`,
`(, window=10, kql=category.keywork: *)`,
`(
, window=10, kql=category.keywork: *
)`,
];
for (const formula of formulas) {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(formula),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual([`The Formula ${formula} cannot be parsed`]);
}
});
it('returns no error for a query wrapped in single quotes but with some whitespaces', () => {
const formulas = [
`count(kql ='category.keyword: *')`,
`count(kql = 'category.keyword: *')`,
`count(kql = 'category.keyword: *')`,
];
for (const formula of formulas) {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(formula),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual(undefined);
}
});
it('returns an error for multiple queries submitted for the same function', () => {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(`count(kql='category.keyword: *', lucene='category.keyword: *')`),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual(['Use only one of kql= or lucene=, not both']);
});
it("returns a clear error when there's a missing field for a function", () => {
for (const fn of ['average', 'terms', 'max', 'sum']) {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(`${fn}()`),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual([`The first argument for ${fn} should be a field name. Found no field`]);
}
});
it("returns a clear error when there's a missing function for a fullReference operation", () => {
for (const fn of ['cumulative_sum', 'derivative']) {
expect(
formulaOperation.getErrorMessage!(
getNewLayerWithFormula(`${fn}()`),
'col1',
indexPattern,
operationDefinitionMap
)
).toEqual([`The first argument for ${fn} should be a operation name. Found no operation`]);
}
});
it('returns no error if a math operation is passed to fullReference operations', () => {
const formulas = [
'derivative(7+1)',

64
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/formula.tsx
View file

@ -10,8 +10,11 @@ import { OperationDefinition } from '../index';
import { ReferenceBasedIndexPatternColumn } from '../column_types';
import { IndexPattern } from '../../../types';
import { runASTValidation, tryToParse } from './validation';
import { MemoizedFormulaEditor } from './editor';
import { regenerateLayerFromAst } from './parse';
import { generateFormula } from './generate';
import { filterByVisibleOperation } from './util';
import { getManagedColumnsFrom } from '../../layer_helpers';
const defaultLabel = i18n.translate('xpack.lens.indexPattern.formulaLabel', {
defaultMessage: 'Formula',
@ -38,7 +41,7 @@ export const formulaOperation: OperationDefinition<
> = {
type: 'formula',
displayName: defaultLabel,
getDefaultLabel: (column, indexPattern) => defaultLabel,
getDefaultLabel: (column, indexPattern) => column.params.formula ?? defaultLabel,
input: 'managedReference',
hidden: true,
getDisabledStatus(indexPattern: IndexPattern) {
@ -49,13 +52,32 @@ export const formulaOperation: OperationDefinition<
if (!column.params.formula || !operationDefinitionMap) {
return;
}
const { root, error } = tryToParse(column.params.formula);
const visibleOperationsMap = filterByVisibleOperation(operationDefinitionMap);
const { root, error } = tryToParse(column.params.formula, visibleOperationsMap);
if (error || !root) {
return [error!.message];
}
const errors = runASTValidation(root, layer, indexPattern, operationDefinitionMap);
return errors.length ? errors.map(({ message }) => message) : undefined;
const errors = runASTValidation(root, layer, indexPattern, visibleOperationsMap);
if (errors.length) {
return errors.map(({ message }) => message);
}
const managedColumns = getManagedColumnsFrom(columnId, layer.columns);
const innerErrors = managedColumns
.flatMap(([id, col]) => {
const def = visibleOperationsMap[col.operationType];
if (def?.getErrorMessage) {
const messages = def.getErrorMessage(layer, id, indexPattern, visibleOperationsMap);
return messages ? { message: messages.join(', ') } : [];
}
return [];
})
.filter((marker) => marker);
return innerErrors.length ? innerErrors.map(({ message }) => message) : undefined;
},
getPossibleOperation() {
return {
@ -72,8 +94,8 @@ export const formulaOperation: OperationDefinition<
const label = !params?.isFormulaBroken
? useDisplayLabel
? currentColumn.label
: params?.formula
: '';
: params?.formula ?? defaultLabel
: defaultLabel;
return [
{
@ -81,21 +103,23 @@ export const formulaOperation: OperationDefinition<
function: 'mapColumn',
arguments: {
id: [columnId],
name: [label || ''],
name: [label || defaultLabel],
exp: [
{
type: 'expression',
chain: [
{
type: 'function',
function: 'math',
arguments: {
expression: [
currentColumn.references.length ? `"${currentColumn.references[0]}"` : ``,
],
},
},
],
chain: currentColumn.references.length
? [
{
type: 'function',
function: 'math',
arguments: {
expression: [
currentColumn.references.length ? `"${currentColumn.references[0]}"` : ``,
],
},
},
]
: [],
},
],
},
@ -119,7 +143,7 @@ export const formulaOperation: OperationDefinition<
prevFormat = { format: previousColumn.params.format };
}
return {
label: 'Formula',
label: previousFormula || defaultLabel,
dataType: 'number',
operationType: 'formula',
isBucketed: false,
@ -152,4 +176,6 @@ export const formulaOperation: OperationDefinition<
);
return newLayer;
},
paramEditor: MemoizedFormulaEditor,
};

28
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/math_examples.md Normal file
View file

@ -0,0 +1,28 @@
Basic numeric functions that we already support in Lens:
count()
count(normalize_unit='1s')
sum(field name)
avg(field name)
moving_average(sum(field name), window=5)
moving_average(sum(field name), window=5, normalize_unit='1s')
counter_rate(field name, normalize_unit='1s')
differences(count())
differences(sum(bytes), normalize_unit='1s')
last_value(bytes, sort=timestamp)
percentile(bytes, percent=95)
Adding features beyond what we already support. New features are:
* Filtering
* Math across series
* Time offset
count() * 100
(count() / count(offset=-7d)) + min(field name)
sum(field name, filter='field.keyword: "KQL autocomplete inside math" AND field.value > 100')
What about custom formatting using string manipulation? Probably not...
(avg(bytes) / 1000) + 'kb'

24
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/parse.ts
View file

@ -5,6 +5,7 @@
* 2.0.
*/
import { i18n } from '@kbn/i18n';
import { isObject } from 'lodash';
import type { TinymathAST, TinymathVariable, TinymathLocation } from '@kbn/tinymath';
import { OperationDefinition, GenericOperationDefinition, IndexPatternColumn } from '../index';
@ -12,7 +13,12 @@ import { IndexPattern, IndexPatternLayer } from '../../../types';
import { mathOperation } from './math';
import { documentField } from '../../../document_field';
import { runASTValidation, shouldHaveFieldArgument, tryToParse } from './validation';
import { findVariables, getOperationParams, groupArgsByType } from './util';
import {
filterByVisibleOperation,
findVariables,
getOperationParams,
groupArgsByType,
} from './util';
import { FormulaIndexPatternColumn } from './formula';
import { getColumnOrder } from '../../layer_helpers';
@ -27,7 +33,7 @@ function parseAndExtract(
indexPattern: IndexPattern,
operationDefinitionMap: Record<string, GenericOperationDefinition>
) {
const { root, error } = tryToParse(text);
const { root, error } = tryToParse(text, operationDefinitionMap);
if (error || !root) {
return { extracted: [], isValid: false };
}
@ -61,9 +67,9 @@ function extractColumns(
const nodeOperation = operations[node.name];
if (!nodeOperation) {
// it's a regular math node
const consumedArgs = node.args.map(parseNode).filter(Boolean) as Array<
number | TinymathVariable
>;
const consumedArgs = node.args
.map(parseNode)
.filter((n) => typeof n !== 'undefined' && n !== null) as Array<number | TinymathVariable>;
return {
...node,
args: consumedArgs,
@ -168,7 +174,7 @@ export function regenerateLayerFromAst(
layer,
columnId,
indexPattern,
operationDefinitionMap
filterByVisibleOperation(operationDefinitionMap)
);
const columns = { ...layer.columns };
@ -188,6 +194,12 @@ export function regenerateLayerFromAst(
columns[columnId] = {
...currentColumn,
label: !currentColumn.customLabel
? text ??
i18n.translate('xpack.lens.indexPattern.formulaLabel', {
defaultMessage: 'Formula',
})
: currentColumn.label,
params: {
...currentColumn.params,
formula: text,

273
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/util.ts
View file

@ -13,7 +13,7 @@ import type {
TinymathNamedArgument,
TinymathVariable,
} from 'packages/kbn-tinymath';
import type { OperationDefinition, IndexPatternColumn } from '../index';
import type { OperationDefinition, IndexPatternColumn, GenericOperationDefinition } from '../index';
import type { GroupedNodes } from './types';
export function groupArgsByType(args: TinymathAST[]) {
@ -66,6 +66,16 @@ export function getOperationParams(
}, {});
}
function getTypeI18n(type: string) {
if (type === 'number') {
return i18n.translate('xpack.lens.formula.number', { defaultMessage: 'number' });
}
if (type === 'string') {
return i18n.translate('xpack.lens.formula.string', { defaultMessage: 'string' });
}
return '';
}
// Todo: i18n everything here
export const tinymathFunctions: Record<
string,
@ -73,145 +83,254 @@ export const tinymathFunctions: Record<
positionalArguments: Array<{
name: string;
optional?: boolean;
defaultValue?: string | number;
type?: string;
}>;
// help: React.ReactElement;
// Help is in Markdown format
help: string;
}
> = {
add: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.left', { defaultMessage: 'left' }) },
{ name: i18n.translate('xpack.lens.formula.right', { defaultMessage: 'right' }) },
{
name: i18n.translate('xpack.lens.formula.left', { defaultMessage: 'left' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.right', { defaultMessage: 'right' }),
type: getTypeI18n('number'),
},
],
help: `
Adds up two numbers.
Also works with + symbol
Example: ${'`count() + sum(bytes)`'}
Example: ${'`add(count(), 5)`'}
Example: Calculate the sum of two fields
${'`sum(price) + sum(tax)`'}
Example: Offset count by a static value
${'`add(count(), 5)`'}
`,
},
subtract: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.left', { defaultMessage: 'left' }) },
{ name: i18n.translate('xpack.lens.formula.right', { defaultMessage: 'right' }) },
{
name: i18n.translate('xpack.lens.formula.left', { defaultMessage: 'left' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.right', { defaultMessage: 'right' }),
type: getTypeI18n('number'),
},
],
help: `
Subtracts the first number from the second number.
Also works with ${'`-`'} symbol
Example: ${'`subtract(sum(bytes), avg(bytes))`'}
Example: Calculate the range of a field
${'`subtract(max(bytes), min(bytes))`'}
`,
},
multiply: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.left', { defaultMessage: 'left' }) },
{ name: i18n.translate('xpack.lens.formula.right', { defaultMessage: 'right' }) },
{
name: i18n.translate('xpack.lens.formula.left', { defaultMessage: 'left' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.right', { defaultMessage: 'right' }),
type: getTypeI18n('number'),
},
],
help: `
Also works with ${'`*`'} symbol
Example: ${'`multiply(sum(bytes), 2)`'}
Multiplies two numbers.
Also works with ${'`*`'} symbol.
Example: Calculate price after current tax rate
${'`sum(bytes) * last_value(tax_rate)`'}
Example: Calculate price after constant tax rate
${'`multiply(sum(price), 1.2)`'}
`,
},
divide: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.left', { defaultMessage: 'left' }) },
{ name: i18n.translate('xpack.lens.formula.right', { defaultMessage: 'right' }) },
{
name: i18n.translate('xpack.lens.formula.left', { defaultMessage: 'left' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.right', { defaultMessage: 'right' }),
type: getTypeI18n('number'),
},
],
help: `
Divides the first number by the second number.
Also works with ${'`/`'} symbol
Example: ${'`ceil(sum(bytes))`'}
Example: Calculate profit margin
${'`sum(profit) / sum(revenue)`'}
Example: ${'`divide(sum(bytes), 2)`'}
`,
},
abs: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
help: `
Absolute value
Example: ${'`abs(sum(bytes))`'}
Calculates absolute value. A negative value is multiplied by -1, a positive value stays the same.
Example: Calculate average distance to sea level ${'`abs(average(altitude))`'}
`,
},
cbrt: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
help: `
Cube root of value
Example: ${'`cbrt(sum(bytes))`'}
Cube root of value.
Example: Calculate side length from volume
${'`cbrt(last_value(volume))`'}
`,
},
ceil: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
// signature: 'ceil(value: number)',
help: `
Ceiling of value, rounds up
Example: ${'`ceil(sum(bytes))`'}
Ceiling of value, rounds up.
Example: Round up price to the next dollar
${'`ceil(sum(price))`'}
`,
},
clamp: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{ name: i18n.translate('xpack.lens.formula.min', { defaultMessage: 'min' }) },
{ name: i18n.translate('xpack.lens.formula.max', { defaultMessage: 'max' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.min', { defaultMessage: 'min' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.max', { defaultMessage: 'max' }),
type: getTypeI18n('number'),
},
],
// signature: 'clamp(value: number, minimum: number, maximum: number)',
help: `
Limits the value from a minimum to maximum
Example: ${'`ceil(sum(bytes))`'}
`,
Limits the value from a minimum to maximum.
Example: Make sure to catch outliers
\`\`\`
clamp(
average(bytes),
percentile(bytes, percentile=5),
percentile(bytes, percentile=95)
)
\`\`\`
`,
},
cube: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
help: `
Limits the value from a minimum to maximum
Example: ${'`ceil(sum(bytes))`'}
Calculates the cube of a number.
Example: Calculate volume from side length
${'`cube(last_value(length))`'}
`,
},
exp: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
help: `
Raises <em>e</em> to the nth power.
Example: ${'`exp(sum(bytes))`'}
Raises *e* to the nth power.
Example: Calculate the natural exponential function
${'`exp(last_value(duration))`'}
`,
},
fix: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
help: `
For positive values, takes the floor. For negative values, takes the ceiling.
Example: ${'`fix(sum(bytes))`'}
Example: Rounding towards zero
${'`fix(sum(profit))`'}
`,
},
floor: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
help: `
Round down to nearest integer value
Example: ${'`floor(sum(bytes))`'}
Example: Round down a price
${'`floor(sum(price))`'}
`,
},
log: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.base', { defaultMessage: 'base' }),
optional: true,
defaultValue: 'e',
type: getTypeI18n('number'),
},
],
help: `
Logarithm with optional base. The natural base <em>e</em> is used as default.
Example: ${'`log(sum(bytes))`'}
Example: ${'`log(sum(bytes), 2)`'}
Logarithm with optional base. The natural base *e* is used as default.
Example: Calculate number of bits required to store values
\`\`\`
log(sum(bytes))
log(sum(bytes), 2)
\`\`\`
`,
},
// TODO: check if this is valid for Tinymath
// log10: {
// positionalArguments: [
// { name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
// { name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }), type: getTypeI18n('number') },
// ],
// help: `
// Base 10 logarithm.
@ -220,59 +339,89 @@ Example: ${'`log(sum(bytes), 2)`'}
// },
mod: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.base', { defaultMessage: 'base' }),
optional: true,
type: getTypeI18n('number'),
},
],
help: `
Remainder after dividing the function by a number
Example: ${'`mod(sum(bytes), 2)`'}
Example: Calculate last three digits of a value
${'`mod(sum(price), 1000)`'}
`,
},
pow: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.base', { defaultMessage: 'base' }),
type: getTypeI18n('number'),
},
],
help: `
Raises the value to a certain power. The second argument is required
Example: ${'`pow(sum(bytes), 3)`'}
Example: Calculate volume based on side length
${'`pow(last_value(length), 3)`'}
`,
},
round: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
{
name: i18n.translate('xpack.lens.formula.decimals', { defaultMessage: 'decimals' }),
optional: true,
defaultValue: 0,
type: getTypeI18n('number'),
},
],
help: `
Rounds to a specific number of decimal places, default of 0
Example: ${'`round(sum(bytes))`'}
Example: ${'`round(sum(bytes), 2)`'}
Examples: Round to the cent
\`\`\`
round(sum(bytes))
round(sum(bytes), 2)
\`\`\`
`,
},
sqrt: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
help: `
Square root of a positive value only
Example: ${'`sqrt(sum(bytes))`'}
Example: Calculate side length based on area
${'`sqrt(last_value(area))`'}
`,
},
square: {
positionalArguments: [
{ name: i18n.translate('xpack.lens.formula.expression', { defaultMessage: 'expression' }) },
{
name: i18n.translate('xpack.lens.formula.value', { defaultMessage: 'value' }),
type: getTypeI18n('number'),
},
],
help: `
Raise the value to the 2nd power
Example: ${'`square(sum(bytes))`'}
Example: Calculate area based on side length
${'`square(last_value(length))`'}
`,
},
};
@ -315,3 +464,11 @@ export function findVariables(node: TinymathAST | string): TinymathVariable[] {
}
return node.args.flatMap(findVariables);
}
export function filterByVisibleOperation(
operationDefinitionMap: Record<string, GenericOperationDefinition>
) {
return Object.fromEntries(
Object.entries(operationDefinitionMap).filter(([, operation]) => !operation.hidden)
);
}

157
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/formula/validation.ts
View file

@ -5,7 +5,7 @@
* 2.0.
*/
import { isObject } from 'lodash';
import { isObject, partition } from 'lodash';
import { i18n } from '@kbn/i18n';
import { parse, TinymathLocation } from '@kbn/tinymath';
import type { TinymathAST, TinymathFunction, TinymathNamedArgument } from '@kbn/tinymath';
@ -58,6 +58,10 @@ interface ValidationErrors {
message: string;
type: { operation: string; count: number; params: string };
};
tooManyQueries: {
message: string;
type: {};
};
}
type ErrorTypes = keyof ValidationErrors;
type ErrorValues<K extends ErrorTypes> = ValidationErrors[K]['type'];
@ -90,15 +94,76 @@ export function hasInvalidOperations(
return {
// avoid duplicates
names: Array.from(new Set(nodes.map(({ name }) => name))),
locations: nodes.map(({ location }) => location),
locations: nodes.map(({ location }) => location).filter((a) => a) as TinymathLocation[],
};
}
export const getRawQueryValidationError = (text: string, operations: Record<string, unknown>) => {
// try to extract the query context here
const singleLine = text.split('\n').join('');
const allArgs = singleLine.split(',').filter((args) => /(kql|lucene)/.test(args));
// check for the presence of a valid ES operation
const containsOneValidOperation = Object.keys(operations).some((operation) =>
singleLine.includes(operation)
);
// no args or no valid operation, no more work to do here
if (allArgs.length === 0 || !containsOneValidOperation) {
return;
}
// at this point each entry in allArgs may contain one or more
// in the worst case it would be a math chain of count operation
// For instance: count(kql=...) + count(lucene=...) - count(kql=...)
// therefore before partition them, split them by "count" keywork and filter only string with a length
const flattenArgs = allArgs.flatMap((arg) =>
arg.split('count').filter((subArg) => /(kql|lucene)/.test(subArg))
);
const [kqlQueries, luceneQueries] = partition(flattenArgs, (arg) => /kql/.test(arg));
const errors = [];
for (const kqlQuery of kqlQueries) {
const result = validateQueryQuotes(kqlQuery, 'kql');
if (result) {
errors.push(result);
}
}
for (const luceneQuery of luceneQueries) {
const result = validateQueryQuotes(luceneQuery, 'lucene');
if (result) {
errors.push(result);
}
}
return errors.length ? errors : undefined;
};
const validateQueryQuotes = (rawQuery: string, language: 'kql' | 'lucene') => {
// check if the raw argument has the minimal requirements
// use the rest operator here to handle cases where comparison operations are used in the query
const [, ...rawValue] = rawQuery.split('=');
const fullRawValue = (rawValue || ['']).join('');
const cleanedRawValue = fullRawValue.trim();
// it must start with a single quote, and quotes must have a closure
if (
cleanedRawValue.length &&
(cleanedRawValue[0] !== "'" || !/'\s*([^']+?)\s*'/.test(fullRawValue)) &&
// there's a special case when it's valid as two single quote strings
cleanedRawValue !== "''"
) {
return i18n.translate('xpack.lens.indexPattern.formulaOperationQueryError', {
defaultMessage: `Single quotes are required for {language}='' at {rawQuery}`,
values: { language, rawQuery },
});
}
};
export const getQueryValidationError = (
query: string,
language: 'kql' | 'lucene',
{ value: query, name: language, text }: TinymathNamedArgument,
indexPattern: IndexPattern
): string | undefined => {
// check if the raw argument has the minimal requirements
const result = validateQueryQuotes(text, language as 'kql' | 'lucene');
// forward the error here is ok?
if (result) {
return result;
}
try {
if (language === 'kql') {
esKuery.toElasticsearchQuery(esKuery.fromKueryExpression(query), indexPattern);
@ -113,7 +178,7 @@ export const getQueryValidationError = (
function getMessageFromId<K extends ErrorTypes>({
messageId,
values: { ...values },
values,
locations,
}: {
messageId: K;
@ -203,6 +268,11 @@ function getMessageFromId<K extends ErrorTypes>({
values: { operation: out.operation, count: out.count, params: out.params },
});
break;
case 'tooManyQueries':
message = i18n.translate('xpack.lens.indexPattern.formulaOperationDoubleQueryError', {
defaultMessage: 'Use only one of kql= or lucene=, not both',
});
break;
// case 'mathRequiresFunction':
// message = i18n.translate('xpack.lens.indexPattern.formulaMathRequiresFunctionLabel', {
// defaultMessage; 'The function {name} requires an Elasticsearch function',
@ -218,12 +288,22 @@ function getMessageFromId<K extends ErrorTypes>({
}
export function tryToParse(
formula: string
formula: string,
operations: Record<string, unknown>
): { root: TinymathAST; error: null } | { root: null; error: ErrorWrapper } {
let root;
try {
root = parse(formula);
} catch (e) {
// A tradeoff is required here, unless we want to reimplement a full parser
// Internally the function has the following logic:
// * if the formula contains no existing ES operation, assume it's a plain parse failure
// * if the formula contains at least one existing operation, check for query problems
const maybeQueryProblems = getRawQueryValidationError(formula, operations);
if (maybeQueryProblems) {
// need to emulate an error shape here
return { root: null, error: { message: maybeQueryProblems[0], locations: [] } };
}
return {
root: null,
error: getMessageFromId({
@ -319,7 +399,10 @@ function getQueryValidationErrors(
const errors: ErrorWrapper[] = [];
(namedArguments ?? []).forEach((arg) => {
if (arg.name === 'kql' || arg.name === 'lucene') {
const message = getQueryValidationError(arg.value, arg.name, indexPattern);
const message = getQueryValidationError(
arg as TinymathNamedArgument & { name: 'kql' | 'lucene' },
indexPattern
);
if (message) {
errors.push({
message,
@ -331,6 +414,12 @@ function getQueryValidationErrors(
return errors;
}
function checkSingleQuery(namedArguments: TinymathNamedArgument[] | undefined) {
return namedArguments
? namedArguments.filter((arg) => arg.name === 'kql' || arg.name === 'lucene').length > 1
: undefined;
}
function validateNameArguments(
node: TinymathFunction,
nodeOperation:
@ -349,7 +438,7 @@ function validateNameArguments(
operation: node.name,
params: missingParams.map(({ name }) => name).join(', '),
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -362,7 +451,7 @@ function validateNameArguments(
operation: node.name,
params: wrongTypeParams.map(({ name }) => name).join(', '),
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -375,7 +464,7 @@ function validateNameArguments(
operation: node.name,
params: duplicateParams.join(', '),
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -383,6 +472,16 @@ function validateNameArguments(
if (queryValidationErrors.length) {
errors.push(...queryValidationErrors);
}
const hasTooManyQueries = checkSingleQuery(namedArguments);
if (hasTooManyQueries) {
errors.push(
getMessageFromId({
messageId: 'tooManyQueries',
values: {},
locations: node.location ? [node.location] : [],
})
);
}
return errors;
}
@ -426,7 +525,7 @@ function runFullASTValidation(
type: 'field',
argument: `math operation`,
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
} else {
@ -436,9 +535,13 @@ function runFullASTValidation(
values: {
operation: node.name,
type: 'field',
argument: getValueOrName(firstArg),
argument:
getValueOrName(firstArg) ||
i18n.translate('xpack.lens.indexPattern.formulaNoFieldForOperation', {
defaultMessage: 'no field',
}),
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -452,7 +555,7 @@ function runFullASTValidation(
values: {
operation: node.name,
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -464,7 +567,7 @@ function runFullASTValidation(
values: {
operation: node.name,
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
} else {
@ -493,9 +596,13 @@ function runFullASTValidation(
values: {
operation: node.name,
type: 'operation',
argument: getValueOrName(firstArg),
argument:
getValueOrName(firstArg) ||
i18n.translate('xpack.lens.indexPattern.formulaNoOperation', {
defaultMessage: 'no operation',
}),
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -506,7 +613,7 @@ function runFullASTValidation(
values: {
operation: node.name,
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
} else {
@ -606,7 +713,11 @@ export function validateParams(
}
export function shouldHaveFieldArgument(node: TinymathFunction) {
return !['count'].includes(node.name);
return hasFunctionFieldArgument(node.name);
}
export function hasFunctionFieldArgument(type: string) {
return !['count'].includes(type);
}
export function isFirstArgumentValidType(arg: TinymathAST, type: TinymathNodeTypes['type']) {
@ -628,7 +739,7 @@ export function validateMathNodes(root: TinymathAST, missingVariableSet: Set<str
type: 'operation',
argument: `()`,
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -640,7 +751,7 @@ export function validateMathNodes(root: TinymathAST, missingVariableSet: Set<str
values: {
operation: node.name,
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -659,7 +770,7 @@ export function validateMathNodes(root: TinymathAST, missingVariableSet: Set<str
values: {
operation: node.name,
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}
@ -678,7 +789,7 @@ export function validateMathNodes(root: TinymathAST, missingVariableSet: Set<str
count: mandatoryArguments.length - node.args.length,
params: missingArgs.map(({ name }) => name).join(', '),
},
locations: [node.location],
locations: node.location ? [node.location] : [],
})
);
}

9
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/index.ts
View file

@ -153,6 +153,9 @@ export interface ParamEditorProps<C> {
updateLayer: (
setter: IndexPatternLayer | ((prevLayer: IndexPatternLayer) => IndexPatternLayer)
) => void;
toggleFullscreen: () => void;
setIsCloseable: (isCloseable: boolean) => void;
isFullscreen: boolean;
columnId: string;
indexPattern: IndexPattern;
uiSettings: IUiSettingsClient;
@ -279,6 +282,11 @@ interface BaseOperationDefinitionProps<C extends BaseIndexPatternColumn> {
* Operations can be used as middleware for other operations, hence not shown in the panel UI
*/
hidden?: boolean;
documentation?: {
signature: string;
description: string;
section: 'elasticsearch' | 'calculation';
};
}
interface BaseBuildColumnArgs {
@ -290,6 +298,7 @@ interface OperationParam {
name: string;
type: string;
required?: boolean;
defaultValue?: string | number;
}
interface FieldlessOperationDefinition<C extends BaseIndexPatternColumn> {

3
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/last_value.test.tsx
View file

@ -30,6 +30,9 @@ const defaultProps = {
hasRestrictions: false,
} as IndexPattern,
operationDefinitionMap: {},
isFullscreen: false,
toggleFullscreen: jest.fn(),
setIsCloseable: jest.fn(),
};
describe('last_value', () => {

16
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/last_value.tsx
View file

@ -277,4 +277,20 @@ export const lastValueOperation: OperationDefinition<LastValueIndexPatternColumn
</>
);
},
documentation: {
section: 'elasticsearch',
signature: i18n.translate('xpack.lens.indexPattern.lastValue.signature', {
defaultMessage: 'field: string',
}),
description: i18n.translate('xpack.lens.indexPattern.lastValue.documentation', {
defaultMessage: `
Returns the value of a field from the last document, ordered by the default time field of the index pattern.
This function is usefull the retrieve the latest state of an entity.
Example: Get the current status of server A:
\`last_value(server.status, kql=\'server.name="A"\')\`
`,
}),
},
};

43
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/metrics.tsx
View file

@ -42,6 +42,7 @@ const supportedTypes = ['number', 'histogram'];
function buildMetricOperation<T extends MetricColumn<string>>({
type,
displayName,
description,
ofName,
priority,
optionalTimeScaling,
@ -51,6 +52,7 @@ function buildMetricOperation<T extends MetricColumn<string>>({
ofName: (name: string) => string;
priority?: number;
optionalTimeScaling?: boolean;
description?: string;
}) {
const labelLookup = (name: string, column?: BaseIndexPatternColumn) => {
const label = ofName(name);
@ -67,6 +69,7 @@ function buildMetricOperation<T extends MetricColumn<string>>({
type,
priority,
displayName,
description,
input: 'field',
timeScalingMode: optionalTimeScaling ? 'optional' : undefined,
getPossibleOperationForField: ({ aggregationRestrictions, aggregatable, type: fieldType }) => {
@ -131,6 +134,26 @@ function buildMetricOperation<T extends MetricColumn<string>>({
getErrorMessage: (layer, columnId, indexPattern) =>
getInvalidFieldMessage(layer.columns[columnId] as FieldBasedIndexPatternColumn, indexPattern),
filterable: true,
documentation: {
section: 'elasticsearch',
signature: i18n.translate('xpack.lens.indexPattern.metric.signature', {
defaultMessage: 'field: string',
}),
description: i18n.translate('xpack.lens.indexPattern.metric.documentation', {
defaultMessage: `
Returns the {metric} of a field. This function only works for number fields.
Example: Get the {metric} of price:
\`{metric}(price)\`
Example: Get the {metric} of price for orders from the UK:
\`{metric}(price, kql='location:UK')\`
`,
values: {
metric: type,
},
}),
},
shiftable: true,
} as OperationDefinition<T, 'field'>;
}
@ -151,6 +174,10 @@ export const minOperation = buildMetricOperation<MinIndexPatternColumn>({
defaultMessage: 'Minimum of {name}',
values: { name },
}),
description: i18n.translate('xpack.lens.indexPattern.min.description', {
defaultMessage:
'A single-value metrics aggregation that returns the minimum value among the numeric values extracted from the aggregated documents.',
}),
});
export const maxOperation = buildMetricOperation<MaxIndexPatternColumn>({
@ -163,6 +190,10 @@ export const maxOperation = buildMetricOperation<MaxIndexPatternColumn>({
defaultMessage: 'Maximum of {name}',
values: { name },
}),
description: i18n.translate('xpack.lens.indexPattern.max.description', {
defaultMessage:
'A single-value metrics aggregation that returns the maximum value among the numeric values extracted from the aggregated documents.',
}),
});
export const averageOperation = buildMetricOperation<AvgIndexPatternColumn>({
@ -176,6 +207,10 @@ export const averageOperation = buildMetricOperation<AvgIndexPatternColumn>({
defaultMessage: 'Average of {name}',
values: { name },
}),
description: i18n.translate('xpack.lens.indexPattern.avg.description', {
defaultMessage:
'A single-value metric aggregation that computes the average of numeric values that are extracted from the aggregated documents',
}),
});
export const sumOperation = buildMetricOperation<SumIndexPatternColumn>({
@ -190,6 +225,10 @@ export const sumOperation = buildMetricOperation<SumIndexPatternColumn>({
values: { name },
}),
optionalTimeScaling: true,
description: i18n.translate('xpack.lens.indexPattern.sum.description', {
defaultMessage:
'A single-value metrics aggregation that sums up numeric values that are extracted from the aggregated documents.',
}),
});
export const medianOperation = buildMetricOperation<MedianIndexPatternColumn>({
@ -203,4 +242,8 @@ export const medianOperation = buildMetricOperation<MedianIndexPatternColumn>({
defaultMessage: 'Median of {name}',
values: { name },
}),
description: i18n.translate('xpack.lens.indexPattern.median.description', {
defaultMessage:
'A single-value metrics aggregation that computes the median value that are extracted from the aggregated documents.',
}),
});

3
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile.test.tsx
View file

@ -32,6 +32,9 @@ const defaultProps = {
hasRestrictions: false,
} as IndexPattern,
operationDefinitionMap: {},
isFullscreen: false,
toggleFullscreen: jest.fn(),
setIsCloseable: jest.fn(),
};
describe('percentile', () => {

18
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/percentile.tsx
View file

@ -59,7 +59,9 @@ export const percentileOperation: OperationDefinition<PercentileIndexPatternColu
defaultMessage: 'Percentile',
}),
input: 'field',
operationParams: [{ name: 'percentile', type: 'number', required: false }],
operationParams: [
{ name: 'percentile', type: 'number', required: false, defaultValue: DEFAULT_PERCENTILE_VALUE },
],
filterable: true,
shiftable: true,
getPossibleOperationForField: ({ aggregationRestrictions, aggregatable, type: fieldType }) => {
@ -213,4 +215,18 @@ export const percentileOperation: OperationDefinition<PercentileIndexPatternColu
</EuiFormRow>
);
},
documentation: {
section: 'elasticsearch',
signature: i18n.translate('xpack.lens.indexPattern.percentile.signature', {
defaultMessage: 'field: string, [percentile]: number',
}),
description: i18n.translate('xpack.lens.indexPattern.percentile.documentation', {
defaultMessage: `
Returns the specified percentile of the values of a field. This is the value n percent of the values occuring in documents are smaller.
Example: Get the number of bytes larger than 95 % of values:
\`percentile(bytes, percentile=95)\`
`,
}),
},
};

3
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/ranges/ranges.test.tsx
View file

@ -102,6 +102,9 @@ const defaultOptions = {
]),
},
operationDefinitionMap: {},
isFullscreen: false,
toggleFullscreen: jest.fn(),
setIsCloseable: jest.fn(),
};
describe('ranges', () => {

3
x-pack/plugins/lens/public/indexpattern_datasource/operations/definitions/terms/terms.test.tsx
View file

@ -35,6 +35,9 @@ const defaultProps = {
http: {} as HttpSetup,
indexPattern: createMockedIndexPattern(),
operationDefinitionMap: {},
isFullscreen: false,
toggleFullscreen: jest.fn(),
setIsCloseable: jest.fn(),
};
describe('terms', () => {

79
x-pack/plugins/lens/public/indexpattern_datasource/operations/layer_helpers.test.ts
View file

@ -24,7 +24,7 @@ import type { IndexPattern, IndexPatternLayer } from '../types';
import { documentField } from '../document_field';
import { getFieldByNameFactory } from '../pure_helpers';
import { generateId } from '../../id_generator';
import { createMockedFullReference } from './mocks';
import { createMockedFullReference, createMockedManagedReference } from './mocks';
jest.mock('../operations');
jest.mock('../../id_generator');
@ -91,10 +91,13 @@ describe('state_helpers', () => {
// @ts-expect-error we are inserting an invalid type
operationDefinitionMap.testReference = createMockedFullReference();
// @ts-expect-error we are inserting an invalid type
operationDefinitionMap.managedReference = createMockedManagedReference();
});
afterEach(() => {
delete operationDefinitionMap.testReference;
delete operationDefinitionMap.managedReference;
});
describe('copyColumn', () => {
@ -102,19 +105,19 @@ describe('state_helpers', () => {
const source = {
dataType: 'number' as const,
isBucketed: false,
label: 'Formula',
label: 'moving_average(sum(bytes), window=5)',
operationType: 'formula' as const,
params: {
formula: 'moving_average(sum(bytes), window=5)',
isFormulaBroken: false,
},
references: ['formulaX3'],
references: ['formulaX1'],
};
const math = {
customLabel: true,
dataType: 'number' as const,
isBucketed: false,
label: 'math',
label: 'formulaX2',
operationType: 'math' as const,
params: { tinymathAst: 'formulaX2' },
references: ['formulaX2'],
@ -135,7 +138,7 @@ describe('state_helpers', () => {
label: 'formulaX2',
operationType: 'moving_average' as const,
params: { window: 5 },
references: ['formulaX1'],
references: ['formulaX0'],
};
expect(
copyColumn({
@ -387,6 +390,42 @@ describe('state_helpers', () => {
).toEqual(expect.objectContaining({ columnOrder: ['col1', 'col2', 'col3'] }));
});
it('should not change order of metrics and references on inserting new buckets', () => {
const layer: IndexPatternLayer = {
indexPatternId: '1',
columnOrder: ['col1', 'col2'],
columns: {
col1: {
label: 'Cumulative sum of count of records',
dataType: 'number',
isBucketed: false,
// Private
operationType: 'cumulative_sum',
references: ['col2'],
},
col2: {
label: 'Count of records',
dataType: 'document',
isBucketed: false,
// Private
operationType: 'count',
sourceField: 'Records',
},
},
};
expect(
insertNewColumn({
layer,
indexPattern,
columnId: 'col3',
op: 'filters',
visualizationGroups: [],
})
).toEqual(expect.objectContaining({ columnOrder: ['col3', 'col1', 'col2'] }));
});
it('should insert both incomplete states if the aggregation does not support the field', () => {
expect(
insertNewColumn({
@ -2655,6 +2694,36 @@ describe('state_helpers', () => {
expect(errors).toHaveLength(1);
});
it('should only collect the top level errors from managed references', () => {
const notCalledMock = jest.fn();
const mock = jest.fn().mockReturnValue(['error 1']);
operationDefinitionMap.testReference.getErrorMessage = notCalledMock;
operationDefinitionMap.managedReference.getErrorMessage = mock;
const errors = getErrorMessages(
{
indexPatternId: '1',
columnOrder: [],
columns: {
col1:
// @ts-expect-error not statically analyzed
{ operationType: 'managedReference', references: ['col2'] },
col2: {
// @ts-expect-error not statically analyzed
operationType: 'testReference',
references: [],
},
},
},
indexPattern,
{},
'1',
{}
);
expect(notCalledMock).not.toHaveBeenCalled();
expect(mock).toHaveBeenCalledTimes(1);
expect(errors).toHaveLength(1);
});
it('should ignore incompleteColumns when checking for errors', () => {
const savedRef = jest.fn().mockReturnValue(['error 1']);
const incompleteRef = jest.fn();

57
x-pack/plugins/lens/public/indexpattern_datasource/operations/layer_helpers.ts
View file

@ -169,6 +169,10 @@ export function insertNewColumn({
if (field) {
throw new Error(`Can't create operation ${op} with the provided field ${field.name}`);
}
if (operationDefinition.input === 'managedReference') {
// TODO: need to create on the fly the new columns for Formula,
// like we do for fullReferences to show a seamless transition
}
const possibleOperation = operationDefinition.getPossibleOperation();
const isBucketed = Boolean(possibleOperation?.isBucketed);
const addOperationFn = isBucketed ? addBucket : addMetric;
@ -358,9 +362,9 @@ export function replaceColumn({
tempLayer = resetIncomplete(tempLayer, columnId);
if (previousDefinition.input === 'managedReference') {
// Every transition away from a managedReference resets it, we don't have a way to keep the state
// If the transition is incomplete, leave the managed state until it's finished.
tempLayer = deleteColumn({ layer: tempLayer, columnId, indexPattern });
return insertNewColumn({
const hypotheticalLayer = insertNewColumn({
layer: tempLayer,
columnId,
indexPattern,
@ -368,6 +372,14 @@ export function replaceColumn({
field,
visualizationGroups,
});
if (hypotheticalLayer.incompleteColumns && hypotheticalLayer.incompleteColumns[columnId]) {
return {
...layer,
incompleteColumns: hypotheticalLayer.incompleteColumns,
};
} else {
return hypotheticalLayer;
}
}
if (operationDefinition.input === 'fullReference') {
@ -859,7 +871,10 @@ function addBucket(
visualizationGroups: VisualizationDimensionGroupConfig[],
targetGroup?: string
): IndexPatternLayer {
const [buckets, metrics, references] = getExistingColumnGroups(layer);
const [buckets, metrics] = partition(
layer.columnOrder,
(colId) => layer.columns[colId].isBucketed
);
const oldDateHistogramIndex = layer.columnOrder.findIndex(
(columnId) => layer.columns[columnId].operationType === 'date_histogram'
@ -873,12 +888,11 @@ function addBucket(
addedColumnId,
...buckets.slice(oldDateHistogramIndex, buckets.length),
...metrics,
...references,
];
} else {
// Insert the new bucket after existing buckets. Users will see the same data
// they already had, with an extra level of detail.
updatedColumnOrder = [...buckets, addedColumnId, ...metrics, ...references];
updatedColumnOrder = [...buckets, addedColumnId, ...metrics];
}
updatedColumnOrder = reorderByGroups(
visualizationGroups,
@ -1169,8 +1183,20 @@ export function getErrorMessages(
}
>
| undefined {
const errors = Object.entries(layer.columns)
const columns = Object.entries(layer.columns);
const visibleManagedReferences = columns.filter(
([columnId, column]) =>
!isReferenced(layer, columnId) &&
operationDefinitionMap[column.operationType].input === 'managedReference'
);
const skippedColumns = visibleManagedReferences.flatMap(([columnId]) =>
getManagedColumnsFrom(columnId, layer.columns).map(([id]) => id)
);
const errors = columns
.flatMap(([columnId, column]) => {
if (skippedColumns.includes(columnId)) {
return;
}
const def = operationDefinitionMap[column.operationType];
if (def.getErrorMessage) {
return def.getErrorMessage(layer, columnId, indexPattern, operationDefinitionMap);
@ -1218,6 +1244,25 @@ export function isReferenced(layer: IndexPatternLayer, columnId: string): boolea
return allReferences.includes(columnId);
}
export function getReferencedColumnIds(layer: IndexPatternLayer, columnId: string): string[] {
const referencedIds: string[] = [];
function collect(id: string) {
const column = layer.columns[id];
if (column && 'references' in column) {
const columnReferences = column.references;
// only record references which have created columns yet
const existingReferences = columnReferences.filter((reference) =>
Boolean(layer.columns[reference])
);
referencedIds.push(...existingReferences);
existingReferences.forEach(collect);
}
}
collect(columnId);
return referencedIds;
}
export function isOperationAllowedAsReference({
operationType,
validation,

25
x-pack/plugins/lens/public/indexpattern_datasource/operations/mocks.ts
View file

@ -40,3 +40,28 @@ export const createMockedFullReference = () => {
getErrorMessage: jest.fn(),
};
};
export const createMockedManagedReference = () => {
return {
input: 'managedReference',
displayName: 'Managed reference test',
type: 'managedReference' as OperationType,
selectionStyle: 'full',
buildColumn: jest.fn((args) => {
return {
label: 'Test reference',
isBucketed: false,
dataType: 'number',
operationType: 'testReference',
references: args.referenceIds,
};
}),
filterable: true,
isTransferable: jest.fn(),
toExpression: jest.fn().mockReturnValue([]),
getPossibleOperation: jest.fn().mockReturnValue({ dataType: 'number', isBucketed: false }),
getDefaultLabel: jest.fn().mockReturnValue('Default label'),
getErrorMessage: jest.fn(),
};
};

2
x-pack/plugins/lens/public/indexpattern_datasource/types.ts
View file

@ -88,6 +88,8 @@ export interface IndexPatternPrivateState {
isFirstExistenceFetch: boolean;
existenceFetchFailed?: boolean;
existenceFetchTimeout?: boolean;
isDimensionClosePrevented?: boolean;
}
export interface IndexPatternRef {

3
x-pack/plugins/lens/public/mocks.tsx
View file

@ -166,6 +166,9 @@ export function mockDataPlugin(sessionIdSubject = new Subject<string>()) {
nowProvider: {
get: jest.fn(),
},
fieldFormats: {
deserialize: jest.fn(),
},
} as unknown) as DataPublicPluginStart;
}

11
x-pack/plugins/lens/public/types.ts
View file

@ -198,6 +198,11 @@ export interface Datasource<T = unknown, P = unknown> {
}
) => { dropTypes: DropType[]; nextLabel?: string } | undefined;
onDrop: (props: DatasourceDimensionDropHandlerProps<T>) => false | true | { deleted: string };
/**
* The datasource is allowed to cancel a close event on the dimension editor,
* mainly used for formulas
*/
canCloseDimensionEditor?: (state: T) => boolean;
getCustomWorkspaceRenderer?: (
state: T,
dragging: DraggingIdentifier
@ -300,11 +305,15 @@ export type DatasourceDimensionEditorProps<T = unknown> = DatasourceDimensionPro
// Not a StateSetter because we have this unique use case of determining valid columns
setState: (
newState: Parameters<StateSetter<T>>[0],
publishToVisualization?: { shouldReplaceDimension?: boolean; shouldRemoveDimension?: boolean }
publishToVisualization?: {
isDimensionComplete?: boolean;
}
) => void;
core: Pick<CoreSetup, 'http' | 'notifications' | 'uiSettings'>;
dateRange: DateRange;
dimensionGroups: VisualizationDimensionGroupConfig[];
toggleFullscreen: () => void;
isFullscreen: boolean;
};
export type DatasourceDimensionTriggerProps<T> = DatasourceDimensionProps<T>;

16
x-pack/plugins/lens/server/usage/schema.ts
View file

@ -14,6 +14,12 @@ const eventsSchema: MakeSchemaFrom<LensUsage['events_30_days']> = {
type: 'long',
_meta: { description: 'Number of times the user opened one of the in-product help popovers.' },
},
toggle_fullscreen_formula: {
type: 'long',
_meta: {
description: 'Number of times the user toggled fullscreen mode on formula.',
},
},
indexpattern_field_info_click: { type: 'long' },
loaded: { type: 'long' },
app_filters_updated: { type: 'long' },
@ -162,6 +168,10 @@ const eventsSchema: MakeSchemaFrom<LensUsage['events_30_days']> = {
type: 'long',
_meta: { description: 'Number of times the moving average function was selected' },
},
indexpattern_dimension_operation_formula: {
type: 'long',
_meta: { description: 'Number of times the formula function was selected' },
},
};
const suggestionEventsSchema: MakeSchemaFrom<LensUsage['suggestion_events_30_days']> = {
@ -183,6 +193,12 @@ const savedSchema: MakeSchemaFrom<LensUsage['saved_overall']> = {
lnsDatatable: { type: 'long' },
lnsPie: { type: 'long' },
lnsMetric: { type: 'long' },
formula: {
type: 'long',
_meta: {
description: 'Number of saved lens visualizations which are using at least one formula',
},
},
};
export const lensUsageSchema: MakeSchemaFrom<LensUsage> = {

36
x-pack/plugins/lens/server/usage/visualization_counts.ts
View file

@ -43,6 +43,31 @@ export async function getVisualizationCounts(
size: 100,
},
},
usesFormula: {
filter: {
match: {
operation_type: 'formula',
},
},
},
},
},
},
runtime_mappings: {
operation_type: {
type: 'keyword',
script: {
lang: 'painless',
source: `try {
if(doc['lens.state'].size() == 0) return;
HashMap layers = params['_source'].get('lens').get('state').get('datasourceStates').get('indexpattern').get('layers');
for(layerId in layers.keySet()) {
HashMap columns = layers.get(layerId).get('columns');
for(columnId in columns.keySet()) {
emit(columns.get(columnId).get('operationType'))
}
}
} catch(Exception e) {}`,
},
},
},
@ -56,16 +81,19 @@ export async function getVisualizationCounts(
// eslint-disable-next-line @typescript-eslint/no-explicit-any
function bucketsToObject(arg: any) {
const obj: Record<string, number> = {};
arg.buckets.forEach((bucket: { key: string; doc_count: number }) => {
arg.byType.buckets.forEach((bucket: { key: string; doc_count: number }) => {
obj[bucket.key] = bucket.doc_count + (obj[bucket.key] ?? 0);
});
if (arg.usesFormula.doc_count > 0) {
obj.formula = arg.usesFormula.doc_count;
}
return obj;
}
return {
saved_overall: bucketsToObject(buckets.overall.byType),
saved_30_days: bucketsToObject(buckets.last30.byType),
saved_90_days: bucketsToObject(buckets.last90.byType),
saved_overall: bucketsToObject(buckets.overall),
saved_30_days: bucketsToObject(buckets.last30),
saved_90_days: bucketsToObject(buckets.last90),
saved_overall_total: buckets.overall.doc_count,
saved_30_days_total: buckets.last30.doc_count,
saved_90_days_total: buckets.last90.doc_count,

42
x-pack/plugins/telemetry_collection_xpack/schema/xpack_plugins.json
View file

@ -2162,6 +2162,12 @@
"description": "Number of times the user opened one of the in-product help popovers."
}
},
"toggle_fullscreen_formula": {
"type": "long",
"_meta": {
"description": "Number of times the user toggled fullscreen mode on formula."
}
},
"indexpattern_field_info_click": {
"type": "long"
},
@ -2371,6 +2377,12 @@
"_meta": {
"description": "Number of times the moving average function was selected"
}
},
"indexpattern_dimension_operation_formula": {
"type": "long",
"_meta": {
"description": "Number of times the formula function was selected"
}
}
}
},
@ -2385,6 +2397,12 @@
"description": "Number of times the user opened one of the in-product help popovers."
}
},
"toggle_fullscreen_formula": {
"type": "long",
"_meta": {
"description": "Number of times the user toggled fullscreen mode on formula."
}
},
"indexpattern_field_info_click": {
"type": "long"
},
@ -2594,6 +2612,12 @@
"_meta": {
"description": "Number of times the moving average function was selected"
}
},
"indexpattern_dimension_operation_formula": {
"type": "long",
"_meta": {
"description": "Number of times the formula function was selected"
}
}
}
},
@ -2666,6 +2690,12 @@
},
"lnsMetric": {
"type": "long"
},
"formula": {
"type": "long",
"_meta": {
"description": "Number of saved lens visualizations which are using at least one formula"
}
}
}
},
@ -2709,6 +2739,12 @@
},
"lnsMetric": {
"type": "long"
},
"formula": {
"type": "long",
"_meta": {
"description": "Number of saved lens visualizations which are using at least one formula"
}
}
}
},
@ -2752,6 +2788,12 @@
},
"lnsMetric": {
"type": "long"
},
"formula": {
"type": "long",
"_meta": {
"description": "Number of saved lens visualizations which are using at least one formula"
}
}
}
}

198
x-pack/test/functional/apps/lens/formula.ts Normal file
View file

@ -0,0 +1,198 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import expect from '@kbn/expect';
import { FtrProviderContext } from '../../ftr_provider_context';
export default function ({ getService, getPageObjects }: FtrProviderContext) {
const PageObjects = getPageObjects(['visualize', 'lens', 'common', 'header']);
const find = getService('find');
const listingTable = getService('listingTable');
const browser = getService('browser');
const testSubjects = getService('testSubjects');
describe('lens formula', () => {
it('should transition from count to formula', async () => {
await PageObjects.visualize.gotoVisualizationLandingPage();
await listingTable.searchForItemWithName('lnsXYvis');
await PageObjects.lens.clickVisualizeListItemTitle('lnsXYvis');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.configureDimension({
dimension: 'lnsXY_yDimensionPanel > lns-dimensionTrigger',
operation: 'average',
field: 'bytes',
keepOpen: true,
});
await PageObjects.lens.switchToFormula();
await PageObjects.header.waitUntilLoadingHasFinished();
// .echLegendItem__title is the only viable way of getting the xy chart's
// legend item(s), so we're using a class selector here.
// 4th item is the other bucket
expect(await find.allByCssSelector('.echLegendItem')).to.have.length(3);
});
it('should update and delete a formula', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.switchToVisualization('lnsDatatable');
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_metrics > lns-empty-dimension',
operation: 'formula',
formula: `count(kql=`,
keepOpen: true,
});
const input = await find.activeElement();
await input.type('*');
await PageObjects.header.waitUntilLoadingHasFinished();
expect(await PageObjects.lens.getDatatableCellText(0, 0)).to.eql('14,005');
});
it('should insert single quotes and escape when needed to create valid KQL', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.switchToVisualization('lnsDatatable');
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_metrics > lns-empty-dimension',
operation: 'formula',
formula: `count(kql=`,
keepOpen: true,
});
let input = await find.activeElement();
await input.type(' ');
await input.pressKeys(browser.keys.ARROW_LEFT);
await input.type(`Men's Clothing`);
await PageObjects.common.sleep(100);
let element = await find.byCssSelector('.monaco-editor');
expect(await element.getVisibleText()).to.equal(`count(kql='Men\\'s Clothing ')`);
await PageObjects.lens.typeFormula('count(kql=');
input = await find.activeElement();
await input.type(`Men\'s Clothing`);
element = await find.byCssSelector('.monaco-editor');
expect(await element.getVisibleText()).to.equal(`count(kql='Men\\'s Clothing')`);
});
it('should persist a broken formula on close', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.switchToVisualization('lnsDatatable');
// Close immediately
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_metrics > lns-empty-dimension',
operation: 'formula',
formula: `asdf`,
});
expect(await PageObjects.lens.getErrorCount()).to.eql(1);
});
it('should keep the formula when entering expanded mode', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.switchToVisualization('lnsDatatable');
// Close immediately
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_metrics > lns-empty-dimension',
operation: 'formula',
formula: `count()`,
keepOpen: true,
});
await PageObjects.lens.toggleFullscreen();
const element = await find.byCssSelector('.monaco-editor');
expect(await element.getVisibleText()).to.equal('count()');
});
it('should allow an empty formula combined with a valid formula', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.switchToVisualization('lnsDatatable');
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_metrics > lns-empty-dimension',
operation: 'formula',
formula: `count()`,
});
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_metrics > lns-empty-dimension',
operation: 'formula',
});
await PageObjects.header.waitUntilLoadingHasFinished();
expect(await PageObjects.lens.getErrorCount()).to.eql(0);
});
it('should duplicate a moving average formula and be a valid table', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.switchToVisualization('lnsDatatable');
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_rows > lns-empty-dimension',
operation: 'date_histogram',
field: '@timestamp',
});
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_metrics > lns-empty-dimension',
operation: 'formula',
formula: `moving_average(sum(bytes), window=5`,
keepOpen: true,
});
await PageObjects.lens.closeDimensionEditor();
await PageObjects.lens.dragDimensionToDimension(
'lnsDatatable_metrics > lns-dimensionTrigger',
'lnsDatatable_metrics > lns-empty-dimension'
);
expect(await PageObjects.lens.getDatatableCellText(1, 1)).to.eql('222420');
expect(await PageObjects.lens.getDatatableCellText(1, 2)).to.eql('222420');
});
it('should keep the formula if the user does not fully transition to a quick function', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.switchToVisualization('lnsDatatable');
await PageObjects.lens.configureDimension({
dimension: 'lnsDatatable_metrics > lns-empty-dimension',
operation: 'formula',
formula: `count()`,
keepOpen: true,
});
await PageObjects.lens.switchToQuickFunctions();
await testSubjects.click(`lns-indexPatternDimension-min incompatible`);
await PageObjects.common.sleep(1000);
await PageObjects.lens.closeDimensionEditor();
expect(await PageObjects.lens.getDimensionTriggerText('lnsDatatable_metrics', 0)).to.eql(
'count()'
);
});
});
}

1
x-pack/test/functional/apps/lens/index.ts
View file

@ -41,6 +41,7 @@ export default function ({ getService, loadTestFile }: FtrProviderContext) {
loadTestFile(require.resolve('./geo_field'));
loadTestFile(require.resolve('./lens_reporting'));
loadTestFile(require.resolve('./lens_tagging'));
loadTestFile(require.resolve('./formula'));
// has to be last one in the suite because it overrides saved objects
loadTestFile(require.resolve('./rollup'));

54
x-pack/test/functional/apps/lens/smokescreen.ts
View file

@ -172,10 +172,11 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await testSubjects.existOrFail('indexPattern-dimension-formatDecimals');
await PageObjects.lens.closeDimensionEditor();
expect(await PageObjects.lens.getDimensionTriggerText('lnsXY_yDimensionPanel')).to.eql(
'Test of label'
);
await PageObjects.lens.closeDimensionEditor();
});
it('should be able to add very long labels and still be able to remove a dimension', async () => {
@ -587,6 +588,57 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
);
});
it('should not leave an incomplete column in the visualization config with field-based operation', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.configureDimension({
dimension: 'lnsXY_yDimensionPanel > lns-empty-dimension',
operation: 'min',
});
expect(await PageObjects.lens.getDimensionTriggerText('lnsXY_yDimensionPanel')).to.eql(
undefined
);
});
it('should not leave an incomplete column in the visualization config with reference-based operations', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');
await PageObjects.lens.goToTimeRange();
await PageObjects.lens.configureDimension({
dimension: 'lnsXY_xDimensionPanel > lns-empty-dimension',
operation: 'date_histogram',
field: '@timestamp',
});
await PageObjects.lens.configureDimension({
dimension: 'lnsXY_yDimensionPanel > lns-empty-dimension',
operation: 'moving_average',
field: 'Records',
});
expect(await PageObjects.lens.getDimensionTriggerText('lnsXY_yDimensionPanel')).to.eql(
'Moving average of Count of records'
);
await PageObjects.lens.configureDimension({
dimension: 'lnsXY_yDimensionPanel > lns-dimensionTrigger',
operation: 'median',
isPreviousIncompatible: true,
keepOpen: true,
});
expect(await PageObjects.lens.isDimensionEditorOpen()).to.eql(true);
await PageObjects.lens.closeDimensionEditor();
expect(await PageObjects.lens.getDimensionTriggerText('lnsXY_yDimensionPanel')).to.eql(
undefined
);
});
it('should transition from unique count to last value', async () => {
await PageObjects.visualize.navigateToNewVisualization();
await PageObjects.visualize.clickVisType('lens');

65
x-pack/test/functional/page_objects/lens_page.ts
View file

@ -107,6 +107,7 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont
isPreviousIncompatible?: boolean;
keepOpen?: boolean;
palette?: string;
formula?: string;
},
layerIndex = 0
) {
@ -114,10 +115,15 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont
await testSubjects.click(`lns-layerPanel-${layerIndex} > ${opts.dimension}`);
await testSubjects.exists(`lns-indexPatternDimension-${opts.operation}`);
});
const operationSelector = opts.isPreviousIncompatible
? `lns-indexPatternDimension-${opts.operation} incompatible`
: `lns-indexPatternDimension-${opts.operation}`;
await testSubjects.click(operationSelector);
if (opts.operation === 'formula') {
await this.switchToFormula();
} else {
const operationSelector = opts.isPreviousIncompatible
? `lns-indexPatternDimension-${opts.operation} incompatible`
: `lns-indexPatternDimension-${opts.operation}`;
await testSubjects.click(operationSelector);
}
if (opts.field) {
const target = await testSubjects.find('indexPattern-dimension-field');
@ -125,6 +131,10 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont
await comboBox.setElement(target, opts.field);
}
if (opts.formula) {
await this.typeFormula(opts.formula);
}
if (opts.palette) {
await this.setPalette(opts.palette);
}
@ -357,7 +367,7 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont
await retry.try(async () => {
await testSubjects.click('lns-palettePicker');
const currentPalette = await (
await find.byCssSelector('[aria-selected=true]')
await find.byCssSelector('[role=option][aria-selected=true]')
).getAttribute('id');
expect(currentPalette).to.equal(palette);
});
@ -379,6 +389,18 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont
});
},
async isDimensionEditorOpen() {
return await testSubjects.exists('lns-indexPattern-dimensionContainerBack');
},
// closes the dimension editor flyout
async closeDimensionEditor() {
await retry.try(async () => {
await testSubjects.click('lns-indexPattern-dimensionContainerBack');
await testSubjects.missingOrFail('lns-indexPattern-dimensionContainerBack');
});
},
async enableTimeShift() {
await testSubjects.click('indexPattern-advanced-popover');
await retry.try(async () => {
@ -398,14 +420,6 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont
await testSubjects.click('errorFixAction');
},
// closes the dimension editor flyout
async closeDimensionEditor() {
await retry.try(async () => {
await testSubjects.click('lns-indexPattern-dimensionContainerBack');
await testSubjects.missingOrFail('lns-indexPattern-dimensionContainerBack');
});
},
async isTopLevelAggregation() {
return await testSubjects.isEuiSwitchChecked('indexPattern-nesting-switch');
},
@ -549,7 +563,8 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont
});
}
const errors = await testSubjects.findAll('configuration-failure-error');
return errors?.length ?? 0;
const expressionErrors = await testSubjects.findAll('expression-failure');
return (errors?.length ?? 0) + (expressionErrors?.length ?? 0);
},
async searchOnChartSwitch(subVisualizationId: string, searchTerm?: string) {
@ -1025,5 +1040,27 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont
);
await PageObjects.header.waitUntilLoadingHasFinished();
},
async switchToQuickFunctions() {
await testSubjects.click('lens-dimensionTabs-quickFunctions');
},
async switchToFormula() {
await testSubjects.click('lens-dimensionTabs-formula');
},
async toggleFullscreen() {
await testSubjects.click('lnsFormula-fullscreen');
},
async typeFormula(formula: string) {
// Formula takes time to open
await PageObjects.common.sleep(500);
await find.byCssSelector('.monaco-editor');
await find.clickByCssSelectorWhenNotDisabled('.monaco-editor');
const input = await find.activeElement();
await input.clearValueWithKeyboard();
await input.type(formula);
},
});
}