* [Discover] redirect to main page on route not found discover error
* [Discover] remove services from dependencies
* [Discover] apply suggestions
* [Discover] return invalid link, but make it not clickable
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* built control group embeddable featuring inline control creation and editing, and DndKit based drag and drop.
Co-authored-by: andreadelrio <delrio.andre@gmail.com>
* remove kbn-legacy-logging package
* remove legacy service
* remove legacy appender
* remove LegacyObjectToConfigAdapter
* gix types
* remove @hapi/good / @hapi/good-squeeze / @hapi/podium
* remove `default` appender validation for `root` logger
* remove old config key from kibana-docker
* fix FTR config
* fix dev server
* remove reference from readme
* fix unit test
* clean CLI args and remove quiet option
* fix type
* fix status test config
* remove from test config
* fix snapshot
* use another regexp
* update generated doc
* fix createRootWithSettings
* fix some integration tests
* another IT fix
* yet another IT fix
* (will be reverted) add assertion for CI failure
* Revert "(will be reverted) add assertion for CI failure"
This reverts commit 78d5560f9e.
* switch back to json layout for test
* remove legacy logging config deprecations
* address some review comments
* update documentation
* update kibana.yml config examples
* add config example for `metrics.ops`
Co-authored-by: Tyler Smalley <tyler.smalley@elastic.co>
* [Discover] fix saved search become active
* [Discover] add another fix to be consistent with data fetching code
* [Discover] simplify solution
* [Discover] add functionals
* [Discover] fix saved query bug, add functionals
* [Discover] fix functionals
* [Discover] fix functional test
* [Discover] split saved query tests
* [Discover] preselect logstash index pattern
* [Discover] remove saved query after test complete
* [Discover] change query fill order
* [Discover] try to fix unrelated functional test
* [Discover] one more fix
* [Discover] try to fix one more problem
* [Discover] fix commonly used time range test
* [Discover] revert uisettings init in before statement, do small adjustments
* [Discover] fix unit test
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
- Add Authorization type to ES UI shared.
- Add convertPrivilegesToArray, patch to also accept privileges that might contain dots in its name, and add tests.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Update the slugs to improve google analytics drilldown tracking
* more slug updates
* Fix some formatting issues in building blocks
* update paths
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [TSVB] Rename Index pattern to Data view
* Update some labels
* Make asterisk bold to indicate it's text input
* Remove redundant translation
* Remove "now" from TSVB callout notification title
* Update variable name to match the text
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* initial pass at moving data views into own plugin
* require expressions plugin, fix META_FIELDS reference
* bundle limits and localization
* fix integration test
* update plugin list and jest config
* type fixes
* search fixes
* fix localization
* fix mocks
* fix mocks
* fix stub
* type fixes
* fix import on test file
* path fixes
* remove shorted dotted from data plugin
* more todo removal
* eslint fixes
* eslint fix
* simplify data views server plugin
* simplify data views server plugin
* simplify data views server plugin
* fix imports on api routes
* fix imports on api routes
* update plugin list
* ts fixes
* ts fixes
* add deprecation notice
* fix circular dependency and api integration test
* fix circular dependency and api integration test
* rename types for better clarity
* path fixes
* jest.config and tsconfig cleanup
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Update ESLint config to disallow usage of the any type
* Remove the any type usages from the expressions plugin
* Update plugins using expressions according to the updated public API
* [TSVB] Update the series and metrics Ids that are numbers to strings
* Minor changes
* Adds a unit test to TSVB plugin to test this case
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Don't trigger autorefresh when there's no time picker
- because there's no UI for that
* Refactor and add test
* Add doc and test
* Refactor
* Remove index pattern without timefield filtering
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Display both beats and epr-packages in the integration browser. When there is overlap, the EPR-package equivalent is displayed. When the EPR-package is not yet ga, the beat-equivalent is displayed.
After precommit hook runs with --fix flag changes are not added to staging. However it also does not
validate staging area since eslint is only looking for last changes on file not staging area this
results fellows to commit with linting errors which fails in CI. This commit resolves this issue by
adding fixed files right after linting to staging area.
Closes#52722
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This fixes two fundamental issues:
* the list of files to be linted is built looking at the diffs of the ref
specified with `--ref` (or otherwise the current index) with those on
the filesystem
* the content to be linted is read from the filesystem instead of the
specified `ref` or the one in the index
* [Discover][Table] Remove unused dependencies of the kibanaLegacy plugin
* More removals of kibanaLegacy plugin dependencies
* Revert discover changes
* Remove the unused functions from the kibana_legacy plugin
* Removes unused translations
Allow selecting any service name, transaction type (where appropriate), and environment when creating and editing rules, both in APM and Stack Management.
- Create /internal/apm/suggestions endpoint that uses `terms_enum`
- Use combo box for environment, service name, and transaction type with suggestions endpoint on all alerts
- Remove "Go to APM" callouts on new alert creation
- Wrap calls to `createCallApmApi` in alert triggers with `useEffect`
- Use `getEnvironmentLabel` for value in environment field expression
- Make all `AlertParams` fields optional (except in latency threshold alert)
- Add e2e tests for creating an alert
- Remove `NewAlertEmptyPrompt` component and `isNewApmRuleFromStackManagement` helper
- Replace `maxServiceEnvironments` and `maxServiceSelections` config options with `maxSuggestions` advanced setting.
Fixes#106786
* Bumping EUI to 38.0.0
* Updating storyshots.test.tsx snapshot
* Removing unneeded ID mock and updating snapshot.
* Fixing panel titles by splitting visible labels.
* Updating one functional test for Discover > Data Grid
* Fixing one test on a Lens > Smokescreen accessible label
* Updating four tests to use visible labels only.
* Adding helper function to retrieve visual text only and fixing three tests.
* Dashboard > Panel Cloning all tests pass.
* Fixing accessible label test
* Fixing data_shared_attributes, two tests
* Updating tests for Discover > Data Grid
* Reverting tests ahead of EUI 38.0.1 backport
* Reverted the SR-only class behavior to use `left: -10000px;`
* Reverting tests to previous assertions
* Bumping configuration files to EUI 38.0.1
* package.json
* package-lock.json
* license-checker
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Write plugin directory doc as part of api docs build.
* Replace service folder with the new name for index patterns
* Updated docs
* small update
* update docs
* [Viz] legend duplicates percentile options when chart has both left & right Y axes
* Update comment for isPercentileIdEqualToSeriesId
* Remove Dimension interface
* Replace partial aspect with whole aspect value
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Add a new plugin `custom_integrations`. This plugin allows for the registration of data-integrations tutorials. The Fleet-integrations app will display these alongside the existing Elastic Agent integrations.
* [TSVB] Series hidden via click reappear on auto refresh
* Move mainAxisGroupId back to render
* Remove blank line in vis.js
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This fixes a regression in Vega Maps that limits users to max zoom 10 regardless of distribution
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* add utility to set current kibana version
* set kibana version on dashboard
* move setVersion function into separate file
* improve setVersion method
* add test mocks
* remove unnecessary any
* narrow down history type
* use new formatting settings
* remove version integration on dashboard
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Refactor executor forking to implement state inheritance
* Fix setup and start contracts typings
* Add support of named forks
* Add expressions service life-cycle assertions
* [DOCS] Update remote cluster and security links
* Updating test link
* Update URL for failing test
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [TSVB] Remove the input string mode
* Fix some tests
* Add some functional tests and fix failing CI
* Update telemetry mappings
* Rename useStringIndices to allowStringIndices, move it from TSVB to Data constants and refactor test
* Apply text suggestions from code review
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
* Apply formatting and remove unused translations
* Fix labels
* Remove unused import
* Move popover toggling to checkIndexPatternSelectionModeSwitchIsEnabled function to prevent flakiness
* Update some visual_builder_page functions
* Remove accidentally added newlines
* Move TSVB ui settings to constants, remove tooltip and update popover text
* Handle the case of editing advanced settings is restricted
* Add requiresPageReload to UI setting and condition for the case the setting is already enabled
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Kaarina Tungseth <kaarina.tungseth@elastic.co>
* only use relative time filter when generating share data
* Added comment on absolute time filter.
Co-authored-by: Tim Sullivan <tsullivan@users.noreply.github.com>
* improve discover search session relative time range test
* update discover tests and types for passing in data plugin to getSharingData function
* updated reporting to pass in data plugin to getSharingData, also updates jest tests
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tim Sullivan <tsullivan@users.noreply.github.com>
Co-authored-by: Anton Dosov <anton.dosov@elastic.co>
* first pass at any cleanup
* fix types on TableListView
* change generic constraint
* fix lint
* fix TS in no data card
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fixed bug with treating null argument as the expression function.
* Added a test for checking if a null argument is processeded correctly.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fix and re-enable Vega test to be consistent with development/CI
* Remove skip comments
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Pie] Loads the no results screen if all slices have zero value
* Add a functional test
* Apply PR changes
* Display no results component if the chart metric has negative values
* Nits
* Apply some of the PR comments
* Change the negative values text
* Update src/plugins/vis_types/pie/public/pie_component.test.tsx
Co-authored-by: Marta Bondyra <marta.bondyra@gmail.com>
* Update src/plugins/vis_types/pie/public/pie_component.test.tsx
Co-authored-by: Marta Bondyra <marta.bondyra@gmail.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marta Bondyra <marta.bondyra@gmail.com>
* Add deprecation warning when unknown types are present
* fix and add service tests
* remove export
* plug deprecation route
* add integration test for new route
* add unit test for getIndexForType
* add unit tests
* improve deprecation messages
* add FTR test
* fix things due to merge
* change the name of the deprecation provider
* improve message
* improve message again
* version 1 of fix: we set the time range on the search source at CSV generation time
* updated jest tests and updated API for getSharingData
* make time range optional for getSharingData
* pivot to updating "getTime" functionality by introducing a new flag
* update jest snapshots
* update comment
* refactored coerceToAbsoluteTime -> coerceRelativeTimeToAbsoluteTime and updated behaviour to be more specific
* fix jest test
* do not change createFilter API, rather create new createRelativeFilter API, also only use this in one place in discover
* update jest tests
* update mock
* update jest test mock
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Use date_histogram instead of auto_date_histogram in pipeline aggregations
* Fix ci
* Fix eslint
* start disable parent pipeline aggs and show error
* Fix CI
* Fix eslint
* Fix CI
* Add functional tests
* Some fixes
* Fix lint
* Use agg_utils
* Fix lint
* Fix text
* Fix lint
* Fix tests
* Fixed condition
* Fix math aggregation
* math should pass panel type as prop
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Use resolve instead of get for saved query service
* Update tests
* Update src/plugins/data/public/query/saved_query/saved_query_service.ts
Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
* Revert step 4
* Make saved queries share-capable
* Fix test
Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
* Use resolve instead of get for saved query service
* Update tests
* Update src/plugins/data/public/query/saved_query/saved_query_service.ts
Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
* Revert step 4
* Fix test
Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
* account for banners when data grid is full screen
* account for banner when canvas is full screen
* change height per feedback
* add withKibana
* rm withKibana; move vars out of Fullscreen
* Use hasHeaderBanner$
* add banner height var comments
* fix ts error
Co-authored-by: Catherine Liu <catherine.liu@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* use correct styles when after split we have only one series
* Fix lint
* Fix comments
* remove unused styles
* Update _vis_with_splits.scss
* Fix lint
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [TSVB] Support custom field format
Add format_label response processor for series vis data and bucket key formatting to process_bucket for table vis data
* Add ignore_field_formatting for series to support value formatting for all panel types except markdown
* Fix type issue for visData and rename getCustomFieldFormatter to createCustomFieldFormatter
* Update vis.test to cover custom field formats logic and add a migration script to set ignore_field_formatting to true for the series
* Move createCustomFieldFormatter to a separate file, make formatting respect only active metrics field name, refactor vis files and fix label formatting only for grouped by terms series
* Remove services, add getFieldFormatsService to use it in format_label and get_table_data, replace getCustomFieldFormatter with createCustomFieldFormatter
* Update plugin.ts
* Update start for plugin.ts
* Add formatting for annotations and markdown values
* Refactor some code
* Update some naming and conditions
* Fix formatting of data type labels
* In process_bucket fix case for no getFieldFormatByName
* Add field formatting functional tests for all panel types
* Update tests to make them run correctly for firefox
* Update _tsvb_markdown test setup
* Move series ignoreFieldFormatting check to a separate file, change convertSeriesToVars signature, update migration script and refactor a bit functional tests
* Fix type check for timeseries_visualization.tsx
* Update migrations.js test expected version to 7.15
* Fix tests in _tsvb_chart.ts
* Fix merge conflict remove process_bucket.js
* Update process_bucket.test.ts
* Fix markdown labels formatting
* Add ignore_field_formatting for annotations, enhanced migration script to set that flag to true, refactor data_format_picker
* Fix migration script and add disabling for ignore component when string index pattern is used
* Add supporting URL and color formatters in tsvb table
* Fix eslint
* Remove ignore formatting component, add field formatting option to TSVB data format picker and make it default, remove migration script, update tests and refactor some files
* Fix failing tests, refactor create_field_formatter and add test to it, update some other files
* Fix series formatting for top hit when it has not numeric result
* Handle no fieldFormatMap case for table/vis.js
* Remove "Default" option form DataFormatPicker when index pattern is string, fix markdown variables issue and refactor some code
* Chore(TSVB): Replace aggregations lookup with map
* Fix types, update test expected data and remove unused translations
* Fix i18 check and useEffect in agg.tsx
* Handle aggregations field formatting case
* Fix agg_utils, vis and check_if_numeric_metric tests
* Correct typo and refactor condition in std_metric
* Fix type check
* Get rid of IFieldType
* Add URL and color formatting for topN and metric tabs, fix setting initial custom formatter and switching formatter in agg.tsx
* Update tsvb.asciidoc
* Remove link icon from Date format field help text, update click logic for top N in case of custom field format and fix CI
* Remove unused import
* Revert top N bar extra logic for click
* Refactor some code in agg.tsx
* Add URL and color formatting to Gauge
* Fix bug with terms formatting, refactor some code, update create_field_formatter
* Add comments to _gauge.scss
* Remove unnecessary await
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Uladzislau Lasitsa <Uladzislau_Lasitsa@epam.com>
* [graph] Make Graph saved object share-capable
Step 4 of https://www.elastic.co/guide/en/kibana/master/sharing-saved-objects.html#sharing-saved-objects-faq-changing-object-ids
* bump migration version to fix the test
* test
* Revert "bump migration version to fix the test"
This reverts commit 04d2f49386.
* Revert "test"
This reverts commit 7b0a74d431.
* Fix Core migration integration tests
The existing tests incorrectly asserted an object's `migrationVersion`
solely based on the registered type's `migration` field; in reality, the
`convertToMultiNamespaceTypeVersion` field is also used when determining
an object's `migrationVersion`. This commit simply updates the test to
reflect that.
Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Remove timelion app and stuff which related to it
* Fix CI
* Fix lint
* Fix tests
* Fix tests
* Fis tests
* Fix some comments
* Clean up
* fix CI
* fix some comments
* Fix deprecation examples
* Return `enabled` property in config for timelion vis
* Remove unused angular lib
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alexey Antonov <alexwizp@gmail.com>
* Move legacy dashboards API to core and adds usage data
* More legacy_export plugin removal
* Log a warning for deprecated dashboard import/export API
* Review comments
## Summary
This PR resolves issue <https://github.com/elastic/kibana/issues/110041> reported by @snide to enable schema-driven sorting descriptions in column headers.
@chandlerprall recommends obtaining a **+1** from the Machine Learning and Observability solutions, because the fix updates an `i18n` constant in Kibana common to all consumers of `EuiDataGrid`.
## Details
Thanks @chandlerprall for paring on this!
The Alerts table, `Host > Events`, and other `EuiDataGrid`-based views in the Security Solution make use of the default [`EuiDataGrid` schemas](https://elastic.github.io/eui/#/tabular-content/data-grid-schemas-and-popovers).
The default schemas enable `EuiDataGrid` to automatically display, for example, `Old-New` and `New-Old` sorting descriptions for datetime fields, as opposed to generic `A-Z` and `Z-A` descriptions.
The following (shared) Kibana `i18n` constant in `src/core/public/i18n/i18n_eui_mapping.tsx` is expected to be rendered a `string` at runtime:
```ts
'euiColumnActions.sort': ({ schemaLabel }: EuiValues) =>
i18n.translate('core.euiColumnActions.sort', {
defaultMessage: 'Sort {schemaLabel}',
values: { schemaLabel },
}),
```
But the constant was rendered in `EuiDataGrid` column headers as `[object Object]` when schemas were enabled, as shown in the screenshot below:
_Above: The `sortTextAsc/Desc` text was rendered as `[object Object]`_
The temporary workaround described by [#110041](https://github.com/elastic/kibana/issues/110041) ensured that `Sort A-Z` and `Sort Z-A` labels were always displayed (in lieu of `[object Object]`), as shown in the screenshot below:
_Above: `Sort A-Z` and `Sort Z-A` labels were always displayed as a workaround_
The fix in this PR updates the following (shared) Kibana `i18n` constant in `src/core/public/i18n/i18n_eui_mapping.tsx` to use a `FormattedMessage`:
```ts
'euiColumnActions.sort': ({ schemaLabel }: EuiValues) => (
<FormattedMessage
id="core.euiColumnActions.sort"
defaultMessage="Sort {schemaLabel}"
values={{ schemaLabel }}
/>
),
```
, which ensures a schema-specific sorting label is displayed as-expected.
It also removes the workaround, as shown in the animated gif below:
_Above: Schema-specific sorting descriptions are displayed for `datetime`, `text`, and `numeric` column headers_
* [sample data] update web log geo.src field to match country code of geo.coordinates
* fix functional tests
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Add verification code protection
* Fix bug where verification code could be less than 6 digits
* Added suggestions from code review
* fix type errors
* Added suggestions from code review
* first iteration of canvas reporting using v2 PDF generator
* updated jest test
* made v2 report URLs compatible with spaces and simplified some code
* remove non-existent import
* updated import of lib
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Added `tagCloud` to canvas.
* Added `icon` to the `tagCloud` element.
* Added column name support at `tag_cloud`.
* Added condition to `vis_dimension` not to pass invalid index.
Added check of accessor index, if such column exists at vis_dimension.
Removed checks of column existance from TagCloudChart.
Added test for accessing data by column name in addition to a column number.
Updated tag_cloud element in Canvas.
Fixed types. Removed almost all `any` and `as` types.
* Added test suites for `vis_dimension` function.
* Added tests for DatatableColumn accessors at tag_cloud_fn and to_ast.
* Refactored metrics, tagcloud and tests.
Added valid functional tests to metrics and tag_cloud.
Fixed types of metrics_vis.
Added handling of empty data at tag_cloud renderer.
* Added storybook ( still doesn't work ).
* Fixed some mistakes.
* Added working storybook with mocks.
* Added clear storybook for tag_cloud_vis_renderer.
* Updated the location of vis_dimension test after movement of the function.
* Fixed unused type.
* Fixed tests and added handling of the column name at `visualizations/**/*/prepare_log_table.ts`
* Reduced the complexity of checking the accessor at `tag_cloud_chart.tsx`
* Added comments at unclear places of code.
* Added the logic for disabling elements for renderers from disabled plugins.
* removed garbage from `kibana.yml`.
* Fixed element_strings.test error.
* Made changes, based on nits.
* Fixed mistake.
* Removed `disabled` flag for `expression_*` plugins.
* recovered lost comments at the unclear places.
* removed dead code.
* fixed test errors.
* Fixed test error, I hope.
* fixed more tests.
* fixed code, based on nits.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Remove kibana.defaultAppId setting
* Fix typings
* Remove plugin dependency
* Use proper navigation method to get to home
* Default route for home
* Address discover new routing code
* Make non existing /kibana URLs working
* Fix space awareness
* Remove documentation
* Remove the setting from docker file
* Make defaultRoute forward work properly
* Add forward_url tests
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* added initial version of locator
* removed unused params and added jest test
* updated functional test to expect PDF reports to be available when vis is new
* fix TS: remove unkown field
* added some docs and removed unused code
* AggsConfigOption -> AggsConfigSerialized
* moved locator to common
* fixed building of "create" path and updated test snapshots
* updated import
* update encoding behaviour
* added time range from timefilter to locator params request
* add index pattern and search id to URL params
* reading index pattern from search source if it is there for the locator
* remove "type" from locator params, update comments and test
* removed duplicate identifier
* remove unused type
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds a workaround for EQL bug: https://github.com/elastic/elasticsearch/issues/77152
Adds the safety feature mentioned here: https://github.com/elastic/kibana/issues/110802
Adds the ability to ignore particular [fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-fields.html#search-fields-param) when the field is merged with [_source](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-fields.html#source-filtering). Also fixes an EQL bug where EQL is introducing the meta field of `_ignored` within the fields and causing documents to not be indexable when we merge with the fields from EQL.
Alerting document creation uses the fields API to get [runtime field](https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime.html), [constant keyword](https://www.elastic.co/guide/en/elasticsearch/reference/master/keyword.html#constant-keyword-field-type), etc... that are only available within the [fields API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-fields.html#search-fields-param) and then merges the field values not found within the `_source` document with the `_source` document and then finally indexes this merged document as an alert document.
This fix/ability is a "safety feature" in that if a problematic [runtime field](https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime.html), [constant keyword](https://www.elastic.co/guide/en/elasticsearch/reference/master/keyword.html#constant-keyword-field-type) is discovered or another bug along the stack we can set a `kibana.yml` key/value pair to ignore the problematic field.
This _WILL NOT_ remove problematic fields from the `_source` document. This will only ignore problematic constant keyword, runtime fields, aliases, or anything else found in the fields API that is causing merge issues.
This PR:
* Adds a `alertIgnoreFields` `kibana.yml` array key with a default of an empty array if not specified.
* Plumbs the `alertIgnoreFields` through the stack and into the fields/_source merge strategies of `missingFields` and `allFields`
* Adds a temporary `isEqlBug77152` where it hard codes an ignore of `_ignored` until the EQL problem is fixed and then we will remove the workaround
* Adds unit tests
* Adds e2e tests which covers the described use cases above.
The `alertIgnoreFields` key/value within `kibana.yml` if set should be an array of strings of each field you want to ignore. This can also contain regular expressions as long as they are of the form, `"/regex/"` in the array.
Example if you want to ignore fields that are problematic called "host.name" and then one in which you want to ignore all fields that start with "user." using a regular expression:
```yml
xpack.securitySolution.alertIgnoreFields: ['host.name', '/user\..*/']
```
Although there are e2e tests which exercise the use cases...
If you want to manual test the EQL bug fix you would add these documents in dev tools:
```json
# Delete and add a mapping with a small ignore_above.
DELETE eql-issue-ignore-fields-delme
PUT eql-issue-ignore-fields-delme
{
"mappings" : {
"dynamic": "strict",
"properties" : {
"@timestamp": {
"type": "date"
},
"some_keyword" : {
"ignore_above": 5,
"type" : "keyword"
},
"other_keyword" : {
"ignore_above": 10,
"type" : "keyword"
}
}
}
}
# Add a single document with one field that will be truncated and a second that will not.
PUT eql-issue-ignore-fields-delme/_doc/1
{
"@timestamp": "2021-09-02T04:13:05.626Z",
"some_keyword": "longer than normal",
"other_keyword": "normal"
}
```
Then create an alert which queries everything from it:
<img width="1155" alt="Screen Shot 2021-09-01 at 10 15 06 PM" src="https://user-images.githubusercontent.com/1151048/131781042-faa424cf-65a5-4ebb-b801-3f188940c81d.png">
and ensure signals are created:
<img width="2214" alt="Screen Shot 2021-09-01 at 10 30 18 PM" src="https://user-images.githubusercontent.com/1151048/131782069-b9ab959c-f22d-44d5-baf0-561fe349c037.png">
To test the manual exclusions of any other problematic fields, create any index which has runtime fields or `constant keywords` but does not have anything within the `_source` document using dev tools. For example you can use `constant keyword` like so
```json
PUT constant-keywords-deleme
{
"mappings": {
"dynamic": "strict",
"properties": {
"@timestamp": {
"type": "date"
},
"testing_ignored": {
"properties": {
"constant": {
"type": "constant_keyword",
"value": "constant_value"
}
}
},
"testing_regex": {
"type": "constant_keyword",
"value": "constant_value"
},
"normal_constant": {
"type": "constant_keyword",
"value": "constant_value"
},
"small_field": {
"type": "keyword",
"ignore_above": 10
}
}
}
}
PUT constant-keywords-deleme/_doc/1
{
"@timestamp": "2021-09-02T04:20:01.760Z"
}
```
Set in your `kibana.yml` the key/value of:
```yml
xpack.securitySolution.alertIgnoreFields: ['testing_ignored.constant', '/.*_regex/']
```
Setup a rule to run:
<img width="1083" alt="Screen Shot 2021-09-01 at 10 23 23 PM" src="https://user-images.githubusercontent.com/1151048/131781696-fea0d421-836f-465c-9be6-5289fbb622a4.png">
Once it runs you should notice that the constant values for testing are not on the signals table since it only typically exists in the fields API:
<img width="1166" alt="Screen Shot 2021-09-01 at 10 26 16 PM" src="https://user-images.githubusercontent.com/1151048/131781782-1684fb1d-bed9-4cf0-be9a-0abe1f0f34d1.png">
But the normal one still exists:
<img width="1136" alt="Screen Shot 2021-09-01 at 10 26 31 PM" src="https://user-images.githubusercontent.com/1151048/131781827-5450c693-de9e-4285-b082-9f7a2cbd5d07.png">
If you change the `xpack.securitySolution.alertIgnoreFields` by removing it and re-generate the signals you will see these values added back.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [x] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/master/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
* remove unnecessary ts-ignore
* add context propagation to x-opaque-id header tests
* run tests on CI
* simplify logging. the action purpose follows from the context name
* extend tests with the assertion against execution_context from the Kibana logs
* split JSON log records only
* apply suggestions proposed by Spencer
* Move to vis_types folder part 2
* fix jest tests
* do some tests
* revert
* Test Tiago's fix
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* We should use 'sort' from terms for 'GroupBy'
* Fix conflicts
* Fix import
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
In #110116 I added the `RedirectAppLinks` component, which makes it so links outside the app automatically do not cause a page reload.
This component inserts a div into the DOM. Apparently this also caused the text areas in the request and response tabs of the request inspector to be collapsed to zero height.
Remove `RedirectAppLinks` and handle the navigation with an `onClick` callback instead.
* [eslint] add rule to prevent export* in plugin index files
* deduplicate export names for types/instances with the same name
* attempt to auto-fix duplicate exports too
* capture exported enums too
* enforce no_export_all for core too
* disable rule by default, allow opting-in for help fixing
* update tests
* reduce yarn.lock duplication
* add rule but no fixes
* disable all existing violations
* update api docs with new line numbers
* revert unnecessary changes to yarn.lock which only had drawbacks
* remove unnecessary eslint-disable
* rework codegen to split type exports and use babel to generate valid code
* check for "export types" deeply
* improve test by using fixtures
* add comments to some helper functions
* disable fix for namespace exports including types
* label all eslint-disable comments with related team-specific issue
* ensure that child exports of `export type` are always tracked as types
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Upgrade EUI to v37.3.1
* Update i18n token mappings
* Skip i18n_eui_mapping defString checks for functions
* Update snapshots
* Update failing Security tests with extra nodes
* Remove hook cleanup now that elastic/eui#5068 is merged
* [i18n PR feedback] Prefer specific token skipping over all functions skipping
* Revert "Remove hook cleanup now that elastic/eui#5068 is merged"
This reverts commit e40ebfa929.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fix logging for existing integration test
* First stab at limiting batches to batchSizeBytes
* Fix tests
* Fix batch size calculation, NDJSON needs to be terminated by an empty line
* Integration tests
* Fix type failures
* rename migration integration tests and log files to be consistent & more descriptive
* Review feedback
* Remove duplication of fatal error reasons
* migrations.maxBatchSizeBytes to docker environment vars
* docs for migrations.maxBatchSizeBytes
* wip to remove rbac
* Revert "[Cases] Include rule registry client for updating alert statuses (#108588)"
This reverts commit 1fd7038b34.
This leaves the rule registry mock changes
* remove rbac on Trend/Count alert
* update detection api for status
* remove @kbn-alerts packages
* fix leftover
* Switching cases to leverage update by query for alert status
* Adding missed files
* fix bad logic
* updating tests for use_alerts_privileges
* remove index alias/fields
* fix types
* fix plugin to get the right index names
* left over of alis on template
* forget to use current user for create/read route index
* updated alerts page to not show table when no privileges and updates to tests
* fix bug when switching between o11y and security solution
* updates tests and move to use privileges page when user tries to access alerts without proper access
* updating jest tests
* pairing with yara
* bring back kbn-alerts after discussion with the team
* fix types
* fix index field for o11y
* fix bug with updating index priv state
* fix i18n issue and update api docs
* fix refresh on alerts
* fix render view on alerts
* updating tests and checking for null in alerts page to not show no privileges page before load
* fix details rules
Co-authored-by: Jonathan Buttner <jonathan.buttner@elastic.co>
Co-authored-by: Yara Tercero <yara.tercero@elastic.co>
When the observability:enableInspectEsQueries advanced setting is enabled, show an inspector that includes all queries through useFetcher.
Remove the callout.
