* POC
* Before and after saving connector callbacks
* Draft callbacks on SN
* Migrate legacy connectors
* Add deprecated connector
* Fix callbacks types
* Pass isEdit to connector forms
* Get application info hook
* Validate instance on save
* Support both legacy and new app
* Seperate SIR
* Log application version & and throw otherwise
* Deprecated tooltip cases
* Deprecated tooltip alerts
* Improve message
* Improve translation
* Change to elastic table & fix types
* Add callbacks to add modal
* Pass new props to tests
* Change health api url to production
* Better installation message
* Migrate connectors functionality
* Change migration version to 7.16
* Fix bug
* Improve message
* Use feature flag
* Create credentials component
* Add form to migration modal
* Improve installation callout
* Improve deprecated callout
* Improve modal
* Improve application required modal
* Improve SN form
* Support both connectors
* Support correlation attributes
* Use same component for SIR
* Prevent using legacy connectors when creating a case
* Add observables
* Unique observables
* Push only if there are observables
* Change labels to plural
* Pass correlation ID and value
* Show errors on the callout
* Improve alerts tooltip
* Improve cases tooltip
* Warning callout on cases configuration page
* Fix tooltip content
* Add help text
* Change from string to array
* Fix i18n
* Fix spelling
* Update incidents for ITSM
* Update incidents for SIR
* Fix types
* Fix backend tests
* Fix frontend tests
* Add service tests
* Fix i18n
* Fix cypress test
* Improve ServiceNow intergration tests
* Fix cases integration tests
* Fix triggers actions ui end to end test
* Fix tests
* Rename modal
* Show error message on modal
* Create useOldConnector helper
* Show the update incident toggle only on new connectors
* Add observables for old connectors
* Fix error when obs are empty
* Enable SIR for alerts
* Fix types
* Improve combineObservables
* Add test for the sir api
* Add test for the sir service
* Add documentation
* PR feedback
* Improve cases deprecated callouts
* Improve observables format
* Add integration tests for SIR
* Fix doc error
* Add config tests
* Add getIncident tests
* Add util tests
* Add migration tests
* Add tests for connectors and improve callouts
* Add more tests
* Add more UI tests
* update connector modal to flyout
* PR feedback
* Test CI
* restore auth callout
* edit connector form spacing
* Improve integration tests
* Add 8 pixels to the left of the connector icon
* update switch to checkboxes
* case detail ui
* Seperate ServiceNow integration tests
* Remove observables fields
* Add correlation values
* Fix merge
* add deprecated text in the dropdown
* update card icon to the right
* new update connetor test and other tests fixes
* PR feedback
* Remove observables from docs
* Remove unused translations
* Using eui theme for styling
* Content feeback
* Add more unit tests
* Fix i18n
* Fix types
* Fixes
* Fixes
* test properly
* fix duplicated translation
* Simplify tooltip
* Writing feedback
Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Jonathan Buttner <jonathan.buttner@elastic.co>
## Summary
Resolves (a portion of) https://github.com/elastic/kibana/issues/107068 for the `siem-detection-engine-rule-status` type by migrating the `alertId` to be within the `SO references[]`. Based on: https://github.com/elastic/kibana/pull/113577
* Migrates the legacy `siem-detection-engine-rule-status` `alertId` to saved object references array
* Adds an e2e test for `siem-detection-engine-rule-status`
* Breaks out `siem-detection-engine-rule-status` & `security-rule` SO's to their own dedicated files/directories, and cleaned up typings/imports
Before migration you can observe the existing data structure of `siem-detection-engine-rule-status` via Dev tools as follows:
```
GET .kibana/_search
{
"size": 10000,
"query": {
"term": {
"type": {
"value": "siem-detection-engine-rule-status"
}
}
}
}
```
``` JSON
{
"_index" : ".kibana-spong_8.0.0_001",
"_id" : "siem-detection-engine-rule-status:d580f1a0-2afe-11ec-8621-8d6bfcdfd75e",
"_score" : 2.150102,
"_source" : {
"siem-detection-engine-rule-status" : {
"alertId" : "d62d2980-27c4-11ec-92b0-f7b47106bb35", <-- alertId which we want in the references array and removed
"statusDate" : "2021-10-12T01:50:52.898Z",
"status" : "failed",
"lastFailureAt" : "2021-10-12T01:50:52.898Z",
"lastSuccessAt" : "2021-10-12T01:18:29.195Z",
"lastFailureMessage" : "6 minutes (385585ms) were not queried between this rule execution and the last execution, so signals may have been missed. Consider increasing your look behind time or adding more Kibana instances. name: \"I am the Host who Names!\" id: \"d62d2980-27c4-11ec-92b0-f7b47106bb35\" rule id: \"214ccef6-e98e-493a-98c5-5bcc2d497b79\" signals index: \".siem-signals-spong-default\"",
"lastSuccessMessage" : "succeeded",
"gap" : "6 minutes",
"lastLookBackDate" : "2021-10-07T23:43:27.961Z"
},
"type" : "siem-detection-engine-rule-status",
"references" : [ ],
"coreMigrationVersion" : "7.14.0",
"updated_at" : "2021-10-12T01:50:53.404Z"
}
}
```
Post migration the data structure should be updated as follows:
``` JSON
{
"_index": ".kibana-spong_8.0.0_001",
"_id": "siem-detection-engine-rule-status:d580f1a0-2afe-11ec-8621-8d6bfcdfd75e",
"_score": 2.1865466,
"_source": {
"siem-detection-engine-rule-status": {
"statusDate": "2021-10-12T01:50:52.898Z", <-- alertId is no more!
"status": "failed",
"lastFailureAt": "2021-10-12T01:50:52.898Z",
"lastSuccessAt": "2021-10-12T01:18:29.195Z",
"lastFailureMessage": "6 minutes (385585ms) were not queried between this rule execution and the last execution, so signals may have been missed. Consider increasing your look behind time or adding more Kibana instances. name: \"I am the Host who Names!\" id: \"d62d2980-27c4-11ec-92b0-f7b47106bb35\" rule id: \"214ccef6-e98e-493a-98c5-5bcc2d497b79\" signals index: \".siem-signals-spong-default\"",
"lastSuccessMessage": "succeeded",
"gap": "6 minutes",
"lastLookBackDate": "2021-10-07T23:43:27.961Z"
},
"type": "siem-detection-engine-rule-status",
"references": [
{
"id": "d62d2980-27c4-11ec-92b0-f7b47106bb35", <-- previous alertId has been converted to references[]
"type": "alert",
"name": "alert_0"
}
],
"migrationVersion": {
"siem-detection-engine-rule-status": "7.16.0"
},
"coreMigrationVersion": "8.0.0",
"updated_at": "2021-10-12T01:50:53.406Z"
}
},
```
#### Manual testing
---
There are e2e tests but for any manual testing or verification you can do the following:
##### Manual upgrade test
If you have a 7.15.0 system and can migrate it forward that is the most straight forward way to ensure this does migrate correctly. You should see that the `Rule Monitoring` table and Rule Details `Failure History` table continue to function without error.
##### Downgrade via script and test migration on kibana reboot
If you have a migrated `Rule Status SO` and want to test the migration, you can run the below script to downgrade the status SO then restart Kibana and observe the migration on startup.
Note: Since this PR removes the mapping, you would need to [update the SO mapping](https://github.com/elastic/kibana/pull/114585/files#r729386126) to include `alertId` again else you will receive a strict/dynamic mapping error.
```json
# Replace id w/ correct Rule Status SO id of existing migrated object
POST .kibana/_update/siem-detection-engine-rule-status:d580ca91-2afe-11ec-8621-8d6bfcdfd75e
{
"script" : {
"source": """
ctx._source.migrationVersion['siem-detection-engine-rule-status'] = "7.15.0";
ctx._source['siem-detection-engine-rule-status'].alertId = ctx._source.references[0].id;
ctx._source.references.remove(0);
""",
"lang": "painless"
}
}
```
Restart Kibana and now it should be migrated correctly and you shouldn't see any errors in your console. You should also see that the `Rule Monitoring` table and Rule Details `Failure History` table continue to function without error.
### Checklist
Delete any items that are not applicable to this PR.
- [ ] ~[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials~
- [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Alerting] Active alerts do not recover after re-enabling a rule
* created reusable lib file for generating event log object
* comment fix
* fixed tests
* fixed tests
* fixed typecheck
* fixed due to comments
* Apply suggestions from code review
Co-authored-by: ymao1 <ying.mao@elastic.co>
* fixed due to comments
* fixed due to comments
* fixed due to comments
* fixed tests
* Update disable.ts
* Update disable.ts
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: ymao1 <ying.mao@elastic.co>
## [Security Solution] Restores Alerts table local storage persistence and the Remove Column action
This PR implements the following changes summarized below to address <https://github.com/elastic/kibana/issues/113090>, as proposed [here](https://github.com/elastic/kibana/issues/113090#issuecomment-935143690):
- Configures the `Columns` popover to be consistent with `Discover`
- Changes the `Hide column` action to `Remove column`, to be consistent with `Discover`
- Persists updates to the `Columns` popover order in `local storage`
- Restores the feature to persist column widths in `local storage`
### Configures the `Columns` popover to be consistent with `Discover`
- We now pass `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid):
This makes all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`'s use of the `EuiDataGrid` `Columns` popover.
In `7.15`, the `Columns` popover includes the _hide column_ toggle, as shown in the screenshot below:
_Above: The `Columns` popover in the `7.15` `Alerts` table_
The `Columns` popover in `Discover`'s `EuiDataGrid`-based table does not display the hide column toggle, as shown the screenshot below:
_Above: The `EuiDataGrid` `Columns` popover in `Discover`, in `master`_
Passing `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API makes the `Columns` popover in all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`, as illustrated by the screenshot below:
_Above: The `Columns` popover is now consistent with `Discover`_
## Changes the `Hide column` action to `Remove column`, to be consistent with `Discover`
- The `Hide column` action shown in the `7.15` alerts table is changed to `Remove column`, making it consistent with `Discover`'s use of `EuiDataGrid`
In `7.15`, the `Alerts` table has a `Hide column` action, as shown in the screenshot below:
_Above: The `Hide Column` action in the `7.15` `Alerts` table_
In `7.15`, clicking the `Hide Column` action shown in the screenshot above hides the column, but does not remove it.
In `7.15`, columns may only be removed by un-checking them in the `Fields` browser, or by un-toggling them in the Alerts / Events details popover. Both of those methods require multiple clicks, and require uses to re-find the field in the modal or popover before it may be toggled for removal.
In `Discover`, users don't hide columns.
In `Discover`, users directly remove columns by clicking the `Remove column` action, shown in the screenshot below:
_Above: The `Remove column` action in `Discover`'s use of `EuiDataGrid` in `master`_
All `EuiDataGrid`-based views in the Security Solution were made consistent with `Discover` by replacing the `Hide column` action with `Remove column`, per the screenshot below:
_Above: The `Remove column` action in the Alerts table_
Note: the `Remove column` action shown above appears as the last item in the popover because it's specified via the `EuiDataGrid` `EuiDataGridColumnActions` > `additonal` API, which appends additonal actions to the end of popover, after the built-in actions:
## Persists updates to the `Columns` popover order in `local storage`
- Persist column order updates to `local storage` when users update the order of columns via the `Columns` popover
The following PR <https://github.com/elastic/kibana/pull/110685> restored partial support for persisting columns across page refreshes via `local storage`, but the Redux store was not updated when users sort columns via the `Columns` popover, an shown in the animated gif below:
_Above: Ordering via the `Columns` popover is not persisted to `local storage` in `7.15`_
This PR utilizes the `setVisibleColumns` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted, which will in-turn update `local storage` to persist the new order across page refreshes:
## Restores the feature to persist column widths in `local storage`
In previous releases, resized column widths were peristed in `local storage` to persist across page refreshes, as documented in <https://github.com/elastic/kibana/issues/110524> :
```
{
"detections-page":{
"id":"detections-page",
"activeTab":"query",
"prevActiveTab":"query",
"columns":[
{
"category":"base",
"columnHeaderType":"not-filtered",
"description":"Date/time when the event originated. This is the date/time extracted from the event, typically representing when the event was generated by the source. If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. Required field for all events.",
"example":"2016-05-23T08:05:34.853Z",
"id":"@timestamp",
"type":"date",
"aggregatable":true,
"width":190
},
{
"category":"cloud",
"columnHeaderType":"not-filtered",
"description":"The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.",
"example":"666777888999",
"id":"cloud.account.id",
"type":"string",
"aggregatable":true,
"width":180
},
{
"category":"cloud",
"columnHeaderType":"not-filtered",
"description":"Availability zone in which this host is running.",
"example":"us-east-1c",
"id":"cloud.availability_zone",
"type":"string",
"aggregatable":true,
"width":180
},
// ...
}
],
// ...
}
}
```
_Above: column widths were persisted to `local storage` in previous release, (going at least back to `7.12`)_
In this PR, we utilize the `onColumnResize` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted via the `Columns` popover. Updating Redux will in-turn update `local storage`, so resized columns widths will persist across page refreshes:
### Other changes
The Alerts page `Trend` chart and table were updated to include the following additional `Stack by` fields (CC @paulewing):
```
process.name
file.name
hash.sha256
```
per the before / after screenshots below:
_Above: The Alerts `Trend` Stack by fields in `7.15` (before)_
_Above: The Alerts `Trend` `Stack by` fields (after the addition of the `process.name`, `file.name`, and `hash.sha256` fields)_
CC: @monina-n @paulewing
## Summary
Removes one liner non-null-assert.
Instead of this line:
```ts
if (rule != null && spacesApi && outcome === 'conflict') {
```
We just check using the `?` operator and type narrowing to remove the possibility of an error
```ts
if (rule?.alias_target_id != null && spacesApi && rule.outcome === 'conflict') {
```
The `rule?.alias_target_id != null` ensures that both `rule` and `alias_target_id` are not `null/undefined`
## Summary
During the work here: https://github.com/elastic/kibana/pull/113577
I accidentally have introduced a bug where on migration I was deleting the attributes of `ruleThrottle` and `alertThrottle` because I was not using splat correctly.
Added unit and e2e tests to fix this.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Fixes:
* Resets happening by adding the throttle to the else switches and error catching. We have to call throttle on every rule execution or we will cause a reset.
* Fixes a case where we were not firing the signal immediately by pushing down the alerts detected. This can cause a reset or a delay of MTTD.
* Adds unit tests for the conditions
* Changes some of the logic to clean things up.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* Re-enable previously disabled APM E2E tests.
* Round to the nearest second in `getComparisonTypes` to avoid cases where a millisecond difference can change which results get shown.
* Simplify error count alert tests to test the "happy path" (#79284 exists in order to expand to more tests for rule editing and creation.)
* Wait for alert list API request to complete before clicking "Create rule" button when running the test to create a rule from the Stack Management UI.
I ran the e2e tests 100 times locally with no failures so I'm confident the flakiness has been addressed.
Fixes#114419.
Fixes#109205.
* Aded some data test subjects for the test.
* Added flush indices test.
* Fixed linting issue.
* Merged test subject PR in and updated tests.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* adds support for saved object based ml models
* adds es asset type and ml model install handler
* wip: handle top level pipeline install
* remove unnecessary mlModel savedObject type
* add package manifest license check
* get modelid from model path
* add fleet api test for ml model
* replace test mlModel for api test with smaller test model
* cleanup install/remove and ensure pipelines are retained when upgrading
* fix types - update test model id
* fix types
* remove hard coded ml category and check top level pipeline on upgrade
* update ml model test file
* ensure deduplicated asset refs are saved
* Fix api integration update test
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
* 🐛 Add padding to the tick label to fit threshold markers
* 🐛 Better icon detection
* 🐛 Fix edge cases with no title or labels
* 📸 Update snapshots
* ✨ Make threshold fit into view automatically
* 🐛 do not compute axis threshold extends if no threshold is present
* ✅ One more fix for 0-based extends and tests
* ✨ Add icon placement flag
* ✨ Sync padding computation with marker positioning
* ✨ compute the default threshold based on data bounds
* 🐛 fix duplicate suggestion issue + missing over time
* 👌 Make disabled when no icon is selected
* ✨ First text on marker implementation
* 🐛 Fix some edge cases with auto positioning
* Update x-pack/plugins/lens/public/xy_visualization/xy_config_panel/threshold_panel.tsx
Co-authored-by: Michael Marcialis <michael@marcial.is>
* 🐛 Fix minor details
* 💄 Small tweak
* ✨ Reduce the padding if no icon is shown on the axis
* 🐛 Fix color fallback for different type of layers
* ✅ Fix broken unit tests
* 🐛 Fix multi layer types issue
* ✅ Fix test
* ✅ Fix other test
* 💄 Fix vertical text centering
* ✨ Rename to reference lines + few fixes
* 🚨 Fix linting issue
* 🐛 Fix issue
* 🐛 Fix computation bug for the initial static value
* ✅ Add new suite of test for static value computation
* 💄 Reorder panel inputs
* 💄 Move styling to sass
* 📝 Keeping up with the renaming
* ✅ Fix functional tests after renaming
* 🐛 Fix duplicate arg from conflict resolution
* 👌 Integrate some follow up feedback
* 📝 Fix typo
* 👌 Integrate feedback
* 🐛 Fix the quick functions transition bug
* 🐛 Fix label issue when updating value
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Michael Marcialis <michael@marcial.is>
* Switch Dashboard to use savedobjects.resolve when loading
* Don't use LegacyURI Redirect if in screenshot mode
* Pass query string on redirects
* Remove unused import
* Fix carrying query params through redirect
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* show loading animation while retrieving policy data based on id
* Move the Not Found logic from the policy details form layout to the policy details page
* Fix unit test error caused by providing empty array to `rightSideItems` of `<AdministrationListPage>`
* Move tests to policy details from policy form layout
