* [NoDataPage] Updating the default messages and reusing `title` as default `button` label
* Update default description for agent
* Overview page i18n and snaps
* use rac alerts bulk_update
* cleanup
* adds replace ALERT_STATUS with ALERT_WORKFLOW_STATUS and updates tests and adds logic for switching between signal.status and workflow status when updating alerts in .siem-signals
* allow object and string types in query param, fixed single update api to use WORKFLOW_STATUS instead of ALERT_STATUS
* adds additional integration test for when query is a DSL object in addtion to KQL string
* optionally use fields api in requests if _source does not contain authz properties
* integrate bulk update to all hook calls
* adds fields support, fixes bug where we were writing to 'signals.status' and not { signals: {status }} in alerts client
* clean up and fixes
* fix a bug where we were not waiting for updates to complete when using ids param in alerts bulk update. Adds integration tests for detection engine testing update alerts with new alerts as data client routes
* take index name from ecsData props
* pr suggestions
* some more type fixes
* refactor and type fixes
* snapshot updated
* add status update actions to row context menu
* refactor to use dispatch function in o11y actions
* comment removed
* bring alertConsumer back
* bring indexNames back
* check capabilities to show status update items
Co-authored-by: Devin Hurley <devin.hurley@elastic.co>
## Summary
This is part 2 to addressing the issue seen here: #101975 (Part 1 #107064)
This adds the alerting migration scripts and unit tests for exception list containers on Kibana startup for `7.15.0`
We only migrate if we find these conditions and cases:
- `exceptionLists` are an `array` and not `null`, `undefined`, or malformed data.
- The exceptionList item is an `object` and its `id` is a `string` and not `null`, `undefined`, or malformed data
- The existing references do not already have an exceptionItem reference already found within it.
We migrate on the common use case
- The saved object references do not exist but we have exceptionList items with the id's to create the saved object references, 👍 so we migrate.
- The alert contains no exception list items, in which case we have nothing to migrate
We do these additional (mis-use) cases and steps as well. These should _NOT_ be common things that happen but we safe guard for them here:
- If the migration is run twice we are idempotent and do _NOT_ add duplicates list items or remove items.
- If the migration was partially successful but re-run a second time, we only add what is missing. Again no duplicates or removed items should occur.
- If the `exceptionLists` contains invalid data shape or not enough information to migrate, we filter it out and ignore it
- If the saved object references already exists and contains a different or foreign value, we will retain the foreign reference(s) and still migrate.
## Manual testing
There are unit tests but for any manual testing or verification you can do the following:
Create a few alerts through the `security_solution` application with exception lists
<img width="1775" alt="Screen Shot 2021-08-11 at 5 42 31 PM" src="https://user-images.githubusercontent.com/1151048/129117377-61b17fcf-ad01-4405-bbfe-42d97a6f7654.png">
Use the dev tools to de-migrate as well as to test end to end like so:
```json
# First get an "_id" with an exceptions list like below. Mine I found was: "alert:38482620-ef1b-11eb-ad71-7de7959be71c":
GET .kibana/_search
{
"query": {
"terms": {
"alert.alertTypeId": [
"siem.signals"
]
}
},
"size": 10000
}
```
With Kibana running downgrade and remove the references as a test:
```json
# Set saved object array references as empty arrays and set our migration version to be 7.14.0
POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c
{
"script" : {
"source": """
ctx._source.migrationVersion.alert = "7.14.0";
ctx._source.references = []
""",
"lang": "painless"
}
}
# Double check the references is empty and the version is 7.14.0
GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c
```
Reload the alert in the `security_solution` and notice you get these errors until you restart Kibana to cause a migration moving forward
```sh
server log [17:35:16.914] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"endpoint_list","namespace_type":"agnostic","id":"endpoint_list","type":"endpoint"}
server log [17:35:16.914] [error][plugins][securitySolution] Cannot get a saved object reference using an index which is larger than the saved object references. Index is:1 which is larger than the savedObjectReferences:[]
server log [17:35:16.915] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"cd152d0d-3590-4a45-a478-eed04da7936b","namespace_type":"single","id":"50e3bd70-ef1b-11eb-ad71-7de7959be71c","type":"detection"}
server log [17:35:16.940] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"endpoint_list","namespace_type":"agnostic","id":"endpoint_list","type":"endpoint"}
server log [17:35:16.940] [error][plugins][securitySolution] Cannot get a saved object reference using an index which is larger than the saved object references. Index is:1 which is larger than the savedObjectReferences:[]
server log [17:35:16.940] [error][plugins][securitySolution] The saved object references were not found for our exception list when we were expecting to find it. Kibana migrations might not have run correctly or someone might have removed the saved object references manually. Returning the last known good exception list id which might not work. exceptionItem with its id being returned is: {"list_id":"cd152d0d-3590-4a45-a478-eed04da7936b","namespace_type":"single","id":"50e3bd70-ef1b-11eb-ad71-7de7959be71c","type":"detection"}
```
Restart Kibana and you should no longer have errors in the Kibana console.
If you do this query in dev tools
```json
# Check that the `migrationVersion` is `7.15.0` and that we have a `references` array filled out with the correct structure
GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c
```
You should notice that you now have a `references` array filled out:
```json
"references" :
[
{
"name" : "param:exceptionsList_0",
"id" : "endpoint_list",
"type" : "exception-list-agnostic"
},
{
"name" : "param:exceptionsList_1",
"id" : "50e3bd70-ef1b-11eb-ad71-7de7959be71c",
"type" : "exception-list"
}
],
"migrationVersion" : {
"alert" : "7.15.0"
}
```
For testing [idempotentence](https://en.wikipedia.org/wiki/Idempotence)
Run just this to downgrade and restart Kibana and you should notice on a GET that we do not have anything extra in the references array:
```json
# Set our migration version to be 7.14.0 only
POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c
{
"script" : {
"source": """
ctx._source.migrationVersion.alert = "7.14.0";
""",
"lang": "painless"
}
}
# Double check the `references` is still there, and we do not get errors or changes to `references` after we restart Kibana
GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c
```
For testing foreign keys:
```json
# Set saved object array references to foreign keys and set our migration version to be 7.14.0
POST .kibana/_update/alert:38482620-ef1b-11eb-ad71-7de7959be71c
{
"script" : {
"source": """
ctx._source.migrationVersion.alert = "7.14.0";
ctx._source.references = [["name" : "foreign", "id" : "123", "type" : "some-type"]];
""",
"lang": "painless"
}
}
```
Restart, ensure no errors in Kibana console and do a get call to ensure we have the foreign mixed with valid values:
```json
GET .kibana/_doc/alert:38482620-ef1b-11eb-ad71-7de7959be71c
```
Should return this data:
```json
"type" : "alert",
"references" :
[
{
"name" : "foreign",
"id" : "123",
"type" : "some-type"
},
{
"name" : "param:exceptionsList_0",
"id" : "endpoint_list",
"type" : "exception-list-agnostic"
},
{
"name" : "param:exceptionsList_1",
"id" : "50e3bd70-ef1b-11eb-ad71-7de7959be71c",
"type" : "exception-list"
}
]
```
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* restore legacy shim
Restore the legacy shim for PDF job creation from the 7.x branch.
* strip relativeUrls from pdf payload
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
- Add a `fixedHeight` prop so tables in panels keep pagination at the bottom but others do not
### Backends inventory
- Remove title from dependency table
- Remove service map link
- Add spacer between the header controls (date picker etc.) and the table for more spacing
### Service dependencies overview
- Don't show dependencies link on table
- Increase spacer size between chart and table from `m` to `l`
### Service page
- Hide dependencies tab for RUM and iOS agents (fixes#108510)
### Backends detail page
- Remove service map link from from dependencies table
- Increase spacer size between charts and table from `m` to `l`
Fixes#108495.
* [ML] Adding reset jobs link to jobs list
* fixing types
* updating types
* improving react code
* adding closed job warning callout
* small code changes after review
* updating comment for api docs
* adding canResetJob to security's emptyMlCapabilities
* updating apidoc
* adding blocked to job summary
* udating test
* adding delayed refresh back in
* updating tests
* adding better reverting controls and labels
* fixing bug in delete modal
* updating job task polling for all blocking tasks
* fixing types after es client update
* one other type correction
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Update signals mappings to include ECS 1.11
* Ensures no constant_keyword mappings
* Bumps index version by 1, since it was already bumped by 10 for 7.15
in #106049
* Remove threat.indicator mappings from signals indices
Until the old, 7.14 enrichment mappings (which define threat.indicator
as nested) are in our rearview, we cannot add the official, non-nested
threat.indicator mappings as they'll conflict.
* Use correct url to management app for observability cases, use normalized ids in timelines
* Update failing test
* Load alert details data to render flyout in case detail view
* Improve timeline title and move description to the notes tab
Truncate the title only in the UI
When the user hover the title we display the full title
Truncate the title if it appears in a table
- Moves APM Latency Correlations from flyout to transactions page.
- Introduces a tab based navigation for `Trace samples` / `Latency correlations` / `Failed transactions correlations`
- For trace samples, the previous low detailed histogram chart gets replaced with the new log log based histogram chart. Users can drag select a range of lantency to filter trace samples.
- Removes code related to the previous distribution chart. Renames `useTransactionDistributionFetcher` to `useTransactionTraceSamplesFetcher`. Instead of a histogram with top hits, this will now just return a sample of traces for the given filters.
* Make indexPattern only read only in Discover permission
* Fix test failures
* Address review comments
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* add alert consumers for useTimelineEventDetails
* set entityType to events
* rename to AlertConsumers
* set entityType to alerts
* send entity type to search strategy
* fix import
* fix import
* fix import
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Add styling to euiPanel
* remove styling on Observability side in favor of separate PR
* Fix merge conflicts
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Decouple job error fetching from the content stream
* Encapsulate content encoding and decoding in the content stream
* Move report size calculation from task runners
* Remove configuration check from the reporting diagnostics
* Add support of chunked export
