## Summary
* Grabbed the ECS mappings from [v1.10.0 tag]( https://github.com/elastic/ecs/blob/v1.10.0/generated/elasticsearch/7/template.json)
* Updated the fields that had `constant_keyword` to `keyword` since we do many to 1 of source to signals index
* Wrote a unit tests which tests to ensure we don't have any `constant_keyword` fields
* Updated the `SIGNALS_TEMPLATE_VERSION` version by an increment of 10.
This should mostly fix:
https://github.com/elastic/kibana/issues/101572
Since agents add their data into `_source` even though they have a `constant_keyword`. When agents do not include the values in `_source` we will have to merge `fields` into `_source` before copying which are still planning on doing before release.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Adds runtime field tests with skips around the tests that we do not support just yet.
* Adds tests around corner cases involving ".", "..", ".foo", "foo.", etc...
* Adds tests around overriding values from the runtime fields within the source document
* Adds tests around ambiguity for when we override an array in a runtime field
* Fixes minor wording around previous tests
* Fixes one line string when we do testing in one area
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* bump @elastic/elasticsearch to canary.7
* address errors in core
* address errors in data plugin
* address errors in Alerting team plugins
* remove outdated messages in Lens
* remove unnecessary comments in ML
* address errors in Observability plugin
* address errors in reporting plugin
* address errors in Rule registry plugin
* fix errors in Security plugins
* fix errors in ES-UI plugin
* remove unnecessary union.
* update core tests
* fix kbn-es-archiver
* update to canary 8
* bump to v9
* use new typings
* fix new errors in core
* fix errors in core typeings
* fix type errors in data plugin
* fix type errors in telemetray plugin
* fix data plugin tests
* fix search examples type error
* fix errors in discover plugin
* fix errors in index_pattern_management
* fix type errors in vis_type_*
* fix errors in typings/elasticsearch
* fix type errors in actions plugin
* fix type errors in alerting and apm plugins
* fix type errors in canvas and cases
* fix errors in event_log
* fix type errors in ILM and ingest_pipelines
* fix errors in lens plugin
* fix errors in lists plugin
* fix errors in logstash
* fix errors in metrics_entities
* fix errors in o11y
* fix errors in watcher
* fix errors in uptime
* fix errors in upgrade_assistant
* fix errors in task_manager
* fix errors in stack_alerts
* fix errors in security_solution
* fix errors in rule_registry
* fix errors in snapshot_restore
* fix remaining errors
* fix search intergration tests
* adjust assetion
* bump version to canary.10
* adapt code to new naming schema
* use mapping types provided by the client library
* Revert "adjust assetion"
This reverts commit 19b8fe0464.
* fix so intergration tests
* fix http integration tests
* bump version to canary 11
* fix login test
* fix http integration test
* fix apm test
* update docs
* fixing some ml types
* fix new errors in data plugin
* fix new errors in alerting plugin
* fix new errors in lists plugin
* fix new errors in reporting
* fix or mute errors in rule_registry plugin
* more ML type fixes
* bump to canary 12
* fix errors after merge conflict
* additional ML fixes
* bump to canary 13
* fix errors in apm plugin
* fix errors in fleet plugin
* fix errors in infra plugin
* fix errors in monitoring plugin
* fix errors in osquery plugin
* fix errors in security solution plugins
* fix errors in transform plugin
* Update type imports for ES
* fix errors in x-pack plugins
* fix errors in tests
* update docs
* fix errors in x-pack/test
* update error description
* fix errors after master merge
* update comment in infra plugin
* fix new errors on xpack tests/
Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
## Summary
Test cases for signals and aliases, including a failure of where we do not copy alias data at the moment even if the target is an ECS compatible field.
For example with this mapping:
```json
{
"dynamic": "strict",
"properties": {
"@timestamp": {
"type": "date"
},
"host": {
"properties": {
"name": {
"type": "alias",
"path": "host_alias.name"
}
}
},
"host_alias": {
"properties": {
"name": {
"type": "keyword"
}
}
}
}
}
```
If we detect this as a signal hit we should be copying over both:
* `host_alias.name` -> `host.name`
* `host_alias.name` -> `host_alias.name`
to the target signal index, but we only copy:
* `host_alias.name` -> `host_alias.name`
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
We have a few bugs where when the source index for detections is not `"strict_date_optional_time"` it is possible that we will misinterpret the format to be epoch milliseconds when it could be epoch seconds or another ambiguous format or blow up when trying to write out the signals index. This fixes it to where we query for the source index format as an ISO8601 and when we copy the date time format we copy it back out as ISO8601 and insert it into the signal index as ISO8601.
See this [gist](https://gist.github.com/FrankHassanabad/f614ec9762d59cd1129b3269f5bae41c) for more details of how this was accidentally introduced when we added support for runtime fields and the general idea of the fix.
* Removes `docvalue_field` and we now only use `fields` in detection engine search requests
* Splits out the timestamp e2e tests into their own file for `timestamps` file
* Adds more tests to ensure we copy what we expect and we are converting to ISO8601 in the signals
* Removes `ts-expect-error` in a lot of areas including tests and then I fix the types and issues once it is removed.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Adds e2e tests for the `constant_keyword` regular `keyword` to compare between the two. Bugs found with these is one where we do not copy `constant_keyword` fields into signals which I added `.skip` to the tests now.
Tested these rule types:
* KQL
* EQL
* Threshold
For the mappings of the `constant_keyword` I use both the `constant_keyword` and the field `alias` like so:
```json
{
"properties": {
"@timestamp": {
"type": "date"
},
"data_stream": {
"properties": {
"dataset": {
"type": "constant_keyword",
"value": "dataset_name_1"
},
"module": {
"type": "constant_keyword",
"value": "module_name_1"
}
}
},
"event": {
"properties": {
"category": {
"type": "keyword"
},
"dataset": {
"type": "alias",
"path": "data_stream.dataset"
},
"module": {
"type": "alias",
"path": "data_stream.module"
}
}
}
}
}
```
To ensure we can detect against fields. I also mix them with regular const keyword fields in another index to ensure they work also in mixed use cases.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Utilizes constants package and deletes duplicate code
* Renames the `securitysolution-constants` to be `securitysolution-list-constants` to be specific
* Deletes duplicated code found during cleanup
* Moves more tests into the packages found along the way with the duplicated code
* Moves `parseScheduleDates` from `@kbn/securitysolution-io-ts-types` to `@kbn/securitysolution-io-ts-utils`
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
* Creates a `securitysolution-list-utils` packaged and moves the first set of utilities into there
* Fixes a slight bug with `kbn-securitysolution-io-ts-list-types` where the wrong name was used
* Moves _all_ of the lists schemas and types into the package `kbn-securitysolution-io-ts-list-types`
* Removes copied code found in a few places
## Tech debt
* Some spots I have to use an `any` in the package as Kibana kbn packages don't have the types I need
* Some spots I copy constants until we can straighten out those pieces.
* I keep copied mock files until we figure out how to share mocks from these packages without adding weight or we create dedicated mock packages for all of this.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Re-adds a test that was skipped. If it goes bonkers again, I will add more debugging information to it. I will keep an eye on the operations channel to see when/if this fails again. Originally this looked to be timeouts waiting, so I increased the global timeout to be 20 seconds instead of the original 10 seconds.
Resolves:
https://github.com/elastic/kibana/issues/89389
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* merge multiple timestamp queries into one single search
* fix types and unit tests
* remove unused code for sending secondary search
* removes unused excludeDocsWithTimestampOverride
* adds integration tests to cover cases that should / should not generate signals when timestamp override is present in rule
* adds integration test to ensure unmapped sort fields do not break search after functionality of detection rules
* Need to figure out why moving the tests around fixed them...
* updates tests with new es archive data and fixes bug where exclusion filter was hardcoded to event.ingested :yikes:
* remove dead commented out code
* fixes typo in test file, removes redundant delete signals call in integration test, fixes logic for possibility of receving a null value in sort ids, removes unused utility function for checking valid sort ids
* a unit test for checking if an empty string of a sort id is present was failing because we moved the logic for checking that out of the build search query function and up into the big loop. So I moved that unit test into the search after bulk create test file.
* fix types
* removes isEmpty since it doesn't check for empty strings
* Add API integration tests for threshold rules and more tests for EQL rules
* Add API more tests for exceptions and value list exceptions
* Fix unit test and add EQL api test checking multiple signal generation
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Adds helper to normalize legacy ML rule field to an array
This will be used on read of rules, to normalize legacy rules while
avoiding an explicit migration.
* Fix our detection-specific ML search function
Luckily this was just a translation layer to our anomaly call, and the
underlying functions already accepted an array of strings.
* WIP: Run rules against multiple ML Job IDs
We don't yet support creation of rules with multiple job ids, either on
the API or the UI, but when we do they will work.
Note: the logic was previously to generate an error if the underlying
job was not running, but to still query and generate alerts. Extending
that logic to multiple jobs: if any are not running, we generate an
error but continue querying and generating alerts.
* WIP: updating ml rule schemas to support multiple job IDs
* Simplify normalization method
We don't care about null or empty string values here; those were
holdovers from copying the logic of normalizeThreshold and don't apply
to this situation.
* Move normalized types to separate file to fix circular dependency
Our use of NonEmptyArray within common/schemas seemed to be causing the
above; this fixes it for now.
* Normalize ML job_ids param at the API layer
Previous changes to the base types already covered the majority of
routes; this updates the miscellaneous helpers that don't leverage those
shared utilities.
At the DB level, the forthcoming migration will ensure that we always
have "normalized" job IDs as an array.
* Count stopped ML Jobs as partial failure during ML Rule execution
Since we continue to query anomalies and potentially generate alerts, a
"failure" status is no longer the most accurate for this situation.
* Update 7.13 alerts migration to allow multi-job ML Rules
This ensures that we can assume string[] for this field during rule
execution.
* Display N job statuses on rule details
* WIP: converts MLJobSelect to a multiselect
Unfortunately, the SuperSelect does not allow multiselect so we need to
convert this to a combobox. Luckily we can reuse most of the code here
and remain relatively clean.
Since all combobox options must be the same (fixed) height, we're
somewhat more limited than before for displaying the rows. The
truncation appears fine, but I need to figure out a way to display the
full description as well.
* Update client-side logic to handle an array of ML job_ids
* Marginally more legible error message
* Conditionally call our normalize helper only if we have a value
This fixes a type error where TS could not infer that the return value
would not be undefined despite knowing that the argument was never
undefined. I tried some fancy conditional generic types, but that didn't
work.
This is more analogous to normalizeThresholdObject now, anyway.
* Fix remaining type error
* Clean up our ML executor tests with existing contract mocks
* Update ML Executor tests with new logic
We now record a partial failure instead of an error.
* Add and update tests for new ML normalization logic
* Add and update integration tests for ML Rules
Ensures that dealing with legacy job formats continues to work in the
API.
* Fix a type error
These params can no longer be strings.
* Update ML cypress test to create a rule with 2 ML jobs
If we can create a rule with 2 jobs, we should also be able to create a
rule with 1 job.
* Remove unused constant
* Persist a partial failure message written by a rule executor
We added the result.warning field as a way to indicate that a partial
failure was written to the rule, but neglected to account for that in the
main rule execution code, which caused a success status to immediately
overwrite the partial failure if the rule execution did not otherwise
fail/short-circuit.
* Use client from branch
* Get type checking working in core
* Fix types in other plugins
* Update client types + remove type errors from core
* migrate Task Manager Elasticsearch typing from legacy library to client library
* use SortOrder instead o string in alerts
* Update client types + fix core type issues
* fix maps ts errors
* Update Lens types
* Convert Search Profiler body from a string to an object to conform to SearchRequest type.
* Fix SOT types
* Fix/mute Security/Spaces plugins type errors.
* Fix bootstrap types
* Fix painless_lab
* corrected es typing in Event Log
* Use new types from client for inferred search responses
* Latest type defs
* Integrate latest type defs for APM/UX
* fix core errors
* fix telemetry errors
* fix canvas errors
* fix data_enhanced errors
* fix event_log errors
* mute lens errors
* fix or mute maps errors
* fix reporting errors
* fix security errors
* mute errors in task_manager
* fix errors in telemetry_collection_xpack
* fix errors in data plugins
* fix errors in alerts
* mute errors in index_management
* fix task_manager errors
* mute or fix lens errors
* fix upgrade_assistant errors
* fix or mute errors in index_lifecycle_management
* fix discover errors
* fix core tests
* ML changes
* fix core type errors
* mute error in kbn-es-archiver
* fix error in data plugin
* fix error in telemetry plugin
* fix error in discover
* fix discover errors
* fix errors in task_manager
* fix security errors
* fix wrong conflict resolution
* address errors with upstream code
* update deps to the last commit
* remove outdated comments
* fix core errors
* fix errors after update
* adding more expect errors to ML
* pull the lastest changes
* fix core errors
* fix errors in infra plugin
* fix errors in uptime plugin
* fix errors in ml
* fix errors in xpack telemetry
* fix or mute errors in transform
* fix errors in upgrade assistant
* fix or mute fleet errors
* start fixing apm errors
* fix errors in osquery
* fix telemetry tests
* core cleanup
* fix asMutableArray imports
* cleanup
* data_enhanced cleanup
* cleanup events_log
* cleaup
* fix error in kbn-es-archiver
* fix errors in kbn-es-archiver
* fix errors in kbn-es-archiver
* fix ES typings for Hit
* fix SO
* fix actions plugin
* fix fleet
* fix maps
* fix stack_alerts
* fix eslint problems
* fix event_log unit tests
* fix failures in data_enhanced tests
* fix test failure in kbn-es-archiver
* fix test failures in index_pattern_management
* fixing ML test
* remove outdated comment in kbn-es-archiver
* fix error type in ml
* fix eslint errors in osquery plugin
* fix runtime error in infra plugin
* revert changes to event_log cluser exist check
* fix eslint error in osquery
* fixing ML endpoint argument types
* fx types
* Update api-extractor docs
* attempt fix for ese test
* Fix lint error
* Fix types for ts refs
* Fix data_enhanced unit test
* fix lens types
* generate docs
* Fix a number of type issues in monitoring and ml
* fix triggers_actions_ui
* Fix ILM functional test
* Put search.d.ts typings back
* fix data plugin
* Update typings in typings/elasticsearch
* Update snapshots
* mute errors in task_manager
* mute fleet errors
* lens. remove unnecessary ts-expect-errors
* fix errors in stack_alerts
* mute errors in osquery
* fix errors in security_solution
* fix errors in lists
* fix errors in cases
* mute errors in search_examples
* use KibanaClient to enforce promise-based API
* fix errors in test/ folder
* update comment
* fix errors in x-pack/test folder
* fix errors in ml plugin
* fix optional fields in ml api_integartoon tests
* fix another casting problem in ml tests
* fix another ml test failure
* fix fleet problem after conflict resolution
* rollback changes in security_solution. trying to fix test
* Update type for discover rows
* uncomment runtime_mappings as its outdated
* address comments from Wylie
* remove eslint error due to any
* mute error due to incompatibility
* Apply suggestions from code review
Co-authored-by: John Schulz <github.com@jfsiii.org>
* fix type error in lens tests
* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* update deps
* fix errors in core types
* fix errors for the new elastic/elasticsearch version
* remove unused type
* remove unnecessary manual type cast and put optional chaining back
* ML: mute Datafeed is missing indices_options
* Apply suggestions from code review
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
* use canary pacakge instead of git commit
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Nathan Reese <reese.nathan@gmail.com>
Co-authored-by: Wylie Conlon <wylieconlon@gmail.com>
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
Co-authored-by: restrry <restrry@gmail.com>
Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: John Schulz <github.com@jfsiii.org>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
## Summary
Fixes and unskips tests. Also makes the tests less picky. Although I enjoy seeing the changes on ES promotions of error messages in case we see something really bad happening, it's too manual of a process to fix and puts too much of a burden on operations to skip the error messages.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* Make indicator enrichment tests order-independent
Due to the fact that we use named queries to determine matches, and the
fact that the order in which named queries are returned is undefined, we
cannot guarantee a consistent ordering of enrichments if a given event
matches multiple named queries.
Because the ordering is not in itself important to enrichment, in order
to assert the multi-match functionality we must make the assertions
order independent.
* PR feedback
* Since we're only looping for side effects, prefer forEach to map for
more idiomatic FP.
* Retrieve SOs by ID in a space-aware manner by using bulkGet
We were previously using a manual invocation of find(), which was a)
tied to the current implementation of how SOs generate their _ids, and
b) didn't respect spaces.
By replacing this with a call to bulkGet, which automatically respects
the space of the current request, and which abstracts away the building
of the actual _id based on the SO ID and type, we address the issues
above.
* Surface SO errors to the finalize/delete APIs
Now that we're using bulkGet, we receive an object with errors if the
object is not found, which by default breaks our subsequent validation.
In order to provider better UX, we re-raise the first of these errors
that we find, if present, and return that to the user.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* fix validation on response of find status route when rule has a partial failure status
* replaces warning in rule status service with partial failure to maintain backwards compatibility from an API standpoint, also displays 'warning' on UI if a rule's status is partial failure
* display partial failure as 'warning' on all rules table and update e2e test to check for partial failure not warning
* add util function, show 'warning' on monitoring table, fix e2e tests
The threat intel data that we are loading for these tests was previously
copy/pasted, resulting in all docs having the same @timestamp value.
This resulted in nondeterministic ordering which manifested as test
failures.
This fixes the issue by giving a unique timestamp to each indicator
document, and then unskips/updates tests according to this new ordering.
* Update copy of rule config
* Encode threat index as part of our named query
* Add index to named query, and enrich both id and index
We still need mappings and to fix integration tests, but this generates
the correct data.
* Update integration tests with new enrichment fields
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Distinguish source and destination config for indicator matches
We were previously conflating the path to retrieve indicator fields with
the path to persist indicator fields, since they were the same value.
To reduce friction in use with the new filebeat modules, we've decided
to make the default source path threatintel.indicator. However, we still
want to persist to threat.indicator, so we add a new constant, here.
* Update our integration tests following change of default
These tests were assuming a default path of threat.indicator. Since that
is the ECS standard, we're not going to rewrite the tests but instead
just add this rule override. In the future if the default changes, this
parameter might be unnecessary.
* DRY up unit tests a bit
* Add a note for future devs
If/when that constant changes, I imagine this will be useful context.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* fix for when search response yields 400 with missing timestamp override field
* prefer includes over strict equality
* adds integration test to check for this case
* adds a unit test and util function to ensure unit test executes properly and waits for rule to complete running
* remove comments from rebase
* removes usage of 'partial failure' status and replaces with a 'warning' status, also adds some logic to be backwards compatible with 'partial failure' statuses
* update integration tests from 'partial failure' to 'warning'
* fix integration test to warn and not error when no index patterns match concrete indices
* fix integration test
* removes outdated comments from the create_rules e2e test
* Adds basic integration test for threat enrichment
* Update signals mappings with indicator fields
* Simplify some ternaries with Math.min
* Remove outdated comments
* Add notes from walkthrough with devin
* Add an enrichment hook to the current signal creation pipeline
When this moves to individual rule-specific data transformations this
will be a little more explicit/configurable; for now to keep changes
minimal, we're using dependency injection to pass a function, which will
default to the identity function (e.g. a no-op).
* Add utility functions for encoding/decoding our threat query
This is what allows us to enrich the threat match signals using only the
signal search response.
* Add a name to each threat match filter clause
This gives us the information we need to enrich our signals after
they've been queried without having to perform a complicated reverse
query.
* Adds functions for signal enrichment of threat indicators
* Wire up threat enrichment to threat match rules
* Fleshes out threat match integration tests
Adds assertions to the existing test, and fleshes out another test for a
multi-match signal.
* Add more test cases to indicator match integration tests
* single indicator matching multiple events
* multiple indicators matching a single event
* multiple indicators, multiple events
* placeholder for deduplication logic
This also adds some descriptions to our threat intel documents, to give
a little context around how they're meant to function within the tests,
particularly as relates to the auditbeat/hosts data on which it is meant
to function.
* Implement signal deduplification
This handles the situation where the indicator match search has returned
the same signal multiple times due to the source event matching
different indicators in different query batches. In this case, we want
to generate a single signal with all matched indicators.
* Move default indicator path to constant
* Testing some edge cases with signal enrichment
* Cover and test edge cases with threat enrichment generation
* Fix logical error in TI enrichment
We were previously adding the indicator's field to matched.field,
instead of the corresponding event field that matched the indicator.
In the normal case, the expectation is that the indicator field is
self-evident, and thus we want to know the other side of the match on
the event itself.
Updates tests accordingly.
* Document behavior when an indicator matched but is absent on enrichment
This could occur if the indicator index is updated while a rule is being
run.
* Add followup note
* Add basic unit test for our enrichment function
This just verifies that the enrichment function gets invoked with search
results.
* Update license headers for new files
* Remove unused threatintel archive
I made both of these before we were clear on the direction we were
taking here.
* Bump signals version to allows some updates in patch releases
* Fix typings of threat list item
We were conflating the type of the underlying document with the type of
the search response for that document. This is now addressed with two
types: ThreatListDoc and ThreatListItem, respectively.
ThreatListDoc isn't the most distinguishing name but it avoids a lot of
unnecessary renaming for the existing concept of ThreatListItem.
* Update test mock to be aware of (but not care about) named queries
* Remove/update outdated comments
This code was modified to perform two searches instead of one; at that
time, a lot of this code was duplicated and modified slightly, and these
misleading comments were a result. I removed the ones that were no
longer relevant, but left a TODO for one that could be a bug.
* Remove outdated comment
Documents will always have _id.
* Update enriched signals' total to account for deduplication
If a given signal matched on multiple indicators in different loops of
our indicator query, it may appear multiple times. Our enrichment
performs the merging of those duplicated results, but did not previously
update the response's total field to account for this.
I don't believe that anything downstream is actually using this field and that we
are instead operating on the length of hits and the response from the
bulk create request, but this keeps things consistent in case that
changes.
* Remove development comments
* Add JSDoc for our special template version constant
* Remove outdated comments
* Add an additional test permutation for error cases
Ensure that we throw an error if the indicator field is either a
primitive or an array of primitives.
* Remove unnecessary coalescing
These values are already defaulted in the parent, and the types are
correct in that these cannot be undefined.
* Move logic to build threat enrichment function into helper
* Refactor code to allow typescript to infer our type narrowing
existingSignalHit could not be undefined on line 30 here, but typescript
could not infer this from the !acc.has() call.
* Use a POJO over a Map
We were using a map previously in order to use .has() for a predicate,
but code has since been refactored to make that unnecessary.
* Explicitly type our enriched signals
These are being typed implicitly and verified against SignalSourceHit[]
on the assignment below, but this makes the types explicit and surfaces
a type error here instead of the subsequent assignment.
* Add an explanatory note about these test results
* Remove unused imports
These references were moved into buildThreatEnrichment
* Remove threat mappings accidentally brought in with indicator work
I copied the entirety of the `threat` mappings in order to get the
`threat.indicator` ones, but it looks like these were added at some
point too.
I'd rather these not be added incidentally. If we need them, we should
do so explicitly.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Add's the list plugins Saved Objects (`exception-list` and `exception-list-agnostic`) to the `Security` feature privilege.
Resolves https://github.com/elastic/kibana/issues/90715
### Test Instructions
Load pre-packaged roles/users, and ensure only those with the Kibana Space privilege `Security:All` have the ability to create/edit rules and exception lists (space-aware/agnostic). Users with `Security:Read` should only be able to view rules/exception lists. Pre-packaged security roles should no longer be granted the `Saved Objects Management` feature privilege, and this feature privilege should no longer be required to use any of the Detections features.
To add test users:
t1_analyst (`"siem": ["read"]`):
``` bash
cd x-pack/plugins/security_solution/server/lib/detection_engine/scripts/
./roles_users/t1_analyst/post_detections_role.sh roles_users/t1_analyst/detections_role.json
./roles_users/t1_analyst/post_detections_user.sh roles_users/t1_analyst/detections_user.json
```
hunter (`"siem": ["all"]`):
``` bash
cd x-pack/plugins/security_solution/server/lib/detection_engine/scripts/
./roles_users/t1_analyst/post_detections_role.sh roles_users/hunter/detections_role.json
./roles_users/t1_analyst/post_detections_user.sh roles_users/hunter/detections_user.json
```
Note: Be sure to remove these users after testing if using a public cluster.
### Checklist
Delete any items that are not applicable to this PR.
- [X] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials -- `docs` label added, will work with @jmikell821 on doc changes
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Adds e2e tests for https://github.com/elastic/kibana/pull/90326
* Adds e2 tests and backfills for updating actions and expected behaviors
* Adds two tests that would fail without the fix and if a regression happens this will trigger on the regression
* Adds two tests to the PATCH for exception lists even though there is no regression there. Reason is to prevent an accidental issue there.
* Adds tests to ensure the version number does not accidentally get bumped if PATCH or UPDATE is called on actions or exceptions for immutable rules.
* Adds utilities for cutting down noise.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
