### Summary
This PR is a follow-up to #89066 - which fixed the same issue occurring with indicator match lists UI. The lack of stable ids for exception item entries resulted in some funky business by where invalid values could overwrite other values when deleting entries in the builder.
Doing a quick refactor to help with #90634 . While working on #90634 found that I would introduce a circular dependency if I didn't refactor the hook used by the exception builder component to make use of the existing useApi hook in the lists plugin.
#90634 adds temporary ids to item entries to mitigate some React key requirements and the logic to remove/add these id's isn't at the boundary but in the hooks (so as not to pollute the data for everyone wanting to use the api.
An upside is that it removed some of the looping and seemed to speed things up a bit. I briefly considered adding the bulk SO endpoints for exception items, but they are still experimental.
Addresses #88450
Issue
Search was not working as expected was because the exception list property name is mapped as a keyword - this means it does not get tokenized which is why one word searches were working but if the name included multiple words and was partial, it was not filtering properly.
* Initial version of adding artifacts per policy to the manifest.
* Minor renaming to convey the purpose of the variable.
* Added ability to override list item mock data.
* Changed function signature to be more reusable.
* Implementationg of support of artifacts per policy in the manifest data structure.
* Added saved objects migrations.
* Renamed the endpoint to reflect that it's artifacts endpoint.
* Fixed tests.
* Fixed the manifest data.
* Fixed linting errors (result of merge).
* Updated ES mappings for manifest in all test setups.
* Updated hash in the mappings.
* Fixed the typo that lead to failing test.
* Fixed the problem with manifest not being dispatched to policies if there are same artifact names but different content. Artifact name in the ManifestSchema is not unique id, hence added decoded_sha256 to the comparison. Added test case to cover this.
* Fixed the problem with the task flow when failure to dispatch to policies will result in commited manifest and no redispatch on next task run. Changed tests to reflect new flow (actually restored previous flow).
* Forgot to commit changes in mock.
* Made other tests more readable using same varialbe naming pattern.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* first pass
* migrate more plugins
* migrate yet more plugins
* more oss plugins
* fix test file
* change Plugin signature on the client-side too
* fix test types
* migrate OSS client-side plugins
* migrate OSS client-side test plugins
* migrate xpack client-side plugins
* revert fix attempt on fleet plugin
* fix presentation start signature
* fix yet another signature
* add warnings for server-side async plugins in dev mode
* remove unused import
* fix isPromise
* Add client-side deprecations
* update migration examples
* update generated doc
* fix xpack unit tests
* nit
* (will be reverted) explicitly await for license to be ready in the auth hook
* Revert "(will be reverted) explicitly await for license to be ready in the auth hook"
This reverts commit fdf73feb
* restore await on on promise contracts
* Revert "(will be reverted) explicitly await for license to be ready in the auth hook"
This reverts commit fdf73feb
* Revert "restore await on on promise contracts"
This reverts commit c5f2fe51
* add delay before starting tests in FTR
* update deprecation ts doc
* add explicit contract for monitoring setup
* migrate monitoring plugin to sync
* change plugin timeout to 10sec
* use delay instead of silence
## Summary
Addresses issue issue 87110
**Issue**
When prepackaged rules were created during the Endpoint Security enrollment flow, the endpoint exceptions list was failing to be created. As a result, when a user navigated to the `Endpoint Security` rule to add an exception it would display errors and not allow a user to add exceptions.
* remove canUserCRUD from signal actions and remove refresh param from open_close_signals route. 'refresh' requires maintenance / manage / all privileges for signals index
* adds 'maintenance' to privileges route
* fix unit teset typing
* update tests, updated lists e2e tests since it relies on the readPrivileges function of SIEM so any changes to the expected response from there must also be changed in the lists privileges route
* update scripts roles to include maintenance for roles that do not have privileges higher than 'maintenance'
* fix open-close signals integration test
### Summary
This PR concentrates on fixing the deletion on the exceptions list table view. This fix is intermediary and a more thorough, backend solution is needed. Currently, if you delete an exception list, it deletes the exception list SO, but does not remove references to it from rules. This PR allows for a quick fix conducting this logic client side.
## Summary
Currently working on issues related to exceptions and it was noted on a separate PR that the request payload validation being done in the client side API calls was unnecessary. It was helpful in development, but not of any added value in production. Not only that, but the extra validations also add to the performance hit.
Removed the payload validation and formatted the code to follow the same pattern as that in the value lists api file.
Tested that exceptions flows not affected by testing out exceptions CRUD flows.
## Summary
This PR addresses a fix on the exceptions list table export functionality. A dedicated route for exception list export needed to be created. List is exported into an `.ndjson` format.
Exception lists consist of two elements - the list itself, and its items. The export file should now contain both these elements, the list followed by its items.
## Summary
Resolves https://github.com/elastic/kibana/issues/77324, https://github.com/elastic/kibana/issues/77325, resolves https://github.com/elastic/kibana/issues/77325, and resolves https://github.com/elastic/kibana/issues/81302
This PR addresses referential integrity issues when deleting value lists. Previously when deleting value lists, any references in Exception Lists/Items would be left behind. This PR introduces a new confirmation modal when deleting value lists that are referenced in either space aware (`simple`) or space `agnostic` exception lists.
Also includes:
* Fixed Lists plugin `quick_start.sh` as it was using endpoint exception list + value lists (unsupported)
* Adds `quick_start_value_list_references.sh` to create exception lists/items, value lists, and references to easily test
* Add support to `findExceptionList` for searching for both `simple` and `agnostic` list types
* Two new query params have been added to the `deleteListRoute`
* `ignoreReferences` (default:false) when true, maintains pre-7.11 behavior of deleting value list without performing any additional checks.
* NOTE: As written, this becomes an API breaking change as existing existing calls to the same API will `409` conflict if references exist. cc @jmikell821 @DonNateR
* `deleteReferences` (default:false) to perform dry run and identify referenced exception lists/items
## Testing
To test, run `quick_start_value_list_references.sh` and it will create all the necessary resources/references to easily exercise the above functionality. The below diagram details the resources created and how the references are wired up.
> Creates three different exception lists and value lists, and associates as
> below to test referential integrity functionality.
>
> NOTE: Endpoint lists don't support value lists, and are not tested here
>
> EL: Exception list
> ELI Exception list Item
> VL: Value list
>
> EL1 EL2 (Agnostic) EL3
> | | |
> ELI1 ELI2 ELI3
> |\ /| |
> | \ / | |
> | \ / | |
> | \ / | |
> | \/ | |
> | /\ | |
> | / \ | |
> | / \ | |
> | / \ | |
> |/ \| |
> VL1 VL2 VL3 VL4
> ips.txt ip_range.txt text.txt hosts.txt
>
Corner cases to be aware of:
* An exception item may have multiple value list entries -- only referenced value list entries should be removed
* There is no API for removing individual entries. If all entries are references the entire item is deleted. If only some entries are references, the item is updated via a `PUT` (no `PATCH` support for exception items)
* It's not possible via the UI to create a space agnostic list that has value list exception items (only agnostic endpoint exception lists can be created and they do not support value lists). Please use above script to exercise this behavior.
Additional notes:
* Once the Exception List table is introduced (https://github.com/elastic/kibana/pull/85465), we can add an enhancement for deeplinking to exception lists from the reference error modal.
* The `deleteListRoute` response has been updated to include the responses from the reference checks to provide maximum flexibility
* There is no bulk API for deleting exception list items, and so they are iterated over via the `deleteExceptionListItem` API.
##### Reference error modal
<p align="center">
<img width="500" src="https://user-images.githubusercontent.com/2946766/102199153-813e1e80-3e80-11eb-8a9b-af116ca13df9.gif" />
</p>
##### Overflow example
<p align="center">
<img width="500" src="https://user-images.githubusercontent.com/2946766/102199032-5784f780-3e80-11eb-81c7-17283d002ce4.gif" />
</p>
### Checklist
Delete any items that are not applicable to this PR.
- [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
### For maintainers
- [X] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
* Adds new SO type for persisting our signals migrations
* WIP: Migration status SO client
Trying to forge a patterrn using io-ts to validate at runtime. I think
I've got it working but I want to refactor the pipeline out into a
reusable function(s).
* Implements our SavedObjects service for signals migrations
* Defines a simple client that delegates to the base SO client with
our SO type
* Defines a service that consumes the simpler client, adding validations
and data transforms on top.
* Refactoring migration code to work with saved objects
As opposed to the previous ephemeral, encoded tokens, we now retrieve migration
info from saved objects.
At the API level, this means that both the create and finalize endpoints
receive a list of concrete indices. No more passing around tokens.
As both endpoints are idempotent, users can hammer them as much as they
want with the same lists of indices. Redundant creates and finalizes
will be met with inline 400 messages, and as one continues to poll the
finalize endpoint they should see more and more indices respond with
"completed: true"
* Fixing integration tests first, and anything upstream breaking them
* Clean up API integration tests
* standardize assignment of responses (with types)
* deletes migration SOs as test cleanup
* Split API tests into separate files
This was getting big and unwieldy; this splits these into one file per
endpoint.
* Refactor: split existing migration service functionality into atomic functions
This will allow us to repurpose the service to compose more
functionality and be more specifically useful, while keeping the
component logic separate.
* WIP: moving logic into migrationService.create
* Splitting get_migration_status into component functions
getMigrationStatus was really two separate aggregations, so I split them
out and we recompose them in the necessary routes.
* Move finalization logic into function
* migrationService exposes this as .finalize()
* adds an error field to our migration SO
* We currently only have one error that we persist there, but it would
be very time-consuming to track down that information were it not
there.
* Adds function for migration "deletion" logic
* migrationService leverages this function
* adds new boolean to our savedObject
* deletes unused function (deleteMigrationSavedObject)
* Adds route for soft-deletion of migrations
* Updating tests related to migration status
* Adding/updating mocks/unit tests necessary to satisfy the things I
need to test
* I mainly wanted to test that the the status endpoint filtered out the
deleted migrations; this was accomplished with a unit test after
fleshing out some mocks/sample data.
* Move old migration service tests to the relevant function tests
This logic was previously moved out into component functions; this moves
the tests accordingly.
* Add some unit tests around our reindex call
* Fix create migration route tests
Mocks out our migration functions, rather than stubbing ES calls
directly.
* Updates finalize route unit tests
Addresses functionality that hasn't been moved to finalizeMigration()
* Unit tests our finalization logic
Fixes a bug where we weren't accounting for soft-deleted migrations.
ALso updates our test migration SO to have a status of 'pending' as
that's a more useful default.
* Fixes finalization integration tests
These were failing due:
* a change in the migration status API response
* a bug I introduced in the finalize route
* Adds tests for our migration deletion endpoint
* unit tests
* API integration tests
* Caught/fixed bug with deleting a successful migration
* Fixes types
Removes unused code.
* Prevent race condition due to template rollover during migration
If a user has an out of date index (v1) relative to the template (v2), but the
template itself is out of date (newest is v3), then it's possible that
the template is rolled over to v3 after the v1-v2 migration has been
created but before the new index has been created.
In such a case, the new index would receive the v3 mappings but would
incorrectl be marked as v2. This shouldn't necessarily be an issue, but
it's an unnecessary state that can easily be prevented with the guard
introduced here.
* Add real usernames to migration savedObjects
In addition to the SOs themselves giving us observability into what
migration actions were performed, this gives us the additional info of
_who_ performed the action.
* Index minimal migration SO fields needed for current functionality
* Add additional migration info to status endpoint
This will allow users to finalize a migration if they've lost the
response to their POST call.
* Finalize endpoint receives an array of migration IDs, not indices
This disambiguates _which_ migrations we were finalizing if you passed
an index (which was previously: the most recent migration).
* Fix type errors in tests after we threaded through username
* Update responsibilities of migration finalize/delete endpoints
Discussions with @marshallmain lead to the following refactor:
* finalize does not delete tasks
* finalize only applies cleanup policy to a failed migration
* delete takes an array of migration ids (like finalize)
* delete hard-deletes the SavedObject of a completed (failed or
successful) migration
This gives a bit more flexibility with the endpoints, as well as
disambiguates the semantics: it just deletes migrations!
* Fix tests that were broken during refactoring
* Fix type errors
I removed some logic here but forgot the imports :(
* Move outdated integration test
In the case of a successful migration, application of the cleanup policy
is done by the deletion endpoint. In the interest of data preservation,
we do not delete a sourceIndex unless it is explicitly deleted.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes different bugs/issues when using exceptions with value based lists for both the UI, the backend, and the large value based lists. See https://github.com/elastic/kibana/issues/79516, but this also fixes several other bugs found mentioned below.
For the front end UI:
* Adds the ability to specify value based lists that are IP Ranges when the source event is an IP. Before you could only match IP to IP and the IP Ranges lists could not be used.
* Breaks down a few functions into smaller functions for unit test writing abilities.
You can now add ip ranges as list values for the UI when before it would not show up:
<img width="1035" alt="Screen Shot 2020-12-07 at 2 15 39 PM" src="https://user-images.githubusercontent.com/1151048/101406552-d6819b00-3896-11eb-9fb5-4c7c2ad93b2e.png">
For value based lists:
* Fixes text data type to use "and" between matching using `operator: 'and'` and changes it from a `terms query to a `match` query
* Adds new API for searching against types called `searchListItemByValues ` so that numeric, text, array based, and other non-stringable types can be sent and then the value based lists will push that to ElasticSearch. This shifts as many corner cases and string/numeric coercions to ElasticSearch rather than Kibana client side code.
* Adds ability to handle arrays within arrays through a `flatten` call.
* Utilizes the `named queries` from ElasticSearch for the new API so that clients can get which parts matched and then use that for their exception list logic rather than in-memory string to string checks. This fixes CIDR and ranges as well as works with arrays.
For the backend exception lists that used value based lists:
* Broke down the `filterEventsAgainstList` function into a folder called `filters` and the functions into other files for better unit based testing.
* Changed the calls from `getListItemByValues` to `searchListItemByValues` which can return exactly what it matched against and this will not break anyone using the existing REST API for `getListItemByValues` since that REST API and client side API stays the same.
* Cleaned up extra promises being used in a few spots that async/await automatically will create.
* Removed the stringabilities and stringify in favor of just a simpler exact check using `JSON.stringify()`
For the tests:
* Adds unit tests to broken down functions
* Adds ip_array, keyword_array, text_array, FTR tests for the backend.
* Adds more CIDR and range based FTR tests for the backend.
* Unskips and fixes all the numeric tests and range tests that could not operate previously from bugs.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
* Separated out service layer for trusted apps.
* Improved the type structure a bit to avoid using explicit string literals and to add possibility to return OS specific parts of trusted app object in type safe manner.
* Added support for mapping of trusted app to exception item and back.
* Changed schema to support signer in the API.
* Renamed utils to mapping.
* Exported some types in lists plugin and used them in trusted apps.
* Added tests for mapping.
* Added tests for service.
* Switched deletion to use exceptions for not found case.
* Added resetting of the mocks in service layer tests.
* Added handlers tests.
* Refactored mapping tests to be more granular based on the case.
* Restored lowercasing of hash.
* Added schema tests for signer field.
* Removed the grouped tests (they were split into tests for separate concerns).
* Corrected the tests.
* Lowercased the hashes in the service test.
* Moved the lowercasing to the right location.
* Fixed the tests.
* Added test for lowercasing hash value.
* Introduced OperatingSystem enum instead of current types.
* Removed os list constant in favour of separate lists in places that use it (each place has own needs to the ordering).
* Fixed the missed OperatingSystem enum usage.
* bump version to 4.1.1-rc
* fix code to run kbn bootstrap
* fix errors
* DO NOT MERGE. mute errors and ping teams to fix them
* Address EuiSelectableProps configuration in discover sidebar
* use explicit type for EuiSelectable
* update to ts v4.1.2
* fix ts error in EuiSelectable
* update docs
* update prettier with ts version support
* Revert "update prettier with ts version support"
This reverts commit 3de48db3ec.
* address another new problem
Co-authored-by: Chandler Prall <chandler.prall@gmail.com>
## Summary
Adds support to the end to end (e2e) functional test runner (FTR) support for rule runtime tests as well as 213 tests for the exception lists which include value based lists. Previously we had limited runtime support, but as I scaled up runtime tests from 5 to 200+ I noticed in a lot of areas we had to use improved techniques for determinism.
The runtime support being added is our next step of tests. Up to now most of our e2e FTR tests have been structural testing of REST and API integration tests. Basically up to now 95% tests are API structural as:
* Call REST input related to a rule such as GET/PUT/POST/PATCH/DELETE.
* Check REST output of the rule, did it match expected output body and status code?
* In some rare cases we check if the the rule can be executed and we get a status of 'succeeded'
With only a small part of our tests ~5%, `generating_signals.ts` was checking the signals being produced. However, we cannot have confidence in runtime based tests until the structural tests have been built up and run through the weeks against PR's to ensure that those are stable and deterministic.
Now that we have confidence and 90%+ coverage of the structural REST based tests, we are building up newer sets of tests which allow us to do runtime based validation tests to increase confidence that:
* Detection engine produces signals as expected
* Structure of the signals are as expected, including signal on signals
* Exceptions to signals are working as expected
* Most runtime bugs can be TDD'ed with e2e FTR's and regressions
* Whack-a-mole will not happen
* Consistency and predictability of signals is validated
* Refactoring can occur with stronger confidence
* Runtime tests are reference points for answering questions about existing bugs or adding new ones to test if users are experiencing unexpected behaviors
* Scaling tests can happen without failures
* Velocity for creating tests increases as the utilities and examples increase
Lastly, this puts us within striking distance of creating FTR's for different common class of runtime situations such as:
* Creating tests that exercise each rule against a set of data criteria and get signal hits
* Creating tests that validate the rule overrides operate as expected against data sets
* Creating tests that validate malfunctions, corner cases, or misuse cases such as data sets that are _all_ arrays or data sets that put numbers as strings or throws in an expected `null` instead of a value.
These tests follow the pattern of:
* Add the smallest data set to a folder in data.json (not gzip format)
* Add the smallest mapping to that folder (mapping.json)
* Call REST input related to exception lists, value lists, adding prepackaged rules, etc...
* Call REST input related endpoint with utilities to create and activate the rule
* Wait for the rule to go into the `succeeded` phase
* Wait for the N exact signals specific to that rule to be available
* Check against the set of signals to ensure that the matches are exactly as expected
Example of one runtime test:
A keyword data set is added to a folder called "keyword" but you can add one anywhere you want under `es_archives`, I just grouped mine depending on the situation of the runtime. Small non-gzipped tests `data.json` and `mappings.json` are the best approach for small focused tests. For _larger_ tests and cases I would and sometimes do use things such as auditbeat but try to avoid using larger data sets in favor of smaller focused test cases to validate the runtime is operating as expected.
```ts
{
"type": "doc",
"value": {
"id": "1",
"index": "long",
"source": {
"@timestamp": "2020-10-28T05:00:53.000Z",
"long": 1
},
"type": "_doc"
}
}
{
"type": "doc",
"value": {
"id": "2",
"index": "long",
"source": {
"@timestamp": "2020-10-28T05:01:53.000Z",
"long": 2
},
"type": "_doc"
}
}
{
"type": "doc",
"value": {
"id": "3",
"index": "long",
"source": {
"@timestamp": "2020-10-28T05:02:53.000Z",
"long": 3
},
"type": "_doc"
}
}
{
"type": "doc",
"value": {
"id": "4",
"index": "long",
"source": {
"@timestamp": "2020-10-28T05:03:53.000Z",
"long": 4
},
"type": "_doc"
}
}
```
Mapping is added. Note that this is "ECS tolerant" but not necessarily all ECS meaning I can and will try to keep things simple where I can, but I have ensured that `"@timestamp"` is at least there.
```ts
{
"type": "index",
"value": {
"index": "long",
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"long": { "type": "long" }
}
},
"settings": {
"index": {
"number_of_replicas": "1",
"number_of_shards": "1"
}
}
}
}
```
Test is written with test utilities where the `beforeEach` and `afterEach` try and clean up the indexes and load/unload the archives to keep one test from effecting another. Note this is never going to be 100% possible so see below on how we add more determinism in case something escapes the sandbox.
```ts
beforeEach(async () => {
await createSignalsIndex(supertest);
await createListsIndex(supertest);
await esArchiver.load('rule_exceptions/keyword');
});
afterEach(async () => {
await deleteSignalsIndex(supertest);
await deleteAllAlerts(supertest);
await deleteAllExceptions(es);
await deleteListsIndex(supertest);
await esArchiver.unload('rule_exceptions/keyword');
});
describe('"is" operator', () => {
it('should filter 1 single keyword if it is set as an exception', async () => {
const rule = getRuleForSignalTesting(['keyword']);
const { id } = await createRuleWithExceptionEntries(supertest, rule, [
[
{
field: 'keyword',
operator: 'included',
type: 'match',
value: 'word one',
},
],
]);
await waitForRuleSuccess(supertest, id);
await waitForSignalsToBePresent(supertest, 3, [id]);
const signalsOpen = await getSignalsById(supertest, id);
const hits = signalsOpen.hits.hits.map((hit) => hit._source.keyword).sort();
expect(hits).to.eql(['word four', 'word three', 'word two']);
});
});
```
### Changes for better determinism
To support more determinism there are changes and utilities added which can be tuned during any sporadic failures we might encounter as well as better support unexpected changes to other Elastic Stack pieces such as alerting, task manager, etc...
Get simple rule and others are now defaulting to false, meaning that the structural tests will no longer activate a rule and run it on task manger. This should cut down on error outputs as well as reduce stress and potentials for left over rules interfering with the runtime rules.
```ts
export const getSimpleRule = (ruleId = 'rule-1', enabled = false): QueryCreateSchema => ({
```
Not mandatory to use, but for most tests that should be runtime based tests, I use this function below which will enable it by default and run it using settings such as `type: 'query'`, `query: '*:*',` `from: '1900-01-01T00:00:00.000Z'`, to cut down on boiler plate noise. However, people can use whatever they want out of the grab bag or if their test is more readable to hand craft a REST request to create signals, or if they just want to call this and override where they want to, then 👍 .
```ts
export const getRuleForSignalTesting = (index: string[], ruleId = 'rule-1', enabled = true)
```
This waits for a rule to succeed before continuing
```ts
await waitForRuleSuccess(supertest, id);
```
I added a required array of id that _waits_ only for that particular id here. This is useful in case another test did not cleanup and you are getting signals being produced or left behind but need to wait specifically for yours.
```ts
await waitForSignalsToBePresent(supertest, 4, [id]);
```
I only get the signals for a particular rule id using either the auto-generated id or the rule_id. It's safer to use the ones from the auto-generated id but either of these are fine if you're careful enough.
```ts
const signalsOpen = await getSignalsById(supertest, id);
const signalsOpen = await getSignalsByIds(supertest, [createdId]);
const signalsOpen = await getSignalsByRuleIds(supertest, ['signal-on-signal']);
```
I delete all alerts now through a series of steps where it properly removes all rules using the rules bulk_delete and does it in such a way that all the API keys and alerting will be the best it can destroyed as well as double check that the alerts are showing up as being cleaned up before continuing.
```ts
deleteAllAlerts()
```
When not explicitly testing something structural, prefer to use the utilities which can and will do retries in case there are over the wire failures or es failures. Examples are:
```ts
installPrePackagedRules()
waitForRuleSuccess()
importFile() // This does a _lot_ of checks to ensure that the file is fully imported before continuing
```
Some of these utilities might still do a `expect(200);` but as we are and should use regular structural tests to cover those problems, these will probably be more and more removed when/if we hit test failures in favor of doing retries, waitFor, and countDowns.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* ExceptionIdentifiers -> ExceptionListIdentifiers
* Pass refreshRule through to alert context menu
* Fix type errors and rename refreshRule to onRuleChange for consistency
* Finish adding .lower to exceptionable fields
* Add back migrations
* .lower -> .caseless
* Add separate field for os type
* updates
* Type updates
* Switch over to osTypes
* get rid of _tags
* Add tests for schema validation
* Remove remaining references to _tags
* Another round of test fixes
* DefaultArray tests
* More test fixes
* Fix remaining test failures
* types / tests
* more test updates
* lowercase os values
* Address feedback + fix test failure
* tests
* Fix integration test
* process.executable.path -> process.executable.caseless
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This is a bug that was introduced by moi in 76537. This previous PR added a sort_field and sort_order to the call for fetching exception lists' items so that the exception item order in the viewer wouldn't jump around any time there was an update. I noticed however that when a rule had both endpoint and detection lists associated with it, when trying to fetch items from both types of lists, the following error shows:
```
"Unable to sort multiple types by field created_at, not a root property"
```
## Summary
Mistmatch caught between the io-ts type and the corresponding typescript enum. Currently, io-ts does not have support for enums - as a workaround I had made a matching typescript enum type. Tests were added to try to ensure the two stayed in sync, but didn't do a straight up comparison of the two.
Updated the tests to now error if the keys do not match.
## Summary
**Components affected:** ExceptionsViewer
**Current behavior:**
- when a user edits an exception item, the order of the exception items in the viewer changes. This creates confusion and looks like updates weren't applied (even though they were, just item order changed)
**New behavior:**
- when a user edits an exception item, the order of the exception items in the viewer don't change. Sort order is now based on `created_at`
* bump ts to v4
* MOAR RAM
* fix type errors for OSS
* first pass on x-pack errors
* second pass on x-pack type errors
* 3rd pass on x-pack type-errors
* mute errors if complex cases
* don't delete if spread suffices
* mute other complex cases
* make User fields optional
* fix optional types
* fix tests
* fix typings for time_range
* fix type errors in x-pack/tests
* rebuild kbn-pm
* remove leftovers from master update
* fix alert tests
* [Telemetry Checker] TS4 Fixes
* bump to 4.0.1-rc
* fix new errors in master
* bump typescript-eslint to version supporting TS v4 syntax
* fix merge commit errors
* update to the stable TS version 4.0.2
* bump ts-eslint to version supporting ts v4
* fix typo
* fix type errors after merge
* update ts in another new package.json
* TEMP: remove me
* Revert "TEMP: remove me"
This reverts commit dc0fc3bae6.
* [Telemetry] Update snapshot for new TS4 SyntaxKind
* bump prettier to support TS v4 syntax
* fix prettier rules
* last style change
* fix new type errors
Co-authored-by: Alejandro Fernández Haro <alejandro.haro@elastic.co>
* Trusted Apps initial setup for route registration
* Added types for TrustedApp entries
* trusted apps list API returns results
* use methods and const from latest PR merge to lists
* a quick generator for trusted apps entries
* support cli options for trusted app data loader
* Add mocked `createTrustedAppsList()` method to `ExceptionListClientMock`
* tests fro trusted apps route handlers
* tests for trusted apps schema
* Correct name of mock method
* Fix service to ensure return value of `getExceptionList` service throws if service not available
* Fix types
* Refactor TrustedApp type + code review feedback
* Simplify our kibana mocks
* Simpler mock factory that returns an object instead of a thunk
* We can use mockReturnValue instead of mockImplementation to
accomplish the same
* Allows us to replace createStartServices mock
* Uses unknown instead of any for mocks
* Clean up our manual use of kibana mocks in tests
* Since our useKibana mock returns a consistent mock, we can modify its
return value instead of re-mocking the entire thing
* Removes unnecessary uses of clearing/resetting mocks
* If your mocks are configured at the beginning of each test this is
usually unnecessary.
* I left one case of clearAllMocks in all_cases/index.test since it
defined several mock functions that were persistent across tests, and
it was easier than moving their definitions to a beforeEach
* Removes some unnecessary overrides that seemed due to storage
previously not being mocked
* Rename some old occurrences of SIEM
* Cross-reference similar hooks via JSDoc
There's a good chance that the consumer might want the OTHER hook, so
let's make that discoverable.
* Adds jest tests for our useListsConfig hook
* adds mocks for the hooks upon which it depends
* Add a mock for our useListsConfig hook
Leverages this mock factory in our manual mock for this hook.
* Remove unneeded eslint exception
* Move kibana_react mocks into their own .mock file
We're trying to consolidate mocks to this pattern so that they're easier
to find and reuse.
* Remove intermediate mock factory
This was only being consumed by our general createStartServicesMock.
* Replace security_solution's alias for a core mock
This is just noise/tech debt, we should use the core mock directly when
we can.
* Remove unnecessary wrapper around core mocks
Instead let's just reference the core mocks themselves.
* Remove outdated references from upstream
* More accurate mock
Throw an error of the same type if an unexpected key is used.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Fixes a bug where the list privileges was returning the `.list` privileges twice instead of returning it once and returning the `.items` privileges second with the call. No UI has to change as the way it was written was dynamic to grab the first key found.
This also adds the functional tests to `x-pack/scripts/functional_tests.js` which was not there originally so the end to tend tests should actually run on the CI machine where it was not running on CI before.
Adds the functional tests to the code owners file as well.
Ensure that you go to the test results page from the Jenkins build:
<img width="901" alt="Screen Shot 2020-08-18 at 1 13 18 AM" src="https://user-images.githubusercontent.com/1151048/90482180-13f7c800-e0f0-11ea-92f2-b30a8fffe84e.png">
And ensure you see the tests under:
```
X-Pack Lists Integration Tests
```
Then click through it and ensure they are shown as running and passing
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* Route options timeout -> timeout.payload
* timeout.idleSocket can now be specified per route
* Removing nested ternary
* Fixing integration tests
* Trying to actually fix the integration tests. Existing tests are hitting
idle socket timeout, not the payload timeout
* Fixing payload post timeout integration test
* Fixing PUT and DELETE payload sending too long tests
* Fixing type-script errors
* GET routes can't specify the payload timeout, they can't accept payloads
* Removing some redundancy in the tests
* Adding 'Transfer-Encoding: chunked' to the POST test
* Fixing POST/GET/PUT quick tests
* Adding idleSocket timeout test
* Removing unnecessary `isSafeMethod` call
* Updating documentation
* Removing PUT/DELETE integration tests
* Working around the HapiJS bug
* Deleting unused type import
* The socket can be undefined...
This occurs when using @hapi/shot directly or indirectly via
Server.inject. In these scenarios, there isn't a socket. This can also
occur when a "fake request" is used by the hacky background jobs:
Reporting and Alerting...
* Update src/core/server/http/http_server.ts
Co-authored-by: Josh Dover <me@joshdover.com>
* Adding payload timeout functional tests
* Adding idle socket timeout functional tests
* Adding better comments, using ?? instead of ||
* Fixing the plugin fixture TS
* Fixing some typescript errors
* Fixing plugin fixture tsconfig.json
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Josh Dover <me@joshdover.com>
* Add separate io-ts types for endpoint exception entries
* Fix text typos
* Fix test
* address review comments
* Add extra entry validation when adding items to endpoint_list
* fix test
* fix tests again
* really actually fix the tests
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This is the basics of end to end tests, so there could be a lot more, but this ties to cover the basics of the tests.
Test with:
```ts
node scripts/functional_tests --config x-pack/test/lists_api_integration/security_and_spaces/config.ts
```
Adds these tests for the route counter parts:
* create_exception_list_items.ts
* create_exception_lists.ts
* delete_exception_list_items.ts
* delete_exception_lists.ts
* find_exception_list_items.ts
* find_exception_lists.ts
* read_exception_list_items.ts
* read_exception_lists.ts
* update_exception_list_items.ts
* update_exception_lists.ts
Fixes a few minor strings, other tests, but no large bugs found with these tests
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Adds initial set of end to end tests for lists
You can run all of these with the command from kibana root:
```ts
node scripts/functional_tests --config x-pack/test/lists_api_integration/security_and_spaces/config.ts
```
Fixes a few minor bugs found such as...
* Validation for importing lists was not checking if the indexes were created first
* Some wording for the error messages had duplicate words within them
### Checklist
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
## Summary
Using the `formatErrors` util would result in duplicate error messages sometimes. Was noticing this in particular when using union types, where the type validation would check every item in a union and report an error for each one. This resulted in large, repeating errors.
Used `uniq` to filter out duplicates. Updated unit tests.
## Summary
Addresses feedback from https://github.com/elastic/kibana/pull/72748
- Updates `plugins/lists` tests text from `should not validate` to `should FAIL validation` after feedback that previous text is a bit confusing and can be interpreted to mean that validation is not conducted
- Remove unnecessary spreads from one of my late night PRs
- Removes `siem_common_deps` in favor of `shared_imports` in `plugins/lists`
- Updates `build_exceptions_query.test.ts` to use existing mocks
## Summary
This PR focuses on addressing issues around the pagination and functionality of rules with numerous (2+) exception lists.
- Updated the `use_exception_list.ts` hook to make use of the new multi list find API
- Updated the viewer to make use of the new multi list find API
- Previously was doing a lot of the filtering and paging manually (and badly) in the UI, now the _find takes care of all that
- Added logic for showing `No results` text if user filter/search returns no items
- Previously would show the `This rule has not exceptions` text
## Summary
This PR addresses the following:
- Adds `list_id` to `rule.exceptions_list` - this is needed in a number of features
- Updated `getExceptions` in `x-pack/plugins/security_solution/server/lib/detection_engine/signals/utils.ts` to use the latest exception item find endpoint that accepts an array of lists (previously was looping through lists and conducting a `find` for each)
- Updated prepackaged rule that makes reference to global endpoint list to include `list_id`
- Updates `formatAboutStepData` in `x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/create/helpers.ts` to include exception list `list_id`
* Validate exception list size when adding new items
* Update comment
* Extract list size validation and apply to endpoint route also
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
