kibana/x-pack/plugins/lists/server
Frank Hassanabad 28738e6b4b
[Security Solution] Fixes CIDR, float, long, integer, array, and text based issues when using value lists in exceptions (#85191)
## Summary

Fixes different bugs/issues when using exceptions with value based lists for both the UI, the backend, and the large value based lists. See https://github.com/elastic/kibana/issues/79516, but this also fixes several other bugs found mentioned below.

For the front end UI:
* Adds the ability to specify value based lists that are IP Ranges when the source event is an IP. Before you could only match IP to IP and the IP Ranges lists could not be used. 
* Breaks down a few functions into smaller functions for unit test writing abilities.

You can now add ip ranges as list values for the UI when before it would not show up:
<img width="1035" alt="Screen Shot 2020-12-07 at 2 15 39 PM" src="https://user-images.githubusercontent.com/1151048/101406552-d6819b00-3896-11eb-9fb5-4c7c2ad93b2e.png">

For value based lists:
* Fixes text data type to use "and" between matching using `operator: 'and'` and changes it from a `terms query to a `match` query
* Adds new API for searching against types called `searchListItemByValues ` so that numeric, text, array based, and other non-stringable types can be sent and then the value based lists will push that to ElasticSearch. This shifts as many corner cases and string/numeric coercions to ElasticSearch rather than Kibana client side code.
* Adds ability to handle arrays within arrays through a `flatten` call.
* Utilizes the `named queries` from ElasticSearch for the new API so that clients can get which parts matched and then use that for their exception list logic rather than in-memory string to string checks. This fixes CIDR and ranges as well as works with arrays.

For the backend exception lists that used value based lists:
* Broke down the `filterEventsAgainstList` function into a folder called `filters` and the functions into other files for better unit based testing.
* Changed the calls from `getListItemByValues` to `searchListItemByValues` which can return exactly what it matched against and this will not break anyone using the existing REST API for `getListItemByValues` since that REST API and client side API stays the same.
* Cleaned up extra promises being used in a few spots that async/await automatically will create. 
* Removed the stringabilities and stringify in favor of just a simpler exact check using `JSON.stringify()`

For the tests:
* Adds unit tests to broken down functions
* Adds ip_array, keyword_array, text_array, FTR tests for the backend.
* Adds more CIDR and range based FTR tests for the backend.
* Unskips and fixes all the numeric tests and range tests that could not operate previously from bugs.

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
2020-12-10 18:07:47 -07:00
..
routes Migrate API keys functionality to a new Elasticsearch client. (#85029) 2020-12-09 20:43:24 +01:00
saved_objects
scripts
services [Security Solution] Fixes CIDR, float, long, integer, array, and text based issues when using value lists in exceptions (#85191) 2020-12-10 18:07:47 -07:00
config.mock.ts
config.test.ts
config.ts
create_config.ts
error_with_status_code.ts
get_space_id.test.ts Migrate API keys functionality to a new Elasticsearch client. (#85029) 2020-12-09 20:43:24 +01:00
get_space_id.ts Migrate API keys functionality to a new Elasticsearch client. (#85029) 2020-12-09 20:43:24 +01:00
get_user.test.ts Migrate API keys functionality to a new Elasticsearch client. (#85029) 2020-12-09 20:43:24 +01:00
get_user.ts Migrate API keys functionality to a new Elasticsearch client. (#85029) 2020-12-09 20:43:24 +01:00
index.ts Trusted Apps signer API. (#83661) 2020-11-30 15:42:31 +01:00
mocks.ts
plugin.ts Migrate API keys functionality to a new Elasticsearch client. (#85029) 2020-12-09 20:43:24 +01:00
siem_server_deps.ts
types.ts Migrate API keys functionality to a new Elasticsearch client. (#85029) 2020-12-09 20:43:24 +01:00