## Summary This PR tries to start to tie together the server and client changes for exceptions lists. - Updates graphql types to allow UI access to a rule's `exceptions_list` property - Updates the exception viewer component to now dynamically take the rule `exceptions_list`, up until now we just had an empty array in it's place - Updates the viewer logic to check if a rule has an endpoint list associated with it. If it does, then it displays both detections and endpoint UIs (in the viewer), if it does not, then it only displays the detections UI - Updates the viewer UI to better deal with spacing when an exception list item only has one or two entries (before the and badge with the antennas was stretching passed the exception items to fill the space) - Updates the detections engine exceptions logic to fetch list items using an exception list's `id` as opposed to it's `list_id`, this now aligns with the UI using the same params on its end - Adds exception list `type` to information kept by the rule for exception lists - Updates the exception list type from `string` to `endpoint | detection` - Updates the exception list _item_ type from `string` to `simple` - Adds unit tests for the detection engine server side util that fetches the exception list items |
||
---|---|---|
.. | ||
common | ||
public | ||
scripts | ||
server | ||
kibana.json | ||
README.md |
README.md for developers working on the backend lists on how to get started using the CURL scripts in the scripts folder.
The scripts rely on CURL and jq:
Install curl and jq (mac instructions)
brew update
brew install curl
brew install jq
Open $HOME/.zshrc
or ${HOME}.bashrc
depending on your SHELL output from echo $SHELL
and add these environment variables:
export ELASTICSEARCH_USERNAME=${user}
export ELASTICSEARCH_PASSWORD=${password}
export ELASTICSEARCH_URL=https://${ip}:9200
export KIBANA_URL=http://localhost:5601
export TASK_MANAGER_INDEX=.kibana-task-manager-${your user id}
export KIBANA_INDEX=.kibana-${your user id}
source $HOME/.zshrc
or ${HOME}.bashrc
to ensure variables are set:
source ~/.zshrc
Open your kibana.dev.yml
file and add these lines:
# Enable lists feature
xpack.lists.enabled: true
xpack.lists.listIndex: '.lists-frank'
xpack.lists.listItemIndex: '.items-frank'
Restart Kibana and ensure that you are using --no-base-path
as changing the base path is a feature but will
get in the way of the CURL scripts written as is.
Go to the scripts folder cd kibana/x-pack/plugins/lists/server/scripts
and run:
./hard_reset.sh
./post_list.sh
which will:
- Delete any existing lists you have
- Delete any existing list items you have
- Delete any existing exception lists you have
- Delete any existing exception list items you have
- Delete any existing mapping, policies, and templates, you might have previously had.
- Add the latest list and list item index and its mappings using your settings from
kibana.dev.yml
environment variable ofxpack.lists.listIndex
andxpack.lists.listItemIndex
. - Posts the sample list from
./lists/new/list_ip.json
Now you can run
./post_list.sh
You should see the new list created like so:
{
"id": "list-ip",
"created_at": "2020-05-28T19:15:22.344Z",
"created_by": "yo",
"description": "This list describes bad internet ip",
"name": "Simple list with an ip",
"tie_breaker_id": "c57efbc4-4977-4a32-995f-cfd296bed521",
"type": "ip",
"updated_at": "2020-05-28T19:15:22.344Z",
"updated_by": "yo"
}
You can add a list item like so:
./post_list_item.sh
You should see the new list item created and attached to the above list like so:
{
"id": "hand_inserted_item_id",
"type": "ip",
"value": "127.0.0.1",
"created_at": "2020-05-28T19:15:49.790Z",
"created_by": "yo",
"list_id": "list-ip",
"tie_breaker_id": "a881bf2e-1e17-4592-bba8-d567cb07d234",
"updated_at": "2020-05-28T19:15:49.790Z",
"updated_by": "yo"
}
If you want to post an exception list it would be like so:
./post_exception_list.sh
You should see the new exception list created like so:
{
"_tags": [
"endpoint",
"process",
"malware",
"os:linux"
],
"created_at": "2020-05-28T19:16:31.052Z",
"created_by": "yo",
"description": "This is a sample endpoint type exception",
"id": "bcb94680-a117-11ea-ad9d-c71f4820e65b",
"list_id": "endpoint_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "86e08c8c-c970-4b08-a6e2-cdba7bb4e023",
"type": "endpoint",
"updated_at": "2020-05-28T19:16:31.080Z",
"updated_by": "yo"
}
And you can attach exception list items like so:
{
"_tags": [
"endpoint",
"process",
"malware",
"os:linux"
],
"comments": [],
"created_at": "2020-05-28T19:17:21.099Z",
"created_by": "yo",
"description": "This is a sample endpoint type exception",
"entries": [
{
"field": "actingProcess.file.signer",
"operator": "included",
"type": "match",
"value": "Elastic, N.V."
},
{
"field": "event.category",
"operator": "included",
"type": "match_any",
"value": [
"process",
"malware"
]
}
],
"id": "da8d3b30-a117-11ea-ad9d-c71f4820e65b",
"item_id": "endpoint_list_item",
"list_id": "endpoint_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "21f84703-9476-4af8-a212-aad31e18dcb9",
"type": "simple",
"updated_at": "2020-05-28T19:17:21.123Z",
"updated_by": "yo"
}
You can then do find for each one like so:
./find_lists.sh
{
"cursor": "WzIwLFsiYzU3ZWZiYzQtNDk3Ny00YTMyLTk5NWYtY2ZkMjk2YmVkNTIxIl1d",
"data": [
{
"id": "list-ip",
"created_at": "2020-05-28T19:15:22.344Z",
"created_by": "yo",
"description": "This list describes bad internet ip",
"name": "Simple list with an ip",
"tie_breaker_id": "c57efbc4-4977-4a32-995f-cfd296bed521",
"type": "ip",
"updated_at": "2020-05-28T19:15:22.344Z",
"updated_by": "yo"
}
],
"page": 1,
"per_page": 20,
"total": 1
}
or for finding exception lists:
./find_exception_lists.sh
{
"data": [
{
"_tags": [
"endpoint",
"process",
"malware",
"os:linux"
],
"created_at": "2020-05-28T19:16:31.052Z",
"created_by": "yo",
"description": "This is a sample endpoint type exception",
"id": "bcb94680-a117-11ea-ad9d-c71f4820e65b",
"list_id": "endpoint_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "86e08c8c-c970-4b08-a6e2-cdba7bb4e023",
"type": "endpoint",
"updated_at": "2020-05-28T19:16:31.080Z",
"updated_by": "yo"
}
],
"page": 1,
"per_page": 20,
"total": 1
}
See the full scripts folder for all the capabilities.