## Summary The io-ts package was too large and needed to broken down more by domain to decrease the lists plugin size and any other plugin wanting to use the packages will not incur big hits as well. Before we had one large io-ts package: ``` @kbn/securitysolution-io-ts-utils ``` Now we have these broken down 4 packages: ``` @kbn/securitysolution-io-ts-utils @kbn/securitysolution-io-ts-types @kbn/securitysolution-io-ts-alerting-types @kbn/securitysolution-io-ts-list-types ``` Deps between these packages are: ``` @kbn/securitysolution-io-ts-utils (none) @kbn/securitysolution-io-ts-types -> @kbn/securitysolution-io-ts-utils @kbn/securitysolution-io-ts-alerting-types -> @kbn/securitysolution-io-ts-types, @kbn/securitysolution-io-ts-utils @kbn/securitysolution-io-ts-list-types -> @kbn/securitysolution-io-ts-types, @kbn/securitysolution-io-ts-utils ``` Short description and function of each (Also in each of their README.md): ``` @kbn/securitysolution-io-ts-utils, Smallest amount of utilities such as format, validate, etc... @kbn/securitysolution-io-ts-types, Base types such as to_number, to_string, etc... @kbn/securitysolution-io-ts-alerting-types, Alerting specific types such as severity, from, to, etc... @kbn/securitysolution-io-ts-list-types, list specific types such as exception lists, exception list types, etc... ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios |
||
---|---|---|
.. | ||
.storybook | ||
common | ||
public | ||
scripts | ||
server | ||
jest.config.js | ||
kibana.json | ||
README.md |
README.md for developers working on the backend lists on how to get started using the CURL scripts in the scripts folder.
The scripts rely on CURL and jq:
Install curl and jq (mac instructions)
brew update
brew install curl
brew install jq
Open $HOME/.zshrc
or ${HOME}.bashrc
depending on your SHELL output from echo $SHELL
and add these environment variables:
export ELASTICSEARCH_USERNAME=${user}
export ELASTICSEARCH_PASSWORD=${password}
export ELASTICSEARCH_URL=https://${ip}:9200
export KIBANA_URL=http://localhost:5601
export TASK_MANAGER_INDEX=.kibana-task-manager-${your user id}
export KIBANA_INDEX=.kibana-${your user id}
source $HOME/.zshrc
or ${HOME}.bashrc
to ensure variables are set:
source ~/.zshrc
Open your kibana.dev.yml
file and add these lines with your name:
xpack.lists.listIndex: '.lists-your-name'
xpack.lists.listItemIndex: '.items-your-name'
Restart Kibana and ensure that you are using --no-base-path
as changing the base path is a feature but will
get in the way of the CURL scripts written as is.
Go to the scripts folder cd kibana/x-pack/plugins/lists/server/scripts
and run:
./hard_reset.sh
./post_list.sh
which will:
- Delete any existing lists you have
- Delete any existing list items you have
- Delete any existing exception lists you have
- Delete any existing exception list items you have
- Delete any existing mapping, policies, and templates, you might have previously had.
- Add the latest list and list item index and its mappings using your settings from
kibana.dev.yml
environment variable ofxpack.lists.listIndex
andxpack.lists.listItemIndex
. - Posts the sample list from
./lists/new/ip_list.json
Now you can run
./post_list.sh
You should see the new list created like so:
{
"id": "ip_list",
"created_at": "2020-05-28T19:15:22.344Z",
"created_by": "yo",
"description": "This list describes bad internet ip",
"name": "Simple list with an ip",
"tie_breaker_id": "c57efbc4-4977-4a32-995f-cfd296bed521",
"type": "ip",
"updated_at": "2020-05-28T19:15:22.344Z",
"updated_by": "yo"
}
You can add a list item like so:
./post_list_item.sh
You should see the new list item created and attached to the above list like so:
{
"id": "hand_inserted_item_id",
"type": "ip",
"value": "127.0.0.1",
"created_at": "2020-05-28T19:15:49.790Z",
"created_by": "yo",
"list_id": "ip_list",
"tie_breaker_id": "a881bf2e-1e17-4592-bba8-d567cb07d234",
"updated_at": "2020-05-28T19:15:49.790Z",
"updated_by": "yo"
}
If you want to post an exception list it would be like so:
./post_exception_list.sh
You should see the new exception list created like so:
{
"created_at": "2020-05-28T19:16:31.052Z",
"created_by": "yo",
"description": "This is a sample endpoint type exception",
"id": "bcb94680-a117-11ea-ad9d-c71f4820e65b",
"list_id": "endpoint_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "86e08c8c-c970-4b08-a6e2-cdba7bb4e023",
"type": "endpoint",
"updated_at": "2020-05-28T19:16:31.080Z",
"updated_by": "yo"
}
And you can attach exception list items like so:
{
"comments": [],
"created_at": "2020-05-28T19:17:21.099Z",
"created_by": "yo",
"description": "This is a sample endpoint type exception",
"entries": [
{
"field": "actingProcess.file.signer",
"operator": "included",
"type": "match",
"value": "Elastic, N.V."
},
{
"field": "event.category",
"operator": "included",
"type": "match_any",
"value": [
"process",
"malware"
]
}
],
"id": "da8d3b30-a117-11ea-ad9d-c71f4820e65b",
"item_id": "endpoint_list_item",
"list_id": "endpoint_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"os_types": ["linux"],
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "21f84703-9476-4af8-a212-aad31e18dcb9",
"type": "simple",
"updated_at": "2020-05-28T19:17:21.123Z",
"updated_by": "yo"
}
You can then do find for each one like so:
./find_lists.sh
{
"cursor": "WzIwLFsiYzU3ZWZiYzQtNDk3Ny00YTMyLTk5NWYtY2ZkMjk2YmVkNTIxIl1d",
"data": [
{
"id": "ip_list",
"created_at": "2020-05-28T19:15:22.344Z",
"created_by": "yo",
"description": "This list describes bad internet ip",
"name": "Simple list with an ip",
"tie_breaker_id": "c57efbc4-4977-4a32-995f-cfd296bed521",
"type": "ip",
"updated_at": "2020-05-28T19:15:22.344Z",
"updated_by": "yo"
}
],
"page": 1,
"per_page": 20,
"total": 1
}
or for finding exception lists:
./find_exception_lists.sh
{
"data": [
{
"created_at": "2020-05-28T19:16:31.052Z",
"created_by": "yo",
"description": "This is a sample endpoint type exception",
"id": "bcb94680-a117-11ea-ad9d-c71f4820e65b",
"list_id": "endpoint_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"os_types": ["linux"],
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "86e08c8c-c970-4b08-a6e2-cdba7bb4e023",
"type": "endpoint",
"updated_at": "2020-05-28T19:16:31.080Z",
"updated_by": "yo"
}
],
"page": 1,
"per_page": 20,
"total": 1
}
See the full scripts folder for all the capabilities.