This PR attempts to batch update tasks in Task Manager in order to avoid overloading the Elasticsearch queue.
This is the 1st PR addressing https://github.com/elastic/kibana/issues/65551
Under the hood we now use a Reactive buffer accumulates all calls to the `update` api in the TaskStore and flushes after 50ms or when as many operations as there are workers have been buffered (whichever comes first).
* Archive e2e test results in ES
* Disable flaky comment feature and PR notifications
* Update .ci/end2end.groovy
Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
* intercept 401 error from new client in routing layer
* improvements
* lint
* fix mocked client construction due to 7.9-rc1 bump
* use default WWW-Authenticate value when not provided by ES 401
* use KibanaClient interface instead of Client
* add back helpers to ElasticsearchClient interface
* use TransportRequestPromise for transport.request
* update generated doc
* [ftr/webdriver] retry on all errors, use Rx so that timers are canceled
* throw if attemptToCreateCommand() aborts by resolving to undefined
Co-authored-by: spalger <spalger@users.noreply.github.com>
## Summary
This PR updates the following:
- `useFetchIndexPatterns` now returns `indexPatterns` whose fields include `esTypes` and `subType`
- Why?? The exceptions builder needs these two fields to determine what fields are of ES type `nested` and parent paths
- exceptions add and edit modals now use the `rule.index` field to pass into `useFetchindexPatterns`
- Before we were using the signals index and alerts index for endpoint, needs to be rule's index patterns
- if no index patterns exist on the rule (if rule created via API, it's not required), then uses `DEFAULT_INDEX_PATTERN`
- updates the autocomplete validation to use `IField.esTypes` to check type instead of `IField.type`
## Summary
* Removes the older wait pattern that does a block no matter what
* Utilizes the improved and better pattern for test-library's waitFor which will test immediately and then poll for results
* Changes everything to put their expect statement within the waitFor
* Once the waitFor is in TypeScript/JS we can change the import statement to use that
If you get a timeout or error this is what it looks like now which improves the developer experience in some ways but does degrade things in others as it suggests that everything is timeout related. However, developers should inspect the values and remove the waitFor() and re-run their tests if they think that they have a real problem during development.
<img width="990" alt="Screen Shot 2020-07-20 at 12 40 39 PM" src="https://user-images.githubusercontent.com/1151048/87975739-4084d980-ca89-11ea-83c9-ba3fb932a175.png">
See the API for more information:
https://testing-library.com/docs/dom-testing-library/api-async#waitfor
But in short we should be using:
```ts
await waitFor(() => expect(...));
```
throughout our code at this point and the waitFor will loop quickly and efficiently until it either times out or gets the condition expected.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
* [Maps] auto-fit to data bounds
* update jest snapshot
* add buffer to fit to bounds
* sync join layers prior to fitting to bounds
* clean-up comment
* better names
* fix tslint errors
* update functional test expect
* add functional tests
* clean-up
* change test run location
* fix test expect
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* removes useSortIds which was leftover from a previous attempt at implementing gap detection mitigation code. This only showed up because I modified the count variable used to determine when we hit maxSignals from utilizing the searchResult hits length to using the count of bulk created items (signals indexed) in this commit 56de45d156
* removes logs and fixes if statement ordering
* adds tests, increases code coverage for search after and bulk create function, updates log statements
* update tests after rebase onto master
* clean up if statements
* fix test data
* merge conflicts are hard
- Also convert api and api_errors services, and improve typing of http service.
- Fix bug where fatalErrors service was improperly consumed in api_errors.
- Improve typing in Rollup api_errors service, for consistency.
## Summary
* Adds list permissions as a feature control to SIEM.
* Separates the controls between two, one of which is `access:lists-all` and the other is `access:lists-read`
* Grants SIEM the ability to utilize both depending on which feature mode the space is in.
## Summary
* Adds conflict versioning by exposing the "_version" from the saved object system. It renames "version" to "_version" so that we can use regular "version" later for versioning things for pre-packaged lists abilities.
* Utilizes `t.OutputOf` in the requests and the data types to give us more correctly types
* Removes the `Identity` utility as that is adding confusion and can confuse vs code rather than improves things
* Removes extra types that were causing confusion which was an idiom from io-ts
* Changes the wording of `Partial` by removing that and instead focuses the request types on either client side or server side at this point.
NOTE: The UI can migrate to holding onto the `_version` and then push it back down when it wants to migrate to using the conflict resolution. If the UI does not push it down, then a value of undefined will be used which is indicating that no conflict errors are wanted.
Output example of posting an exception list:
❯ ./post_exception_list.sh
```ts
{
"_tags": [
"endpoint",
"process",
"malware",
"os:linux"
],
"_version": "Wzk4NiwxXQ==",
"created_at": "2020-07-17T18:59:22.872Z",
"created_by": "yo",
"description": "This is a sample endpoint type exception",
"id": "a08795b0-c85f-11ea-b1a6-c155df988a92",
"list_id": "simple_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "b789ec05-3e0f-4344-a156-0c0f5b6e2f9c",
"type": "detection",
"updated_at": "2020-07-17T18:59:22.891Z",
"updated_by": "yo"
}
```
Output example of posting an exception list item
❯ ./post_exception_list_item.sh
```ts
{
"_tags": [
"endpoint",
"process",
"malware",
"os:linux"
],
"_version": "Wzk4NywxXQ==",
"comments": [],
"created_at": "2020-07-17T18:59:30.286Z",
"created_by": "yo",
"description": "This is a sample endpoint type exception",
"entries": [
{
"field": "actingProcess.file.signer",
"operator": "excluded",
"type": "exists"
},
{
"field": "host.name",
"operator": "included",
"type": "match_any",
"value": [
"some host",
"another host"
]
}
],
"id": "a4f2b800-c85f-11ea-b1a6-c155df988a92",
"item_id": "simple_list_item",
"list_id": "simple_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "1dc456bc-7aa9-44b4-bca3-131689cf729f",
"type": "simple",
"updated_at": "2020-07-17T18:59:30.304Z",
"updated_by": "yo"
}
```
Output example of when you get an exception list:
❯ ./get_exception_list.sh simple_list
```ts
{
"_tags": [
"endpoint",
"process",
"malware",
"os:linux"
],
"_version": "WzEwNzcsMV0=",
"created_at": "2020-07-17T18:59:22.872Z",
"created_by": "yo",
"description": "Different description",
"id": "a08795b0-c85f-11ea-b1a6-c155df988a92",
"list_id": "simple_list",
"name": "Sample Endpoint Exception List",
"namespace_type": "single",
"tags": [
"user added string for a tag",
"malware"
],
"tie_breaker_id": "b789ec05-3e0f-4344-a156-0c0f5b6e2f9c",
"type": "endpoint",
"updated_at": "2020-07-17T20:01:24.958Z",
"updated_by": "yo"
}
```
Example of the error you get if you do an update of an exception list and someone else has changed it:
```ts
{
"message": "[exception-list:a08795b0-c85f-11ea-b1a6-c155df988a92]: version conflict, required seqNo [1074], primary term [1]. current document has seqNo [1077] and primary term [1]: [version_conflict_engine_exception] [exception-list:a08795b0-c85f-11ea-b1a6-c155df988a92]: version conflict, required seqNo [1074], primary term [1]. current document has seqNo [1077] and primary term [1], with { index_uuid=\"a2mgXBO6Tl2ULDq-MTs1Tw\" & shard=\"0\" & index=\".kibana-hassanabad_1\" }",
"status_code": 409
}
```
Lists are the same way and flavor, they encode the _version the same way that saved objects do. To see those work you run these scripts:
```ts
./post_list.sh
./post_list_item.sh
./find_list.sh
./find_list_item.sh
```
### Checklist
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
* fix nav with no-data
* fix rules action alerts
* fix raw alert event type
* snapshot
* can only use network-only
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
* first implementation of tooltip
* Add processor tooltip component files
* remove init position from code for now
* colocate on change handler and make code a bit cleaner
* removed document.body.appendChild logic because EuiPortal does that for us
* use correct toggle button api
* added test to check button disabled while editing
* remove cursor not allowed
* simplify logic
* assert if against positive
* remove unused variable
* Remove unused actions const
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
