resolves https://github.com/elastic/kibana/issues/88333
Fixed:
- add note that `secure: false` will use TLS, but after an initial connection
with TCP; we have been getting questions from customers who believed that
`secure: false` implied TLS was not used at all.
- added a link to the nodemailer "well-known services" module, to allow
customers to see examples of other email service configurations
- updated the Outlook config example to use the current nodemailer values
- couple of other small tweaks
* Revert "Revert "Migrations v2: don't auto-create indices + FTR/esArchiver support (#85778)""
This reverts commit f97958043f.
* Fix flaky saved objects management test #89953
* If a clone target exists, wait for yellow, not green, index status
* Fix test after master merge
* Fix types
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Search Sessions: Unskip Flaky Functional Test
* Save all search sessions and then manage them based on their persisted state
* Get default search session expiration from config
* randomize sleep time
* fix test
* fix test
* Make sure we poll, and dont persist, searches not in the context of a session
* Added keepalive unit tests
* fix ts
* code review @lukasolson
* ts
* More tests, rename onScreenTimeout to completedTimeout
* lint
* lint
* Delete async seaches
* Support saved object pagination
Fix get search status tests
* better PersistedSearchSessionSavedObjectAttributes ts
* test titles
* Remove runAt from monitoring task
Increase testing trackingInterval (caused bug)
* support workload histograms that take into account overdue tasks
* Update touched when changing session status to complete \ error
* removed test
* Updated management test data
* Rename configs
* delete tap first
add comments
* Use sync config in data-enhanced plugin
* fix merge
* fix merge
* ts
* code review
Co-authored-by: Timothy Sullivan <tsullivan@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Anton Dosov <anton.dosov@elastic.co>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
* chore(NA): simple changes on bazelrc
* chore(NA): integrate bazel tools with BuildBuddy and remote cache service
* chore(NA) fix bazelrc line config
* chore(NA): move non auth settings out of bazelrc.auth
* chore(NA): output home dir
* chore(NA): load .bazelrc-ci.auth from /Users/tiagocosta dir
* chore(NA): remove bazelrc auth file and append directly into home bazelrc
* chore(NA): comment announce option
* chore(NA): integrate build buddy metadata
* chore(NA): update src/dev/ci_setup/.bazelrc-ci
Co-authored-by: Tyler Smalley <tylersmalley@me.com>
* chore(NA): move build metadata integation to common confdig
* chore(NA): fix problem on bazel file location
* chore(NA): correct sh file permissions
* chore(NA): only get host on CI
* chore(NA): add cores into host info on CI
* chore(NA): sync with last settings to setup bazelisk tools on ci
* chore(NA): sync last changes on ci setup env
* chore(NA): sync settings on ci setup with the other PR
* chore(NA): remove yarn export
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tyler Smalley <tylersmalley@me.com>
* Convert user values back to string after yaml template compilation if they were strings originally
* Add better test cases and adjust patch
* Fix when field is undefined
* Handle array of strings too
* chore(NA): build bazel projects all at once in the distributable build process
* chore(NA): make sure bazelisk is installed
* chore(NA): install bazelisk using npm
* chore(NA): remove extra spac
* chore(NA): test yarn path exports
* chore(NA): add direct global dir
* chore(NA): some more debug steps
* chore(NA): remove one statement
* chore(NA): comment one more line out for testing purposes
* chore(NA): export the correct yarn bin location into the PATH
* chore(NA): cleaning implementation
* chore(NA): move installation process of bazelisk into npm
* chore(NA): add missing type
## [Security Solution] [Timeline] Endpoint row renderers (1st batch)
This PR implements the 1st batch of Endpoint (`event.module: "endpoint"`) row renderers by updating and enhancing some of the existing "Endgame" (`event.module: "endgame"`) row renderers to use the latest [ECS fields](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html).
The following Endpoint events will be rendered via row renderers in Timeline:
| event.dataset | event.action |
|--------------------------|---------------------|
| endpoint.events.file | creation |
| endpoint.events.file | deletion |
| endpoint.events.process | start |
| endpoint.events.process | end |
| endpoint.events.network | lookup_requested |
| endpoint.events.network | lookup_result |
| endpoint.events.network | connection_accepted |
| endpoint.events.network | disconnect_received |
| endpoint.events.security | log_on |
| endpoint.events.security | log_off |
## File (FIM) Creation events
Endpoint File (FIM) Creation events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.file and event.action: creation
```
### Sample rendered File (FIM) Creation event
![endpoint_file_creation](https://user-images.githubusercontent.com/4459398/106036793-ff522f80-6092-11eb-9e3b-c24538129bea.png)
Each field with `this formatting` is draggable (to pivot a search) in the row-rendered event:
`SYSTEM` \ `NT AUTHORITY` @ `win2019-endpoint` created a file `WimProvider.dll` in `C:\Windows\TEMP\F590BACBAE94\WimProvider.dll` via `MsMpEng.exe` `(2424)`
### Fields in a File (FIM) Creation event
`user.name` \ `user.domain` @ `host.name` created a file `file.name` in `file.path` via `process.name` `(process.pid)`
## File (FIM) Deletion events
Endpoint File (FIM) Deletion events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.file and event.action: deletion
```
### Sample rendered File (FIM) Deletion event
![endpoint_file_deletion](https://user-images.githubusercontent.com/4459398/106037520-088fcc00-6094-11eb-985d-ba8cead9fec9.png)
`SYSTEM` \ `NT AUTHORITY` @ `windows-endpoint-1` deleted a file `AM_Delta_Patch_1.329.2793.0.exe` in `C:\Windows\SoftwareDistribution\Download\Install\AM_Delta_Patch_1.329.2793.0.exe` via `svchost.exe` `(1728)`
### Fields in a File (FIM) Deletion event
`user.name` \ `user.domain` @ `host.name` deleted a file `file.name` in `file.path` via `process.name` `(process.pid)`
## Process Start events
Endpoint Process Start events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.process and event.action: start
```
### Sample rendered Process Start event
![creation-event](https://user-images.githubusercontent.com/4459398/106061579-c7f37b00-60b2-11eb-9bc4-224e671baa4a.png)
`SYSTEM` \ `NT AUTHORITY` @ `win2019-endpoint` started process `conhost.exe` (`376`) `C:\Windows\system32\conhost.exe` `0xffffffff` `-ForceV1` via parent process `sshd.exe` (`6460`)
`sha256 697334c236cce7d4c9e223146ee683a1219adced9729d4ae771fd6a1502a6b63`
`sha1 e19da2c35ba1c38adf12d1a472c1fcf1f1a811a7`
`md5 1b0e9b5fcb62de0787235ecca560b610`
### Fields in a Process Start event
The following fields will be used to render a Process Start event:
`user.name` \ `user.domain` @ `host.name` started process `process.name` (`process.pid`) `process.args` via parent process `process.parent.name` (`process.parent.pid`)
`process.hash.sha256`
`process.hash.sha1`
`process.hash.md5`
## Process End events
Endpoint Process End events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.process and event.action: end
```
### Sample rendered Process End event
![endpoint_process_end](https://user-images.githubusercontent.com/4459398/106076527-f1b99b80-60cc-11eb-8ff8-2da78a1fcb8f.png)
`SYSTEM` \ `NT AUTHORITY` @ `win2019-endpoint` terminated process `svchost.exe` (`10392`) `C:\Windows\System32\svchost.exe` `-k` `netsvcs` `-p` `-s` `NetSetupSvc` with exit code `0` via parent process `services.exe` `(568)`
`7fd065bac18c5278777ae44908101cdfed72d26fa741367f0ad4d02020787ab6`
`a1385ce20ad79f55df235effd9780c31442aa234`
`8a0a29438052faed8a2532da50455756`
### Fields in a Process End event
The following fields will be used to render a Process End event:
`user.name` \ `user.domain` @ `host.name` terminated process `process.name` (`process.pid`) with exit code `process.exit_code` via parent process `process.parent.name` (`process.parent.pid`)
`process.hash.sha256`
`process.hash.sha1`
`process.hash.md5`
## Network (DNS) Lookup Requested events
Endpoint Network (DNS) Lookup Requested events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.network and event.action: lookup_requested
```
### Runtime matching criteria
All Network Lookup Requested events, including Endpoint and non-Endpoint DNS events matching the following criteria will be rendered:
```
dns.question.type: * and dns.question.name: *
```
### Sample rendered Network Lookup Requested event
![network_lookup_requested](https://user-images.githubusercontent.com/4459398/106191208-cdf76380-6167-11eb-9be7-aaf78e4cfdd3.png)
`SYSTEM` \ `NT AUTHORITY` @ `windows-endpoint-1` asked for `logging.googleapis.com` with question type `A` via `google_osconfig_agent.exe` `(4064)` `dns`
### Fields in a Network Lookup Requested event
The following fields will be used to render a Network Lookup Request event:
`user.name` \ `user.domain` @ `host.name` asked for `dns.question.name` with question type `dns.question.type` via `process.name` `(process.pid)` `network.protocol`
## Network Lookup Result events
Endpoint Network (DNS) Lookup Result events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.network and event.action: lookup_result
```
### Runtime matching criteria
All Network Lookup Result events, including Endpoint and non-Endpoint DNS events matching the following criteria will be rendered:
```
dns.question.type: * and dns.question.name: *
```
### Sample rendered Network Lookup Result event
![network_lookup_result](https://user-images.githubusercontent.com/4459398/106192595-a43f3c00-6169-11eb-95bc-4ebe331f1231.png)
`SYSTEM` \ `NT AUTHORITY` @ `windows-endpoint-1` asked for `logging.googleapis.com` with question type `AAAA` via `GCEWindowsAgent.exe` `(684)` `dns`
### Fields in a Network Lookup Result event
The following fields will be used to render a Network Lookup Result event:
`user.name` \ `user.domain` @ `host.name` asked for `dns.question.name` with question type `dns.question.type` via `process.name` `(process.pid)` `network.protocol`
## Network Connection Accepted events
Endpoint Network Connection Accepted events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.network and event.action: connection_accepted
````
### Sample rendered Network Connection Accepted event
![network_connection_accepted](https://user-images.githubusercontent.com/4459398/106200497-4f54f300-6174-11eb-8879-06b7bfc88edf.png)
Network Connection Accepted events, like the one in the screenshot above, are also rendered by the _Netflow_ row renderer, which displays information that includes the directionality of the connection, protocol, and source / destination details.
`NETWORK SERVICE` \ `NT AUTHORITY` @ `windows-endpoint-1` accepted a connection via `svchost.exe` `(328)` with result `success`
### Fields in a Network Connection Accepted event
`user.name` \ `user.domain` @ `host.name` accepted a connection via `process.name` `(process.pid)` with result `event.outcome`
## Network Disconnect Received events
Endpoint Network Disconnect Received events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.network and event.action: disconnect_received
````
### Sample rendered Network Disconnect Received event
![network_disconnect_received](https://user-images.githubusercontent.com/4459398/106205196-56cbca80-617b-11eb-83d3-26aa9670f114.png)
Network Disconnect Received events, like the one in the screenshot above, are also rendered by the _Netflow_ row renderer, which displays information that includes the directionality of the connection, protocol, and source / destination details.
`NETWORK SERVICE` \ `NT AUTHORITY` @ `windows-endpoint-1` disconnected via `svchost.exe` `(328)`
### Fields in a Network Disconnect Received event
`user.name` \ `user.domain` @ `host.name` disconnected via `process.name` `(process.pid)`
## Security Log On events
Endpoint Security Log On events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.security and event.action: log_on
```
### `event.outcome: "success"` vs `event.outcome: "failure"`
The row renderer for Security Log On events uses the `event.outcome` field to display different results for events matching:
```
event.dataset: endpoint.events.security and event.action: log_on and event.outcome: success
```
vs events matching:
```
event.dataset: endpoint.events.security and event.action: log_on and event.outcome: failure
```
### Sample rendered Security Log On / `event.outcome: "success"` event
![security_log_on_success](https://user-images.githubusercontent.com/4459398/106210917-fcd00280-6184-11eb-9c1c-564cfb375539.png)
`SYSTEM` \ `NT AUTHORITY` @ `win2019-endpoint` successfully logged in via `C:\Program Files\OpenSSH-Win64\sshd.exe`
### Fields in an Security Log On / `event.outcome: "success"` event
`user.name` \ `user.domain` @ `host.name` successfully logged in via `process.name` (`process.pid`)
### Sample rendered Security Log On / `event.outcome: "failure"` event
![security_log_on_failure](https://user-images.githubusercontent.com/4459398/106211893-b2e81c00-6186-11eb-9c34-43227c15a1f0.png)
`SYSTEM` \ `NT AUTHORITY` @ `win2019-endpoint` failed to log in via `C:\Program Files\OpenSSH-Win64\sshd.exe`
### Fields in an Security Log On / `event.outcome: "failure"` event
`user.name` \ `user.domain` @ `host.name` failed to log in via `process.name` (`process.pid`)
## Security Log Off events
Endpoint Security Log Off events with the following `event.dataset` and `event.action` will be rendered in Timeline via row renderers:
```
event.dataset: endpoint.events.security and event.action: log_off
```
### Sample rendered Security Log Off event
![security_log_off](https://user-images.githubusercontent.com/4459398/106212499-0018bd80-6188-11eb-9e91-971f360ee87a.png)
`SYSTEM` \ `NT AUTHORITY` @ `win2019-endpoint` logged off via `C:\Program Files\OpenSSH-Win64\sshd.exe`
### Fields in a Security Log Off event
`user.name` \ `user.domain` @ `host.name` logged off via `process.name` (`process.pid`)
* Add eslint rule for linting unnecessary backticks
This needs to be below the Prettier overrides at the bottom of the file to override Prettier
* Run --fix
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Setup] Split rule that explicitly allows `any` in test/mock files into its own section
- so that the rules we're about to add apply correctly to all files
* Add react/jsx-boolean-value rule
* Run --fix
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Visualize] Removes the dashboard callout for users without permission
* Check if the user has the createNew permission
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* inital setup and experiments.
* Cast into ML job metric.
* Update mappings file.
* small refactor. add basic test to build on.
* mock out anomoly detector for testing from the usage collector.
* [PH JD] collect first set of ml job stats.
* Update telemetry schema.
* Include create and finished time.
* Cache datafeed calls and find / filter by naming convention.
* Fix jest test temp.
* [PH JD] Add datafeed to the usage collector payload.
* Get e2e test working.
* Update time complexity detail / df stats lookup. O(n) -> O(1)
* Update var names.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds e2e tests for https://github.com/elastic/kibana/pull/90326
* Adds e2 tests and backfills for updating actions and expected behaviors
* Adds two tests that would fail without the fix and if a regression happens this will trigger on the regression
* Adds two tests to the PATCH for exception lists even though there is no regression there. Reason is to prevent an accidental issue there.
* Adds tests to ensure the version number does not accidentally get bumped if PATCH or UPDATE is called on actions or exceptions for immutable rules.
* Adds utilities for cutting down noise.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Fixes regression: https://github.com/elastic/kibana/issues/90319 that has not been released where in some cases such as adding actions to a rule through an update we can and will update an immutable rule and do not expect the immutable to turn into a mutable through the tags.
Simple one-liner fix, I will update in a follow on PR with a regression test for this particular use case of actions but not with this one since we optimizing for speed of pull request to back-port.
Criticality is high and impact is high as this is data bug which can cause a lot of headaches and migrations if this goes out.
### Checklist
No unit test for this one, but a functional test will be added in a follow up
- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* [maps] Top hits per entity--change to title to use recent, minor edits
* Updated TopHitsPerEntity title and description to use the term relevant
* updating top hits per entity topic to new title
Co-authored-by: Kent Marten <kmartastic@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>