Commit graph

4598 commits

Author SHA1 Message Date
Andrew Goldstein
d503b7268a
[SIEM] Endgame events on the SIEM Overview page (#47774) (#47904)
## Summary

* Adds Endgame events to the SIEM Overview page, per the following screenshot:

<img width="1680" alt="overview-chrome" src="https://user-images.githubusercontent.com/4459398/66524250-26a47800-eaaf-11e9-8ff9-311c031e5d00.png">

* Adds `endgame-*` to the default SIEM index pattern, per the following screenshot:

<img width="1665" alt="siem-advanced-settings" src="https://user-images.githubusercontent.com/4459398/66524300-45a30a00-eaaf-11e9-93c3-dce74917e73a.png">

RELEASE NOTE: To view Endgame events in existing SIEM deployments, you must manually add `endgame-*` to the SIEM index pattern in `Kibana Management > Advanced Settings > SIEM > Elasticsearch indices`. Also note that the `Reset to default` feature for this setting in the Advanced Settings page now includes `endgame-*`.

* Adds the GraphQL plumbing for rendering Endgame data in the Timeline via row renderers (in an upcoming PR), with the introduction of the following fields:

```
dns.question.name
dns.question.type
dns.resolved_ip
dns.response_code
endgame.exit_code
endgame.file_name
endgame.file_path
endgame.logon_type
endgame.parent_process_name
endgame.pid
endgame.process_name
endgame.subject_domain_name
endgame.subject_logon_id
endgame.subject_user_name
endgame.target_domain_name
endgame.target_logon_id
endgame.target_user_name
event.code
file.name
process.hash.md5
process.hash.sha1
process.hash.sha256
user.domain
winlog.event_id
```

## Testing

### Cypress
The `smoke_tests/overview/overview.spec.ts` Cypress test was updated to include the new counts on the Overview page, per the screenshot below:

![cypress-overview-spec](https://user-images.githubusercontent.com/4459398/66529142-8c98fb80-eabf-11e9-800e-a0d9e1e51d6d.png)

### API Integration test

The Overview page API integration test `xpack/test/api_integration/apis/siem/overview_host.ts` was updated to include counts of mock Endgame data added to `test/functional/es_archives/auditbeat/overview/data.json.gz`

### Unit tests

Overview page unit tests were updated to include the new Endgame event counts

### Desk testing

* Desk tested by hand-editing `components/page/overview/overview_host/index.tsx` and setting the `endDate` and `startDate` values below to a fixed datetime:

```
<OverviewHostQuery endDate={endDate} sourceId="default" startDate={startDate}>
```

The counts shown on the overview page where then compared to the counts shown in the timeline in the same date period, to verify the counts match 1:1.

* The additional fields mentioned above in this PR (e.g. `dns.question.name`,`endgame.target_domain_name`) that are now being requested via GraphQL can be seen via the Timeline Inspect (query) feature:

1) Enter `event.module: endgame` in the Timeline KQL bar. (Adjust the date range if necessary.)
2) After Endgame events are displayed in the timeline, click the Inspect button in the Timeline settings gear.

The additional fields (and values) will be included in the Inspect query Request / Response tabs.

### Cross-browser dark/light testing
#### Firefox

<img width="1680" alt="overview-firefox" src="https://user-images.githubusercontent.com/4459398/66524773-9c5d1380-eab0-11e9-9383-c155872881b0.png">

#### Safari

<img width="1680" alt="overview-safari" src="https://user-images.githubusercontent.com/4459398/66524790-a54de500-eab0-11e9-9786-aa7dbe18c1bf.png">

#### IE11

This PR was *not* tested in IE11 due to the current blocker with `react-reverse-portal`

https://github.com/elastic/siem-team/issues/465
https://github.com/elastic/ecs-dev/issues/178
2019-10-11 08:42:58 -06:00
Poff Poffenberger
e5b6e90fa7
[Canvas] Adding i18n for transitions (#47626) (#47894)
* Adding i18n for transitions

* removing function

* Slide copy fix
2019-10-11 09:25:26 -05:00
Pete Harverson
56be26c99e
[ML] Adds new SIEM auditbeat, winlogbeat and packetbeat modules (#47848) (#47945)
* [ML] Adds new SIEM auditbeat, winlogbeat and packetbeat modules

* [ML] Removed rare_country_for_user jobs

* [ML] Removed rare_country_for_user jobs from manifests
2019-10-11 14:20:17 +01:00
Dima Arnautov
6c577688d1
[ML] Enhance created_by check for custom rules (#47825) (#47943)
* [ML] delete created_by only for wizards

* [ML] use CREATED_BY_LABEL enum

* [ML] move processCreatedBy
2019-10-11 15:12:38 +02:00
Liza Katz
730bff388c
Reconstruct setup\start data shim APIs (due to savedObject N/A in setup) (#47851) (#47924)
* No saved objects client in data.setup

* replace old loading of data/public/setup

* test fixes

* fix list_control_factory

* range_control_factory
2019-10-11 09:55:31 +03:00
Yulong
bb92b8b8b5
[Code] replace nodegit with native git (#45491) (#47920)
* [Code]  use native git to iterate git files
* [Code] use native git to clone/update repository
* [Code] git history using native git
* [Code] use native git to read file tree and file content
* [Code] fix the 'bad file' warning from status api
* [Code] use native git to handle worktree
* [Code] use native git to resolve references
* [Code] use native git to handle blame / diff
* [Code] patch git binaries in kibana build script
* [Code] migrate unit tests to use native git
2019-10-11 12:50:06 +08:00
Thomas Neirynck
936e018172
[SIEM][Maps] Use vector tiles for rendering EMS basemap (#47867) (#47911) 2019-10-10 22:31:09 -04:00
Nathan Reese
8cb0b75485
[Maps] retrieve geo_point value from docvalue_fields instead of _source (#47389) (#47917)
* [Maps] retrieve geo_point value from docvalue_fields instead of _source

* add functional test ensuring _search request only pulls what is needed

* clean up i18n translation fiels
2019-10-10 20:15:46 -06:00
Steph Milovic
3d9fb52cee
[SIEM] type fix (#47708) (#47828) 2019-10-10 15:19:39 -07:00
Wylie Conlon
43e8c4420c
[lens] Basic usage telemetry for total visualizations, and by type (#47597) (#47899)
* [lens] Basic usage telemetry for total visualizations, and by type

* Implement overall, 30 day, 90 day tracking for saved visualizations

* Add forgotten file

* Implement collection using scripted metrics

* Add functional test to ensure painless script is working
2019-10-10 17:55:26 -04:00
Corey Robertson
09d72bb9b5
[Canvas] Translate More Expression Arguments (#47028) (#47844)
* Translate other canvas arguments

* Update Snapshot

* Fix incorrect i18n key
2019-10-10 16:54:22 -04:00
Ryland Herrick
2fe225009d
[Code] Export integration components (#47636) (#47880)
* Expose CodeBlockPanel component

This separates the current CodeBlock component into two:
* CodeBlock, which simply renders the code view without
padding/margin
* CodeBlockPanel which wraps the CodeBlock in an EUIPanel and allows
overrides

It seems like APM will want to use the former for their integration,
while the latter is currently used internally by Code. It's very simple,
though, and could absolutely be inlined.

* Update demo page to use CodeBlock

This has no styling, and so a header could go right against it, it
could be shown/hidden distinct from the header, etc.

* Export our current integration components from main index

Adds a 'shared' manifest that does all the reaching in; the main one
just re-exports that.

* Move shared exports to the frontend manifest

This was incorrectly placed a level too high, in the plugin itself.

* Rename to better reflect relationship

CodeBlockPanel = EuiPanel + CodeBlock

* Distinguish monaco CSS overrides with page-specific layout
2019-10-10 15:12:06 -05:00
Poff Poffenberger
297ef13e96
[7.x] [Canvas] i18n for all known canvas errors (#46945) (#47838)
* [Canvas] i18n for all known canvas errors (#46945)

* i18n for all known canvas errors

* Removing error message not seen by users

* Fixing weird merge
2019-10-10 15:10:33 -05:00
Nathan L Smith
1008c9d07e
[7.x] [APM] Keywords to select legend color in TPM graph (#476… (#47874)
If the result contains "ok" or "success", use the same color we use for HTTP 200.

If the result contains "error" or "failure", use the same color we use for HTTP 500.

Fixes #18707
2019-10-10 14:15:13 -05:00
Nathan L Smith
6b57c79f11
[7.x] [APM] Experimental Service Map front end (#46497) (#47840)
Add service map tabs on the main APM screen and for individual services.

This is not yet hooked up to work with back-end data, so it always shows the same hard-coded graph.

This is experimental, so you must have x-pack.apm.serviceMapEnabled: true in your Kibana config for it to show up.

Also add "PSF" to the list of allowed licenses since a new dependency added uses this license (it's on the [green list](https://github.com/elastic/open-source/blob/master/elastic-product-policy.md#green-list).)

Fixes #44890
Fixes #44853
2019-10-10 13:02:54 -05:00
Nathan L Smith
67ce81bf53
[7.x] [APM] Correctly plurarlize view in discover link (#47618) (#47836)
Fixes #46549
2019-10-10 13:02:33 -05:00
Nathan L Smith
8e64935e9e
[7.x] [APM] Documentation and alignment for impact column (#47… (#47839)
In the traces and transactions tables, left align the impact column heading and add a tooltip explaining what it means.

Fixes #44313
Fixes #28559
2019-10-10 13:01:47 -05:00
Nathan L Smith
c00d1558c3
[7.x] [APM] Add docs about running in VSCode to README (#47536) (#47833) 2019-10-10 13:01:18 -05:00
Nathan L Smith
f8da8f3744
[7.x] [APM] Add User agent to trace summary (#47526) (#47837) 2019-10-10 13:00:51 -05:00
Chris Davies
3329e5f72c
[Lens] Lazy load Lens index patterns (#46785) (#47749) 2019-10-10 13:15:54 -04:00
Chris Roberson
51a7bb22be
Use fixed_interval instead of interval (#47725) (#47786) 2019-10-10 10:55:03 -04:00
Joe Reuter
06764edfc7
improve wording of save modal and listing page (#47443) (#47823) 2019-10-10 16:42:44 +02:00
Joe Reuter
4a1dc494b8
[Graph] Style fixes (#47667) (#47824) 2019-10-10 16:42:37 +02:00
patrykkopycinski
061c0d47c0
[SIEM] Fix GraphQL type generation fails to generate types (#47638) (#47748) 2019-10-10 15:29:40 +02:00
Aleh Zasypkin
da745fad97
[7.x] Preserve URL fragment during SAML handshake. (#47742) 2019-10-10 14:36:49 +02:00
Joe Reuter
2405752e17
[Graph] Do not carry over diversity field when switching data source (#47560) (#47799) 2019-10-10 14:06:32 +02:00
Søren Louv-Jansen
0d77169b64
[7.x] [APM] Use new platform for toast notifications (#47276) | [APM] Agent configuration phase 2 (#46995) (#47806)
* [APM] Use new platform for toast notifications (#47276)

* [APM] Use new platform for toast notifications

* fix more tests

* remove comment

* [APM] Agent configuration phase 2 (#46995)

* [APM] Agent Config Management Phase 2

* Add status indicator

* Extract TimestampTooltip component

* Remove unused StickyTransactionProperties component

* Fix snapshot and minor cleanup

* Minor cleanup

* Display settings conditionally by agent name

* Fix client

* Format timestamp

* Minor design feedback

* Clear cache when clicking refresh

* Fix test

* Revert t() short hand

* Fix translations

* Add support for “all” option

* Fix API tests

* Move delete button to footer

* Fix snapshots

* Add API tests

* Fix toasts

* Address feedback and ensure order when searching for configs

* Fix snapshots

* Remove timeout
2019-10-10 13:51:15 +02:00
Cauê Marcondes
9b8979424a
[APM]Show span.http.response.status_code and span.http object in Span details flyout (#47322) (#47793)
* Extracting HttpStatusBadge component, in order to be used on SpanFlyout

* Changing tab labels to metadata

* Changing tab labels to metadata

* Refactoring section labels

* Refactoring section labels

* Refactoring section labels

* Refactoring http info component

* adding new translation

* Refactoring section

* fixing type check on test

* Using lodash pick to get the correct properties
2019-10-10 13:37:29 +02:00
Dima Arnautov
e56e60dd0a
[ML] Converts Recognizer job page to React (#47429) (#47804)
* [ML] wip recognize job

* [ML] error handling

* [ML] layout enhancements

* [ML] errors handling

* [ML] fix badges alignment and error message

* [ML] components split

* [ML] load module from the component, reset support

* [ML] validators

* [ML] check view route and resolver

* [ML] validation

* [ML] remove angular code

* [ML] change layout, fix patterns

* [ML] job response fix

* [ML] directive test

* [ML] directive test

* [ML] remove lookbehind regexp

* [ML] fix IE11 layout issues

* [ML] remove form tag, refactor to formState

* [ML] aria labels, remove unused i18n

* [ML] align kibana objects and reset button

* [ML] column layout for jobs response

* [ML] align icons and buttons

* [ML] check existing kibana objects

* [ML] don't display kibana objects panel if empty

* [ML] result and reset button order

* [ML] add EuiHorizontalRule

* [ML] use time range and full index dat

* [ML] no check icon for existing objects

* [ML] toast for exception during a jobs setup

* [ML] disable panels growth, advanced settings with described form

* [ML] fix timeRange for results url

* [ML] fix i18n

* [ML] use EuiSwitch

* [ML] PR remarks

* [ML] job settings form component

* [ML] i18n

* [ML] onChange fix

* [ML] custom hook for partial state update

* [ML] jobGroups update

* [ML] PR remarks

* [ML] fix imports
2019-10-10 13:27:41 +02:00
Tim Roes
d29b1846be
Prepare TS for 3.7 upgrade (#47703) (#47792) 2019-10-10 10:27:02 +02:00
Dario Gieselaar
7e9964d17b
[7.x] [APM] Garbage collection metrics charts (#47023) (#47675)
* [APM] Garbage collection metrics charts

Closes #36320.

* Review feedback

* Display average of delta in gc chart
2019-10-10 09:26:24 +02:00
Tim Roes
7033c32ae9
Prepare for TS 3.7 upgrade (#47683) (#47734) 2019-10-10 08:22:35 +02:00
Nathan Reese
5ac60370ff
[Maps] provide drag-n-drop support to order tooltip properties (#46631) (#47771)
* [Maps] tooltip custom labels

* add drag handlers for re-ordering tooltip property order

* add trash button to remove property

* add jest tests for AddTooltipFieldPopover

* sort EMS file tooltip properties

* update TooltipSelector jest test

* clean up AddTooltipFieldPopover field sorting

* remove console statements

* add more styles when row is getting dragged

* change reorder aria label

* move css changes into seperate file

* allow adding multiple fields before closing popover

* clear checked state on Add

* update jest snapshots

* use FieldIcon to display field type as icon

* add bottom border to tooltip field

* avoid flash after drag and drop

* Tooltip styles (#32)

* update TooltipSelector snapshot

* replace 24px with
2019-10-09 18:11:34 -06:00
Tim Sullivan
76cce0063e
[Reporting] Performance instrumenting 1 (#47081) (#47729) 2019-10-09 14:37:53 -07:00
Walter Rafelsberger
b6e7432e8b
[transform] Move ML "Data Frame Transforms" to Kibana management section "Transforms". (#45880) (#47745)
Moves "Data frame transforms" from the ML plugin to its own "transform" plugin within the Kibana management section.
2019-10-09 12:21:47 -07:00
Poff Poffenberger
06f4e0a86e
[Canvas] i18n for TextStylePicker and Expression components (#46981) (#47707)
* i18n Text Style Picker

* i18n expression component

* Cleanup
2019-10-09 13:27:08 -05:00
Nathan Reese
b05c68b0e5
[Maps] Move sort out of top hits configuration for ES documents source (#47361) (#47732)
* [Maps] Move sort out of top hits configuration for ES documents source

* add migration script to convert topHitsTimeField to sortField

* update i18n translations

* add jest test for es docs source UpdateSourceEditor component

* remove time configuration from top hits docs

* update migrations integration expect statement

* review feedback

* reverse hits list so top documents by sort are drawn on top

* update functional test expect to account for reversing hits order

* update another functional test expect clause for reversing hits
2019-10-09 11:52:04 -06:00
Melissa Alvarez
cb63f5e9a7
[ML] Overview Page - follow-ups (#47637) (#47709)
* add tooltip, fix sort, style tweaks

* show alert icon if group max score error

* remove unnecessary comment

* update max score tooltip text
2019-10-09 12:26:32 -04:00
Liza Katz
f5430748a2
Remove document links dependency in data plugin (#44574) (#47702)
* Use docLinks service

* Update tests to use docLinks mock service

* Test fixes

* Fixed jest test to work with functional tests

* snaps

* removed clicking on button test - as it actually tests EuiPopover
2019-10-09 18:30:52 +03:00
Joe Reuter
421110f3cf
[Graph] Fix functional tests (#47053) (#47704) 2019-10-09 17:30:15 +02:00
Daniil Suleiman
e849663cff
[Vis: Default Editor] Euificate table options tab (#46013) (#47666)
* Euificate table options tab

* Add validation

* Use SchemaConfig for dimensions

* Update snapshots
2019-10-09 18:03:36 +03:00
Mike Côté
8832a5a869
Add missing API documentation for alerting APIs (#47515) (#47695) 2019-10-09 11:00:45 -04:00
Clint Andrew Hall
600a035ab2 [i18n][chore] Sort the i18n Dictionaries (#47487) (#47640) 2019-10-09 09:33:49 -05:00
Mike Côté
601b0fbb2b
Cleanup alerting API calls on update (#47548) (#47694) 2019-10-09 10:19:32 -04:00
Mike Côté
d79af38b27
Add KQL support to alert and actions find API (#47530) (#47693)
* Add KQL support to alert and actions find API

* Fix failing jest tests
2019-10-09 10:18:38 -04:00
Mike Côté
05450ea2f6
Add docs mentioning required actionGroups for alert types (#47513) (#47691) 2019-10-09 10:16:36 -04:00
Mike Côté
cd4d020db9
Handle scenario security plugin is enabled but Elasticsearch security is disabled (#47504) (#47690) 2019-10-09 10:13:59 -04:00
Dima Arnautov
341005cb4d
[ML] Reorder buttons for calendar form (#47594) (#47682)
* [ML] reorder buttons for calendar form

* [ML] update snapshot
2019-10-09 15:34:24 +02:00
Elizabet Oliveira
111e5d1d06
[Maps] Cleanup SASS and add linting (#47211) (#47611)
* Enabling sass lint compressed maps

* Fixing linting error in all maps sass files
2019-10-09 14:28:21 +01:00
Yulong
617e8f17c6
[Code] fix binary file download (#47650) (#47679) 2019-10-09 20:42:29 +08:00