* [ML] Sets max height and preserve whitespace in categorization wizard examples
* [ML] Edit following review
Co-authored-by: Pete Harverson <peteharverson@users.noreply.github.com>
* [SecuritySolutions] [Navigation] Prepare new routing and migrate overview (#101733)
* prepare new routing and migrate overview
* test fix and todo comments identified
* telemetry using app views
* navigation groups implemented
* cleaning
* export subplugin routes as route props array
* [Security Solution][Navigation] Migrate Security Solutions 'explore' tab group to deep link navigation (#102306)
* Update navigateToApp and getUrlForApp to provide the deepLinkId
* Update Hosts and Network routes to start from /hosts and /network
* Add Hosts and Network to side nav menu under "Explore" menu group
* Delete Hosts and Network old menu code
* Fix broken tests
* [SecuritySolution] Add detections subplugin to deeplink (#101791)
* prepare new routing and migrate overview
* init nav deeplink
* split detections into rules and alerts
* init exception link
* init detections
* link to rules creation page
* link to rules creation page
* rename detections to alerts
* fix unit tests
* fix rules creation page
* remove console
* fix lint error
* fix unit tests
* fix unit tests
* isolating rules and exceptions page
* replace history push with navigateToApp
* fix unit test
* temporary fix for createCoreStartMock
* update cypress
* skip failing cypress
* skip failing cypress
Co-authored-by: semd <sergi.massaneda@elastic.co>
* Migrate "Investigate" tab group to new side navigation (#102705)
* Migrate "Investigate" tab group to new side navigation
It includes:
* Timelines
* Cases
* Quick fix useFormatUrl and HeaderPage navigation
* [Security Solutions] Management navigation (#102685)
* prepare new routing and migrate overview
* test fix and todo comments identified
* telemetry using app views
* navigation groups implemented
* cleaning
* export subplugin routes as route props array
* breadcrumbs changes and sidenav generation improvements
* jest tests for breadcrumbs and navigation changes
* retrocompatibility for sections that are not yet migrated to deepLinks
* management deepLinks and plugin refactoring
* home navigation changes
* management navigation migrated to deeplinks
* jest tests fixed
* header page back link improved and tests fixed
* type errors fixes
* improve home navigation encapsulation
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fix type checking
* export header page
* fix padding
* add redirect routes
* unskip detection cypress
* fix i18n
* fix create your own rules btn
* fix cancel button on rules creation page
* test fixes
* fix breadcrumbs for rules pages
* unit test fixes
* additional fixes
* [Security Solutions] Navigation usage tracker and general changes (#103271)
* [Security Solutions] use of currentAppId$ migrated. and some small fixes
* unused constants removed
* remove unused constant
* test fix and types
* fix cypress
* fix cypress tests
* Fix case navTab permission and tests
* Revert 'timeline.isOpen' breadcrumb code that was deleted during merge
* Fix useInsertTimeline test by removing '/'
* change global navigation visible deeplinks
* fix /admininstration top level redirect to
* fix global search icon, nav order and overview hosts link
* update start a new case link
* fix rules link in exception list table
* unskip cypress tests
* update rules link
* fix full screen timeline
* fixing broken links and administration telemetry split
* remove unused comments
* remove timeline z-index and cleanup global header component
* some minor fixes
* add unit tests for detections breadcrumbs
* remove case to global/search nav when cases is none
* rename test scenario
* fix side_panel flyout
* fix cases use cases between search/gobal nav
* timeline snapshot regenerated and cypres test fixed
* rollback management tracking split as it causes unexpected errors on the telemetry component
Co-authored-by: Pablo Machado <pablo.nevesmachado@elastic.co>
Co-authored-by: Angela Chuang <6295984+angorayc@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co>
Co-authored-by: Angela Chuang <yi-chun.chuang@elastic.co>
Co-authored-by: Xavier Mouligneau <189600+XavierM@users.noreply.github.com>
Co-authored-by: Pablo Machado <pablo.nevesmachado@elastic.co>
Co-authored-by: Angela Chuang <6295984+angorayc@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co>
Co-authored-by: Angela Chuang <yi-chun.chuang@elastic.co>
Co-authored-by: Xavier Mouligneau <189600+XavierM@users.noreply.github.com>
* Add new runtime types for parsing on client/server.
* Add more runtime types.
* Remove dead code.
* Mark parameter as unused.
* Improve typing for failed journey request function.
* Add new API functions, improve typing in a few others.
* Modify API calls to work with new screenshot_ref data.
* Fix untested refactor error.
* Add required fields to runtime type.
* Update typing in failed steps component.
* Adapt client to work with old screenshots as well as new screenshot_ref.
* Refactor composite code to reusable hook.
* Implement screenshot blocks endpoint.
* Define runtime types for full-size screenshots.
* Delete dedicated screenshot and ref queries.
* Optimize screenshot endpoint by combining queries.
* Handle parsing error.
* Clean up screenshot/ref typings.
* Remove dead types. DRY a type out.
* Simplify types.
* Improve typing in step screenshot components.
* Prefer PNG to JPG for canvas composite op.
* Simplify and clean up some code.
* Remove reliance on `Ping` type, clean up types.
* Add a comment.
* Add a comment.
* Fix typing for `FailedStep` component.
* Standardize loading spinner sizes.
* Add comments to composite code.
* Remove unnecessary optional chaining.
* Reformat error string.
* Remove unneeded key from request return object.
* Add a comment to a return object explaining very large cache value.
* Make type annotation more accurate.
* Resolve some type and test errors.
* Clean up remaining type errors.
* Move type definitions to simplify imports.
* Simplify `PingTimestamp` interface.
* Refactor failing unit test to use RTL and actually test things.
* Add tests for new helper functions.
* Add a comment.
* Test `PingTimestamp` for screenshot ref data.
* Test `StepImageCaption` for ref data.
* Improve typing for step list column definitions.
* Harden a test.
* Extract code to avoid repeated declarations.
* Create centralized mock for `useCompositeImage`.
* Add test for ref to `StepScreenshotDisplay`.
* Add tests for `getJourneyDetails`.
* Extract search results wrapper to helper lib.
* Add tests for `getJourneyFailedSteps`.
* Add support for aggs to result helper wrapper.
* Write tests for `getJourneyScreenshot` and simplify type checking.
* Write tests for `getJourneyScreenshotBlocks`.
* Simplify prop types for `FailedStep`.
* Remove unused type.
* Fix regression in step navigating for new style screenshots.
* Implement PR feedback.
* Implement PR feedback.
* Implement PR feedback.
* Reduce limit of screenshot block queries from 10k to 1k.
* Remove redundant field selection from ES query.
* Implement PR feedback.
* Fix regression that caused "Last successful step" to not show an image.
* Delete unused props from `Ping` runtime type.
* More precise naming.
* Naming improvements. Add `useCallback` to prevent callback re-declaration.
* Prefer explicit props to `{...spread}` syntax.
* Remove redundant type checking.
* Delete obsolete unit tests.
* Fix a regression.
* Add effect to `useEffect`.
Co-authored-by: Justin Kambic <justin.kambic@elastic.co>
* fix up CCR centered sates in tabs content
* update snapshots list
* fix lint errors
* Change tab states for all pages in snapshot+restore
* Remove unnecessary variables
* Seems we dont need the class wrapper
* fix broken type
* Fix bug in ILM table when filtering it down
* center align search box
* fix linter errors
* fix prettier warnings
* revert content var refactor and just focus on ux
* add breakword class to paragraph so we avoid text overflowing
* fix prettier errors
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Removing feature flag changes
* Adding isExportable flag to rule type definition
* Adding isExportable flag to rule type definition
* Adding isExportable flag to rule type definition
* Filtering rule on export by rule type isExportable flag
* Fixing types
* Adding docs
* Fix condition when exportCount is 0
* Unit test for fix condition when exportCount is 0
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: ymao1 <ying.mao@elastic.co>
* Remove role region on flash messages
- just `aria-live` is enough for screen readers to read it out, and `role` was causing "Flash messages" to get read out loud repeatedly between page navigation even when empty which was annoying and not good
* Further a11y attribute recommendations from @myasonik
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
This PR fixes an edge case where a race condition mught cause the total_results from a federated content source to come back null from the server. This PR tells the server to expect null in those edge cases to prevent browser errors
Co-authored-by: Scotty Bollinger <scotty.bollinger@elastic.co>
## Summary
This adds utilities and two strategies for merging using the [fields API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-fields.html) and the `_source` document during signal generation. This gives us the ability to support `constant_keyword`, field alias value support, some runtime fields support, and `copy_to` support. Previously we did not copy any of these values and only generated signals based on the `_source` record values. This changes the behavior to allow us to copy some of the mentioned values above.
The folder of `source_fields_merging` contains a `strategy` folder and a `utils` folder which contains both the strategies and the utilities for this implementation. The two strategies are `merge_all_fields_with_source` and `merge_missing_fields_with_source`. The defaulted choice for this PR is we use `merge_missing_fields_with_source` and not the `merge_all_fields_with_source`. The reasoning is that this is much lower risk and lower behavior changes to the signals detection engine.
The main driving force behind this PR is that ECS has introduced `constant_keyword` and that field has the possibility of only showing up in the fields section of a document and not `_source` when index authors do not push the `constant_keyword` into the `_source` section. The secondary driving forces behind this behavioral change is that some users have been expecting their runtime fields, `copy_to` fields, and field alias values of their indexes to be copied into the signals index.
Both strategies of `merge_missing_fields_with_source` and `merge_all_fields_with_source` are considered Best Effort meaning that both strategies will not always merge as expected when they encounter ambiguous use cases as outlined in the `README.md` text at the top of `source_fields_merging` in detail.
The default used strategy of `merge_missing_fields_with_source` which has the simplest behavior will work in most common use cases. This is simply if the `_source` document is missing a value that is present in the `fields`, and the `fields` value is a primitive concrete value such as a `string` or `number` or `boolean` and the `_source` document does not contain an existing object or ambiguous array, then the value will be merged into `_source` and a new reference is returned. If you call the strategy twice it should be idempotent meaning that the second call will detect a value is now present in `_source` and not re-merge a second time.
* 301 unit tests were added
* Extensive README.md docs are added
* e2e tests are updated to test scenarios and ambiguity and conflicts from previously to support this effort.
* Other e2e tests were updated
* One bug with EQL and fields was found with a workaround implemented. See https://github.com/elastic/elasticsearch/issues/74582
* SearchTypes adjusted to use recursive TypeScript types
* Changed deprecated for `@deprecated` in a few spots
* Removed some `ts-expect-error` in favor of `??` in a few areas
* Added a new handling of epoch strings and tests to `detection_engine/signals/utils.ts` since fields returns `epoch_millis` as a string instead of as a number.
* Uses lodash safer set to reduce changes of prototype pollution
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
### Risk Matrix
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Prototype pollution | Low | High | Used lodash safer set |
| Users which have existing rules that work, upgrade and now we do not generate signals due to bad merging of fields and _source | Mid | High | We use the safer strategy method, `merge_missing_fields_with_source `, that is lighter weight to start with. We might add a follow up PR which enables a key in Kibana to turn off merging of fields with source. We added extensive unit tests and e2e tests. However, unexpected unknowns and behaviors from runtime fields and fields API such as geo-points looking like nested fields or `epoch_milliseconds` being a string value or runtime fields allowing invalid values were uncovered and tests and utilities around that have been added which makes this PR risky |
| Found a bug with using fields and EQL which caused EQL rules to not run. | Low | High | Implemented workaround for tests to pass and created an Elastic ticket and communicated the bug to EQL developers. |
Co-authored-by: Frank Hassanabad <frank.hassanabad@elastic.co>
* Update Enterprise Search request handler to send back an error connecting header
- vs only distinguishing error connecting issues by 502 status
+ clarify comment where this.handleConnectionError is called - for the most part, auth issues should already be caught by 401s in logic above
* Update HttpLogic to set errorConnecting state based on header
+ update tests etc to match read-only-mode state
* [Tech debt] Gracefully handle invalid HTTP responses
I've noticed this error a few times after Kibana gets shut down (http.response is undefined) so figured I would catch it here
* Fix missing try/catch/flashAPIErrors on engines overview
- This is the only http call I found missing a try/catch across our codebase, so we should be set for all views correctly flashing an API error that receive a 5xx response from ent-search
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
* Set up new BetaNotification component
* Update shared page template to append new beta notification item to side nav
NOTE: I'm mutating the array because:
- returning a new instance leads to a lot of really annoying type errors
- the side nav's we're getting are entirely static with predictable items & and always come from us anyway
- this is eventually going to get removed, and I'm optimizing for easy-to-remove code
* Add beta notification to error connecting state
- to help users/SDH cases where users cannot connect at all
* Fix type error
- sideNav itself can be undefined but not `sideNav.items`
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
Default to either the installed version of an integration, or the latest
available version based on installation status when a version is not
included in the integration details URL.
Closes#93393
Co-authored-by: Kyle Pollich <kyle.pollich@elastic.co>
* move content from modal to flyout with message table below chart
* update file name from modal to flyout
* update messages endpoint for range to use with chart range
* add show in chart action for messages table
* add job messages title and make flyout smaller
* cancel the previous session
* split to 3 tasks
* fixes
* cancellation
* updated tests
* split out and improve jest tests
* cleanup previous session properly
* don't fail delete and cancel if item was already cleaned up
* test
* test
* ignore resource_not_found_exception when deleting an already cleared \ expired async search
* jest
* update jest
* api int
* fix jest
* testssss
* Code review @dosant
* types
* remove any
* Fix merge
* type
* test
* jest
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Liza Katz <lizka.k@gmail.com>
Co-authored-by: Liza K <liza.katz@elastic.co>
* Remove es sql strategy from behind Labs project, remove legacy essql code, remove last spot of legacy elasticsearch client from canvas
* clean up test
* fix es field test
* remove comment
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Poff Poffenberger <poffdeluxe@gmail.com>
Closes#103061. We use polished.tint() in several places to have a subdued highlight for an element. With the polished upgrade to 3.x from several weeks ago came a bug fix for tint() that applied the tint fraction in a different way. The fix for us to invert those fractions (eg 0.1 becomes 0.9).
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
