* Update EPM package registry typings to reflect registry changes
* Change `dataset.id` references to `dataset.name`
* Fix RegistryStream
* Fix packageToConfigDatasourceInputs service
* Fix assignPackageStream service
* Fix validateDatasource service
* Fix configure data source components
* Fix variable
* Fix stream template installation
* Add support for `input[].dataset.type` and change `stream.dataset` mapping to be object containing `name` with instead of just a string
* Nest package information under `meta` in agent config yaml
* Move `dataset.type` to stream level instead of input level
* Make single call to fetch registry package information instead of doing it per stream
* Fix type issues
* Update endpoint test assertion
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [ui-shared-deps] include styled-components
* update snapshots caused by react-is upgrade
* fix jest snapshots
* declare react-is dep in x-pack too so it uses the right version
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* fixes 'sets and reads the url state for timeline by id' test
* fixes merge issue
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* refactor: 💡 rename folder to "explore_data"
* style: 💄 check for "share" plugin in more semantic way
"explore data" actions use Discover URL generator, which is registered
in "share" plugin, which is optional plugin, so we check for its
existance, because otherwise URL generator is not available.
* refactor: 💡 move KibanaURL to a separate file
* feat: 🎸 add "Explore underlying data" in-chart action
* fix: 🐛 fix imports after refactor
* feat: 🎸 add start.filtersFromContext to embeddable plugin
* feat: 🎸 add type checkers to data plugin
* feat: 🎸 better handle empty filters in Discover URL generator
* feat: 🎸 implement .getUrl() method of explore data in-chart act
* feat: 🎸 add embeddable.filtersAndTimeRangeFromContext()
* feat: 🎸 improve getUrl() method of explore data action
* test: 💍 update test mock
* fix possible stale hashHistory.location in discover
* style: 💄 ensureHashHistoryLocation -> syncHistoryLocations
* docs: ✏️ update autogenerated docs
* test: 💍 add in-chart "Explore underlying data" unit tests
* test: 💍 add in-chart "Explore underlying data" functional tests
* test: 💍 clean-up custom time range after panel action tests
* chore: 🤖 fix embeddable plugin mocks
* chore: 🤖 fix another mock
* test: 💍 add support for new action to pie chart service
* feat: 🎸 add kibana.yml to disable in-chart "explore data" actio
Co-authored-by: Anton Dosov <anton.dosov@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* move last snapshot to inline
* move legacy files to legacy subfolder
* move request types out of legacy
* export Headers from http instead of elasticsearch
* renaming - first pass
* renaming - second pass
* fix core mocks
* adapt new calls
* update generated doc
* fix IT test mocks
* fix new usages
* Update Canvas Plugin documentation
* Small fix
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* Update x-pack/plugins/canvas/README.md
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
* rollover data stream when index template mappings are not compatible
* update error messages
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Partially disable test files.
* Use DockerServers in EPM tests.
* Only run tests when DockerServers have been set up
* Reenable ingest manager API integration tests
* Pass new test_packages to registry container
* Enable DockerServers tests in CI.
* Correctly serve filetest package for file tests.
* Add helper to skip test and log warning.
* Reenable further file tests.
* Add developer documentation about Docker in Kibana CI.
* Document use of yarn test:ftr
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [APM] Use licensing from context
We added the usage of `featureUsage.notifyUsage` from the licensing plugin in #69455.
This required us to use `getStartServices to add `licensing` to `context.plugins`.
In #69838 `featureUsage` was added to `context.licensing`, so we don't need to add it to `context.plugins`.
## [Security] `Investigate in Resolver` Timeline Integration
This PR adds a new `Investigate in Resolver` action to the Timeline, and all timeline-based views, including:
- Timeline
- Alert list (i.e. Signals)
- Hosts > Events
- Hosts > External alerts
- Network > External alerts
### Resolver Overlay
When the `Investigate in Resolver` action is clicked, Resolver is displayed in an overlay over the events. The screenshot below has placeholder text where Resolver will be rendered:
The Resolver overlay is closed by clicking the `< Back to events` button shown in the screenshot above.
The state of the timeline is restored when the overlay is closed. The scroll position (within the events), any expanded events, etc, will appear exactly as they were before the Resolver overlay was displayed.
### Case Integration
Users may link directly to a Timeline Resolver view from cases via the `Attach to new case` and `Attach to existing case...` actions show in the screenshot below:
When users click the link in a case, Timeline will automatically open to the Resolver view in the link.
### URL State
Users can directly share Resolver views (in saved Timelines) with other users by copying the Kibana URL to the clipboard when Resolver is open.
When another user pastes the URL in their browser, Timeline will automatically open and display the Resolver view in the URL.
### Enabling the `Investigate in Resolver` action
In this PR, the `Investigate in Resolver` action is only enabled for events where all of the following are true:
- `agent.type` is `endpoint`
- `process.entity_id` exists
### Context passed to Resolver
The only context passed to `Resolver` is the `_id` of the event (when the user clicks `Investigate in Resolver`)
### What's next?
- @oatkiller will replace the placeholder text shown in the screenshots above with the actual call to Resolver in a separate PR
- I will follow-up this PR with additional tests
- The action text `Investigate in Resolver` may be changed in a future PR
- Hide the `Add to case` action in timeline-based views (it's currently visible, but disabled)
* hide timeline on Management pages
* adjust managment page view styles
* Added additional tests for validating no timeline button on management views
* centralize API Path responses and reuse across some tests
* Fix state being reset incorrectly
### Summary
This PR is a follow up to #68864 . That PR used a partial to differentiate between new and existing comments, this meant that comments could be updated when they shouldn't. It was decided in our discussion of exception list schemas that comments should be append only. This PR assures that's the case, but also leaves it open to editing comments (via API). It checks to make sure that users can only update their own comments.
This pr adds convenient license support to dynamic uiActions in x-pack.
Works for actions created with action factories & drilldowns.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* First round of UX tweaks
- Fixed potential text overflow issue on descriptions
- Removed border around text input when editing description
* Updated the on-failure pipeline description copy
* Properly encode URI component pipeline names
* use xjson editor in flyout
* also hide the test flyout if we are editing a component
* add much stronger dimming effect when in edit mode
* also added dimming effect to moving state
* remove box shadow if dimmed
* add tooltips to dropzones
* fix CITs after master merge
* fix nested rendering of processors tree
* only show the tooltip when the dropzone is unavaiable and visible
* keep white background on dim
* hide controls when moving
* fix on blur bug
* Rename variables and prefix booleans with "is"
* Remove box shadow on all nested tree items
* use classNames as it is intended to be used
* Refactor SCSS values to variables
* Added cancel move button
- also hide the description in move mode when it is empty
- update and refactor some shared sass variables
- some number of sass changes to make labels play nice in move
mode
- changed the logic to not render the buttons when in move mode
instead of display: none on them. The issue is with the tooltip
not hiding when when we change to move mode and the mouse event
"leave" does get through the tooltip element causing tooltips
to hang even though the mouse has left them.
* Fixes for monaco XJSON grammar parser and update form copy
- Monaco XJSON worker was not handling trailing whitespace
- Update copy in the processor configuration form
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [Endpoint] use rbush to only render resolver nodes that are in view in the DOM
* Add related events code back
* Change processNodePositionsAndEdgeLineSegments selector to return a function that takes optional bounding box
* Refactor selectors to not break original, and not run as often
* Memoize rtree search selector, fix tests
* Update node styles to use style hook, update jest tests
* Fix type change issue in jest test
* redirect app/security to app/security/overview
* missing re-naming initialization
* add unit test for intialization value of indicesExists
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* refactor: 💡 rename folder to "explore_data"
* style: 💄 check for "share" plugin in more semantic way
"explore data" actions use Discover URL generator, which is registered
in "share" plugin, which is optional plugin, so we check for its
existance, because otherwise URL generator is not available.
* refactor: 💡 move KibanaURL to a separate file
* feat: 🎸 add "Explore underlying data" in-chart action
* fix: 🐛 fix imports after refactor
* feat: 🎸 add start.filtersFromContext to embeddable plugin
* feat: 🎸 add type checkers to data plugin
* feat: 🎸 better handle empty filters in Discover URL generator
* feat: 🎸 implement .getUrl() method of explore data in-chart act
* feat: 🎸 add embeddable.filtersAndTimeRangeFromContext()
* feat: 🎸 improve getUrl() method of explore data action
* test: 💍 update test mock
* fix possible stale hashHistory.location in discover
* style: 💄 ensureHashHistoryLocation -> syncHistoryLocations
* docs: ✏️ update autogenerated docs
* test: 💍 add in-chart "Explore underlying data" unit tests
* test: 💍 add in-chart "Explore underlying data" functional tests
* test: 💍 clean-up custom time range after panel action tests
* chore: 🤖 fix embeddable plugin mocks
* chore: 🤖 fix another mock
* test: 💍 add support for new action to pie chart service
Co-authored-by: Anton Dosov <anton.dosov@elastic.co>
This PR migrates all old alerts with the `alerting` consumer to have `alerts` instead.
This is because in 7.9 we changed the feature ID and we need these to remain in sync otherwise the RBAC work (https://github.com/elastic/kibana/pull/67157) will break old alerts.
* Add store class
* fix tests
* fix the createIndex bug
* add reportingstore test
* change function args
* nits
* add test for automatic index creation failure recovery
* [APM] Add callout
Showing a callout to inform the user we have detected a high cardinality in unique transaction names and enabling them how to fix it.
* Changed color and icon
* Updated copy and styling
* Check number of returned buckets
* Add translations and docs
* Update docs link
Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>
* Fix tests
Co-authored-by: Casper Hübertz <casper@formgeist.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Brandon Morelli <bmorelli25@gmail.com>
Introduces migrations into Encrypted Saved Objects.
The two main changes here are:
1. The addition of a createMigration api on the EncryptedSavedObjectsPluginSetup.
2. A change in SavedObjects migration to ensure they don't block the event loop.
Relates to #69642. If the user doesn't have the appropriate privileges for the annotations index, instead of failing with a 500, we now catch the error and log a warning to the console.
* Ingest: Rename datasource Layout prop to `onCancel`
* Endpoint: Policy list - swap use of endpoint package hook for redux middleware
* Endpoint: Add tests cases for `sendGetEndpointSecurityPackage()` method
* Endpoint: add policy list store tests for new action
* Updates list entry schema, exposes exception list client, updates tests
* create new de list schema and unit tests
* updated route unit tests and types to match new list schema
* updated existing DE exceptions code so it should now work as is with updated schema
* test and types cleanup
* cleanup
* update unit test
* updates per feedback
This modifies the routes related to log rate and category analysis to use the new programmatic APIs provided by the `ml` plugin to access the results index and job info. Because that access is facilitated via the request context, the log analysis lib was converted from classes to plain functions.
At the same time the routes have been updated to use the most recent validation and error handling patterns.
* Pulls out existing ML integration from the service maps
* - removes ML job creation flyout in integrations menu on the service details UI
- removes ML searches and transforms in the transaction charts API
- removes unused shared functions and types related to the legacy ML integration
* removes unused translations for APM anomaly detection
* Adds tags to TODOs for easy searching later
* Ingest: Expose `registerExternalCallback()` method out of Ingest server `start` lifecycle
* Ingest: Add support for External Callbacks on REST `createDatasourceHandler()`
* Ingest: expose DatasourceServices to Plugin start interface
* Endpoint: Added Endpoint Ingest handler for Create Datasources
- Also moved the temporary logic from the middleware
to the handler (still temporary)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* add progress indicator to creation wizard page
* only show progress bar if job is started immediately
* add title and switch to timeout
* fix progress check
* clean up interval on unmount
* fix types
* clear interval if stats undefined. show progress if job created
* Exclude glob-only (*) index pattern from map layers
This pattern is a special case that our map should ignore, as including
it causes all indexes to be queried.
* Ignore CCS glob pattern in our embedded map
Users may have this pattern for cross-cluster search, and it should
similarly be excluded when matching Security indexes.
As security_solution continues to integrate with lists, the absents of
these types will lead to lots of implicit anys and false positives.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [APM] Storybook theme fixes
The changes adding theme support in #69362 broke some of the Storybook stories.
Add decorators to wrap some of the stories in the theme context.
This should be done in a global decorator, but our current storybook setup doesn't support this. It also would be nice to be able to switch between light/dark mode, but that's something we can add in the future.
* Remove unused import
* Adds missing decorator to cytoscape examples + adds a new real-world example
Co-authored-by: Oliver Gupte <olivergupte@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Return attributes when telemetry created instead of whole saved object. Update integration test
* Change 'maps-telemetry' to 'maps'
* No need to create a saved object anymore. This is leftover from task manager telemetry mgmt
* Add test confirming attrs undefined. Change tests to check for 'maps' iso 'maps-telemetry'
* Add two more tests confirming expected telemetry shape
* Review feedback. Use TELEMETRY_TYPE constant and set to APP_ID
* Fixing resolver alert generation
* Splitting indices up
* Removing tests that could randomly fail because of the generation code
* Adding support for multiple indices
* Updating archives with the new index names
* Removing alerts data stream
* Switching to process instead of fake
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Adding specific apis for each plugin
* adding metric hosts stat
* addressing PR comment
* addressing PR comments
* changing series to key/value
* exporting interfaces
* adding label to stat
* refactoring types
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Change the bootstrap of the app
* rename SiemPageName to SecurityPageName
* modify alerts routes
* modify cases routes
* modify hosts routes
* modify network routes
* modify overview routes
* modify timelines routes
* wip change management route
* change route for common
* some fixing from the first commit
* modify route for management
* update url format hook to use history
* bug when you click on external alerts from host or network
* improvement from josh feedback
* redirect siem to security solution
* a little clean up
* Fix types
* fix breadcrumbs
* fix unit test
* Update index.tsx
* Fix cypress
* bug remove timeline when you are in case configure
* Fix functionel test for management
* Fix redirect siem + ml
* fixes some cypress tests
* adds 'URL compatibility' test
* bring ml back to alerts
* review I
* Fix memory leak in timelines page
* fix storage bug for timeline search bar
* fix endpoint merge + functional test
* avoid timeline flyout toggle
* Fix link to ml score
* Fix breadcrumb
* Fix management url
* fix unit test
* fixes typecheck issue
* fixes remaining url cypress tests
* fixes timeline scenario
* fix link to details rule from timeline
* review remove absolute path for consistency
* Fixing resolver alert generation (#69587)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [Security_Solution][Endpoint] Resolver leverage ancestry array for queries (#69264)
* Adding alerts route
* Adding related alerts generator changes, tests, and script updates
* Fixing missed parameter
* Aligning the AlertEvent and ResolverEvent definition
* Fixing type errors
* Fixing import error
* Adding ancestry functionality in generator
* Creating some tests for ancestry field
* Making progress on the ancestry
* Fixing the ancestry verification
* Fixing existing tests
* Removing unused code and fixing test
* Adding more comments
* Fixing endgame queries
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* fix cypress test
* skip failing suite (#69595)
* [Endpoint] Fix flaky endpoints list unit test (#69591)
* Fix flaky endpoints list unit test
* un-skip test
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* remove flaky test
Co-authored-by: patrykkopycinski <contact@patrykkopycinski.com>
Co-authored-by: Gloria Hornero <snootchie.boochies@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Jonathan Buttner <56361221+jonathan-buttner@users.noreply.github.com>
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>
* Adding alerts route
* Adding related alerts generator changes, tests, and script updates
* Fixing missed parameter
* Aligning the AlertEvent and ResolverEvent definition
* Fixing type errors
* Fixing import error
* Adding ancestry functionality in generator
* Creating some tests for ancestry field
* Making progress on the ancestry
* Fixing the ancestry verification
* Fixing existing tests
* Removing unused code and fixing test
* Adding more comments
* Fixing endgame queries
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Use theme colors for stacktrace
* [APM] Use theme for all components
* Switch HoC withTheme to useTheme and convert classes to hooks
* Change hardcoded white to euiColorEmptyShade for ServiceMap
* Snapshots and Legends fix
* Switch to context and add test helper
* Fix tests and update snaps
* New snaps + new anomaly detection
* Remove shallow from testHelpers
* Remove commented tests
* Fix prettier
* Pass correct theme to cytoscape
* Fix ServiceMap
* fixes some rendering issues in service maps
* removed the old anomaly detection logic from service map popover contents, since it's been moved to a new component.
* Fix eslint, tsc lint issues and unit tests
* Remove types for styled-components default theme
* Update x-pack/plugins/apm/public/components/shared/KueryBar/Typeahead/Suggestions.js
Co-authored-by: Casper Hübertz <casper@formgeist.com>
* fix OuterTheme
* Ise function declaration instead of expression
Co-authored-by: Balthazar Gronon <git@balthazar.dev>
Co-authored-by: Balthazar Gronon <bgronon@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Oliver Gupte <olivergupte@gmail.com>
Co-authored-by: Casper Hübertz <casper@formgeist.com>
Adds support for histogram charts to data grid columns.
- Adds a toggle button to the data grid's header to enabled/disable column charts.
- When enabled, the charts get rendered as part of the data grid header.
- Histogram charts will get rendered for fields based on date, number, string and boolean.
## Summary
* https://github.com/elastic/kibana/issues/69463
* See here for manual backport to 7.8: https://github.com/elastic/kibana/pull/69434
This fixes a bug where if you import rules and set your overwrite to `true` multiple times in a row within 7.7 you can end up with a lists array. When upgrading to 7.8, we change the name of `lists` to `exceptions_lists` and suddenly when you enable/disable a rule you can get the following error below:
The fix is to allow the lists array still if it is present within saved objects to avoid seeing this error screen and being tolerant. We also fix the area of code that is causing the data bug so it cannot happen again with `exceptions_list` which is what the name of lists was renamed to causing this problem.
Note that this has unit tests and I also manually tested this by intentionally injecting a `lists` and `exceptions_lists` and using the UI to verify there wasn't another validation spot that needed to be relaxed to allow for the data.
### Checklist
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
* start redoing types
* finish redoing types
* fix bad test
* rework tests
* fix more types
* fix test
* Fix endpoints test and render error
* add deletePolicyStream to alerts api tests
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Paul Tavares <paul.tavares@elastic.co>
### Summary
This PR is meant to update the `ExceptionListItemSchema.entries` structure to align with the most recent conversations regarding the need for a more explicit depiction of `nested` fields. To summarize:
- Adds schema validation for requests and responses within `lists/public/exceptions/api.ts`. It was super helpful in catching existing bugs. Anyone that uses the api will run through this validation. If the client tries to send up a malformed request, the request will not be made and an error returned. If the request is successful, but somehow the response is malformed, an error is returned. There may be some UX things to figure out about how to best communicate these errors to the user, or if surfacing the raw error is fine.
- Updates `entries` structure in lists plugin api
- Updates hooks and tests within `lists/public` that make reference to new structure
- Updates and adds unit tests for updated schemas
- Removes unused temporary types in `security_solution/public/common/components/exceptions/` to now reference updated schema
- Updates UI tests
- Updates `lists/server/scripts`
* [APM] Service map download in debug mode
Add a download button when debug mode is enabled that downloads JSON of
the map.
Add an upload button to the Storybook.
## Summary
Fixes io-ts formatter bugs for REST and validation by:
* First trying to get the correct key from the io-ts context. If no keys are found, then it will fall back on trying to get the first name from the context.
* If the key is a value and an object then this will do a `JSON.stringify()` on the value object
* This fixes a few places where `formatError` was not being used within the code base resulting in `[object Object]` within the validations to show up.
### Checklist
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
* convert kbn test config into TS
* add test for Kibana embedded in iframe
* run embedded tests in functional suite
* ignore tls errors in functional tests by default
* switch test to https
* remove env vars mutation
* allow to pass ssl config to Kibana
* pass ssl config to axios
* adopt KbnClient interfaces
* adopt KibanaServer
* use KbnRequester in security service
* set sameSiteCookies:None in test
* acceptInsecureCerts in chrome
* remove leftovers
* fix type error
* remove unnecessary field
* address comments
* refactor plugin
* refactor test
* make acceptInsecureCerts configurable
* run firefox tests on ci
* up TS version
* fix firefox.sh script
* fix path
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* adapt KibanaDatabaseAdapter to use core ES service
* remove unused `exposeStaticDir` method
* create empty KP plugin
* remove unused and broken tests
* delete unused wallaby config file
* delete unused utils
* delete unused d.ts and move formsy definition to new plugin
* fix findNonExistentItems
* remove dead code and useless exports from common package
* delete non migratable test suite + remove unused test adapter
* remove yet another unused adapter
* restore awaits in KibanaDatabaseAdapter
* added doc links to processor types
* Bring doc links in line with mappings editor
Also refactor the processors type map
* remove helpText prop from Custom field
* fix i18n
* rename doc button and refactor type map const name
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* docs: ✏️ add drilldown user docs link to REAMDE
* feat: 🎸 wire drilldown docs link into <HelloBar> component
* feat: 🎸 set new page attributes on drilldown docs link
* feat: 🎸 add external prop to link to show icon
* fix: 🐛 remove rel= attribute
* fix: 🐛 add TypeScript type for drilldown docLink
* Closes#69092 by replacing direct queries on ml indices with seaching
via the `mlAnomalySearch` client API + job_id filters. Also removes
`getMlIndex` since it is no longer relevant.
* Use the mlCapabilities API to ensure the required license is active for ml queries
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Ingest: add data-test-subj prop support to Ingest components
* Ingest: Add Context Provider to bridge History from Kibana to Hash router
* Ingest: Added support for route state in Create Datasource page
* Endpoint: Add Create button to Polices List header
* Endpoint: Added support for passing of state from endpoint to ingest on policy
* Endpoint: additional functional test cases
With the improvements in the model memory estimation for DFA jobs, the required memory limit could be reduced, which allows execution in a low-memory environment (like the 1GB ML node in a cloud trial).
* Closes#69238 by handling 404 thrown error from the query for APM ML jobs.
* Improved coded readability
* moved anomaly job fetch to new function getApmAnomalyDetectionJobs for
improved readability and only handle a 404 status code response else throw
* [search] Refactor the way search strategies are registered/retrieved on the server
* Fix types and tests and update docs
* Fix failing test
* Fix build of example plugin
* Fix functional test
* Make server strategies sync
Co-authored-by: Liza K <liza.katz@elastic.co>
## Summary
Fixes a one liner bug where we were not using waitFor and could have stale views of data from the timeline. This removes that by adding a `waitFor` when setting the signals to be closed, open, in-progress, etc...
This also adds a new `waitFor` for the end to end tests and fixes other tests to use that rather than doing a wait for 5 seconds. This should keep the end to end backend tests fast.
Before this you could sometimes try to re-open 3 signals like below and it would not change the signals to being open but rather re-query the stale view and show the same signals as being closed when they are not closed:
Now with this PR, those signals will show as being re-opened and the closed signals will update correctly within the same UI on the same tab.
### Checklist
- [x] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
* initial plugin setup
* add smoke test
* fix license check
* refactor plugin setup
* Server-side create/update ingest pipelines (#62744)
* List pipelines (#62785)
* First iteration of ingest table
* Add action placeholders
* Refactor list and address PR feedback
Refactored the list into smaller pieces and assemble in main.tsx
Also addressed feedback on copy, removed unused notifications dep
* WiP on flyout
Showing name in title
* Add reload button
* Finish first version of flyout
* Slight update to copy
* `delete` -> `edit`
* Address PR feedback
Copy and a11y updates
* Add on failure JSON to flyout if it is available
* Add details json block file and remove ununsed import
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [Ingest pipelines] Create pipeline UI (#63017)
* First vertical slice of pipeline editor component
* Made a space for common parameters
* [Ingest pipelines] Edit pipeline page (#63522)
* First iteration of CRUD functionality working
* WiP on moving the pipeline editor to pipeline processor editor
* Finish refactor to work with passing state out
* Refactor and fix tests
* [Ingest pipelines] Polish details panel and empty list (#63926)
* Address some early feedback and use FormDataProvider
FormDataProvider gives a more declarative way to listen to form
state updates.
Also refactored the state reader mechanism.
* [Ingest pipelines] Delete pipeline (#63635)
* [Ingest Node Pipelines] Clone Pipeline (#64049)
* First iteration of clone functionality
Wired up for both the list table and the details flyout in the
list section.
* satisfy eslint
* Turn on sorting for the list table
* Clean up const declarations
* Address PR feedback
Sentence-casify and update some other copy.
* Mark edit and delete as primary actions in list table
* Handle URI encoded chars in pipeline name when cloning
* Update to using the more flexible controlled component pattern
To make this component and mappings editor more consistent, both
will expose a more traditional controlled component interface to
consumers.
In the current implementation this requires that consumers use
`useCallback` for `onUpdate` handlers to avoid an infinite
rendering cycle and to avoid staleness bugs in their `onUpdate`
handlers they need to think about what might make the handler
stale.
This approach comes with the benefits and flexibility of
controlled components at the cost of slightly more complex
consumption of the components.
In future, we can explore adding the uncontrolled component
interface too so that consumers have the option to more simply
render and pull data out only when needed
* Handle sub-form validity
The pipelines processor editor not emits overall validity to
consumers
* Fix Jest test
* Refactor some names
prepareDataOut -> serialize
prepareDataIn -> deserialize
EditorProcessor -> ProcessorInternal
* Mark as private
* Major WiP
Started working on the drag-and-drop-tree and updated some
typings stuff
* [Ingest node pipelines] Privileges (#63850)
* Create privileges check for ingest pipelines app
Also moved the public side logic for checking and rendering
privilege related messages to es_ui_shared/public following the
new __packages_do_not_import__ convention.
* Add ,
* Fix import paths
* Address PR feedback
Fix i18n strings (remove reference to snapshot and restore) and
fix copy referencing snapshot and restore - all copy-pasta errors.
Also remove unused field from missing privileges object.
* Fix issue from resolving merge conflicts
* Add missing app privilege
* Use non-deprecated privilige name
* First iteration of drag and drop tree on feature parity
* First steps toward add on failure handler
Updated reducer logic to create the next copy value using immer.
* First iteration of nested tree with add on failure working
* Refactor and some UI layout updates
- Remove the "id" field on processors for now
- Implement the nested remove and update functionality again
* Remove immer (not call stack safe)
Refac to remove immer and reimplemented the immutable set and
get functions.
* [Ingest Node Pipelines] More lenient treatment of on-failure value (#64411)
* Move file to components folder
* [Ingest pipelines] Simulate pipeline (#64223)
* Updated tree rendering
- turn off dropzones for children
- fixed up some padding and margins
- fixed integration with pipeline_form.tsx
The current implementation still has a lot of jank stemming from
the UX with DnD. Unfortunately the nesting has opened a number
of issues here.
* [Ingest Node Pipelines] Show flyout after editing or creating a pipeline (#64409)
* Show flyout after editing or creating a pipeline
* JSX comment
* Show not found flyout
Copied from CCR
* update not found flyout and fix behavior when viewing details from table
* Reset pipeline name in URI when closing flyout
* Remove encodeURI
Already using encodingURIComponent for unsafe string.
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* Clarification of terms
- addProcessor -> addTopLevelProcessor (same in editor modes)
- Expanded comment on ProcessorSelector
* Implement move between lists functionality
Added tests to the reducer for moving in and out of a recusrsive
structure.
* fix TS
* Prevent nesting a parent inside of its own child
* Add comment
* [Ingest pipelines] Cleanup (#64794)
* address review feedback
* remove unused import
* Big refactor to tree rendering structure
DnD tree now converts the nested tree to a flat one and only
consists of _1_ droppable area with a flat array of draggables
that can be combined.
Using the existing logic in the reducer combined with translating
the flat structure changes to a format the nested reducer can
understand looks like a really promising avenue.
There still seems to be a bug with a longer list where items do
not interact properly.
* Remove unused component
* A number of NB changes
- Fixed a subtle serialisation bug with on_failure in options
- Fixed a performance bug by using useRef in pipeline form
- Memoized the drag and drop tree
- Removed use of "isValid" from form lib... I think this should
be removed entirely.
* fix bad conflict resolution
* Implemented a slightly better destination resolution algo.
Also added tests.
* Fix subtle staleness bug, whitelist keys for setValue
* NB styling fix!!
Due to a parent's setting of overflow: hidden the drag and drop
tree had a dead zone that would equal the page overflow limit,
unsetting on the component itself (overflow: auto) relaxes this
limitation.
* Fix stale delete bug too
* Update naming of editor modes and update comments
* Use field types combo box
* Add delete confirmation modal
* Refactor delete modal component file name
* Better visual integration with existing form
* Update layout and styling of form
- added some padding around the new processors editor field
- Updated use of flex box in the pipeline form fields
* Move pipeline processor copy into pipeline processor component
The test button is also now inside of the pipeline processor
editor. Eventually all of this functionality should be moved
into the pipeline processor editor.
* First step of refactor to moving between trees
* First iteration of x-tree drag and drop
* Remove unused import
* Fix jest test types
* Fix up minor i18n issues and fix up layout of on failure
* Remove unnecessary prop
* Update spacing above add processor button to make it more center
* Fix destination resolution algo
* Update dragging resolver unit tests and add a lot more comments
* Use one sorting algo (removed use of euiReorder for now)
* Add placeholder tests and update comments
* Quite a big refactor
- Remove DraggableLocation entirely, only use ProcessorSelector
this simplifies mapping back to reducer instructions quite a lot
- Add tests for special case dragging behaviour
* Fix off by one bug in tests and implementation 🤦🏼♂
- Also move processor reducer to it's own folder
* Update behaviour for dragging up across trees and add tests
* Fix combine instruction
* Fix test and i18n issues
* Remove background color
* Fix selector after selector refactor
* A major performance
- Do not re-render the entire tree when editing settings
* Fix component smoke test
* Fix reading value from processor state using processor selector
* [Ingest pipelines] Custom processor form (#66022)
* Re add background color and refactor name of processor item
* Fix file naming and refactor 🚜 dnd tree rendering
- deserialze and serialize were backwards, fixed
- the dnd tree was rendering a flat tree which needed to be
mapped back to the nested tree for updates. This is still the
case but we do use recursive rendering. This enables tree nodes
to better hold local state like whether they are collapsed or not.
* Fix getting of initial on failure processors value
* Update padding styles for containers
* A lot of styling updates to get closer to look of mockup
* A WiP version of th click-tree an alternative to dnd
As a response to really long pipelines we may pivot away from
dnd.
* Remove dnd tree
* clean up reamining dnd tree references
* Clean up and refactor of tree component
To simplify the logic of the tree component processor id's
were added. Also had to update the jest spec to support this
* Added the ability to duplicate a processor
* Fix types in test
* Added duplicate functionality to ui
* Memoize tree components
* address es lint issues
* remove unused import
* Fix editing of custom json
* Address form performance issues
* Add all known missing processors
* Add ability to cancel move
* Fix staleness in test and view request flyouts
* fix type issue
* Remove unused translations and skip funcitonal test for now
* add todo comment
* Fix type issues
* remove poc styles
* disable other move buttons if we have a selected processor
* Refactor drop zone pin to button and add some styling
* Refactor processor editor item
* Update styling and use icon for cancel move action too
* fix nasty integration bug
* some minor optimizations
* prevent parent from being placed in own on failure handler
* Re-add cancel button
* Re-introduce failure processors toggle
* Fix typo
* Add Handler types for processor editor item
* Fix staleness bug for type, refactor classname and fix duplicate
bug not immutably copying values
* Experimenting with padels (revert this to undo if no further
changes have been made)
* Add description and unique ids
* Share links via component-wide context rather than props
* Virtualized list and back to outline dropzones
* Refactor id getter to a service and make it an incrementing number
* Temporary fix for double rendering issue.
This means the pipeline processors component is not controllable
but fixes the double rendering even when processors have not
changed. This is a problem coming from the ingest pipelines
plugin form system outside of this component
* add todo comment
* remove euicode element
* properly handle duplicate flow
* attempt to fix i18n
* split private_tree into it's own component and add comments
* refactor 🚜. rename Tree to ProcessorsTree and move things around
* do not delete the top level arrays for processors and onfailures
* fix typescript error
* Move duplicate, addOnFailure and delete actions into ctx menu
* remove unused import
* add support for pressing esc key to cancel move
* Add outside click listener
* always prompt before deleting a processor
* refactor remove distinction between adding top level and on fail
* add processor button to tree
* Hide the add on failure context menu item for processors with
failure handlers
* Reinstated x-tree moving and highlight and disable for buttons
on move and on edit
* removing ids step 1: remove idGenerator
* remove ids step 2: added inline text input
Also refactored a lot of the tree actions away. Now using
context and the processors editor tree item component for
dispatching actions. The tree item has access to the dispatch
and to the selector which makes it a well positioned component.
Also saves on some props drilling.
* Slight improvement to styling of text input (border)
* Re-implement missing failure toggle test
* address type todo
* Address many type issues and fix yarn.lock
* re-enable create pipeline functional test
* prevent multiple flyouts from opening
* change flyout title when editing an on-failure processor
* absolutely position the failure handlers label
when we render a dropzone on the label, then we move the label
up without affecting overall component height
* fix description behaviour not removing tag if empty
* some minor clean up
* add onflyoutopen cb to tests
* refactor processors editor item to multiple files
also refactored i18n into it's own file. would be good to come
up with a pttaern for doing this more broadly.
* fix add on-failure handler in context menu after refactor
* tag -> new description field
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Alison Goryachev <alison.goryachev@elastic.co>
* [ML] Model snapshot management
* more updates
* adding calendar range
* updating layout
* multiple calendars
* moving calendar creator
* fixing chart issues
* fixing chart issues
* improving calendar rendering
* adding capabilities checks
* code clean up
* fixing end time argument type
* fix translations
* code clean up
* comments based on review
* changes based on review
* fixing include
* adding useMemo to theme function
* update vega version
* a11y skip test
* a11y skip test attempt 2
* adding back all a11y tests
* fix failed karma test
* remove extra 0BSD license
* coalesce yarn.lock versions a little
* update kbn/pm dist
* fix CI
* fix Vega View for ML
Co-authored-by: Michail Yasonik <michail.yasonik@elastic.co>
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR adds an API to the AlertsClient which allows users to execute actions immediately, rather than via TaskManager.
This also moves the TaskManager based execution to a new api on ActionsClient along side the immediate execution.
* Add copy agent config endpoint, schema types, and request hook
* Add copy config to context menu and modal
* Change data sources to be copied via bulk instead of singles in parallel
* Add copy endpoint to openapi spec file
* Add api integration tests
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Don't use filter to get default config & output
the KQL parsing takes up a lot of CPU time. Avoid it with a search string
Also add a `refresh: false` as suggested by @bkobel
* Does restoring await fix CI?
* Remove 'refresh: false' from agent enroll
Does this this fix the failing test? It does keep the PR more focused
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* set wrap to the errorLink so it doesnt go outside of box
* updating
* set wrap to the errorLink so it doesnt go outside of box
* updating
* Show full message on hover
Co-authored-by: Nathan L Smith <smith@nlsmith.com>
[SECURITYSOLUTION][INGEST] Fixes endpoint link, adds link from endpoint details to fleet
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [APM] Changes to duration formatting
* Change the threshold for showing microseconds to 1 millisecond instead of 10. This means you now get "900 µs/1.3 ms/20.0 ms" instead of "900 µs/1300 µs/20 ms."
* Change milliseconds formatted with `asDuration` to be `asDecimal` instead of `asInteger`. That means you get "0.0 ms/2.5 ms/3.0 ms" instead of "0 ms/2 ms/3 ms."
* Tables were all using their own module-scoped functions that called `asDuration` to format things as decimal milliseconds. Extract this to an `asMillisecondDuration` function exported from the duration helpers and use it in all the tables.
* Change `getResponseTimeseries` in chart selectors to use `asDuration` to make all chart timeseries units consistent.
* Don't export `convertTo` from the duration helpers as it's now not used anywhere. Always use a more specific exported function for more consistency.
* Change ">=" to "≥" in the ML flyout text.
* Update e2e snapshot
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Ensure we use existing Elasticsearch config
* Use separate type for this to ensure custom properties work
* PR suggestions
* PR feedback
* PR feedback
* Fix type issues
* PR feedback
