* chore(NA): upgrade oss to lodash4
chore(NA): migrate cli, cli_plugin, cli_keystore, dev, test_utils and apm src script to lodash4
chore(NA): missing file for cli plugin
chore(NA): add src core
chore(NA): es archiver and fixtures
chore(NA): try to fix functional test failure
chore(NA): migrate src/legacy entirely to lodash4 except src/legacy/core_plugins
chore(NA): move legacy core plugins to lodash4
chore(NA): upgrade optimize to lodash4
chore(NA): upgrade to lodash4 on advanced_settings, charts, console and dashboard
chore(NA): migrate to lodash4 on dev_tools, discover, embeddable, es_ui)shared, expressions, home plugins
chore(NA): upgrade data plugin to lodash4
chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4
chore(NA): missing data upgrades to lodash4
Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4"
This reverts commit 137055c5fed2fc52bb26547e0bc1ad2e3d4fe309.
Revert "Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4""
This reverts commit f7e73688782998513d9fb6d7e8f0765e9beb28d1.
Revert "chore(NA): missing data upgrades to lodash4"
This reverts commit 92b85bf947a89bfc70cc4052738a6b2128ffb076.
Revert "chore(NA): upgrade data plugin to lodash4"
This reverts commit 88fdb075ee1e26c4ac979b6681d8a2b002df74c6.
chore(NA): upgrade idx_pattern_mgt, input_control_vis, inspector, kbn_legacy, kbn_react, kbn_usage_collections, kbn_utils, management and maps_legacy to lodash4
chore(NA): map src plugin data to lodash3
chore(NA): missing lodash.clonedeep dep
chore(NA): change packages kbn-config-schema deps
chore(NA): update renovate config
chore(NA): upgrade vis_type plugins to lodash4
chore(NA): move vis_type_vislib to lodash3
chore(NA): update visualizations and visualize to lodash4
chore(NA): remove lodash 3 types from src and move test to lodash4
chore(NA): move home, usage_collection and management to lodash 3
Revert "chore(NA): move home, usage_collection and management to lodash 3"
This reverts commit f86e8585f02d21550746569af54215b076a79a3d.
chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3
chore(NA): update x-pack test to mock lodash4
Revert "chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3"
This reverts commit 2d10fe450533e1b36db21d99cfae3ce996a244e0.
* chore(NA): move x-pack and packages to lodash 4
* chore(NA): remove mention to lodash from main package.json
* chore(NA): remove helper alias for lodash4 and make it the default lodash
* chore(NA): fix last failing types in the repo
* chore(NA): fix public api
* chore(NA): fix types for agg_row.tsx
* chore(NA): fix increment of optimizer modules in the rollup plugin
* chore(NA): migrate `src/core/public/http/fetch.ts` (#5)
* omit undefined query props
* just remove merge usage
* fix types
* chore(NA): fixes for feedback from apm team
* chore(NA): recover old behaviour on apm LoadingIndeicatorContext.tsx
* chore(NA): fixes for feedback from watson
* Platform lodash4 tweaks (#6)
* chore(NA): fix types and behaviour on src/core/server/elasticsearch/errors.ts
* Canvas fixes for lodash upgrade
* [APM] Adds unit test for APM service maps transform (#7)
* Adds a snapshot unit test for getConnections and rearranges some code to make testing easier
* reverts `ArrayList` back to `String[]` in the painless script within `fetch_service_paths_from_trace_ids.ts`
* chore(NA): update yarn.lock
* chore(NA): remove any and use a real type for alerts task runner
Co-authored-by: Gidi Meir Morris <github@gidi.io>
* chore(NA): used named import for triggers_actions_ui file
* chore(NA): fix eslint
* chore(NA): fix types
* Delete most uptime lodash references.
* Simplify. Clean up types.
* [Uptime] Delete most uptime lodash references (#8)
* Delete most uptime lodash references.
* Simplify. Clean up types.
* chore(NA): add eslint rule to avoid using lodash3
* chore(NA): apply changes on feedback from es-ui team
* fix some types (#9)
* Clean up some expressions types.
* chore(NA): missing ts-expect-error statements
* Upgrade lodash 4 vislib (#11)
* replace lodash 3 with lodash 4 on vislib plugin
* Further changes
* further replacement of lodash3 to 4
* further work on upgrading to lodash 4
* final changes to update lodash
* chore(NA): upgrade data plugin to lodash4
chore(NA): upgrade data plugin public to lodash4
chore(NA): fix typecheck task
chore(NA): fix agg_config with hasIn
chore(NA): assign to assignIn and has to hasIn
chore(NA): upgrade data plugin server to lodash4
chore(NA): new signature for core api
fix(NA): match behaviour between lodash3 and lodash4 for set in search_source
* chore(NA): remove lodash3 completely from the repo
* chore(NA): fix x-pack/test/api_integration/apis/metrics_ui/snapshot.ts missing content
* chore(NA): fix lodash usage on apm
* chore(NA): fix typecheck for maps
* Patch lodash template (#12)
* Applying changes from https://github.com/elastic/kibana/pull/64985
* Using isIterateeCall, because it seems less brittle
* Also patching `lodash/template` and `lodash/fp/template`
* Reorganizing some files...
* Revising comment
* Ends up `_` is a function also... I hate JavaScript
Co-authored-by: Pierre Gayvallet <pierre.gayvallet@gmail.com>
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
Co-authored-by: Oliver Gupte <ogupte@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Justin Kambic <justin.kambic@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
Co-authored-by: Luke Elmers <luke.elmers@elastic.co>
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
Co-authored-by: kobelb <brandon.kobel@elastic.co>
* implements recursive scanning in plugin discovery system
* update optimizer to find plugins in sub-directories
* update renovate
* update optimizer IT snapshot
* refactor processPluginSearchPaths$ and add test for inaccessible manifest
* add symlink test
* add maxDepth to the optimizer
* adapt mockFs definitions
* remove `flat` usage
* [ui-shared-deps] include styled-components
* update snapshots caused by react-is upgrade
* fix jest snapshots
* declare react-is dep in x-pack too so it uses the right version
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [Endpoint] use rbush to only render resolver nodes that are in view in the DOM
* Add related events code back
* Change processNodePositionsAndEdgeLineSegments selector to return a function that takes optional bounding box
* Refactor selectors to not break original, and not run as often
* Memoize rtree search selector, fix tests
* Update node styles to use style hook, update jest tests
* Fix type change issue in jest test
Introduces migrations into Encrypted Saved Objects.
The two main changes here are:
1. The addition of a createMigration api on the EncryptedSavedObjectsPluginSetup.
2. A change in SavedObjects migration to ensure they don't block the event loop.
* update vega version
* a11y skip test
* a11y skip test attempt 2
* adding back all a11y tests
* fix failed karma test
* remove extra 0BSD license
* coalesce yarn.lock versions a little
* update kbn/pm dist
* fix CI
* fix Vega View for ML
Co-authored-by: Michail Yasonik <michail.yasonik@elastic.co>
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Added example for using dashboard container by value
1.1 Refactored embeddable explorer e2e test to use new example, removed not needed kbn_tp_embeddable_explorer plugin.
For embeddable explorer examples went away from using getFactoryById() to improve type checks
There is new component a replacement for EmbeddableFactoryRenderer with slightly more flexible api: EmbeddableRenderer.
3.1 We can improve it going forward to support more use case
* add babel support for export type
* bump ts version to 3.9.3
* rebuild kbn-pm
* bump typescript-eslint
* fix error in security plugin UI
* check export as works
* fix app migration type
* use correct test subj attribute
* fix errors from the old PR
* embeddable is already passed in props
* explicitly define type of fetch
* add some types for viz
* fix fetch type p.2
* add null to allow spreading without type errors due to override
* add type guard to fix type error
* cast to any, since cannot assign unknown
* add timestamp to known types
* fix type error in fetch
* fix type error. id is always defined in attibutes
* declare a type
* move ts-ignore to the lines with errors
* declare tuple type explicitly
* mute type error. cannot assign unknown
* fix errors. id is always defined
* fix error type
* fix override errors. id is always defined
* fix error. extends any doesn't work anymore
* fix type error. type is always defined
* env doesn't always contain values
* fix type error
* cast to string
* add: logs is already declared in getNodeLogsUrl
* state is already passed in props
* fix some errors in timelion
* number of fragments is always defined
* 'absolute' is not just string, but value
* TEMP: option is always defined
* always true if cast to promise manually
* both props are always defined
* explicitly define returned SO type
* workaround type
* bump tslib to be compatible with ts v3.9
* test private property
* rebuild kbn-pm
* Fix ts errors for beats management
* Fix type inference broken by the TS 3.9 upgrade
* Fix ingest manager saved object attributes typings
* Fix TS errors in cross_cluster_replication and index_management.
* Fix TS error in Watcher.
* roll back colorRange wrong type
* fix security plugin types
* TypeScript 3.9 fixes for APM
* Fix ColorRange types.
* fix actions & alerts errors. ByGidi
* fix lists error
* More APM fixes
* Remove paramaterization from `removeEmpty in agent config SettingsPage component (it's only used there and doesn't need to be parameterized.)
* Add option chain for case in registerTransactionDurationAlertType
* Cast `overallValue` in transform_metrics_chart
* Use more specific type for custom link filters
* Add more option chaining for local UI filters buckets response
* Remove unused parameters from routes
* Fix getProjection type parameter
* Use destructuring in serviceNodesLocalFiltersRoute to hide `never` error
* Revert `UnionToIntersection` change in `AggregationResponseMap`
Fixes#67804.
* fix platform type error
* Fix visualizations types.
* Fix data plugin types.
* bump TS version to 3.9.5
* Fix telemetry TS errors
* Fix dashboard code
* Adding Canvas Fixes for TS 3.9
* Fix case and security_solution types
* roll back to the old export syntax. new one might cause problems in api-extractor
* update docs
* Fix timelion code
* Fix meta
* Fix types
* fix type errors om ingest_manager
* bump babel deps
* enable private props & methods syntax
* update kbn-pm dist
* whitelist 0BSD license
* use @babel/plugin-proposal-private-methods in default set as well
* disable new babel plugins
* Revert "disable new babel plugins"
This reverts commit 04d959431d.
* cleanup security_solution types
* Fixes type error for newer TypeScript
* update docs
Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
Co-authored-by: Felix Stürmer <stuermer@weltenwort.de>
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: Larry Gregory <larry.gregory@elastic.co>
Co-authored-by: Nathan L Smith <smith@nlsmith.com>
Co-authored-by: Walter Rafelsberger <walter@elastic.co>
Co-authored-by: Luke Elmers <luke.elmers@elastic.co>
Co-authored-by: Alejandro Fernández Haro <alejandro.haro@elastic.co>
Co-authored-by: Tim Roes <tim.roes@elastic.co>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Co-authored-by: FrankHassanabad <frank.hassanabad@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Revert "Revert "Using re2 for Timelion regular expressions (#55208)""
This reverts commit c90293d03f.
* Updating re2 to 1.14.0. Still need to update build patching
* Extract the gzip to the destination, supporting multiple extract methods
* Adding 'node' to jest's moduleFileExtensions
'node' is in the defaults, not sure why we aren't using the defaults...
https://jestjs.io/docs/en/configuration#modulefileextensions-arraystring
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* fix: rum agent should work correctly on new platform
* chore: add IS_KIBANA_DISTRIBUTABLE flag back
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR migrates the vast majority of Alerting legacy code to the Kibana Platform.
This includes:
1. Removed legacy Task Manager
2. Migrates Fixture plugins in Alerting, Triggers UI and Task Manager Perf
This does not includes:
1. The PagerDuty simulator due to a lack of support for custom responses in the platform. issue opened. https://github.com/elastic/kibana/issues/65045
2. The Webhooks simulator due to a lack of support for custom authorisation. Requires investigation.
* feat(NA): added new build step to generate a static file system for node_modules and monkey patch the bin start files to include the static file system loader on begining.
* chore(NA): updated lockfile.
* chore(NA): replace import to the correct static-fs files.
* fix(NA): resolve for a correct static fs loader.
* chore(NA): last static-fs working version.
* feat(NA): changed patched entryPoints.
* chore(NA): update to last static fs.
* fix(NA): fix imports on webpack shims. chore(NA): added new static-fs version.
* chore(NA): update to static-fs with stat and statSync.
* fix(NA): vendor chunk filtering in order to include default node modules and static node modules.
* refact(NA): major refact on create static modules fs task.
* fix(NA): remove global from clean empty folders task.
* chore(NA): change args order for deleteAll.
* chore(NA): move to the new static fs
* feat(NA): migrate to last apis of static fs
* fix(NA): patch correct entry file.
* chore(NA): last stable integration with static-fs task
* chore(NA): rollback some unecessary changes
* chore(NA): remove changes on sinon webpackshim
* chore(NA): integrate with official static-fs version
* chore(NA): integrate last thread-loader version
* chore(NA): added last static-fs version
* chore(NA): upgrade to last static-fs version
* chore(NA): upgrade to last static-fs version 1.1.1
* chore(NA): rever static-fs version to 1.1.0
* chore(NA): upgrade static-fs to 1.1.1
* chore(NA): upgrade static-fs for last 1.2.0 version
* chore(NA): update package.json
* chore(NA): run apm after patched environment
* chore(NA): specify rule to disable eslint for
* chore(NA): remove changes in the src/apm file
* chore(NA): change apm order in the dev scripts
* chore(na): update static-fs versin
* chore(NA): bump static fs version
* chore(NA): correctly lint task file
* chore(NA): move away from rimraf to del
* chore(NA): bump static-fs version
* chore(NA): bump static-fs version
* chore(NA): bump static-fs version
* chore(NA): bump static-fs version
* chore(NA): bump to last static-fs version
* chore(NA): apply changed according PR review feedback
* chore(NA): remove changes to base optimizer
* chore(NA): fix discover pattern
* chore(NA): bump static-fs version
* chore(NA): bump static fs to 1.6.3
* chore(NA): bump static-fs to 1.6.4
* chore(NA): bump static-fs to last version
* chore(NA): bump static-fs to last version
* chore(NA): bump static-fs to last version
* chore(NA): bump static-fs to last version
* chore(NA): bump static-fs to last version
* chore(NA): bump static-fs to last version
* chore(NA): bump static-fs for 1.8.3
* chore(NA): bump static-fs to 1.9.0
* chore(NA): bump static-fs to 1.9.1
* chore(NA): update to last static-fs version 1.10.0
* chore(NA): moving to @elastic/static-fs package
* fix(NA): change import to the new package
* chore(NA): bump elastic static-fs to 1.0.1
Co-authored-by: spalger <email@spalger.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Add set-value as an explicit dependency
This is a more robust solution than lodash's set().
* Replace lodash.set() with set-value's equivalent
* Rebuild renovate config
We added set-value to our dependencies.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Move SIEM public/ folder to NP plugin
This is solely renames; fixes come next.
* Update relative imports in our API tests
* Fix linter errors following move to NP folder
These paths got a little shorter, so some lines could be collapsed.
* Move client dependencies to NP package.json
I'm removing the @types/js-yaml for now because I'm not sure we need it;
I'll add it back later if we do.
* Fix relative imports to other plugins
* Fix errant uses of ui/chrome
* Remove legacy plugin shim
* Move feature registration into plugin
This previously had to be part of legacy bootstrapping due to an order
of operations issue.
* Disconnect legacy plugin
The index file should now be redundant with what's in the plugin:
* app registration
* feature registration
* Move public gitattributes
* Remove references to legacy embeddables
We can now use the NP API. Maps embeddable will not work here until
their work is merged, but this should prevent us from importing legacy
code and thus breaking the build.
* Add our frontend dependencies to kibana.json
These are all required for now, because that's how they're typed. If
they _should_ be optional (and I think several should), we need to
update the type and handle the null case within the app.
* Replace use of ui/new_platform mocks in embeddable utils
* Fix remaining jest tests
* Replace build-breaking ui/new_platform mocks with equivalents in core
proper
* Remove unnecessary mocks of ui/new_platform
* Remove references to legacy SIEM folder
* I left the reference in CODEOWNERS in case someone tries to sneak
something back
* I left the .gitignore reference for the same reason
* Fix mocks of relative paths
These were not caught by typescript and were causing test failures.
* Export our client plugin contracts
They're empty for now.
* Move from deprecated appmount API
The new one dropped a param we weren't using.
* Add missing mock causing test failures
* Don't re-export core types from our plugin
Import them from core where we need them, instead
* Move Actions UI registry outside of mount
This is already imported, there's no benefit (and potential timing
issues) with doing this inside the mount.
* Add security's setup contract to our StartServices
This doesn't change what's used, only how we're typing it. The types are now a
little more truthful as:
* our StartPlugins don't include setup contracts
* our StartServices includes everything we use at Start time, including
the one setup plugin.
* Add order and icon back to the sidebar link
* Replace plugin class properties with constants
These are shared, and should be consistent.
* Enable our UI on NP
* Add missed plugin dependencies
We're not using their contracts, but we are importing code from them.
* Revert use of constant in translation
Can't do that, whoops
* i18n our feature catalogue entry
* Remove unnecessary array from single element
* Remove unused keys
These were the legacy translations used... well, I don't know where they
were used.
* Ignore circular dependencies in external plugins
* Normalize exclusions
* Add undeclared dependencies to kibana.json
We import our maps embeddable from maps, and we pass inspector to the
embeddable. I just missed these in my audit. This was causing errors in
the map embeddable.
* Await our call to setLayerList
This is an async call that we need to complete before we can render.
* Reduce siem plugin size
When we load our initial plugin (before our app is loaded), were were
implicitly importing all of kibana_react with this import. While a
global module prevents this from affecting our bundle size currently,
that could change in the future. Since we only need a reference to our
class, we just import that instead.
* chore(NA): reduce siem bundle size using babel-plugin-transform-imports for lodash
* fix(NA): dont use preventFullImport
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* snapshot updates
* bump to eui 22.3.0
* snapshots
* Fix css selector in test to match EUI internal DOM change
* Updated snapshots
* Insert react-beautiful-dnd@10 into top-level kibana to force siem's node_modules to hold its correct version
* More snapshot updates, resolved a typescript issue
* Move react-beautiful-dnd dependency around to make things happy again
* Use SearchFilterConfig from the top-level eui export
* Swallow 'ResizeObserver error does not have an err object' error in canvas
* Swallow ResizeObserver error cross-browser
* bump dependency, add edge support in ftr services
* add config files
* fix browser version for msedge
* use npm ms-chromium-edge-driver
* download edge driver aside from session creation
* move dependency to dev
* update dist/index file
* bump edge-driver version
* change type to msedge to match w3c spec
* fix discover tests for Edge
* Revert "fix discover tests for Edge"
This reverts commit 87e7fdd256.
* bump driver version up
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* update crypto packages
* as type for return value
* get default export
* add if checks
* wrap errors in i18n
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [kbn/ui-shared-deps] expand and split
* add two import styles for eui/react-dom that are new
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Make global service map zoom toward center
Calculate the center of the graph when there's not a primary node.
Also update to latest version of types.
Fixes#61176.
* Fix types
* update to eui 21.0.1
* most changes needed for search bar ts changes
* Passing types
* snapshots
* jest tests
* Removed IQuery placeholder types
* Updated functional test to only look at table headers with content
* Moved 'filters' definition around in api docs
* Update types
* update snapshot
* typo
* Move DATA_FRAME_TASK_STATE enum to its own file to fix x-pack functional test config imports
* merge public api 'changes'
Co-authored-by: patrykkopycinski <patryk.kopycinski@elastic.co>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Kibana uses Node.js v10.19.0. The closest version of @types/node to this
version is currently v10.17.17.
This commit updates the resolutions field in package.json to ensure that
the latest version less than 10.20.0 is always used.
* Update the ems-client dependency
This PR adds the `appName` and `appVersion` parameters used by ems-client. The `appVersion` parameter replaces the now deprecated `kbnVersion` parameter in ems-client.
* Review feedback
* Fix borked merge
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [EPM] Documentation of HTTP routes & TS types for Ingest (#48798)
* Add beginning models and two routes for Ingest
* Update types & models per discussion w/Ruflin
Also reviewed data structures listed at https://docs.google.com/document/d/1IBR3f9dpHqJmXYEdg06WV34KSMd3g5k4aMGa4jde_Eg/edit#
* Update: /policies always returns array. /policy returns single policy
* Add pagination for /policy & /datasources. Uses per_page & page params
* Add API metadata. Standardize policy_id param name.
* Update descriptions to match Google Doc. Move use case to Policy.
Disabled the '@typescript-eslint/array-type' rule because it was going around in circles. It didn't like Datasource[] or Array<Datasource>
* Return to initial TS annotation for Arrays
Remove the line disabling @typescript-eslint/array-type now that it's behaving normally again 🤷
* [EPM] Add directory structure for server/lib. (#50469)
* Add directory structure for server/lib.
* 'tests' seems to be more common than 'test'
* Make CI happy
* [EPM] Add basic documentation directory (#50478)
* [EPM] Add basic documentation directory
Having the doc directory around allows us to easily add docs from here on to document how EPM works.
To run the docs build, use the following command from the kibana directory:
```
../docs/build_docs --doc docs/epm/index.asciidoc --open
```
The above assumes that docs (https://github.com/elastic/docs) are checked out in the same directory as Kibana.
With this change, the EPM docs build is not included yet in the overall docs build. For this adjustments to https://github.com/elastic/docs/blob/master/conf.yaml must be made.
* [EPM] Add basic index template (#50471)
This PR adds the very basic index template we will use for the packages. It contains all the basic settings and some examples. The examples will be remove as soon as we have an actual implementation with packages but for now is convenient to see if it is a valid package.
This code is put into the lib directory as it does not tie directly into any handlers.
It also adds an functional tests for loading a template. This means we have a way to check if a template is valid in Elasticsearch. Based on this we can check in the future all our generated templates for validity with Elasticsearch.
To run the functional test, go to the Kibana x-pack directory. Start the first command:
```
node scripts/functional_tests_server.js --config test/epm_api_integration/config.ts
```
Keep the above running and switch to an other Terminal. Now run:
```
node scripts/functional_test_runner.js --config x-pack/test/epm_api_integration/config.ts
```
* 40752 rewrite ingest pipeline (#50627)
* Add directory structure for server/lib.
* 'tests' seems to be more common than 'test'
* Make CI happy
* Implement pipeline rewriting.
* Add more testcases
* For posterity (comment change)
* Allow beats-style template delimiters
* Be more succinct
* Document better
* Replace AssetType enum with union type (#50696)
See https://github.com/elastic/kibana/pull/50609#discussion_r346080439
Discussed in Slack and agree to revert for now. Can track down the issues & restore later
* Remove unnecessary await if we can return the promise (#50329)
* Fix whitespace per figma comments. Closes#47348 (#47350)
* Add fix & comment for TS 3.7.2 regression
* [EPM] cleanup assets, filter assets for those currently supported (#50609)
* cleanup assets, filter assets for those currently supported
* removed unused type
* fix type
* add comment for better type
* change type name to be more descriptive
* hardcode image width for ie11 (#49796)
* hardcode image width for ie11
* eslint
* improve comment
* add maxWidth
* [EPM] useKibana hook & render in plugin.start (#50110)
* plugin.start now does reactdom.render vs returning react element
export plugin function from public/index
* Move setClient call from plugin.start to plugin.setup
* Use `useUiSetting$` from `useKibana` hooks
* Fix broken app due to bad hooks usage
Can't use useKibana outside a React component.
Reverting to prior approach since it's still NP. Can revisit context usage in a followup PR
* [EPM] Install package from detail view on button click (#50735)
* Support basic "click button -> show spinner -> installed" install flow
* Remove incorrect comments. Add TS return types to data functions.
* [EPM] Use NP feature_catalogue.register (#50108)
* Use NP feature_catalogue.register
* Use type from NP plugin
* fix linting
* fix types
* fix headers in Fleet
* skipping test due to ES param change
* Revert "skipping test due to ES param change"
This reverts commit d05f20decf.
* remove type field
* remove unused import
* [EPM] Final(?) update from integrations_manager -> EPM (#50976)
* Update (all remaining?) references from integrations_manager to EPM
* Update path in i18n file
* [EPM] update compatibility section (#50975)
* change min max version to single range value
* add elastic stack icon and change text
* remove badge from version and use code font
* remove euistyled
* add back version component lost in merge
* remove euiStyled
* remove old file
* Restore RequirementVersionRange type
* Disable test for elasticsearch username.
Temporary work around until we know how the stack wants to add the permissions we need. Either adding to the kibana user or creating a new user.
* Revert "Disable test for elasticsearch username."
This reverts commit f1020e4eab.
* Disable test for elasticsearch username.
Temporary work around until we know how the stack wants to add the permissions we need. Either adding to the kibana user or creating a new user.
* Fix EPM typing issues in register feature
* Fix typings after master merge
* [EPM] CI fixes (#51284)
* Initialize es in test.
* Add it(), no es.init()
* Clean up.
* [EPM] add confirmation modal (#51172)
* add confirmation modal, move install state to Header
* update callout to use title
* move components only used in detail view to detail dir
* use better variable names
* update to more descriptive variable names
* Restore prior response vaulues for install & delete package (#51252)
Discussed this with @skh https://github.com/elastic/kibana/pull/51112#commitcomment-35961413 & https://github.com/elastic/kibana/pull/51112#commitcomment-35970664 as well as in a video call
Also added some TS type annotations for data fetching functions to make the contracts more explicit
* [EPM] /package API only lists installable assets. Restore enums. (#51414)
* API only shows installable assets. Server types to own file. Restore enums
* Fix type imports
* Only return installable asset types (kibana for now)
* server/types now only has code from hapi which shouldn't go to client
* Add more restricted TS types to DisplayAssets object
* Flip order of arguments to Extract
In these cases it still works the same, but looking at https://www.typescriptlang.org/docs/handbook/advanced-types.html the signature is
`Extract<T, U>` - Extract from `T` those types that are assignable to `U`
so the larger set should be first
* [Fleet] Enrollment api key UI (#51495)
* Make button pretty in dark mode as well. (#51610)
* [EPM] Add docs entry about registryUrl config (#51697)
This documentation is at the moment mainly for internal use. I found myself searching for this URL several times in the code or PRs so I thought I rather add it to the docs for now.
* [EPM] Remove encoding of Kibana objects as not needed anymore
* [Fleet] Move agent status server side and API to get aggregated status for a policy (#51673)
* [EPM] Add basic docs around install/delete API endpoint (#51728)
This is mainly for internal usage at the moment to look up.
* Ingest/policy (#51741)
* wip policy
* tweaks
* tweaks
* FIX TYPOS
* WIP move policy => agent config conversion to fleet, WIP policy changed method
* fix tests and bugs
* updates tests and snaps
* more fixes
* use AGENT_POLLING_INTERVAL
* cleanup and fix some formatting
* Update x-pack/legacy/plugins/ingest/server/libs/datasources.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/ingest/server/libs/datasources.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/ingest/server/libs/outputs.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/fleet/server/libs/policy.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* fix things broken by PR review suggestions
* remove unused field
* fix types
* fix mappings
* add datasource mappings
* Fix mappings and remove get full policy from checkin
* Fix ingest api integration tests
* run es-lint to fix fleet
* Fix typescript issues
* [EPM] Track package install state and add toast notification (#51734)
* add notifications from core to plugin
* add package install state hook
* fix type error
* use toMountPoint helper to add jsx to notification
* add warning notification to failed install
* make notifications dependency explicit prop
* move PackageInstall provider lower
* add comment about InstallStatus type overlapping InstallationStatus
* use InstallStatus type in InstallationButton component
* fix type
* [Ingest] Adds support for a working default output (#51841)
* aadding config
* add working settings to the default output
* remove default username and password
* update libs
* [EPM] Add basics for creating the ILM setup (#50474)
This contains the basic objects to setup ILM
* Create index and alias with a write index
* Get the policy
The code does not contain any functional tests yet as it is still open on how to do it best. I suggest to get this in as a foundation and then iterate on top of it.
* [EPM] Add datasource (ingest pipeline) from package (#51851)
## Summary
This mixes a few concerns but I think it's worth it to show the parts working together.
Take a look at the individual commits for a better separation of features.
This adds
- the `/datasource/install/{pkgkey}` endpoint which installs ingest pipelines from a package into ES and saves a reference to them in the EPM state Saved Object
- Connects the "Add datasource" button in the successful installation Toast to the new API
- Adds a toast notification to inform the user the datasource was added correctly
- Adds a "Delete Package" button on the details page so we can uninstall a package while we're waiting for the separate view which allows deletes
- b99eda6 Pushes logic that was in the detail view into `InstallationButton`. This consolidates the logic in one component (or one component & the existing hook) and, iiic, means we can put `<InstallationButton package={...} />` on any view and get the same behavior
I'm marking this as a normal PR so people can merge if they wish
* [EPM] Add basic processing of fields.yml file (#51148)
The fields.yml is used to generate the Elasticsearch template and Kibana index pattern. This PR adds a very basic implementation of processing the fields.yml and then create an Elasticsearch template out of it. The only fields that are supported at the moment are keyword fields, more will be added as a follow up.
The testing was implemented with a golden file. The output from the method is compared to a json file. If the input is changed or the method is changed, it is possible to regenerate the files with the `-generate` flag as following:
```
node scripts/jest ./legacy/plugins/epm/server/lib/template/template.test.ts -generate
```
This will allow us to quickly test many inputs / outputs in the future, make adjustments to the existing files and generate the new outputs. We then can compare it in the diff it the changes make sense.
* [EPM] Create basic implementation to merge input template and dataset manifest (#51803)
* [EPM] Create basic implementation to merge input template and dataset manifest
With this code it is possible to take an input template for the agent and merge it with the config variables from the dataset manifest file. Currently only the name and the default value are merged. Later on we must implement to be able to pass user configured variables to it and make decision based on OS selection.
Closing https://github.com/elastic/kibana/issues/51794
* [EPM] Refactoring of lib structure (#51885)
This refactors the structure of lib. As so far all the lib parts are related to assets in the package, it is organised the same way with the same structure. Each directory has its own tests directory if it needs one. This makes it possible to (almost) not need relative paths for tests.
* [EPM] Allow to read files from fields directory (#51958)
This change allows to also extract files from the `fields` directory. Previously this was not possible because it always assumed a service must be there.
* [EPM] Install Elasticsearch Index Template for data source (#51878)
This installs the Elasticsearch index template for each dataset in a package. For now the names are hardcoded based on package key and dataset name but will be more dynamic later on when we pass the full dataset information.
The dataset extractions is a bit "hacky" at the moment and we should get a full implementation of dataset at a later stage and replace this code.
* [Fleet] Policy list, details, create, edit UIs (#51950)
* Set up simple policies list view
* Adjust spec to return single policy
* Set up simple policy details page
* Add demo stats/chart to policy details
* Add description string
* Initial setup of policy form and create policy UI
* Policy create/edit form; integrate policy list api
* Integrate create policy api
* Integrate policy detail, agent status, and policy edit APIs; adjust policy list api integration in agent enrollment
* Fix edit policy mock meta
* Fix policy list search bar
* PR and linting fixes; use typings from ingest plugin
* Fix i18n
* [EPM] Add datasource saved object type (#51871)
## Summary
This PR makes a few assumptions, and contains a lot of refactoring. It might be beneficial to look at the resulting directory structure under `server` first to get the (new) big picture.
Assumptions:
- our API deals with several concerns, for now these are packages and datasources
- we manage our own HTTP API endpoints for these concerns (in particular, don't use the ingest plugin for that)
- we manage (for now) the Kibana saved object in which datasources are saved. importing and calling methods from the ingest plugin to do that down the road will (hopefully) be a manageable change
This led to the following decisions:
- the code is separated into subdirectories by concern, containing all the route handlers and tightly coupled code
- for now, these directories are in `server/packages` and `server/datasources`. I'm tempted to move them into `server/api/{packages,datasources}` but wanted to limit the amount of refactoring in one PR
- shared code lives in `server/lib`
- some code from `server/packages` has been almost duplicated to handle saving to Datasource saved objects, some has been refactored and is used from both places. The deduplication needs further improvement
- maybe `server/registry` should also move under `server/lib` (but see above, I'm trying to not move everything around all at once)
Testing:
* Please note that this is a breaking change because the saved object type for package information
has also been renamed. You'll need to start with a fresh `.kibana-*` index. Restarting `yarn es snapshot` (withouth specifying a data directory) should do the trick.
* Package installation should still work, e.g. with a GET request to `http://localhost:5601/api/epm/package/coredns-1.0.1`. The saved objects for packages can be inspected with a GET request to `http://localhost:5601/api/saved_objects/epm-package/$PKG_KEY`, e.g. `http://localhost:5601/api/saved_objects/epm-package/coredns-1.0.1`
* Datasource creation should still work, e.g. with a GET request to `http://localhost:5601/api/epm/datasource/install/coredns-1.0.1`. The saved objects for datasources can be inspected with a GET to `http://localhost:5601/api/saved_objects/epm-datasource/$PKG_KEY`, e.g. `http://localhost:5601/api/saved_objects/epm-datasource/coredns-1.0.1`
* [Fleet] Expose policy during agent checkin (#51968)
* [EPM] Add /epr prefix to the tar.gz download path (#51881)
The registry slightly changed the .tar.gz path because of download stats reason. This adjusts for it. See https://github.com/elastic/package-registry/pull/169
* [EPM] Move template installation to lib and add asset helper (#52049)
* [EPM] Move template installation to lib and add asset helper
All the logic related to the installation of the templates for a package should be inside the template library folder. This moves the logic into this folder.
A few refactorings were made to simplify installation:
* Introduction of DataSet interface: This interface is needed to extract the data sets inside a package and install one template per data set.
* Pass package instead of package key to installation process: Passing the package instead of the package key means fetching of package information is decoupled from the installation process and abstracted. This separates the two concerns and should simplify testing.
* getAsssets method: The getAssets methods works on top of the package object to extract asset paths. It is inspired by get_objects methods but supports passing a package and a dataset.
Currently one problem with testing that exists is that to fetch the content of an asset is not decoupled yet.
* [EPM] Reduce data source to one type (#52061)
Between Fleet / Ingest / EPM there had been several interface definitions of Datasource and the related types. This reduces it to one place for the definition. The same applies to the policy definition.
The goal of this is that from now on we all rely on the same definition. If we make changes, we make them in all parts of the code.
In this PR is only the minimal change needed to get us all on one interface. Further changes will be needed that we all rely on the same saved objects etc.
* add export command
* revert 2 more files to rely on export
* revert imports
* Fix types for Datasource Saved Object
* merge in master
* fix type check
* Run VSCode's organize imports on EPM files (#52234)
Learned about it on Slack from https://twitter.com/ryanchenkie/status/1201883268527927301
Blog at https://code.visualstudio.com/updates/v1_23#_run-code-actions-on-save
Basically does the order we've been loosely following (3rd party, then relative) & alphabetic by location and variable name.
It's not customizable but it's reasonable and, afaict, consistent.
* [EPM] More realistic datasource SO. Error if package not installed. (#52229)
* Move cache 'hack' into getAssetsData
* p -> pkg. package is reserved. pkgkey is used in many places
* Remove unnecessary type cast
* Clarify reasons behind asset path manipulation
* Return the Datasource; not the Saved Object.
* Use real values from package in fake datasource SO
* Error if /datasource/install before /package/install
```
> curl --user elastic:changeme localhost:5601/api/epm/datasource/install/coredns-1.0.1
{
"statusCode": 403,
"error": "Forbidden",
"message": "coredns-1.0.1 is not installed"
}
> curl --user elastic:changeme localhost:5601/api/epm/install/coredns-1.0.1
[
{
"id": "53aa1f70-443e-11e9-8548-ab7fbe04f038",
"type": "dashboard"
},
{
"id": "Metricbeat-CoreDNS-Dashboard-ecs",
"type": "dashboard"
},
{
"id": "75743f70-443c-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "36e08510-53c4-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "277fc650-67a9-11e9-a534-715561d0bf42",
"type": "visualization"
},
{
"id": "cfde7fb0-443d-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "a19df590-53c4-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "a58345f0-7298-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "9dc640e0-4432-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "3ad75810-4429-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "57c74300-7308-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "27da53f0-53d5-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "86177430-728d-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "4804eaa0-7315-11e9-b0d0-414c3011ddbb",
"type": "visualization"
}
]
> curl --user elastic:changeme localhost:5601/api/epm/datasource/install/coredns-1.0.1
[
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_plaintext_json",
"type": "ingest-pipeline"
},
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_json_json",
"type": "ingest-pipeline"
},
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_entry_json",
"type": "ingest-pipeline"
}
]
```
* fix duplicated imports
* [EPM] Move golden files generation over to jest snapshot (#52203)
* [EPM] Move golden files generation over to jest snapshot
I initially used my own implementation to write the generated files. It runs out jest has a feature to write snapshots which simplifies the code a lot.
I added a loop with an additional test file so in the future we can just keep adding test files without having to modify the test code.
To updated the snapshots, the param `-u` has to be used:
```
node scripts/jest legacy/plugins/epm/server/lib/fields/field.test.ts -u
```
* [EPM] Create metrics-* and logs-* Kibana index pattern (#52277)
This creates the very basic Kibana index patterns metrics-* and logs-* for Kibana. At the moment it is overwritten every time. We need to change this in the future to take the fields from all installed data sources and regenerate it.
* [EPM] Create helper for elasticsearch asset names (#52265)
Most of the Elasticsearch assets have the same base name. This creates a helper to get the base name for the assets. In case we decide to change the base name in the future, we can change it in one place.
* fix tests and destructing
* [EPM] Update Registry types. Prevent errors installing certain datasources. (#52285)
* Update RegistryPackage type
* Use download key from EPR to fetch archive
* Fix errors caused by correcting the Registry types.
The issues were largely that some Registry types like `title, `datasets` and `assets` where marked as required, but are actually optional. This highlighted area in the code were we relied on them always being present.
We added to the issue by wrapping Registry types in `Required` which made those items which were correctly listed as optional, required for EPM code. Updated EPM types to reflect the largely pass-through nature of the EPM types.
There are two properties which we ensure are in every EPM response, those were put into their own (unexported) type.
Confirm by trying to add a datasource to a package which has no datasources, like apache-1.0.1 or system-2.0.1. In `feature-ingest` and the earlier version of this PR, the `/datasource/install` call returns a 500. In this PR it succeeds.
* [EPM] Add setup of default ILM policies (#52272)
This creates two ILM policies: logs-default and metrics-default. These are the default ILM policies used. Currently the policy content is hardcoded in the code but should be fetched from the base package in the future. The setup happens as part of the datasource installation. When a data source is installed it is a good time to check if the assets are there but we might extract this to a better place in the future.
* [EPM] 52075 add data source first page (#52320)
* Update RegistryPackage type
* add first page of add data source
* fix for ie11 flex min width bug
* remove toDetailViewRelative
* remove unneeded spread
* Update TS type names for EPR search results (#52512)
* `RegistryList -> RegistrySearchResults`
* `RegistryListItem -> RegistrySearchResult`
* Restore import sort order from #52234 (#52548)
Many of the changes from #52234 were lost. Presumably due to PR(s) merging which were based on branches which had the previous unsorted order.
* [EPM] Replace wildcard export (#52554)
* PackageNotInstalledError -> packages/index.ts
* pkgToPkgKey -> registry/index.ts (will convert existing `${name}-${version}` instances later)
* Replace export * from packages.
There's an argument that the import sites should be updated to import from `packages/get`, `packages/install`, etc but that can wait for a later PR.
* [EPM] Reduce usage of epm-package SavedObject (#52576)
* Delete existing Installation type. Rename InstallationAttributes to Installation
* Reduce usage of EPM SO. Add getInstallation().
Replaced two calls of getInstallationObject() with getInstallation().
Two less places with knowledge of SO internals.
Lots of potential improvements for EPM TS types remain (refactoring/removing Installable, etc), but this is a good incremental step, IMO
* [EPM] Fix missing export link (#52628)
Without it, things break. I am surprised CI did not catch this.
* [EPM] Cleanup ILM loading (#52632)
Before the check for the ILM policy to exist triggered an exception. With this change it is a normal response also if the policy does not exist yet.
A follow up issue will be created in Elasticsearch to get a HEAD request for this available.
* [EPM] Switch to staging URL for registry (#52626)
The old cluster with the registry will be removed as soon as this is merged.
* [EPM] Use Dataset interface to generate template (#52255)
This will make sure we have to pass much feature params and can fully rely on the datasource object to create names for assets.
* [Fleet] Use agent events to compute agent health (#52513)
* [EPM] Data source integration tests (#52542)
* Add fixtures for data source integration test.
* Move test setup to beforeEach
* Add test for datasource creation
* Handle pipelines in yml format.
* Make integration test for adding a data source pass.
* Use EPR staging URL with CDN. (#52776)
See https://github.com/elastic/kibana/pull/52626#pullrequestreview-330622868
* [EPM] Add Data Source page updates (#52705)
* remove dupe type RegistryPackage
* change switches to checkboxes, use datasets to create checkboxes, add some local form state
* update types
* [EPM] redirect after package install (#52771)
* add callback after successful installation and redirect
* add temp data sources tab content to access add data source page
* remove assets tab for mvp
* hide data sources link and redirect from data sources tab if package not installed
* change callback name
* remove commented out assets logic
* add redirect to hook
* fix type
* Use ingest datasource api (#52964)
Incremental change. Uses HTTP API for datasource creation. Will do follow-up PR which uses JS function instead
* Remove duplicate fetchInfo & installTemplates
I think this was from a bad merge, but pretty sure we don't want these functions called twice in the same function
* WIP. Pushing so others can see
* Improve correctness/flexibility of absolute URL
* Disable datasource test & template installation
* [Ingest] Data source APIs (#52448)
* Clean up ingest imports and remove unneeded mock_spec files
* Initial pass at datasources lib and API endpoints
* Add add/remove datasource to/from policy API endpoints
* Add datasource contract tests and related policy contract tests; update snapshots
* Fix tests
* Fix tests again
* Fix tests 3
* Adjust routes, PR feedback
* modify epm createDatasource endpoint to use user data (#52971)
* change epm/datasource/install/{pkg} to POST, send user data to endpoint, install pipelines and templates based on user selected datasets
* change test to post for installing a datasource
* change some names and types around
* delete request.headers['transfer-encoding'] being passed through from epm request
* [EPM] Don't share CreateFakeDatasource type (#53068)
It's not shared between client & server so it doesn't need to be in common. Also, it imports server code which would try to bring server types to the client. It's types so they're compiled away but it's important to keep common to what's truly common. Breaking this separation is why we thought enums broke the client. A lint rule just landed in master to prevent this.
* [EPM] Index template generation fixes (#53104)
* Only add keyword type field to mappings.
* Index template installation
* Handle empty fields definition files
* Re-enable index template installation
* [Fleet] Assign/Unassign data source from policy UI (#53058)
* Add index files to export various modules; normalize imports
* Clean up unused files; extract datasources table component from policy details page
* Expose http client to frontend libs; remove unused types; import ES UI's useRequest lib
* Adjust shape of rest api adapter interface to better match with rest of kibana; remove unused node adapter; change per_page param to perPage in agent events route
* Initial pass at assign data sources flyout
* Initial pass at unassigning data sources from policy
* Make data sources table searchable by package values
* Fix enrollment key lib for rest adapter param changes
* Fix imports and types
* `yarn.lock` changes after bootstrapping
* [EPM] Implement getConfig for dataset (#53261)
* [EPM] Implement getConfig for dataset
* Implements a getConfig method on a dataset object.
* Build the configuration for each dataset in a package.
* construct and save streams into datasource saved object
* [EPM] Fix template installation (#53272)
As dataset.package was not set, the installed templates contained undefined in the template name. This changes fixes this.
* [EM] Refactor ingest pipeline installation (#53309)
* Refactor ingest pipeline installation
* Only install index templates for requested datasets
* Add index.default_pipeline to index template
* Hook up pipeline rewriting
* Add correct types.
* change POST create datasources path (#53165)
* change POST create datasources path
* remove pkgkey from params
* Fix creation of a data source with a custom ID (#53537)
* [Ingest] Return associated policy IDs in data source info (#53350)
* Return number of policies from data source, surface in assign data source UI
* Update snapshots
* [EPM]: Assign data source to policy in UI (#53597)
* Let ES generate source ids. Refactor along the way.
* Datasource.id isn't optional. It's just missing before we send to Ingest
* Delete EPM's mapping of datasources saved object. Ingest handles that.
* Keep datasource object-related work in constructDatasource
* Move asset installation into own function. Keep entry point high-level.
* More descriptive (less ambiguous) names for these two functions
* Use enum values from Ingest instead of plain strings
* Limit the 'type' key of references to known asset types.
* Update variable names to clarify that we're merging arrays of references
* Use [].flat instead .reduce + .concat to avoid error on empty arrays.
* Pass PackageInfo value directly to component vs pulling off n properties
* Name handlers/options based on the data, not the UI element
* Populate policy combo box based on values from Ingest policy API
* Mark Dataset.vars as optional.
* Add TODOs
* Add commands to run API tests to README (#53847)
* Limit functions to 3 params max. Update those which used more (#53848)
* [EPM] Code in 'common' directories shouldn't import server code (#53854)
* [Fleet] Code in 'common' directories shouldn't import server code (#53938)
* [Fleet] Remove server code from common folder in fleet
* [Fleet] Fix typescript issues after master merge
* [EPM] Fix typescript issues after master merge
* Fix eslint issues
* Fix typescript issues after merge
* Fix merge master missing line
* Fix merge conflict
* [Fleet] Fix registration of Ingest management section (#54065)
* Fix registration of Ingest management section
* Fix i18n key
* [Fleet] Remove server code from common folder in ingest (#53969)
* [Fleet] Connect fleet to policy change update (#53201)
* [Fleet] Send created event when a policy is created
* [Fleet] updated created event when a policy is created
* [Fleet] Send deleted event when a policy is deleted
* [Fleet] Rename output.url => output.hosts (#54258)
* [Ingest] Remove policies UI (#54308)
* Remove meta field UI from policy add/edit form
* Initial pass at policy bulk+single delete UI and API
* Adjust policy links from agent list and detail pages so that links are only active if policy exists
* Add delete policy UI to policy detail page
* Disable policy delete button for default policy
* Commit updated kbn-pm artifact.
CI is failing with messages like
14:52:28 ERROR: 'yarn kbn run build -i @kbn/pm' caused changes to the following files:
14:52:28
14:52:28 packages/kbn-pm/dist/index.js
Following advice from https://elastic.slack.com/archives/C0D8P2XK5/p1570032166063400 and running/committing build
* Update kbn/pm package
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* [EPM] create logs metrics index patterns (#54037)
* fixes bug in for loop returning too early and not looping through all yaml files creating incomplete index template, move loading yaml files to own function, other cleanup
* use reduce in place of for loop
* basic functionality for creating index patterns
* separate logs and metrics index patterns
* dedupe fields
* adjust flattenFields to rename nested fields with parent name path
* some tests
* use yml files for tests
* add awaits as part of installing the package
* optimize loading of yaml files
* fix typo
* change type packageName to package
* update tests to use all files from the beginning
* fix type errors
* fix test
* Use dataset.package from registry
https://github.com/elastic/kibana/pull/54037#pullrequestreview-340362812
* Form validation on add datasource page. (#53920)
* [Ingest] Add support for policy `label` field (#54413)
* Allow `label` field in policy APIs, update UIs to support `label` field
* PR review changes, typo fixes, update tests
* [Fleet] Fix api key creation (#54498)
* [Fleet] Move agent acks to his own endpoint (#54401)
* [Ingest] Fix MaxListenersExceededWarning during kibana boot (#54745)
* [Fleet] Create a default api key for the default output (#54658)
* [EPM] update index pattern fields (#54862)
* add DatasetType type
* move loadFieldsFromYaml to fields
* add logic to determine field values
* update tests
* add back accidentally removed readFromDocValues
* remove DatasetType and add IndexPatternType
* rename dedup to dedupe
* group tests
* use null coalescing operator
* Fix typing issues
* [Fleet] Support agent event subtype STOPPING (#55231)
* [EPM] Prevent double submit when creating data sources
Update installationRequested state after submit (#55100)
* [EPM] handle alias fields when creating kibana index pattern (#55254)
* add alias support, update flattenFields to handle alias copying, update and add tests
* update snapshot
* update findFieldByPath to return undefined if not leaf node
* remove temporary alias type from map
* [EPM] handle multi fields when creating index patterns (#55554)
* handle mult_fields
* move nested function out
* [Fleet] Fleet/spec docs (#55619)
* [EPM] Start to document definitions (#55361)
This is a first stab at creating a place where we define the terms we use across ingest management. This is not complete but defines a place where we can add all the future defintions.
* [EPM] Indexing strategy docs (#55301)
* [EPM] Indexing strategy docs
This documentation is to start documenting our new indexing strategy in a public place and have it versioned. This will allow us to share the current state of the indexing strategy more easily in a single place, track it when updated and also already have it ready for our future users to look it up and understand the benefits of it.
* update typos
* fix one more typo
* apply review feedback
* skip fields that are disabled (#55735)
* [Fleet] Fix fleet typing issues after merging master
* [EPM] Create fieldFormatMap in kibana index pattern (#55892)
* add support for fieldFormatMap
* output params must be camel case
* fix case
* add url_template param
* [Fleet] Use user from saved object to create apiKeys (#55458)
* [EPM] Document package upgrade behaviour (#56138)
This PR adds more detailed documentation on what should happen when a package is upgraded.
* Remove some explicit typing to pass type checks.
This abstraction will be removed in the single plugin going into master.
* remove readFromDocValues (#56227)
* Add symlink to yarn.lock to fix(?) CI
https://github.com/elastic/kibana/pull/56443/checks?check_run_id=418123781 failed saying EPM directory "MUST have a 'yarn.lock' symlink"
Seems to have originated with https://github.com/elastic/kibana/pull/55440
Following example from other legacy/plugins/* in that PR
* Like 441d9ed, but correct
* [Ingest] Convert `ingest` plugin to new platform `ingest_manager` plugin (#56262)
* Seed Ingest Manager as a new NP plugin
* Add contexts for core, deps, and config. Begin routing and nav UI
* Export NP ready request from top-level es_ui_shared/public/
* Add nav styling w/ theming, add useRequest hook
* Set up license and config server-side services; add test routes
* Move most types and constants into /common
* Initial pass at:
* data stream and agent config models
* data stream routes and schemas
* Initial pass at agent config api route handlers
* Change plugin id to camel case
* Fix circular schema dependency, add security as optional plugin
* Create appContext service, use request user info in agent config routes + libs
* Create default agent config
* Add default output host config, output typings, and create default output and its api key
* Move saved object mapping to new plugin
* Change data streams -> datasources
* Add legacy plugin to bootstrap mappings
* Adjust fleet's ingest dependencies
* Disable policies UI in Fleet
* Adjust EPM's ingest dependencies
* Adjust ingest manager base API route
* Adjust fleet's client side ingest dependencies
* Remove more ingest dependencies from fleet
* REMOVE MOST OF LEGACY INGEST PLUGIN
* Add section for agent configs in UI nav
* Allow useRequest and sendRequest consumers to specify typing for response
* Initial pass at porting over agent config list UI
* Port over agent config creation
* Port over delete agent config functionality
* Fix app routing
* Port over fleet setup routes
* Adjust fleet's ingest dependencies
* Make fleet happy path work, skip some tests (MESSY! :))
* Remove policy list UI code from fleet
* Change useRequestResponse error type
* Add missing agent config schemas and hooks
* Fix type check issues
* Register IM under management
* Fix type issues as a result of changes to use/sendRequest interfaces
* Make all ingest saved objects *not* space-aware
* Fix i18n path
* Fix app categories import
* Fix datasource package assets schema (array of asset objects)
* Seed Ingest Manager privileges to fix tests
* Change `features` to optional plugin instead of required
* Fix security privileges tests
* Fix feature test
* Fix duplicate enrollment key created for default agent config
* Fix fleet agent enrollment by catching agent config 404
* PR feedback
* [Fleet] Detailed docs of fleet <-> agents interactions (#56212)
* [EPM] update index patterns on install/uninstall of package (#56591)
* create index patterns functionality on install/uninstall of package
* update snapshots
* [Fleet] Generate an ES api key per agent per output (#56637)
* [Fleet] Remove unused enrollement rules (#56753)
* remove files related to creating data source (#56745)
* Fix typing issues after mergin master
* Fix api key authentication after master merge
* [EPM] NP Migration: Move server files and route handlers to ingest_manager (#56854)
* initial pass moving registry and categories endpoint
* moves all needed server files and gets list endpoint working
* add route list validation schema
* remove epm config
* use config to get the registryUrl
* clean up registry url
* gets file endpoint working
* add info endpoint
* get install package endpoint working
* support uninstall package endpoint
* add API response types
* move epm types to models
* move AssetType to IngestAsset type from ingest plugin
* remove redundant export
* update epm_api_integration tests to new endpoint paths in ingest manager
* fix imports
* [Ingest Management] Change indexing from {type}-{namespace}-{dataset} to {type}-{dataset}-{namespace} (#56132)
Currently we have the indexing strategy defined as `{type}-{namespace}-{dataset}`. In this PR I propose to change this to `{type}-{dataset}-{namespace}`. As all 3 fields are constant keyword fields, the orders does not matter in most cases.
The reason I propose this change is to better align the name of indices with the name of the other assets:
* ingest pipeline: {type}-{dataset}
* index template: {type}-{dataset}
* Index pattern: {type}-{dataset}-*
* alias name: {type}-{dataset}-{namespace}
This makes it easier to remember the asset names conventions (at least for me).
It makes a difference when specifying security per namespace: To lock down security, previously it is `/(logs|metrics)-prod-$/` and becomes `/(logs|metrics)-[^-]+-prod-$/`. In any case, we should help / assist the user to get this right.
* [EPM] Remove epm plugin and directory (#57309)
* delete server files, move over epm saved object schemas and mapping, stop epm plugin loading in xpack
* updated yarn
* change to updated name
* remove epm dir, copy readme to ingest_manager
* move package.json over and update yarn.lock
* update package name
* add yarn.lock symlink
* fix yarn.lock symlink
* remove epm from security privileges map
* remove epm from feature endpoint and i18nrc
* [Fleet] Move fleet to the new platform and to ingest_manager plugin (#56803)
* [Fleet] NP migration public part (#57567)
* [EPM] EPM to new plugin, UI part (#56882)
* Move EPM home / list view over to ingest-manager
* Use react-router-dom in epm section.
* WIP: add package detail view.
* Use correct route.
* Only import needed types to public
* Remove obsolete file.
* Import type correctly
* Revert "Remove obsolete file."
This reverts commit 4b061102ebc62b49e7d1291060405ea8d23a3a8a.
* Routes are still needed, fix them.
* Import types correctly
* More type import fixes.
* update get categories hook
* remove no longer used getCategories function
* get list packages hook working
* delete routes.tsx, cleanup links
* add the usePackageInstall hook
* replace rest of api calls with use/send request
* remove tmp_routes
* bring over breadcrumbs
* remove comments and get styles working
* get ride side col loading
* temp type fix
* remove useCore
* add assets
* remove comment
* add public directory to legacy ingest_manager and update asset path
* Fix PackageInfo type. Use for API & UI vs a saved object type.
The `as PackageInfo` cast was required because the pipeline was typed as returning `Installed | NotInstalled` which are saved object response.
Updating that to PackageInfo allows the `as` to be removed but revealed an incompatibility between the `assets` properties of RegistryPackage and PackageInfo
```
Types of property 'assets' are incompatible.
Type 'Record<"kibana", Record<KibanaAssetType, KibanaAssetParts[]>>' is missing the following properties from type 'string[]': length, pop, push, concat, and 28 more.
```
It seems the `RegistryPackage & PackageAdditions` didn't cause the PackageAdditions.assets to replace the RegistryPackage.assets property.
I changed the definition of PackageInfo to do what I initially thought it was doing. See comments in models/epm.ts for more about how the new type is constructed.
* remove comment
* fix paths
* fix public paths
* fix path
* remove ui types file
* fix types
Co-authored-by: Sandra Gonzales <neptunian@users.noreply.github.com>
Co-authored-by: John Schulz <github.com@jfsiii.org>
* [Fleet] AgentEvent change agent_id and remove data (#57818)
* Remove legacy `ingest` plugin completely (#58056)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* WIP. 1 type error (but >1 bugs) remaining.
* Add `callCluster` accessors for using ES vs appContext.getClusterClient()
* Undo (?) changes to kbn-pm/dist/index.js
* Run scripts/build_renovate_config for @types/tar
* Replace data w/ agent_id in server schema
* Different way to declare a saved object type
* Use a more specific path to the agent script
* Replace data with agent_id for agent checkin
* Restore internalSavedObjectsClient in app context
* Use project & HTTP TS types in scripts/dev_agent
* Remove ingestManager from FTR features
The plugin is disabled by default and not currently running the FTR tests. I believe we'll add this back when we restore the EPM integration tests.
* Move more variables to common/{constants,types}
* Remove ingestManager from default expected features
* Enable conditional routes. Adjust integration tests
EPM routes currently return a 500 for these tests. For now they `.expect(500)` when enabled and `.expect(404)` when disabled. We can look into the issue and get them to `.expect(200)` in later tests.
* Replace React.FC with React.FunctionComponent following new repo pattern
* Enable Fleet & Ingest FTR tests
* Remove duplicate *Response entries from server/types
* Update README instructions for CLI flags. Rearrange sections (#58363)
* Add instructions for CLI flags. Rearrange sections
Also added some more information about the plugin behavior with links to relevant code
* Add instructions for Ingest & Fleet FTR tests
* Restore search to EPM list page
* [Fleet] Allow to configure CA sha for kibana and elasticsearch (#58186)
* [Ingest] Adjust saved object mappings and rename policy -> config (#58670)
* Replace all reference to (agent) "policy"/"policies" with "config"/"configs"
* Adjust output and datasource saved object mappings
* Update schemas and types to match SO changes
* Fix type check
* Adjust default output object
* Fix property names in tests
* Move installing of index templates and ingest pipelines to package installation (#58619)
* update template and pipeline asset names, install on package install
* fix package install error handling messaging
* save references to installations to package saved object
* add epm.enabled flag for epm functional test runner
* don't add suffix to pipeline entry
* [Fleet] Add a schema of all of our saved objects (#58769)
* [Fleet] Do not use default id for saved objects that need to be encrypted (#57876)
* Adding events for index pattern generation (#58908)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Fix yarn.lock after merge
* [Fleet] create agent config before enabling fleet user (#59166)
* support for top level elasticsearch assets installation (#58869)
* add support for ilm policy installation
* check if ilm policy exists
* handle prebuilt index templates, update tests
* cleanup
* update type in install
* fix installing index templates to create multiple ones for inheritance
* [Fleet] Create default output while creating default config (#59223)
* Use the new definitions from package-registry#176 (#59311)
* [Fleet] Remove our custom API key authentication (#59212)
* [Ingest] Full agent config schema & API (#59262)
* Add schema for full agent config and business logic to convert SO agent config -> full agent config
* Adjust output properties in full agent config
* Whitelist full agent config output fields
* [Fleet] UI Agent enrollment flyout (#58524)
* Use POST vs GET for EPM install & remove (#59367)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* install default packages during setup (#59330)
* install default packages during setup
* check if package is already installed
* [Ingest] Updates to Agent Configuration List UI (#59374)
- Added search bar
- Sync of columns to design
- Actions per row in popup menu
- Connect pagination and per-page count to API request
- Support for `kuery` url param
* Update docs from /api/:section to /api/ingest_manager/:section (#59422)
* [EPM] Add Streams TS type to mirror EPR's (#59446)
* Add Streams TS type in EPM to mirror EPR's
Follows the changes add in https://github.com/elastic/package-registry/pull/230/files#diff-7dea786222588c32c19238bffffee9c2
* Add RegistryPackage.datasources
* Add more detailed shape for Registry Vars
* Don't code in modify src/plugins/management
See PR convo at c47975535f (r389042975)
* [Ingest] Add agent counts to each agent config output of `/agent_configs` API (#59552)
* Added agent counts to getAgentConfigsHandler
* Show agent counts on Agent Config List
* [Fleet] Agent list header (#59487)
* [EPM] Use /packages & /packages/{pkgkey} (#59550)
* Use /packages & /packages/{pkgkey}
* Update paths in skipped tests
* Docs use /packages vs /package/{install,delete}
Some copy & code sample changes.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [Ingest] Design sync for Agent Configuration Create Flyout (#59479)
* Match form to design
* Support URI route param to open flyout
* [Ingest] Create data source step 1 & 2 (#59590)
* Add Error and PackageIcon components
* Add create datasource layout+navigation, and select package step
* Add description field to datasource, remove assets and description from datasource package
* Add temporary datasources typing for EPM package info
* Initial pass at configure datasources step. Stream vars only (no input vars yet)
* First pass at input vars config; separate components
* Fix issue with adding more than one datasource to a config
* Add shell review step; save datasouce
* Remove assign/unassign datasource from agent config details UI, replace with add datasource buttons
* Remove actions column from datasource table
* Initial pass at create datasource from package
* Move package to config service to /common, add tests
* Rename VarsEntry to RegistryVarsEntry, add datasets and datasources to RegistrySearchResult definition
* Add typings to create datasource flow
* Add real count of agents to select agent config list
* Ensure the necessary package is installed at time of datasource creation
* Use lowercase pkgkey for consistency
* Update EPM file path to use /packages (#59693)
* fixed header padding (#59711)
* [EPM] Use icons from packages, if present (#59765)
* [Ingest] Create data source step 3 (#59822)
* Make app setup loading state prettier
* Add review step of datasource wizard
* Change name to ID in agent config datasources field
* Fix types
* Add stored datasource to agent datasource unit tests
* Adjustment of registry typings and which registry copy fields to show to sync with elastic/package-registry#242
* Fix `multi` vars not populating with array: elastic/kibana#59724
* Account for if a stream is enabled by default from registry package definition: elastic/kibana#59724
* Adjust tests to account for last two commits
* Fix review page back link
* Fix d'oh typo
* [Ingest] Agent Config Details header and sub tabs navigation (#59783)
- Syncs Agent Config Details header to design
- Includes sub navigation tabs connected to route URL
- Agent Config List Create data source row action enabled
* [Fleet] update agent list UI (#59685)
* [Fleet] ensure default packages are added to the default config (#59759)
* [Fleet] fix output rename api_token => api_key (#60001)
* [Ingest] Address #59376 feedback (#59961)
* Disable create/destroy CTAs if no write capability
Use `core.application.capabilities.ingestManager.write` to test user permissions
* Add -all & -read tags for HTTP routes
* Update test .expect() to match description
* Add useCapabilities hook. Fix two issues with hiding/disabling CTA.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Missed one in e12a8ad8a4
* Use package icon as default when no other can be found (#60025)
* Remove duplicate xpack setting
FTR tests failing to start ES with error
```
ERROR ERROR: setting [xpack.security.authc.api_key.enabled] already set, saw [true] and [true]
```
https://github.com/elastic/kibana/pull/59376/checks?check_run_id=503930031 & https://github.com/elastic/kibana/pull/59376/checks?check_run_id=503975576 etc
It appears the xpack.security.authc.api_key.enabled flag was recently added to master in another part, so remove our instance of the setting to prevent the error
* Update EPM file tests to use /packages/{pkgkey}
These should have failed when the routes were changed. Will go back and see what happened.
* [Ingest] Add `revision` to agent configs & data sources (#59848)
* Add revision to agent config and datasource saved objects, add delete datasource service and datasource
* Add revision to full agent config output
* PR feedback
Co-authored-by: Sonja Krause-Harder <sonja.krause-harder@elastic.co>
Co-authored-by: Nicolas Ruflin <spam@ruflin.com>
Co-authored-by: Sandra Gonzales <neptunian@users.noreply.github.com>
Co-authored-by: Matt Apperson <me@mattapperson.com>
Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
Co-authored-by: Nicolas Chaulet <n.chaulet@gmail.com>
Co-authored-by: Matt Apperson <matthew.apperson@elastic.co>
Co-authored-by: Jen Huang <its.jenetic@gmail.com>
Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
Co-authored-by: Tyler Smalley <tyler.smalley@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: neptunian <sandra.gonzales@elastic.co>
Co-authored-by: Jonathan Buttner <56361221+jonathan-buttner@users.noreply.github.com>
Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>
Co-authored-by: Henry <henry.harding@elastic.co>
* Using re2 for Timelion's regexs
* Patching native modules
* Restructuring to be more generic
* Fixing download_node_builds_task tests
* Updating linux sha after properly gzipping the archive
* Using a Centos7 machine with devtoolset-6. That's what node does
* Using new archives which Travis built for us
* Not using a standard import to prevent Kibana from not starting up
If the "portable" version of RE2 for some reason isn't truly portable
and can't load, we don't want to prevent the rest of Kibana from working
properly. This will only prevent Timelion from working, which isn't
great, but is less worse
* Isolating the require even further
* Detecting the package version mismatches, thanks Larry!
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Updating svgo -> css-tree -> mdn-data, all so we get mdn-data > 2.0
* fix extra entry in yarn.lock file
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: spalger <spalger@users.noreply.github.com>
* scaffolding and notes.md
* add skeleton event generator to kibana
* add optional entityID param to generateEvent
* add tree generation
* add tests
* working tests
* fix up tests
* fix linting
* fix event types
* make process parent types consistent
* make generator match types
* move test resolver node out of common types
* fix random string generation
* fix typecheck errors
* remove extraneous stuff
* address PR comments
* add test for full resolver tree
* cleanup
* make tests clearer
* add seedrandom to endpoint plugin. contains DONOTMERGE example code
* remove robs test
* start replacing random with seedrandom
* use seeded random for uuidv4
* separate out IP randomization
* typecheck fixes
Co-authored-by: oatkiller <robert.austin@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Updated EUI to 20.0.1; updated typescript usage
* snapshots
* Upgrade to eui 20.0.2, fix one more type
* PR feedback
* Update EUI icon usage to the correct types
* Updated with master
* Addresses #55544.
- uses the core breadthfirst cytoscape layout
- rotates elements by -90degrees
- selects rum nodes as roots
- implements hover styles to show connected nodes
- fixes flash of unstyled cytoscape elements on initial load
* PR review feedback
* adds canned response for testing cytoscape layout in storybook
* update dep snapshot for removing cytoscape-dagre
* [kbn/optimizer] fix ui/* url rewrites in dist
* add tests to verify styles are built correctly and ui-rewrites are happening
* clarify change to dirs creation
* create tested & shared parsePath helper
* update renovate config
* split implementation of parsePath for dir and file paths
* switch to valid css property
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* adding comprehensive dashboard tests
* fixing delete and adding dima changes
* Fixing some of the a11y test failures
* Fixing i18n issue
* Extracting exit fullscreen logic in a separate function
* Fixing typo
* Upgrading axe
* Fixing failing jest tests
* Removing main tag as it was causing a test to fail
* Adding focusable=false to a range control as well
* Update test/accessibility/apps/dashboard.ts
Co-Authored-By: Michail Yasonik <michail.yasonik@elastic.co>
* Fixing linting error
* Update src/legacy/core_plugins/input_control_vis/public/components/vis/list_control.tsx
Co-Authored-By: Michail Yasonik <michail.yasonik@elastic.co>
* Add comments
Co-authored-by: Bhavya RM <bhavya@elastic.co>
Co-authored-by: Michail Yasonik <michail@yasonik.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Fully migrates metrics and logs to the NP
Co-authored-by: Jason Rhodes <jason.matthew.rhodes@gmail.com>
Co-authored-by: John Schulz <github.com@jfsiii.org>
Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
* eui to v19.0.0
* typescript updates; idAria removal
* src snapshot updates
* mock euicode and euicodeblock for jest
* x-pack snapshot updates
* mock euicode for jest
* more euicode snapshots
* mock euicode in storyshots
* types/enzyme yarn.lock
* sidenav type update
This PR adds a sidebar navigation. clicking the icons in the nav will focus the camera on the different nodes. There is an animation effect when the camera moves.
* build immutable bundles for new platform plugins
* only inspect workers if configured to do so
* [navigation] use an index.scss file
* add yarn.lock symlink
* set pluginScanDirs in test so fixtures stay consistent
* cleanup helpers a little
* fix type error
* support KBN_OPTIMIZER_MAX_WORKERS for limiting workers via env
* test support for KBN_OPTIMIZER_MAX_WORKERS
* expand the available memory for workers when only running one or two
* add docs about KBN_OPTIMIZER_MAX_WORKERS environment variable
* fix README link
* update kbn/pm dist
* implement bundle caching/reuse
* update kbn/pm dist
* don't check for cache if --no-cache is passed
* update renovate config
* standardize on index.scss, move console styles over
* add support for --no-cache to cli
* include worker config vars in optimizer version
* ignore concatenated modules
* update integration test
* add safari to browserslist to avoid user-agent warnings in dev
* update docs, clean up optimizer message/misc naming
* always handle initialized messages, don't ignore states that are attached to specific events
* reword caching docs, add environment var to disable caching
* tweak logging and don't use optimizer.useBundleCache as that's disabled in dev
* handle change notifications
* batch changes for 1 second
* rename CompilerState type to CompilerMsg
* getChanges() no longer needs to assign changes to dirs
* remove unused deps
* split up run_worker.ts and share cacheKey generation logic
* add a couple docs
* update tests and remove unused imports
* specify files when creating bundle cache key
* remove one more unused import
* match existing dev cli output more closely
* update kbn/pm dist
* set KBN_NP_PLUGINS_BUILT to avoid warning in CI
* avoid extending global window type
* add note to keep pluginScanDirs in sync
* pass browserslistEnv in workerConfig so it is used for cache key
* load commons.bundle.js in parallel too
* emit initialized+success states if all bundles are cached
* load bootstraps as quickly as possible
* skip flaky suite
* bump
* update jest snapshots
* remove hashing from cache key generation
* remove unnecessary non-null assertion
* improve docs and break up Optimizer#run()
* remove unused import
* refactor kbn/optimizer to break up observable logic, implement more helpful cache invalidation logic with logging
* fix tests
* add initializing phase
* avoid rxjs observable constructor
* remove unnecessary rxjs helper, add tests for bundle cache
* update consumers of optimizer
* update readme with new call style
* replace "new platform" with "kibana platform"
* fix a couple more renames
* add support for several plain-text file formats
* fix naming of OptimizerMsg => OptimizerUpdate, use "store" naming too
* one more OptimizerMsg update
* ensure bundles are not cached when cache config is false
* test for initializing states and bundle cache events
* remove unnecessary timeout change
* Remove unnecessary helpers
* Add tests for BundleCache class
* Add tests for Bundle class
* test summarizeEvent$
* missing paths are no longer listed in mtimes map
* add tests for optimizer/cache_keys
* Add some extra docs
* Remove labeled loop
* add integration test for kbn-optimizer watcher components
* querystring-browser removed
* tweak logging a smidge, improve info and final message
* remove unused imports
* remove duplication of getModuleCount() method
* move type annotation that validates things
* clear up the build completion message
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* chore(NA): introduce futureEmitAssets option. chore(NA): bump related webpack dependencies.
* chore(NA): remove minimizer compress on dll compiler.
* chore(NA): re enable dead code removal.
* chore(NA): remove not needed file
* docs(NA): add todo on dll compiler to remove this on webpack 5
We're seeing occasional "Error: Call retries were exceeded" exception
with Terser, which should be resolved by
abfd950620,
first included in 2.3.4
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* Remove angular dependencey from vis/tooltip
* Move tooltip logic into vislib
* Remove and fix all ngMock refs in vislib tests
* Add numeral to renovate config
* Add vis_type_vislib to codeowners
* Move vis_legend into vislib and fix errors
* vis_type_vislib/public imports to be only top-level
Resolver is a map. It shows processes that ran on a computer. The processes are drawn as nodes and lines connect processes with their parents.
Resolver is not yet implemented in Kibana. This PR adds a 'map' type UX. The user can click and drag to pan the map and zoom using trackpad pinching (or ctrl and mousewheel.)
There is no code providing actual data. Sample data is included. The sample data is used to draw a map. The fundamental info needed is:
process names
the parent of a process
With this info we can topologically lay out the processes. The sample data isn't yet in a realistic format. We'll be fixing that soon.
Related issue: elastic/endpoint-app-team#30
* Upgraded EUI to 18.0.0
* Fix breaks from `palette._.colors` changes
* snapshots
* Updated hard coded hex color codes in tests, fixed TS errors
* Updated a functional test's selector; added (BSD-3-Clause AND Apache-2.0) to license checker whitelist
* Functional test selector update
* Updated vega browser-ci tests for palette changes
* rebased on master
* One more location for EUI package number update and yarn lock
* Fixed lurking [but introduced] TypeScript logic bug
* Swap a prop definition for the same value but tied closer to its source
Co-authored-by: Caroline Horn <549577+cchaos@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* feat(NA): implement dll split feature
* chore(NA): improve logic to split dlls
* feat(NA): support multiple manifests in the clean dll task
* fix(NA): patch clean dll task to support multiple manifest files
* feat(NA): shuffle seed before split dll in chunks
* chore(NA): correctly load chunks into the templates
* chore(NA): correctly load chunks on karma tests
* docs(NA): add explanation why we need a single runtime for dll chunks
* teste(NA): fix jest tests
Some maintenance and minor fixes to state containers based on experience while working with them in #53582
Patch unit tests to use current "terminology" (e.g. "transition" vs "mutation")
Fix docs where "store" was used instead of "state container"
Allow to create state container without transition.
Fix freeze function to deeply freeze objects.
Restrict State to BaseState with extends object.
in set() function, make sure the flow goes through dispatch to make sure middleware see this update
Improve type inference for useTransition()
Improve type inference for createStateContainer().
Other issues noticed, but didn't fix in reasonable time:
Can't use addMiddleware without explicit type casting #54438
Transitions and Selectors allow any state, not bind to container's state #54439
* share specific instances of some ui packages
* remove unnecessary eslint changes, every package will define deps anyway
* remove mentions of moment webpackShims in eslint resolver
* remove use of lodash
* list angular as dep for x-pack
* add operations as codeowner of shared-deps pkg
Today, apps rely on AppState and GlobalState in the ui/state_management module to deal with internal (app) and shared (global) state. These classes give apps an ability to read/write state, when is then synced to the URL as well as sessionStorage. They also react to changes in the URL and automatically update state & emit events when changes occur.
This PR introduces new state synching utilities, which together with state containers src/plugins/kibana_utils/public/state_containers will be a replacement for AppState and GlobalState in New Platform.
* Move fp-ts and immer to "." package.json
* Revert "Move fp-ts and immer to "." package.json"
This reverts commit b876df0d54.
* Second attempt, fp-ts and immer -> root
* fp-ts -> 2.3.1
* Revert x-pack/package.json
* Update fp-ts in x-pack/package.json
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
As we recently added react-use as a dependency, makes sense to clean up those generic hooks from Kibana repo.
Removed custom hooks from kibana_react and other places:
useObservable
useUnmount
useShallowCompareEffect
react-use should be used instead:
import useObservable from 'react-use/lib/useObservable'
* Shim input_control_vis
* Convert input_control_vis src files to typescript
* Add Required, Optional, Required and Class types to kbn-utility-types
* Collect all ui/* imports into legacy imports file
* Pass down plugin deps from top level
* Add timeout and terminate_after options to SearchSourceFields
* Instrument Kibana with APM RUM agent
* make route-change transaction work with properl url
* extract page-load transaction url from app link
* check if app is hidden and set active:false
* make distributed tracing work and merge config
* remove config/apm.js and address review
* address review comments
* add apm.js to build tassks
* move apm from dev to src
* add @types/hoist-non-react-statics which is required by react rum
* apply changes correctly from master
Adds a `runNow` api to Task Manager, allowing us to force the refresh of a recurring task.
This PR includes a couple of sustainability changes as well as the feature itself.
1. **Declarative query composition.** At the moment the queries in the TaskStore are huge JSON objects that are hard to maintain and understand. This PR introduces a pattern where the different parts of the query are composed out of type-checked functions, making it easier to maintain and to construct dynamically as needs change. _This was included in this PR as the **markAvailableTasksAsClaimed** query needs different query clauses depending on whether there are specific Tasks we wish to claim first.
2. **Refactoring of the Task Poller** As the `runNow` api is introduced we find Task Manager's lifecycle in a weird state where it has both a _pull_ model, where timeouts & callbacks interact without having to responsd to any external requests, and a _push_ model where requests are made to the new `runNow` api. Balancing these two proved error prone, hard to maintain and had the potential of _lossy_ behaviour where requests are dropped accidentally. To address this TaskPoller has been refactored using Rxjs observables, remodelling the existing _pull_ mechanism as a _push_ mechanism so Task Manager can _respond_ to both _polling_ calls and _runNow_ in a similar fashion.
And ofcourse the main feature of this PR:
3. **runNow api** An api on TaskManager that takes a _task ID_ and attempts to run the task. The call returns a promise which resolves with a result which notifies the caller when the task has either completed successfully, or result in an error.
Ensure no deprecated Node.js core API's are used in Kibana. This is
achieved by throwing an error in either development mode or in CI if one
of the deprecated API's is called, and as such, new PR's should no
longer be able to be merged if they use deprecated API's.
Some of these API's (like the `Buffer` constructor`) is a security risk.
Instruments Kibana with Elastic APM by adding the Node.js agent to the
source code. The agent is not turned on by default but can be enabled by
setting the environment variable `ELASTIC_APM_ACTIVE=true` or by
creating an apm config file called `config/apm.dev.js` and setting
`active: true` inside of it.
This implementation is not meant to be used by end-users of Kibana as it
lacks integration with the regular Kibana config file. For now, this is
meant as a useful internal tool for Elastic employees when developing
Kibana.
By default, it's pre-configured with a `serverUrl` pointing to an APM
Server hosted on Elastic Cloud. The data is stored in an ES cluster
accessible only by Elastic employees. These defaults can easily be
overwritten using environment variables or via the custom config file.
This adds an absolute session timeout (lifespan) to user sessions.
It also improves the existing session timeout toast and the overall
user experience in several ways.
* run mocha tests from x-pack with root mocha script
* Only run Karma tests in xpack intake job
* disable failing suites
* fix typo
* skip correct suite (there are multiple root suites)
* support disabling junit reporting with $DISABLE_JUNIT_REPORTER
* don't generate junit in ispec_plugin tests
* eui to 16.0.0
* type defs
* card click
* snapshot updates
* time format test updates
* ts return
* integration fix
* async
* more time format fixes
* more time format fixes
* test updates
* add uisettings dateformat timepicker test
* after method to clear setting
* initial push
* self code review
* ignore node-fetch type
* usageFetcher api
* user agent metric
* telemetry plugin collector
* remove extra unused method
* remove unused import
* type check
* fix collections tests
* pass kfetch as dep
* add ui metrics integration test for user agent
* dont start ui metrics when not authenticated
* user agent count always 1
* fix broken ui-metric integration tests
* try using config.get
* avoid fetching configs if sending
* type unknown -> string
* check if fetcher is causing the issue
* disable ui_metric from functional tests
* enable ui_metric back again
* ignore keyword above 256
* check requesting app first
* clean up after all the debugging :)
* fix tests
* always return 200 for ui metric reporting
* remove boom import
* logout after removing role/user
* undo some changes in tests
* inside try catch
* prevent potential race conditions in priorities with =
* use snake_case for telemetry plugin collection
* usageFetcher -> sendUsageFrom
* more replacements
* remove extra unused route
* config() -> config
* Update src/legacy/core_plugins/telemetry/index.ts
Co-Authored-By: Mike Côté <mikecote@users.noreply.github.com>
* Update src/legacy/core_plugins/ui_metric/server/routes/api/ui_metric.ts
Co-Authored-By: Mike Côté <mikecote@users.noreply.github.com>
* config() -> config
* fix SO update logic given the current changes
* fix opt in check
* triple check
* check for non boolean
* take into account older settings
* import TelemetryOptInProvider
* update test case
* [a11y] add initial accessibility functional tests
* add accessibility jobs
* fix config path
* remove percy setup from scripts
* disable color-contrast rule
* apply changes from @myasonik
* define aria-controls/owns props even when suggestions aren't visible
* [ftr/a11y] only throw error when there are errors
* adding tests for management page
* add a11y test for management page
* adding ignore rules' to a11y
* accessibility test for kibana home
* 7 passing tests, 0 failures
* jest snapshot update
* support a11y test in pipeline job
* update a11y test script for pipelines
* use oss compatible ci setup
* fix exclude syntax
* add default exclusion syntax
This PR include three key changes:
1. Run tasks as soon as they have been marked as running, rather than wait for the whole batch to me marked
2. Use a custom refresh setting of refresh: false where possible, in place of wait_for, in order to speed up Task Manager's internal workflow
3. Instrumentation of Task Manager exposing Activity / Inactivity metrics in Performance test runs
* Update dependency rxjs to ^6.5.3
* move argument type def up for better coverage
* adapt to possibly undefined helpExtension
* complete definition of deprecation$ type
* define types that bindNodeCallback can no longer infer
* define more types that bindNodeCallback can't infer
* be more explicit and accurate about the types for a subject/observable pair
* fix interface error, TS now identifies it
* ignore a return type flaw because types are not being managed properly
* avoid overspecifying types
* allow types to be inferred where possible
* remove unnecessary withLatestFrom()
* reduce number of rxjs versions installed
* update kbn/pm dist
* Update moment related packages
* unify moment version in yarn.lock
* adapt ro pluralization fixes
* update some bad pluralization in fixtures/snapshots
* [npm] Removes react-clipboard.js
Added in the original commit of APM, but does not appear to still be used.
* [npm] Remove redux-test-utils
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Originally added in https://github.com/elastic/kibana/pull/19236 as part
of the notification service, however it's no longer used.
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* [npm] Remove babel-plugin-mock-imports
Originally added as part of
https://github.com/elastic/kibana/pull/22695, however, it doesn't appear
to have ever actually been used.
* [npm] Removes gulp-multi-process
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Originally added in https://github.com/elastic/kibana/pull/6791 as part
of custom notification banners, however it's no longer used.
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Added when X-Pack was closed-source and
import/no-extraneous-dependencies was implemented. Does not appear to
still be used.
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* Update dependency del to v5
* unify del/rimraf usage with del v5
* update yarn.lock
* update kbn-pm distributable
* remove poorly transformed code
* force some deletes from tmp
* mock less of the fs module
* force tmp deletion in a few more suites
* please make this the last force
* Update eslint related packages
* add errorMessage to no-restricted-paths validation
* add license to eslintrc file
* ignore broken plugin fixture
* upgrade tangled peer deps
* give name to unnamed func to avoid typescript-eslint/typescript-eslint#1078
* migrate away from removed prefer-interface
* migrate away from removed no-angle-bracket-type-assertions rule
* migrate away from removed no-triple-slash-reference rule
* old config was invalid and made this rule a noop
* fix array-type rule config
* fix readonly paramProperties that weren't being checked
* fix propType ordering
* remove references to non-existent rules
* remove unnecessary disable that is improperly formatted
* fix comment-related spacing
* fix typo/invalid namespace overrides
* rename fn to avoid hook-alike behavior
* remove reference to old removed rule
* use caret version for prettier
* reference babel-eslint with absolute path
This removes the outdated and unused 0.x version of `constate` and instead depends on a recent version. It also removes the aliasing to `constate-latest`.
closes#48606
* Add file parse chunking, update component on progress
* Clean up clean and validate and redo to process single features
* Add oboe dependency
* Prevent state updates on cancel
* Handle new files added mid-way through parsing another file
* Fix issue where subsequent index name is wiped out when previous file cancelled
* Remove unneeded oboe abort logic
* Dice parsing logic up further for testing
* Clean up
* Revert "Fix issue where subsequent index name is wiped out when previous file cancelled" (covered in separate PR)
This reverts commit 0688e73ffc.
* Update file parse test to focus on different stream states
* Update clean and validate tests to reflect function input/output changes
* Bump up file buffer. Simplify ui update logic, not neceesary to throttle with less frequent callbacks
* Show features parsed on UI rather than percentage
* Remove extra mock reset
* Review feedback. Add localized feature tracking callback
* Review feedback. Add comment explaining progress update throttling. Also, use debounce to throttle
* Remove console log
* Consolidate feature handling into one function passed to oboeStream node
* Abstract oboe logic to separate class and import for use in file parser
* Update file parser test to mock PatternReader import
* Prevent file parse active flag from resetting if another file is in progress
* Don't pass back result if no features found on complete, throw error with feedback. Add clean-up for prev PatternReader
* Use singleton version of jsts reader & writer. Pass back unmodified feature if clean returns nothing
* Make fileHandler function async
* Return null if no geometry
* Handle single features differently. Fixes functional test error
* Update jest test to use unique instances & counts of readers
* Review feedback
* Review feedback
* Review feedback. Add error-handling for null geom
* Fix i18n error
* Clean up handling of cancelled/replaced files to account for changed fileHandler return type
* [Code] use native git to iterate git files
* [Code] use native git to clone/update repository
* [Code] git history using native git
* [Code] use native git to read file tree and file content
* [Code] fix the 'bad file' warning from status api
* [Code] use native git to handle worktree
* [Code] use native git to resolve references
* [Code] use native git to handle blame / diff
* [Code] patch git binaries in kibana build script
* [Code] migrate unit tests to use native git
* get coverage for console functional test
* instrument kibana
* collect coverage stats after each test, on url navigation, refresh and back
* switch to babel-plugin-istanbul
* switch to babel-plugin-istanbul
* [coverage service] basic version
* trigger final coverage flush when ftr finishes, wait for all logs before exiting
* add coverage report generation
* increase optimizer timeout, re-run idx/babel plugin before istanbul
* increase memory usage for node
* put istanbul preset in the beginning to run as the last one
* bump babel-plugin-istanbul up to 5.2.0
* cleanup
* save unique json files with coverage
* [functional test coverage] update coverage.json path
* run code coverage on CI
* increase max-old-space-size with env variable
* fix coverage folder creation
* use env variable to configure code coverage
* revert ci config changes
* remove duplicate plugin
* remove comments
* fixes
* refactor failed_tests_reporter to use TS, no octokit
* update renovate config
* ensure that all kbn-test files are in ts project
* fix some type errors
* add some more tests
* [kbn-test/githubapi] cleanup and document
* collect log messages as strings instead of message objects
* ensure issue is open when updating body
* improve readability of getKibanaIssues
* expose axios helpers from dev-utils
* fix request params for fetching github issues and validate locally
* include a README for failed_tests_reporter
* improve axios error helpers
Add service map tabs on the main APM screen and for individual services.
This is not yet hooked up to work with back-end data, so it always shows the same hard-coded graph.
This is experimental, so you must have x-pack.apm.serviceMapEnabled: true in your Kibana config for it to show up.
Also add "PSF" to the list of allowed licenses since a new dependency added uses this license (it's on the [green list](https://github.com/elastic/open-source/blob/master/elastic-product-policy.md#green-list).)
Fixes#44890Fixes#44853
* Add option for Internet Explorer IE11
* WIP trying to figure out IE options/capabilities
* more attempts to get IE tests working
* more work, but still can't set font size in Dev Tools Console
* two changes for IE
* revert skip in console test
* this change *appears* to set absolute times but they don't stick
* Shakespeare passing on all 3 browsers
* add config.ie.js, change 'internet explorer' to 'ie'
* increase notifications:lifetime so ew have a chance to see the message
* fix lint error remove unused Options
* updates from PR review, remove console.log
* cleanup + reuse existing code
* make iedriver path work for x-pack tests
* try to avoid changing notification timeout
* resolve path and add correct delimiter
* remove unused code, make timepicker work on IE11
* work-around 45333 for IE11
* [WIP] removing some work-arounds
* revert work-arounds
* cleanup
* remove debugging code
* remove debugging code
* removed unused code
* reverting 2 files that didn't have any material changes
* adds linter rule to disallow undeclared variables
* adds cypress eslint plugin, remove redundant packages from siem/package.json
* removes comment from json and adds a line about cypress linting to the cypress README
* adds upgraded package back to siem/package.json
* updates cypress run scripts since cypress package is now located in the x-pack/node_modules
* needed to yarn kbn bootstrap after rebasing with master
* use new setScreenshotSize
* change tolerance from 1% to 2% (originally 5%)
* retake baseline without notification dialog
* close add and save notifications
* wait for render complete when going in to FullScreen mode
* add 5 second sleep to see if it ever goes to full screen mode
* add debugging screenshots and sleeps
* try reloading the Kibana index before this test
* 30 iterations with only the first test enabled
* revert multi-run changes
* ran node scripts/build_renovate_config
* fix tslint errors
* fix tslint error
* update xpack dependency to match oss
* update yarn.lock
* revert notifications:lifetime:info change by reducing some other timeouts
* remove commented-out code
* fixed and Unskipped 2nd test
* re-arrange order of dashboard tests to pass :-(
* removed commented-out sleep
Hopefully this won't be an issue
* Add comment about the order of tests
```
loadTestFile(require.resolve('./view_edit'));
// Order of test suites *shouldn't* be important but there's a bug for the view_edit test above
// https://github.com/elastic/kibana/issues/46752
// The dashboard_snapshot test below requires the timestamped URL which breaks the view_edit test.
// If we don't use the timestamp in the URL, the colors in the charts will be different.
loadTestFile(require.resolve('./dashboard_snapshots'));
```
* Revert "Updating load-grunt-config to ^3.0.1 (#46408)"
This reverts commit 2733c093e0.
* adapt changes from #46408 to not invalidate yarn state on re-installs
* Update angular related packages
* upgrade jQuery too
* don't delete screenshots in pipeline jobs
* use ng-change so that listener is triggered after ng-model is updated
* match @types/angular versions
* update yarn.lock
* fix test
$routeChangeSuccess is no longer broadcast when resolving a redirect route, so this test has been adjusted to validate that
* Update jest related packages
* use correct type
* adapt spy and describe.each types
* update yarn.lock
* update snapshot
* define types before passing to describe.each
