2018-04-20 21:13:37 +02:00
|
|
|
/*
|
|
|
|
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
|
|
|
* or more contributor license agreements. Licensed under the Elastic License;
|
|
|
|
* you may not use this file except in compliance with the Elastic License.
|
|
|
|
*/
|
|
|
|
|
2019-06-18 20:45:53 +02:00
|
|
|
import { xpackMain } from './legacy/plugins/xpack_main';
|
|
|
|
import { graph } from './legacy/plugins/graph';
|
|
|
|
import { monitoring } from './legacy/plugins/monitoring';
|
|
|
|
import { reporting } from './legacy/plugins/reporting';
|
|
|
|
import { security } from './legacy/plugins/security';
|
|
|
|
import { tilemap } from './legacy/plugins/tilemap';
|
|
|
|
import { dashboardMode } from './legacy/plugins/dashboard_mode';
|
|
|
|
import { logstash } from './legacy/plugins/logstash';
|
|
|
|
import { beats } from './legacy/plugins/beats_management';
|
|
|
|
import { apm } from './legacy/plugins/apm';
|
|
|
|
import { maps } from './legacy/plugins/maps';
|
|
|
|
import { indexManagement } from './legacy/plugins/index_management';
|
|
|
|
import { spaces } from './legacy/plugins/spaces';
|
|
|
|
import { canvas } from './legacy/plugins/canvas';
|
|
|
|
import { infra } from './legacy/plugins/infra';
|
|
|
|
import { taskManager } from './legacy/plugins/task_manager';
|
|
|
|
import { rollup } from './legacy/plugins/rollup';
|
|
|
|
import { siem } from './legacy/plugins/siem';
|
|
|
|
import { remoteClusters } from './legacy/plugins/remote_clusters';
|
|
|
|
import { crossClusterReplication } from './legacy/plugins/cross_cluster_replication';
|
|
|
|
import { upgradeAssistant } from './legacy/plugins/upgrade_assistant';
|
|
|
|
import { uptime } from './legacy/plugins/uptime';
|
|
|
|
import { encryptedSavedObjects } from './legacy/plugins/encrypted_saved_objects';
|
Introduce basic alerting and actions plugin (#37042)
* Create actions plugin (#35679)
* Basic alerting plugin with actions
* Remove relative imports
* Code cleanup
* Split service into 3 parts, change connector structure
* Ability to disable plugin, ability to get actions
* Add slack connector
* Add email connector
* Ability to validate params and connector options
* Remove connectorOptionsSecrets for now
* Fix plugin config validation
* Add tests for slack connector
* Default connectors register on plugin init, console renamed to log, slack to message_slack
* Add remaining API endpoints for action CRUD
* Add list connectors API
* Change actions CRUD APIs to be closer with saved objects structure
* WIP
* Fix broken tests
* Add encrypted attribute support
* Add params and connectorOptions for email
* WIP
* Remove action's ability to have custom ids
* Remove ts-ignore
* Fix broken test
* Remove default connectors from this branch
* Fix API integration tests to use fixture connector
* Rename connector terminology to action type
* Rename actionTypeOptions to actionTypeConfig
* Code cleanup
* Fix broken tests
* Rename alerting plugin to actions
* Some code cleanup and add API unit tests
* Change signature of action type service execute function
* Add some plugin api integration tests
* Fix type check failure
* Code cleanup
* Create an actions client instead of an action service
* Apply Bill's PR feedback
* Fix broken test
* Find function to have destructured params
* Add tests to ensure encrypted attributes are not returned
* Fix broken test
* Add tests for validation
* Ensure actions can be updated without re-passing the config
* Remove dead code
* Test cleanup
* Fix eslint issue
* Apply Peter's PR feedback
* Code cleanup and fix broken tests
* Apply Brandon's PR feedback
* Add namespace support
* Fix broken test
* Pass services to action executors (#37194)
* Pass services to action executors
* Fix tests
* Apply PR feedback
* Apply PR feedback pt2
* Cleanup actions plugin (#37250)
* Cleanup actions, move code from alerting plugin PR
* Rename service terminology to registry
* Use static encryption key for encrypted attributes plugin inside of tests
* Empty data after create test is done running
* Fix type checks
* Fix inconsistent naming
* add server log action for alerting (#37530)
adds the first "builtin" alertType for performing a `server.log()`
* Create alerting plugin (#37043)
* WIP
* Rename fire function and remove @ts-ignore in all places
* Change naming in alerting service
* Remove alert instance class for now, support interval configuration
* Cleanup TS
* Split alerting between registry and client
* Use saved object alongside task manager instance
* Add remaining alerting APIs
* Change create structure
* Rename some variables, change actionGroups structure
* Use handlebars for templating strings at fire time
* Fix params given to alert type execute function
* Use alert instance class
* Alert instances support meta attributes
* Move alert instances deserialization
* Change interval to be ms
* Rename actions es archive
* Fix tests to use encrypted esArchive for action record
* Add create alert test to demo end to end flow
* Fix type check issue
* Alerts to use references to action objects
* Only update task manager tasks after saved objects are fully updated
* Use scope in task manager
* Fix type check
* Use task manager to execute actions
* Convert ids into references and back
* Apply PR feedback
* Fix broken test
* Fix some bugs
* Fix test errors
* Alert interval to be previous runAt + interval instead of now + interval
* Add range support
* Remove extra line
* Cleanup
* Add alert_instance.test.ts
* Add alert_type_registry.test.ts
* Move tests around
* Create generic task manager mock
* Add note about saved objects client mock
* Create alert_type_registry.mock.ts
* Add alerts_client.test.ts
* Add create_alert_instance_factory.test.ts
* Add create_fire_handler.test.ts
* WIP
* Fix get_create_task_runner_function.test.ts and make test pass
* Make get_create_task_runner_function.test.ts 100% coverage
* Add unit tests for routes
* Move files around
* Created transform_action_params.ts
* Add get_next_run_at.ts
* Add comment explaining why we copy nextRunAt
* Re-use state within alert instance
* Finalize code coverage in unit tests
* Create base api integration tests
* Add a test that ensures end to end functionality of an alert
* Fix ui capabilities test
* Fix broken plugin api integration test
* Fix jest tests with new saved objects client
* Fix broken integration tests
* Change api integration test fixture to make more sense, add functions for future tests
* Move alerts integration testing into own file, prep to add more tests
* Add tests to ensure failed task instances get retried
* Add get_create_task_runner_function.test.ts for actions, create encrypted saved objects mock
* Add action validation tests
* Ensure action type validation occurs on update
* Test 400 on unregistered alert types
* Ensure alertTypeId can't be updated
* Add validation test for alert create / update
* Fix broken checks / tests
* Skip failing test for now
* Cleanup jest tests
* Ensure action objects can be updated while keeping encrypted attributes readable
* Remove partial update sopport, remove ability to change actionTypeId, require config
* Ensure actionTypeConfig is validated on create and update
* Add alertTypeParams validation support
* Fix failing tests
* Ensure alert cleanup errors don't replace the original error
* Pass callCluster as a service to alerts and actions
* Only pass log to alerts client
* Pass savedObjectsClient as a service to alerting and actions
* Fix failing tests
* Remove range support, provide when current and previous task got scheduled
* Ensure Joi validation happens before every execute
* Remove skipped tests, to be done in future PR
* Apply self feedback pt1
* Apply self feedback pt2
* Fix broken tests
* Apply PR feedback
* PR feedback pt1
* Apply security team PR feedback
* PR feedback pt1
* PR feedback pt2
* PR feedback pt3
* Fix broken tests
* Fix callCluster to have signature
* Revert f11a6aeb06c65e5ab54bf3cc36694845c812b6a7
* PR feedback pt4
* Remove __jest__ folders
* PR feedback pt5
* Fix Joi from leaking secrets
* Fire instance actions in parallel instead of series
* Fix failing jest tests
* Accept core api changes
* Fix saved objects client mock
* PR feedback pt1
* Fix eslint issues
* Throw error when alert instance already fired (#39251)
* Throw error when alert instance already fired
* shouldFire doesn't need its own boolean value
* Actions & alerting getting started user guides (#39093)
* Initial user guides
* Cleanup
* Typos, example changes
* Switch to tables, use ordered list for usage
* Start docs around alert instances and templating
* Documentation changes
* Some adjustments
* Apply PR feedback
* Apply suggestions from code review
Co-Authored-By: gchaps <33642766+gchaps@users.noreply.github.com>
* PR feedback pt2
* Provide better examples for alert types
* Apply PR feedback
* Update README locations
2019-06-21 01:37:17 +02:00
|
|
|
import { actions } from './legacy/plugins/actions';
|
|
|
|
import { alerting } from './legacy/plugins/alerting';
|
2019-09-17 20:57:53 +02:00
|
|
|
import { lens } from './legacy/plugins/lens';
|
Add Ingest Manager plugin to master (#56567)
* [Fleet] Add permissions checks (#48143)
* Add permissions check to API
* Add permissions check to UI
* Undo changes
* Update index.tsx
* Update enroll.ts
* add API tests for permissions
* add fleet to trial license check
* Agent enrollment flyout (#48173)
* Fix agent table empty prompt
* First pass at agent enrollment flyout with placeholders
* Move enrollment commands into separate file and enable templated vars
* Use default double braces for templating
* [Fleet] unenroll agent from the details page (#48286)
* [Fleet] add API to unenroll agents
* [Fleet] return a 403 for inactive agent
* [Fleet] UI to unenroll an agent from the detail page
* [Fleet] Filter inactive agent from the listing (#48466)
* [Fleet] Expose policy change method from fleet plugins (#48562)
* [IM] add screenshot detail view and image endpoint (#48149)
* add api endpoint to handle images
* #47978 add screenshots component with single image
* update padding around screenshot
* import existing API_ROOT
* move ImageRequestParams interface and fix type
* pass the content-type through and add test
* fix ie11 issues with nested flex items, change radius to use eui variable
* use eui variables for padding
* add aria label and image description
* [IM] Use EPM in variables & types (#48453)
* IntegrationsManager -> EPM
* integrationsManager -> epm
* [iI]ntegration -> [pP]ackage. Update tests.
* Don't rename integrations registry URL.
* Update i18n key in x-pack/.i18nrc.json
* Update path to functional test config
* Add epm to recently re-enabled privileges test
* Update two values recently added in main feature branch
* Move/update screenshot tests
* [Fleet] Allow to edit metadata (#48682)
* [Fleet] Add a toggle to show Inactive users (#48917)
* Multiselect to unenroll agents (#48852)
* Initial pass at multiselect to unenroll agents
* Adjust loading state button text
* Fix types; use unenroll provider in agent details
* Update select all agents kuery
* Prevent inactive agents from being selected
* [Fleet] Fix privilege tests after merge of master (#49118)
* [Fleet] Temporary use the elastic user as kibana user to be able to create API keys (#49037)
* [EPM] Fix package version requirement structure (#49172)
* [EPM] Fix package version requirement structure
In https://github.com/elastic/integrations-registry/pull/134 the API structure changes. This PR should adjust to the new structure.
* Update x-pack/legacy/plugins/integrations_manager/common/types.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/integrations_manager/common/types.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/integrations_manager/common/types.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/integrations_manager/common/types.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* [EPM] 48799 markdown into readme (#49180)
* add markdown component and fetch markdown
* add package.json file with react-markdown dependency
* add markdown renderers, use readme path
* remove description and rename to readme
* remove dropshadow on images
* add loading component for readme request
* comment on loading components
* Add command to spin up a mock server for ingest endpoints (#49259)
* 48654 files from registry (#49580)
* Replace image path/handler with generic file one
* Incorporate tests from #48696
* [EPM] Make screenshot image captions optional (#48912)
* Make screenshot image captions optional
* fix typos, type, rename variable
* [EPM] update to layout and and spacing (#49413)
* fix height and color issues
* make container full height
* remove panels per design update and make white bg color
* adjust spacing
* check for empty response
* fix eslint issue
* fix layout in safari
* remove unused component
* [EPM] Rename registry url to new path (#49598)
The integrations-registry repository was renamed to package-registry and with it also all the assets inside. The url under which the service is served will also be changed (not done yet). As soon as this is done, this PR should be merged.
* [EPM] rewrite relative image paths (#49637)
* rewrite relative image paths
* remove EuiImage, use transformImageUri, clean up
* [EPM] Use NP registerFeature (#49625)
* Use NP registerFeature
* Added features plugin to deps. Used it in setup.
Moved `registerFeature` from `init` to plugin setup.
Moved rest of init into `createSetupShim`. `init` is now
```ts
init(server: Legacy.Server) {
const { initializerContext, coreSetup, pluginsSetup } = createSetupShim(server);
new Plugin(initializerContext).setup(coreSetup, pluginsSetup);
},
```
Moved feature (argument for `registerFeature`) to separate file.
Renamed plugin-specific `CoreSetup` to `EPMCoreSetup`
* [Fleet] Filter agents list table by policy name (#49968)
* Add policies lib and adapters
* Fix mock server headers
* Initial pass at policy filter dropdown by manipulating kuery string
* Adjust spec return values
* Use separate state for policy filter; fix typings and disable some eslint rules
* Fix react-related eslint errors
/Users/jfsiii/work/kibana/x-pack/legacy/plugins/integrations_manager/public/hooks/use_links.tsx
26:20 error React Hook "useCore" is called in function "addBasePath" which is neither a React function component or a custom React Hook function react-hooks/rules-of-hooks
/Users/jfsiii/work/kibana/x-pack/legacy/plugins/integrations_manager/public/screens/detail/readme.tsx
26:35 error React Hook "useLinks" cannot be called inside a callback. React Hooks must be called in a React function component or a custom React Hook function react-hooks/rules-of-hooks
39:6 error React Hook useEffect has a missing dependency: 'readmePath'. Either include it or remove the dependency array react-hooks/exhaustive-deps
/Users/jfsiii/work/kibana/x-pack/legacy/plugins/integrations_manager/public/screens/home/search_results.tsx
27:42 error `"` can be escaped with `"`, `“`, `"`, `”` react/no-unescaped-entities
27:49 error `"` can be escaped with `"`, `“`, `"`, `”` react/no-unescaped-entities
✖ 5 problems (5 errors, 0 warnings)
1 error and 0 warnings potentially fixable with the `--fix` option.
* Remove ui/* imports (#50094)
* [Fleet] Use ES api keys for agent authentication (#49639)
* add collapsible read me component (#49990)
* add collapsible read me component
* Update x-pack/legacy/plugins/integrations_manager/public/components/content_collapse.tsx
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* add empty array as second argument to useLayoutEffect
* [EPM] Make API consistent for package installation and removal (#48745)
* Adjust type names
* Install package in one go.
* Integration -> Package
* Really install all known assets
* Remove ingest pipelines on package deletion.
* [EPM] Replace image paths/handlers with generic ones for file (#48688)
* Replace image path/handler with generic file one
* Incorporate tests from #48696
* Fix merge error.
* Type tuning
* Adjust types
* Remove asset types from routes altogether
* Add timelion-sheet back to types.
* Respond with full package info to install/delete
* Be specific in return type.
* Keep installAssets() as separate step
* [EPM] Add unit tests for pathParts (#50279)
* [EPM] Add unit tests for pathParts
This adds unit tests for the pathParts function. I initially wanted to enhance the function to also support paths like `iptables-1.0.4/dataset/log/elasticsearch/ingest-pipeline/pipeline.json`. But without unit tests it was hard for me to make the changes.
This adds a unit test driven by a table so we can extend it later. Adding these tests also helps me to better understand the code.
As a note for myself, the command to run these tests is:
```
node scripts/jest --watch ./legacy/plugins/integrations_manager/server/registry/index.test.ts
```
* Update docs for merging master into feature branch (#50396)
* [Fleet] Remove in memory repository (#50431)
* [EPM] Documentation of HTTP routes & TS types for Ingest (#48798)
* Add beginning models and two routes for Ingest
* Update types & models per discussion w/Ruflin
Also reviewed data structures listed at https://docs.google.com/document/d/1IBR3f9dpHqJmXYEdg06WV34KSMd3g5k4aMGa4jde_Eg/edit#
* Update: /policies always returns array. /policy returns single policy
* Add pagination for /policy & /datasources. Uses per_page & page params
* Add API metadata. Standardize policy_id param name.
* Update descriptions to match Google Doc. Move use case to Policy.
Disabled the '@typescript-eslint/array-type' rule because it was going around in circles. It didn't like Datasource[] or Array<Datasource>
* Return to initial TS annotation for Arrays
Remove the line disabling @typescript-eslint/array-type now that it's behaving normally again :shrug:
* [EPM] Add directory structure for server/lib. (#50469)
* Add directory structure for server/lib.
* 'tests' seems to be more common than 'test'
* Make CI happy
* [EPM] Add basic documentation directory (#50478)
* [EPM] Add basic documentation directory
Having the doc directory around allows us to easily add docs from here on to document how EPM works.
To run the docs build, use the following command from the kibana directory:
```
../docs/build_docs --doc docs/epm/index.asciidoc --open
```
The above assumes that docs (https://github.com/elastic/docs) are checked out in the same directory as Kibana.
With this change, the EPM docs build is not included yet in the overall docs build. For this adjustments to https://github.com/elastic/docs/blob/master/conf.yaml must be made.
* [EPM] Add basic index template (#50471)
This PR adds the very basic index template we will use for the packages. It contains all the basic settings and some examples. The examples will be remove as soon as we have an actual implementation with packages but for now is convenient to see if it is a valid package.
This code is put into the lib directory as it does not tie directly into any handlers.
It also adds an functional tests for loading a template. This means we have a way to check if a template is valid in Elasticsearch. Based on this we can check in the future all our generated templates for validity with Elasticsearch.
To run the functional test, go to the Kibana x-pack directory. Start the first command:
```
node scripts/functional_tests_server.js --config test/epm_api_integration/config.ts
```
Keep the above running and switch to an other Terminal. Now run:
```
node scripts/functional_test_runner.js --config x-pack/test/epm_api_integration/config.ts
```
* 40752 rewrite ingest pipeline (#50627)
* Add directory structure for server/lib.
* 'tests' seems to be more common than 'test'
* Make CI happy
* Implement pipeline rewriting.
* Add more testcases
* For posterity (comment change)
* Allow beats-style template delimiters
* Be more succinct
* Document better
* Replace AssetType enum with union type (#50696)
See https://github.com/elastic/kibana/pull/50609#discussion_r346080439
Discussed in Slack and agree to revert for now. Can track down the issues & restore later
* Remove unnecessary await if we can return the promise (#50329)
* Fix whitespace per figma comments. Closes #47348 (#47350)
* Add fix & comment for TS 3.7.2 regression
* [EPM] cleanup assets, filter assets for those currently supported (#50609)
* cleanup assets, filter assets for those currently supported
* removed unused type
* fix type
* add comment for better type
* change type name to be more descriptive
* hardcode image width for ie11 (#49796)
* hardcode image width for ie11
* eslint
* improve comment
* add maxWidth
* [EPM] useKibana hook & render in plugin.start (#50110)
* plugin.start now does reactdom.render vs returning react element
export plugin function from public/index
* Move setClient call from plugin.start to plugin.setup
* Use `useUiSetting$` from `useKibana` hooks
* Fix broken app due to bad hooks usage
Can't use useKibana outside a React component.
Reverting to prior approach since it's still NP. Can revisit context usage in a followup PR
* [EPM] Install package from detail view on button click (#50735)
* Support basic "click button -> show spinner -> installed" install flow
* Remove incorrect comments. Add TS return types to data functions.
* [EPM] Use NP feature_catalogue.register (#50108)
* Use NP feature_catalogue.register
* Use type from NP plugin
* fix linting
* fix types
* fix headers in Fleet
* skipping test due to ES param change
* Revert "skipping test due to ES param change"
This reverts commit d05f20decfbfc4d91069816a6f8dfde26b5bd6bc.
* remove type field
* remove unused import
* [EPM] Final(?) update from integrations_manager -> EPM (#50976)
* Update (all remaining?) references from integrations_manager to EPM
* Update path in i18n file
* [EPM] update compatibility section (#50975)
* change min max version to single range value
* add elastic stack icon and change text
* remove badge from version and use code font
* remove euistyled
* add back version component lost in merge
* remove euiStyled
* remove old file
* Restore RequirementVersionRange type
* Disable test for elasticsearch username.
Temporary work around until we know how the stack wants to add the permissions we need. Either adding to the kibana user or creating a new user.
* Revert "Disable test for elasticsearch username."
This reverts commit f1020e4eab2ada5d854eacc44cdb6d5bd23c267f.
* Disable test for elasticsearch username.
Temporary work around until we know how the stack wants to add the permissions we need. Either adding to the kibana user or creating a new user.
* Fix EPM typing issues in register feature
* Fix typings after master merge
* [EPM] CI fixes (#51284)
* Initialize es in test.
* Add it(), no es.init()
* Clean up.
* [EPM] add confirmation modal (#51172)
* add confirmation modal, move install state to Header
* update callout to use title
* move components only used in detail view to detail dir
* use better variable names
* update to more descriptive variable names
* Restore prior response vaulues for install & delete package (#51252)
Discussed this with @skh https://github.com/elastic/kibana/pull/51112#commitcomment-35961413 & https://github.com/elastic/kibana/pull/51112#commitcomment-35970664 as well as in a video call
Also added some TS type annotations for data fetching functions to make the contracts more explicit
* [EPM] /package API only lists installable assets. Restore enums. (#51414)
* API only shows installable assets. Server types to own file. Restore enums
* Fix type imports
* Only return installable asset types (kibana for now)
* server/types now only has code from hapi which shouldn't go to client
* Add more restricted TS types to DisplayAssets object
* Flip order of arguments to Extract
In these cases it still works the same, but looking at https://www.typescriptlang.org/docs/handbook/advanced-types.html the signature is
`Extract<T, U>` - Extract from `T` those types that are assignable to `U`
so the larger set should be first
* [Fleet] Enrollment api key UI (#51495)
* Make button pretty in dark mode as well. (#51610)
* [EPM] Add docs entry about registryUrl config (#51697)
This documentation is at the moment mainly for internal use. I found myself searching for this URL several times in the code or PRs so I thought I rather add it to the docs for now.
* [EPM] Remove encoding of Kibana objects as not needed anymore
* [Fleet] Move agent status server side and API to get aggregated status for a policy (#51673)
* [EPM] Add basic docs around install/delete API endpoint (#51728)
This is mainly for internal usage at the moment to look up.
* Ingest/policy (#51741)
* wip policy
* tweaks
* tweaks
* FIX TYPOS
* WIP move policy => agent config conversion to fleet, WIP policy changed method
* fix tests and bugs
* updates tests and snaps
* more fixes
* use AGENT_POLLING_INTERVAL
* cleanup and fix some formatting
* Update x-pack/legacy/plugins/ingest/server/libs/datasources.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/ingest/server/libs/datasources.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/ingest/server/libs/outputs.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/fleet/server/libs/policy.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* fix things broken by PR review suggestions
* remove unused field
* fix types
* fix mappings
* add datasource mappings
* Fix mappings and remove get full policy from checkin
* Fix ingest api integration tests
* run es-lint to fix fleet
* Fix typescript issues
* [EPM] Track package install state and add toast notification (#51734)
* add notifications from core to plugin
* add package install state hook
* fix type error
* use toMountPoint helper to add jsx to notification
* add warning notification to failed install
* make notifications dependency explicit prop
* move PackageInstall provider lower
* add comment about InstallStatus type overlapping InstallationStatus
* use InstallStatus type in InstallationButton component
* fix type
* [Ingest] Adds support for a working default output (#51841)
* aadding config
* add working settings to the default output
* remove default username and password
* update libs
* [EPM] Add basics for creating the ILM setup (#50474)
This contains the basic objects to setup ILM
* Create index and alias with a write index
* Get the policy
The code does not contain any functional tests yet as it is still open on how to do it best. I suggest to get this in as a foundation and then iterate on top of it.
* [EPM] Add datasource (ingest pipeline) from package (#51851)
## Summary
This mixes a few concerns but I think it's worth it to show the parts working together.
Take a look at the individual commits for a better separation of features.
This adds
- the `/datasource/install/{pkgkey}` endpoint which installs ingest pipelines from a package into ES and saves a reference to them in the EPM state Saved Object
- Connects the "Add datasource" button in the successful installation Toast to the new API
- Adds a toast notification to inform the user the datasource was added correctly
- Adds a "Delete Package" button on the details page so we can uninstall a package while we're waiting for the separate view which allows deletes
- b99eda6 Pushes logic that was in the detail view into `InstallationButton`. This consolidates the logic in one component (or one component & the existing hook) and, iiic, means we can put `<InstallationButton package={...} />` on any view and get the same behavior
I'm marking this as a normal PR so people can merge if they wish
![add-datasource-delete-package-small](https://user-images.githubusercontent.com/57655/69775686-7fb39280-1167-11ea-8d41-e2b8a02252a1.gif)
* [EPM] Add basic processing of fields.yml file (#51148)
The fields.yml is used to generate the Elasticsearch template and Kibana index pattern. This PR adds a very basic implementation of processing the fields.yml and then create an Elasticsearch template out of it. The only fields that are supported at the moment are keyword fields, more will be added as a follow up.
The testing was implemented with a golden file. The output from the method is compared to a json file. If the input is changed or the method is changed, it is possible to regenerate the files with the `-generate` flag as following:
```
node scripts/jest ./legacy/plugins/epm/server/lib/template/template.test.ts -generate
```
This will allow us to quickly test many inputs / outputs in the future, make adjustments to the existing files and generate the new outputs. We then can compare it in the diff it the changes make sense.
* [EPM] Create basic implementation to merge input template and dataset manifest (#51803)
* [EPM] Create basic implementation to merge input template and dataset manifest
With this code it is possible to take an input template for the agent and merge it with the config variables from the dataset manifest file. Currently only the name and the default value are merged. Later on we must implement to be able to pass user configured variables to it and make decision based on OS selection.
Closing https://github.com/elastic/kibana/issues/51794
* [EPM] Refactoring of lib structure (#51885)
This refactors the structure of lib. As so far all the lib parts are related to assets in the package, it is organised the same way with the same structure. Each directory has its own tests directory if it needs one. This makes it possible to (almost) not need relative paths for tests.
* [EPM] Allow to read files from fields directory (#51958)
This change allows to also extract files from the `fields` directory. Previously this was not possible because it always assumed a service must be there.
* [EPM] Install Elasticsearch Index Template for data source (#51878)
This installs the Elasticsearch index template for each dataset in a package. For now the names are hardcoded based on package key and dataset name but will be more dynamic later on when we pass the full dataset information.
The dataset extractions is a bit "hacky" at the moment and we should get a full implementation of dataset at a later stage and replace this code.
* [Fleet] Policy list, details, create, edit UIs (#51950)
* Set up simple policies list view
* Adjust spec to return single policy
* Set up simple policy details page
* Add demo stats/chart to policy details
* Add description string
* Initial setup of policy form and create policy UI
* Policy create/edit form; integrate policy list api
* Integrate create policy api
* Integrate policy detail, agent status, and policy edit APIs; adjust policy list api integration in agent enrollment
* Fix edit policy mock meta
* Fix policy list search bar
* PR and linting fixes; use typings from ingest plugin
* Fix i18n
* [EPM] Add datasource saved object type (#51871)
## Summary
This PR makes a few assumptions, and contains a lot of refactoring. It might be beneficial to look at the resulting directory structure under `server` first to get the (new) big picture.
Assumptions:
- our API deals with several concerns, for now these are packages and datasources
- we manage our own HTTP API endpoints for these concerns (in particular, don't use the ingest plugin for that)
- we manage (for now) the Kibana saved object in which datasources are saved. importing and calling methods from the ingest plugin to do that down the road will (hopefully) be a manageable change
This led to the following decisions:
- the code is separated into subdirectories by concern, containing all the route handlers and tightly coupled code
- for now, these directories are in `server/packages` and `server/datasources`. I'm tempted to move them into `server/api/{packages,datasources}` but wanted to limit the amount of refactoring in one PR
- shared code lives in `server/lib`
- some code from `server/packages` has been almost duplicated to handle saving to Datasource saved objects, some has been refactored and is used from both places. The deduplication needs further improvement
- maybe `server/registry` should also move under `server/lib` (but see above, I'm trying to not move everything around all at once)
Testing:
* Please note that this is a breaking change because the saved object type for package information
has also been renamed. You'll need to start with a fresh `.kibana-*` index. Restarting `yarn es snapshot` (withouth specifying a data directory) should do the trick.
* Package installation should still work, e.g. with a GET request to `http://localhost:5601/api/epm/package/coredns-1.0.1`. The saved objects for packages can be inspected with a GET request to `http://localhost:5601/api/saved_objects/epm-package/$PKG_KEY`, e.g. `http://localhost:5601/api/saved_objects/epm-package/coredns-1.0.1`
* Datasource creation should still work, e.g. with a GET request to `http://localhost:5601/api/epm/datasource/install/coredns-1.0.1`. The saved objects for datasources can be inspected with a GET to `http://localhost:5601/api/saved_objects/epm-datasource/$PKG_KEY`, e.g. `http://localhost:5601/api/saved_objects/epm-datasource/coredns-1.0.1`
* [Fleet] Expose policy during agent checkin (#51968)
* [EPM] Add /epr prefix to the tar.gz download path (#51881)
The registry slightly changed the .tar.gz path because of download stats reason. This adjusts for it. See https://github.com/elastic/package-registry/pull/169
* [EPM] Move template installation to lib and add asset helper (#52049)
* [EPM] Move template installation to lib and add asset helper
All the logic related to the installation of the templates for a package should be inside the template library folder. This moves the logic into this folder.
A few refactorings were made to simplify installation:
* Introduction of DataSet interface: This interface is needed to extract the data sets inside a package and install one template per data set.
* Pass package instead of package key to installation process: Passing the package instead of the package key means fetching of package information is decoupled from the installation process and abstracted. This separates the two concerns and should simplify testing.
* getAsssets method: The getAssets methods works on top of the package object to extract asset paths. It is inspired by get_objects methods but supports passing a package and a dataset.
Currently one problem with testing that exists is that to fetch the content of an asset is not decoupled yet.
* [EPM] Reduce data source to one type (#52061)
Between Fleet / Ingest / EPM there had been several interface definitions of Datasource and the related types. This reduces it to one place for the definition. The same applies to the policy definition.
The goal of this is that from now on we all rely on the same definition. If we make changes, we make them in all parts of the code.
In this PR is only the minimal change needed to get us all on one interface. Further changes will be needed that we all rely on the same saved objects etc.
* add export command
* revert 2 more files to rely on export
* revert imports
* Fix types for Datasource Saved Object
* merge in master
* fix type check
* Run VSCode's organize imports on EPM files (#52234)
Learned about it on Slack from https://twitter.com/ryanchenkie/status/1201883268527927301
Blog at https://code.visualstudio.com/updates/v1_23#_run-code-actions-on-save
Basically does the order we've been loosely following (3rd party, then relative) & alphabetic by location and variable name.
It's not customizable but it's reasonable and, afaict, consistent.
* [EPM] More realistic datasource SO. Error if package not installed. (#52229)
* Move cache 'hack' into getAssetsData
* p -> pkg. package is reserved. pkgkey is used in many places
* Remove unnecessary type cast
* Clarify reasons behind asset path manipulation
* Return the Datasource; not the Saved Object.
* Use real values from package in fake datasource SO
* Error if /datasource/install before /package/install
```
> curl --user elastic:changeme localhost:5601/api/epm/datasource/install/coredns-1.0.1
{
"statusCode": 403,
"error": "Forbidden",
"message": "coredns-1.0.1 is not installed"
}
> curl --user elastic:changeme localhost:5601/api/epm/install/coredns-1.0.1
[
{
"id": "53aa1f70-443e-11e9-8548-ab7fbe04f038",
"type": "dashboard"
},
{
"id": "Metricbeat-CoreDNS-Dashboard-ecs",
"type": "dashboard"
},
{
"id": "75743f70-443c-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "36e08510-53c4-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "277fc650-67a9-11e9-a534-715561d0bf42",
"type": "visualization"
},
{
"id": "cfde7fb0-443d-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "a19df590-53c4-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "a58345f0-7298-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "9dc640e0-4432-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "3ad75810-4429-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "57c74300-7308-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "27da53f0-53d5-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "86177430-728d-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "4804eaa0-7315-11e9-b0d0-414c3011ddbb",
"type": "visualization"
}
]
> curl --user elastic:changeme localhost:5601/api/epm/datasource/install/coredns-1.0.1
[
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_plaintext_json",
"type": "ingest-pipeline"
},
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_json_json",
"type": "ingest-pipeline"
},
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_entry_json",
"type": "ingest-pipeline"
}
]
```
* fix duplicated imports
* [EPM] Move golden files generation over to jest snapshot (#52203)
* [EPM] Move golden files generation over to jest snapshot
I initially used my own implementation to write the generated files. It runs out jest has a feature to write snapshots which simplifies the code a lot.
I added a loop with an additional test file so in the future we can just keep adding test files without having to modify the test code.
To updated the snapshots, the param `-u` has to be used:
```
node scripts/jest legacy/plugins/epm/server/lib/fields/field.test.ts -u
```
* [EPM] Create metrics-* and logs-* Kibana index pattern (#52277)
This creates the very basic Kibana index patterns metrics-* and logs-* for Kibana. At the moment it is overwritten every time. We need to change this in the future to take the fields from all installed data sources and regenerate it.
* [EPM] Create helper for elasticsearch asset names (#52265)
Most of the Elasticsearch assets have the same base name. This creates a helper to get the base name for the assets. In case we decide to change the base name in the future, we can change it in one place.
* fix tests and destructing
* [EPM] Update Registry types. Prevent errors installing certain datasources. (#52285)
* Update RegistryPackage type
* Use download key from EPR to fetch archive
* Fix errors caused by correcting the Registry types.
The issues were largely that some Registry types like `title, `datasets` and `assets` where marked as required, but are actually optional. This highlighted area in the code were we relied on them always being present.
We added to the issue by wrapping Registry types in `Required` which made those items which were correctly listed as optional, required for EPM code. Updated EPM types to reflect the largely pass-through nature of the EPM types.
There are two properties which we ensure are in every EPM response, those were put into their own (unexported) type.
Confirm by trying to add a datasource to a package which has no datasources, like apache-1.0.1 or system-2.0.1. In `feature-ingest` and the earlier version of this PR, the `/datasource/install` call returns a 500. In this PR it succeeds.
* [EPM] Add setup of default ILM policies (#52272)
This creates two ILM policies: logs-default and metrics-default. These are the default ILM policies used. Currently the policy content is hardcoded in the code but should be fetched from the base package in the future. The setup happens as part of the datasource installation. When a data source is installed it is a good time to check if the assets are there but we might extract this to a better place in the future.
* [EPM] 52075 add data source first page (#52320)
* Update RegistryPackage type
* add first page of add data source
* fix for ie11 flex min width bug
* remove toDetailViewRelative
* remove unneeded spread
* Update TS type names for EPR search results (#52512)
* `RegistryList -> RegistrySearchResults`
* `RegistryListItem -> RegistrySearchResult`
* Restore import sort order from #52234 (#52548)
Many of the changes from #52234 were lost. Presumably due to PR(s) merging which were based on branches which had the previous unsorted order.
* [EPM] Replace wildcard export (#52554)
* PackageNotInstalledError -> packages/index.ts
* pkgToPkgKey -> registry/index.ts (will convert existing `${name}-${version}` instances later)
* Replace export * from packages.
There's an argument that the import sites should be updated to import from `packages/get`, `packages/install`, etc but that can wait for a later PR.
* [EPM] Reduce usage of epm-package SavedObject (#52576)
* Delete existing Installation type. Rename InstallationAttributes to Installation
* Reduce usage of EPM SO. Add getInstallation().
Replaced two calls of getInstallationObject() with getInstallation().
Two less places with knowledge of SO internals.
Lots of potential improvements for EPM TS types remain (refactoring/removing Installable, etc), but this is a good incremental step, IMO
* [EPM] Fix missing export link (#52628)
Without it, things break. I am surprised CI did not catch this.
* [EPM] Cleanup ILM loading (#52632)
Before the check for the ILM policy to exist triggered an exception. With this change it is a normal response also if the policy does not exist yet.
A follow up issue will be created in Elasticsearch to get a HEAD request for this available.
* [EPM] Switch to staging URL for registry (#52626)
The old cluster with the registry will be removed as soon as this is merged.
* [EPM] Use Dataset interface to generate template (#52255)
This will make sure we have to pass much feature params and can fully rely on the datasource object to create names for assets.
* [Fleet] Use agent events to compute agent health (#52513)
* [EPM] Data source integration tests (#52542)
* Add fixtures for data source integration test.
* Move test setup to beforeEach
* Add test for datasource creation
* Handle pipelines in yml format.
* Make integration test for adding a data source pass.
* Use EPR staging URL with CDN. (#52776)
See https://github.com/elastic/kibana/pull/52626#pullrequestreview-330622868
* [EPM] Add Data Source page updates (#52705)
* remove dupe type RegistryPackage
* change switches to checkboxes, use datasets to create checkboxes, add some local form state
* update types
* [EPM] redirect after package install (#52771)
* add callback after successful installation and redirect
* add temp data sources tab content to access add data source page
* remove assets tab for mvp
* hide data sources link and redirect from data sources tab if package not installed
* change callback name
* remove commented out assets logic
* add redirect to hook
* fix type
* Use ingest datasource api (#52964)
Incremental change. Uses HTTP API for datasource creation. Will do follow-up PR which uses JS function instead
* Remove duplicate fetchInfo & installTemplates
I think this was from a bad merge, but pretty sure we don't want these functions called twice in the same function
* WIP. Pushing so others can see
* Improve correctness/flexibility of absolute URL
* Disable datasource test & template installation
* [Ingest] Data source APIs (#52448)
* Clean up ingest imports and remove unneeded mock_spec files
* Initial pass at datasources lib and API endpoints
* Add add/remove datasource to/from policy API endpoints
* Add datasource contract tests and related policy contract tests; update snapshots
* Fix tests
* Fix tests again
* Fix tests 3
* Adjust routes, PR feedback
* modify epm createDatasource endpoint to use user data (#52971)
* change epm/datasource/install/{pkg} to POST, send user data to endpoint, install pipelines and templates based on user selected datasets
* change test to post for installing a datasource
* change some names and types around
* delete request.headers['transfer-encoding'] being passed through from epm request
* [EPM] Don't share CreateFakeDatasource type (#53068)
It's not shared between client & server so it doesn't need to be in common. Also, it imports server code which would try to bring server types to the client. It's types so they're compiled away but it's important to keep common to what's truly common. Breaking this separation is why we thought enums broke the client. A lint rule just landed in master to prevent this.
* [EPM] Index template generation fixes (#53104)
* Only add keyword type field to mappings.
* Index template installation
* Handle empty fields definition files
* Re-enable index template installation
* [Fleet] Assign/Unassign data source from policy UI (#53058)
* Add index files to export various modules; normalize imports
* Clean up unused files; extract datasources table component from policy details page
* Expose http client to frontend libs; remove unused types; import ES UI's useRequest lib
* Adjust shape of rest api adapter interface to better match with rest of kibana; remove unused node adapter; change per_page param to perPage in agent events route
* Initial pass at assign data sources flyout
* Initial pass at unassigning data sources from policy
* Make data sources table searchable by package values
* Fix enrollment key lib for rest adapter param changes
* Fix imports and types
* `yarn.lock` changes after bootstrapping
* [EPM] Implement getConfig for dataset (#53261)
* [EPM] Implement getConfig for dataset
* Implements a getConfig method on a dataset object.
* Build the configuration for each dataset in a package.
* construct and save streams into datasource saved object
* [EPM] Fix template installation (#53272)
As dataset.package was not set, the installed templates contained undefined in the template name. This changes fixes this.
* [EM] Refactor ingest pipeline installation (#53309)
* Refactor ingest pipeline installation
* Only install index templates for requested datasets
* Add index.default_pipeline to index template
* Hook up pipeline rewriting
* Add correct types.
* change POST create datasources path (#53165)
* change POST create datasources path
* remove pkgkey from params
* Fix creation of a data source with a custom ID (#53537)
* [Ingest] Return associated policy IDs in data source info (#53350)
* Return number of policies from data source, surface in assign data source UI
* Update snapshots
* [EPM]: Assign data source to policy in UI (#53597)
* Let ES generate source ids. Refactor along the way.
* Datasource.id isn't optional. It's just missing before we send to Ingest
* Delete EPM's mapping of datasources saved object. Ingest handles that.
* Keep datasource object-related work in constructDatasource
* Move asset installation into own function. Keep entry point high-level.
* More descriptive (less ambiguous) names for these two functions
* Use enum values from Ingest instead of plain strings
* Limit the 'type' key of references to known asset types.
* Update variable names to clarify that we're merging arrays of references
* Use [].flat instead .reduce + .concat to avoid error on empty arrays.
* Pass PackageInfo value directly to component vs pulling off n properties
* Name handlers/options based on the data, not the UI element
* Populate policy combo box based on values from Ingest policy API
* Mark Dataset.vars as optional.
* Add TODOs
* Add commands to run API tests to README (#53847)
* Limit functions to 3 params max. Update those which used more (#53848)
* [EPM] Code in 'common' directories shouldn't import server code (#53854)
* [Fleet] Code in 'common' directories shouldn't import server code (#53938)
* [Fleet] Remove server code from common folder in fleet
* [Fleet] Fix typescript issues after master merge
* [EPM] Fix typescript issues after master merge
* Fix eslint issues
* Fix typescript issues after merge
* Fix merge master missing line
* Fix merge conflict
* [Fleet] Fix registration of Ingest management section (#54065)
* Fix registration of Ingest management section
* Fix i18n key
* [Fleet] Remove server code from common folder in ingest (#53969)
* [Fleet] Connect fleet to policy change update (#53201)
* [Fleet] Send created event when a policy is created
* [Fleet] updated created event when a policy is created
* [Fleet] Send deleted event when a policy is deleted
* [Fleet] Rename output.url => output.hosts (#54258)
* [Ingest] Remove policies UI (#54308)
* Remove meta field UI from policy add/edit form
* Initial pass at policy bulk+single delete UI and API
* Adjust policy links from agent list and detail pages so that links are only active if policy exists
* Add delete policy UI to policy detail page
* Disable policy delete button for default policy
* Commit updated kbn-pm artifact.
CI is failing with messages like
14:52:28 ERROR: 'yarn kbn run build -i @kbn/pm' caused changes to the following files:
14:52:28
14:52:28 packages/kbn-pm/dist/index.js
Following advice from https://elastic.slack.com/archives/C0D8P2XK5/p1570032166063400 and running/committing build
* Update kbn/pm package
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* [EPM] create logs metrics index patterns (#54037)
* fixes bug in for loop returning too early and not looping through all yaml files creating incomplete index template, move loading yaml files to own function, other cleanup
* use reduce in place of for loop
* basic functionality for creating index patterns
* separate logs and metrics index patterns
* dedupe fields
* adjust flattenFields to rename nested fields with parent name path
* some tests
* use yml files for tests
* add awaits as part of installing the package
* optimize loading of yaml files
* fix typo
* change type packageName to package
* update tests to use all files from the beginning
* fix type errors
* fix test
* Use dataset.package from registry
https://github.com/elastic/kibana/pull/54037#pullrequestreview-340362812
* Form validation on add datasource page. (#53920)
* [Ingest] Add support for policy `label` field (#54413)
* Allow `label` field in policy APIs, update UIs to support `label` field
* PR review changes, typo fixes, update tests
* [Fleet] Fix api key creation (#54498)
* [Fleet] Move agent acks to his own endpoint (#54401)
* [Ingest] Fix MaxListenersExceededWarning during kibana boot (#54745)
* [Fleet] Create a default api key for the default output (#54658)
* [EPM] update index pattern fields (#54862)
* add DatasetType type
* move loadFieldsFromYaml to fields
* add logic to determine field values
* update tests
* add back accidentally removed readFromDocValues
* remove DatasetType and add IndexPatternType
* rename dedup to dedupe
* group tests
* use null coalescing operator
* Fix typing issues
* [Fleet] Support agent event subtype STOPPING (#55231)
* [EPM] Prevent double submit when creating data sources
Update installationRequested state after submit (#55100)
* [EPM] handle alias fields when creating kibana index pattern (#55254)
* add alias support, update flattenFields to handle alias copying, update and add tests
* update snapshot
* update findFieldByPath to return undefined if not leaf node
* remove temporary alias type from map
* [EPM] handle multi fields when creating index patterns (#55554)
* handle mult_fields
* move nested function out
* [Fleet] Fleet/spec docs (#55619)
* [EPM] Start to document definitions (#55361)
This is a first stab at creating a place where we define the terms we use across ingest management. This is not complete but defines a place where we can add all the future defintions.
* [EPM] Indexing strategy docs (#55301)
* [EPM] Indexing strategy docs
This documentation is to start documenting our new indexing strategy in a public place and have it versioned. This will allow us to share the current state of the indexing strategy more easily in a single place, track it when updated and also already have it ready for our future users to look it up and understand the benefits of it.
* update typos
* fix one more typo
* apply review feedback
* skip fields that are disabled (#55735)
* [Fleet] Fix fleet typing issues after merging master
* [EPM] Create fieldFormatMap in kibana index pattern (#55892)
* add support for fieldFormatMap
* output params must be camel case
* fix case
* add url_template param
* [Fleet] Use user from saved object to create apiKeys (#55458)
* [EPM] Document package upgrade behaviour (#56138)
This PR adds more detailed documentation on what should happen when a package is upgraded.
* Remove some explicit typing to pass type checks.
This abstraction will be removed in the single plugin going into master.
* remove readFromDocValues (#56227)
* Add symlink to yarn.lock to fix(?) CI
https://github.com/elastic/kibana/pull/56443/checks?check_run_id=418123781 failed saying EPM directory "MUST have a 'yarn.lock' symlink"
Seems to have originated with https://github.com/elastic/kibana/pull/55440
Following example from other legacy/plugins/* in that PR
* Like 441d9ed, but correct
* [Ingest] Convert `ingest` plugin to new platform `ingest_manager` plugin (#56262)
* Seed Ingest Manager as a new NP plugin
* Add contexts for core, deps, and config. Begin routing and nav UI
* Export NP ready request from top-level es_ui_shared/public/
* Add nav styling w/ theming, add useRequest hook
* Set up license and config server-side services; add test routes
* Move most types and constants into /common
* Initial pass at:
* data stream and agent config models
* data stream routes and schemas
* Initial pass at agent config api route handlers
* Change plugin id to camel case
* Fix circular schema dependency, add security as optional plugin
* Create appContext service, use request user info in agent config routes + libs
* Create default agent config
* Add default output host config, output typings, and create default output and its api key
* Move saved object mapping to new plugin
* Change data streams -> datasources
* Add legacy plugin to bootstrap mappings
* Adjust fleet's ingest dependencies
* Disable policies UI in Fleet
* Adjust EPM's ingest dependencies
* Adjust ingest manager base API route
* Adjust fleet's client side ingest dependencies
* Remove more ingest dependencies from fleet
* REMOVE MOST OF LEGACY INGEST PLUGIN
* Add section for agent configs in UI nav
* Allow useRequest and sendRequest consumers to specify typing for response
* Initial pass at porting over agent config list UI
* Port over agent config creation
* Port over delete agent config functionality
* Fix app routing
* Port over fleet setup routes
* Adjust fleet's ingest dependencies
* Make fleet happy path work, skip some tests (MESSY! :))
* Remove policy list UI code from fleet
* Change useRequestResponse error type
* Add missing agent config schemas and hooks
* Fix type check issues
* Register IM under management
* Fix type issues as a result of changes to use/sendRequest interfaces
* Make all ingest saved objects *not* space-aware
* Fix i18n path
* Fix app categories import
* Fix datasource package assets schema (array of asset objects)
* Seed Ingest Manager privileges to fix tests
* Change `features` to optional plugin instead of required
* Fix security privileges tests
* Fix feature test
* Fix duplicate enrollment key created for default agent config
* Fix fleet agent enrollment by catching agent config 404
* PR feedback
* First pass at deleting everything related to EPM & Fleet plugins
* Remove EPM-specific react-markdown
* Remove some unwanted differences from master.
Removed more EPM and Fleet changes
* React.FC -> React.FunctionComponent to match the pattern in master
* Selectively enable features based on config values
* Fix typing and(?) CI
* Enable xpack.ingestManger during tests
Follow example from xpack.endpoint flags
* Don't create BASE_URL value based on PLUGIN_ID
* Don't add optional routes. Use config for registry url
* Add README with some skeleton docs on tests, etc
* Undo changes to some files based on PR feedback
* Commit the result of 'yarn kbn run build -i @kbn/pm'
* Commit the result of 'yarn kbn run build -i @kbn/pm'
* WIP. But add some passing api integration tests
└-: apis
└-> "before all" hook
└-: Ingest Manager plugin
└-> "before all" hook
└-: /agent_configs
└-> "before all" hook
└-> should get agent configs
└-> "before each" hook: global before each
└- ✓ pass (42ms) "apis Ingest Manager plugin /agent_configs should get agent configs"
└-> should create agent config
└-> "before each" hook: global before each
└- ✓ pass (63ms) "apis Ingest Manager plugin /agent_configs should create agent config"
└-> should have new agent config
└-> "before each" hook: global before each
└- ✓ pass (41ms) "apis Ingest Manager plugin /agent_configs should have new agent config"
└-> should delete that new config
└-> "before each" hook: global before each
└- ✓ pass (1.0s) "apis Ingest Manager plugin /agent_configs should delete that new config"
└-> should get datasources
└-> "before each" hook: global before each
└- ✓ pass (26ms) "apis Ingest Manager plugin /agent_configs should get datasources"
└-> should 404 from EPM package api
└-> "before each" hook: global before each
└- ✓ pass (41ms) "apis Ingest Manager plugin /agent_configs should 404 from EPM package api"
└-> "after all" hook
└-> "after all" hook
└-> "after all" hook
6 passing (1.2s)
* WIP
* Fix TS type errors.
Types are created tests. Should they come from test project?
* Add route to delete datasources (mirrors agent_configs api)
* Remove old ESArchiver files
* Add integration tests for each /agent_config & /datasources route
* Update README
Remove bash integration test scripts
* More updates to README
* Remove xpack.kueryAutocomplete from x-pack/.i18n.json
* Use default appRoute. We can't use /app/ingest (at least for now).
Looking at the app_router it seems like any route starting with `/app` must be `/app/:appId`.
I also don't think appRoute is fully supported. I'll follow up with Platform.
* Use appBasePath from app mount params for React Router basename
* Remove out-of-date comments
* Use the ES client from core
* Move License & Config services WIP. Pushing to share
* Move config & license services into App Context service.
* Remove FTR tests for Ingest Manager
* Remove license-related propperties
* Remove clusterClient properties
* Don't set this.security if not in dependencies
* WIP (failing) integration_test
* No more unknown key errors. Timeouts now
* Test routes with default (disabled) and manager-only flags
* Add tests to cover all permutations of flags (A, AB, AC, ABC)
* Use TS types vs config-schema in common & public
Move all @kbn/config-schema usage to server
Co-authored-by: Matt Apperson <me@mattapperson.com>
Co-authored-by: Jen Huang <its.jenetic@gmail.com>
Co-authored-by: Nicolas Chaulet <n.chaulet@gmail.com>
Co-authored-by: Sandra Gonzales <neptunian@users.noreply.github.com>
Co-authored-by: Nicolas Ruflin <spam@ruflin.com>
Co-authored-by: Sonja Krause-Harder <krauseha@gmail.com>
Co-authored-by: Brian Seeders <seeders@gmail.com>
Co-authored-by: Tyler Smalley <tylersmalley@me.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-02-19 03:30:38 +01:00
|
|
|
import { ingestManager } from './legacy/plugins/ingest_manager';
|
2020-01-14 01:16:20 +01:00
|
|
|
import { triggersActionsUI } from './legacy/plugins/triggers_actions_ui';
|
2018-04-20 21:13:37 +02:00
|
|
|
|
2019-12-13 23:44:04 +01:00
|
|
|
module.exports = function(kibana) {
|
2018-04-20 21:13:37 +02:00
|
|
|
return [
|
|
|
|
xpackMain(kibana),
|
|
|
|
graph(kibana),
|
|
|
|
monitoring(kibana),
|
|
|
|
reporting(kibana),
|
Spaces Phase 1 (#21408)
### Review notes
This is generally ready for review. We are awaiting https://github.com/elastic/elasticsearch/issues/32777 to improve handling when users do not have any access to Kibana, but this should not hold up the overall review for this PR.
This PR is massive, there's no denying that. Here's what to focus on:
1) `x-pack/plugins/spaces`: This is, well, the Spaces plugin. Everything in here is brand new. The server code is arguably more important, but feel free to review whatever you see fit.
2) `x-pack/plugins/security`: There are large and significant changes here to allow Spaces to be securable. To save a bit of time, you are free to ignore changes in `x-pack/plugins/security/public`: These are the UI changes for the role management screen, which were previously reviewed by both us and the design team.
3) `x-pack/test/saved_object_api_integration` and `x-pack/test/spaces_api_integration`: These are the API test suites which verify functionality for:
a) Both security and spaces enabled
b) Only security enabled
c) Only spaces enabled
What to ignore:
1) As mentioned above, you are free to ignore changes in `x-pack/plugins/security/public`
2) Changes to `kibana/src/server/*`: These changes are part of a [different PR that we're targeting against master](https://github.com/elastic/kibana/pull/23378) for easier review.
## Saved Objects Client Extensions
A bulk of the changes to the saved objects service are in the namespaces PR, but we have a couple of important changes included here.
### Priority Queue for wrappers
We have implemented a priority queue which allows plugins to specify the order in which their SOC wrapper should be applied: `kibana/src/server/saved_objects/service/lib/priority_collection.ts`. We are leveraging this to ensure that both the security SOC wrapper and the spaces SOC wrapper are applied in the correct order (more details below).
### Spaces SOC Wrapper
This wrapper is very simple, and it is only responsible for two things:
1) Prevent users from interacting with any `space` objects (use the Spaces client instead, described below)
2) Provide a `namespace` to the underlying Saved Objects Client, and ensure that no other wrappers/callers have provided a namespace. In order to accomplish this, the Spaces wrapper uses the priority queue to ensure that it is the last wrapper invoked before calling the underlying client.
### Security SOC Wrapper
This wrapper is responsible for performing authorization checks. It uses the priority queue to ensure that it is the first wrapper invoked. To say another way, if the authorization checks fail, then no other wrappers will be called, and the base client will not be called either. This wrapper authorizes users in one of two ways: RBAC or Legacy. More details on this are below.
### Examples:
`GET /s/marketing/api/saved_objects/index-pattern/foo`
**When both Security and Spaces are enabled:**
1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function
2) The Security wrapper is invoked.
a) Authorization checks are performed to ensure user can access this particular saved object at this space.
3) The Spaces wrapper is invoked.
a) Spaces applies a `namespace` to be used by the underlying client
4) The underlying client/repository are invoked to retrieve the object from ES.
**When only Spaces are enabled:**
1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function
2) The Spaces wrapper is invoked.
a) Spaces applies a `namespace` to be used by the underlying client
3) The underlying client/repository are invoked to retrieve the object from ES.
**When only Security is enabled:**
(assume `/s/marketing` is no longer part of the request)
1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function
2) The Security wrapper is invoked.
a) Authorization checks are performed to ensure user can access this particular saved object globally.
3) The underlying client/repository are invoked to retrieve the object from ES.
## Authorization
Authorization changes for this project are centered around Saved Objects, and builds on the work introduced in RBAC Phase 1.
### Saved objects client
#### Security without spaces
When security is enabled, but spaces is disabled, then the authorization model behaves the same way as before: If the user is taking advantage of Kibana Privileges, then we check their privileges "globally" before proceeding. A "global" privilege check specifies `resources: ['*']` when calling the [ES _has_privileges api.](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-has-privileges.html). Legacy users (non-rbac) will continue to use the underlying index privileges for authorization.
#### Security with spaces
When both plugins are enabled, then the authorization model becomes more fine-tuned. Rather than checking privileges globally, the privileges are checked against a specific resource that matches the user's active space. In order to accomplish this, the Security plugin needs to know if Spaces is enabled, and if so, it needs to ask Spaces for the user's active space. The subsequent call to the `ES _has_privileges api` would use `resources: ['space:marketing']` to verify that the user is authorized at the `marketing` space. Legacy users (non-rbac) will continue to use the underlying index privileges for authorization. **NOTE** The legacy behavior implies that those users will have access to all spaces. The read/write restrictions are still enforced, but there is no way to restrict access to a specific space for legacy auth users.
#### Spaces without security
No authorization performed. Everyone can access everything.
### Spaces client
Spaces, when enabled, prevents saved objects of type `space` from being CRUD'd via the Saved Objects Client. Instead, the only "approved" way to work with these objects is through the new Spaces client (`kibana/x-pack/plugins/spaces/lib/spaces_client.ts`).
When security is enabled, the Spaces client performs its own set of authorization checks before allowing the request to proceed. The Spaces client knows which authorization checks need to happen for a particular request, but it doesn't know _how_ to check privileges. To accomplish this, the spaces client will delegate the check security's authorization service.
#### FAQ: Why oh why can't you used the Saved Objects Client instead!?
That's a great question! We did this primarily to simplify the authorization model (at least for our initial release). Accessing regular saved objects follows a predictible authorization pattern (described above). Spaces themselves inform the authorization model, and this interplay would have greatly increased the complexity. We are brainstorming ideas to obselete the Spaces client in favor of using the Saved Objects Client everywhere, but that's certainly out of scope for this release.
## Test Coverage
### Saved Objects API
A bulk of the changes to enable spaces are centered around saved objects, so we have spent a majority of our time automating tests against the saved objects api.
**`x-pack/test/saved_object_api_integration/`** contains the test suites for the saved objects api. There is a `common/suites` subfolder which contains a bulk of the test logic. The suites defined here are used in the following test configurations:
1) Spaces only: `./spaces_only`
2) Security and spaces: `./security_and_spaces`
3) Security only: `./security_only`
Each of these test configurations will start up ES/Kibana with the appropriate license and plugin set. Each set runs through the entire test suite described in `common/suites`. Each test with in each suite is run multiple times with different inputs, to test the various permutations of authentication, authorization type (legacy vs RBAC), space-level privileges, and the user's active space.
### Spaces API
Spaces provides an experimental public API.
**`x-pack/test/spaces_api_integration`** contains the test suites for the Spaces API. Similar to the Saved Objects API tests described above, there is a `common/suites` folder which contains a bulk of the test logic. The suites defined here are used in the following test configurations:
1) Spaces only: `./spaces_only`
2) Security and spaces: `./security_and_spaces`
### Role Management UI
We did not provide any new functional UI tests for role management, but the existing suite was updated to accomidate the screen rewrite.
We do have a decent suite of jest unit tests for the various components that make up the new role management screen. They're nested within `kibana/x-pack/plugins/security/public/views/management/edit_role`
### Spaces Management UI
We did not provide any new functional UI tests for spaces management, but the components that make up the screens are well-tested, and can be found within `kibana/x-pack/plugins/spaces/public/views/management/edit_space`
### Spaces Functional UI Tests
There are a couple of UI tests that verify _basic_ functionality. They assert that a user can login, select a space, and then choose a different space once inside: `kibana/x-pack/test/functional/apps/spaces`
## Reference
Notable child PRs are listed below for easier digesting. Note that some of these PRs are built on other PRs, so the deltas in the links below may be outdated. Cross reference with this PR when in doubt.
### UI
- Reactify Role Management Screen: https://github.com/elastic/kibana/pull/19035
- Space Aware Privileges UI: https://github.com/elastic/kibana/pull/21049
- Space Selector (in Kibana Nav): https://github.com/elastic/kibana/pull/19497
- Recently viewed Widget: https://github.com/elastic/kibana/pull/22492
- Support Space rename/delete: https://github.com/elastic/kibana/pull/22586
### Saved Objects Client
- ~~Space Aware Saved Objects: https://github.com/elastic/kibana/pull/18862~~
- ~~Add Space ID to document id: https://github.com/elastic/kibana/pull/21372~~
- Saved object namespaces (supercedes #18862 and #21372): https://github.com/elastic/kibana/pull/22357
- Securing saved objects: https://github.com/elastic/kibana/pull/21995
- Dedicated Spaces client (w/ security): https://github.com/elastic/kibana/pull/21995
### Other
- Public Spaces API (experimental): https://github.com/elastic/kibana/pull/22501
- Telemetry: https://github.com/elastic/kibana/pull/20581
- Reporting: https://github.com/elastic/kibana/pull/21457
- Spencer's original Spaces work: https://github.com/elastic/kibana/pull/18664
- Expose `spaceId` to "Add Data" tutorials: https://github.com/elastic/kibana/pull/22760
Closes #18948
"Release Note: Create spaces within Kibana to organize dashboards, visualizations, and other saved objects. Secure access to each space when X-Pack Security is enabled"
2018-10-01 13:09:33 +02:00
|
|
|
spaces(kibana),
|
2018-04-20 21:13:37 +02:00
|
|
|
security(kibana),
|
|
|
|
tilemap(kibana),
|
|
|
|
dashboardMode(kibana),
|
|
|
|
logstash(kibana),
|
Beats Management (#23819)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* [Beats Management] Move to Ingest UI arch and initial TS effort (#20039)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* Ported over base types and arch structure
* move management of installIndexTemplate into the framework adapter
* ts-lint fix
* tslint fixes
* more ts tweaks
* fix paths
* added several working endpoints
* add more routes and bug fixes
* fix linting
* fix type remove CRUFT
* remove more cruft
* remove more CRUFT
* added comments, change plurality
* add tsconfig file
* add extends path
* fixed typo
* serveral PR review fixes
* fixed lodash type version
* “fix” types by applying a lot of any
* [Beats Management] Move tokens to use JWT, add more complete test suite (#20317)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* fix broken test, this is beats CM not logstash 😊
* added readme
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* [Beats Management] add more tests, update types, break out ES into it's own adapter (#20566)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* Add initial efforts for backend framework adapter testing
* move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES
* re-typed
* renamed types to match pattern
* aditional renames
* adapter tests should always just use adapterSetup();
* database now uses InternalRequest
* corrected spelling of framework
* fix typings
* remove CRUFT
* RequestOrInternal
* Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible
* fix tests, add test, removed extra comment
* fix auth
* updated lock file
* [Beats Management] add get beat endpoint (#20603)
* [Beats Management] Move tokens to use JWT, add more complete test suite (#20317)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* fix broken test, this is beats CM not logstash 😊
* added readme
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* Add initial efforts for backend framework adapter testing
* move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES
* re-typed
* renamed types to match pattern
* aditional renames
* adapter tests should always just use adapterSetup();
* database now uses InternalRequest
* corrected spelling of framework
* fix typings
* remove CRUFT
* RequestOrInternal
* Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible
* fix tests, add test, removed extra comment
* Moved critical path code from route, to more easeley tested domain
* fix auth
* remove beat verification, added get beat endpoint to return configs
* fix type
* update createGetBeatConfigurationRoute URL
* rename method
* update to match PR #20566
* updated lock file
* fix bad merge
* update TSLinting
* fix bad rebase
* [Beats Management] [WIP] Create public resources for management plugin (#20864)
* Init plugin public resources.
* rename beats to beats_management
* rendering react now
* Beats/initial ui (#20994)
* initial layout and main nav
* modal UI and pattern for UI established
* fix path
* wire up in-memroy adapters
* tweak adapters
* add getAll method to tags adapter (#21287)
* Beats/real adapters (#21481)
* add initial real adapters, and nulled data where we need endpoints
* UI adapters and needed endpoints added (though not tested)
* prep for route tests and some cleanup
* move files
* [Beats Management] Add BeatsTable/Bulk Action Search Component (#21182)
* Add BeatsTable and control bar components.
* Clean yarn.lock.
* Move raw numbers/strings to constants. Remove obsolete state/props.
* Update/add tests.
* Change prop name from "items" to "beats".
* Rename some variables.
* Move search bar filter definitions to table render.
* Update table to support assignment options.
* Update action control position.
* Refactor split render function into custom components.
* Beats/basic use cases (#21660)
* tweak adapter responses / types. re-add enroll ui
* routes enabled, enroll now pings the server
* full enrollment path now working
* improved pinging for beat enrollment
* fix location of history call
* reload beats list on beat enrollment completion
* [Beats Management] Add Tags List (#21274)
* Add BeatsTable and control bar components.
* Clean yarn.lock.
* Move raw numbers/strings to constants. Remove obsolete state/props.
* Update/add tests.
* Change prop name from "items" to "beats".
* Add TagsTable component and associated search/action bar.
* Rename some variables.
* Add constant after forgetting to save file.
* Fix design mistake in table component.
* Disable delete button when no tags selected.
* Export tags table from index.ts.
* Move search bar filter definitions to table render.
* Update table to support assignment options.
* Update action control position.
* Refactor split render function into custom components.
* Add assignment options to Tags List.
* Remove obsolete code.
* Move tooltips for tag icons to top position.
* Beats/update (#21702)
* [ML] Fixing issue with historical job audit messages (#21718)
* Add proper aria-label for close inspector (#21719)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* [Beats Management] Move to Ingest UI arch and initial TS effort (#20039)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* Ported over base types and arch structure
* move management of installIndexTemplate into the framework adapter
* ts-lint fix
* tslint fixes
* more ts tweaks
* fix paths
* added several working endpoints
* add more routes and bug fixes
* fix linting
* fix type remove CRUFT
* remove more cruft
* remove more CRUFT
* added comments, change plurality
* add tsconfig file
* add extends path
* fixed typo
* serveral PR review fixes
* fixed lodash type version
* “fix” types by applying a lot of any
* [Beats Management] Move tokens to use JWT, add more complete test suite (#20317)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* fix broken test, this is beats CM not logstash 😊
* added readme
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* [Beats Management] add more tests, update types, break out ES into it's own adapter (#20566)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* Add initial efforts for backend framework adapter testing
* move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES
* re-typed
* renamed types to match pattern
* aditional renames
* adapter tests should always just use adapterSetup();
* database now uses InternalRequest
* corrected spelling of framework
* fix typings
* remove CRUFT
* RequestOrInternal
* Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible
* fix tests, add test, removed extra comment
* fix auth
* updated lock file
* [Beats Management] add get beat endpoint (#20603)
* [Beats Management] Move tokens to use JWT, add more complete test suite (#20317)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* fix broken test, this is beats CM not logstash 😊
* added readme
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* Add initial efforts for backend framework adapter testing
* move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES
* re-typed
* renamed types to match pattern
* aditional renames
* adapter tests should always just use adapterSetup();
* database now uses InternalRequest
* corrected spelling of framework
* fix typings
* remove CRUFT
* RequestOrInternal
* Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible
* fix tests, add test, removed extra comment
* Moved critical path code from route, to more easeley tested domain
* fix auth
* remove beat verification, added get beat endpoint to return configs
* fix type
* update createGetBeatConfigurationRoute URL
* rename method
* update to match PR #20566
* updated lock file
* fix bad merge
* update TSLinting
* fix bad rebase
* [Beats Management] [WIP] Create public resources for management plugin (#20864)
* Init plugin public resources.
* rename beats to beats_management
* rendering react now
* Beats/initial ui (#20994)
* initial layout and main nav
* modal UI and pattern for UI established
* fix path
* wire up in-memroy adapters
* tweak adapters
* add getAll method to tags adapter (#21287)
* Beats/real adapters (#21481)
* add initial real adapters, and nulled data where we need endpoints
* UI adapters and needed endpoints added (though not tested)
* prep for route tests and some cleanup
* move files
* [Beats Management] Add BeatsTable/Bulk Action Search Component (#21182)
* Add BeatsTable and control bar components.
* Clean yarn.lock.
* Move raw numbers/strings to constants. Remove obsolete state/props.
* Update/add tests.
* Change prop name from "items" to "beats".
* Rename some variables.
* Move search bar filter definitions to table render.
* Update table to support assignment options.
* Update action control position.
* Refactor split render function into custom components.
* Beats/basic use cases (#21660)
* tweak adapter responses / types. re-add enroll ui
* routes enabled, enroll now pings the server
* full enrollment path now working
* improved pinging for beat enrollment
* fix location of history call
* reload beats list on beat enrollment completion
* add update on client side, expand update on server to allow for partial data, and user auth
* remove double beat lookup
* fix tests
* only return active beats
* disenroll now working
* fig getAll query
* re-enrolling a beat will now work
* fix types
* fix types
* update deps
* update kibana API for version
* [Beats CM] Manage Tags (#21776)
* [ML] Fixing issue with historical job audit messages (#21718)
* Add proper aria-label for close inspector (#21719)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* [Beats Management] Move to Ingest UI arch and initial TS effort (#20039)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* Ported over base types and arch structure
* move management of installIndexTemplate into the framework adapter
* ts-lint fix
* tslint fixes
* more ts tweaks
* fix paths
* added several working endpoints
* add more routes and bug fixes
* fix linting
* fix type remove CRUFT
* remove more cruft
* remove more CRUFT
* added comments, change plurality
* add tsconfig file
* add extends path
* fixed typo
* serveral PR review fixes
* fixed lodash type version
* “fix” types by applying a lot of any
* [Beats Management] Move tokens to use JWT, add more complete test suite (#20317)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* fix broken test, this is beats CM not logstash 😊
* added readme
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* [Beats Management] add more tests, update types, break out ES into it's own adapter (#20566)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* Add initial efforts for backend framework adapter testing
* move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES
* re-typed
* renamed types to match pattern
* aditional renames
* adapter tests should always just use adapterSetup();
* database now uses InternalRequest
* corrected spelling of framework
* fix typings
* remove CRUFT
* RequestOrInternal
* Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible
* fix tests, add test, removed extra comment
* fix auth
* updated lock file
* [Beats Management] add get beat endpoint (#20603)
* [Beats Management] Move tokens to use JWT, add more complete test suite (#20317)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* fix broken test, this is beats CM not logstash 😊
* added readme
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* Add initial efforts for backend framework adapter testing
* move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES
* re-typed
* renamed types to match pattern
* aditional renames
* adapter tests should always just use adapterSetup();
* database now uses InternalRequest
* corrected spelling of framework
* fix typings
* remove CRUFT
* RequestOrInternal
* Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible
* fix tests, add test, removed extra comment
* Moved critical path code from route, to more easeley tested domain
* fix auth
* remove beat verification, added get beat endpoint to return configs
* fix type
* update createGetBeatConfigurationRoute URL
* rename method
* update to match PR #20566
* updated lock file
* fix bad merge
* update TSLinting
* fix bad rebase
* [Beats Management] [WIP] Create public resources for management plugin (#20864)
* Init plugin public resources.
* rename beats to beats_management
* rendering react now
* Beats/initial ui (#20994)
* initial layout and main nav
* modal UI and pattern for UI established
* fix path
* wire up in-memroy adapters
* tweak adapters
* add getAll method to tags adapter (#21287)
* Beats/real adapters (#21481)
* add initial real adapters, and nulled data where we need endpoints
* UI adapters and needed endpoints added (though not tested)
* prep for route tests and some cleanup
* move files
* [Beats Management] Add BeatsTable/Bulk Action Search Component (#21182)
* Add BeatsTable and control bar components.
* Clean yarn.lock.
* Move raw numbers/strings to constants. Remove obsolete state/props.
* Update/add tests.
* Change prop name from "items" to "beats".
* Rename some variables.
* Move search bar filter definitions to table render.
* Update table to support assignment options.
* Update action control position.
* Refactor split render function into custom components.
* Beats/basic use cases (#21660)
* tweak adapter responses / types. re-add enroll ui
* routes enabled, enroll now pings the server
* full enrollment path now working
* improved pinging for beat enrollment
* fix location of history call
* reload beats list on beat enrollment completion
* add update on client side, expand update on server to allow for partial data, and user auth
* remove double beat lookup
* fix tests
* only return active beats
* disenroll now working
* fig getAll query
* re-enrolling a beat will now work
* fix types
* Add create tags view.
* fix types
* update deps
* update kibana API for version
* Added component/config interface for editing/creating tags. Added separate pages for create/edit tags.
* Fixup.
* Beats/beat tags workflow (#21923)
* [Beats Management] Move to Ingest UI arch and initial TS effort (#20039)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* Ported over base types and arch structure
* move management of installIndexTemplate into the framework adapter
* ts-lint fix
* tslint fixes
* more ts tweaks
* fix paths
* added several working endpoints
* add more routes and bug fixes
* fix linting
* fix type remove CRUFT
* remove more cruft
* remove more CRUFT
* added comments, change plurality
* add tsconfig file
* add extends path
* fixed typo
* serveral PR review fixes
* fixed lodash type version
* “fix” types by applying a lot of any
* add details page, re-configure routes
* move tag crud to new route stuff
* update tag create/edit component api
* tags creation now working
* bunch of stuff I should have split up better…
* fixed perf bug, selected items that are removed are no longer phantom selected
* fix rendering of assignments
* remove assign to beats, the UX was too poor
* Beats/config view (#22177)
* [Beats Management] Move to Ingest UI arch and initial TS effort (#20039)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* Ported over base types and arch structure
* move management of installIndexTemplate into the framework adapter
* ts-lint fix
* tslint fixes
* more ts tweaks
* fix paths
* added several working endpoints
* add more routes and bug fixes
* fix linting
* fix type remove CRUFT
* remove more cruft
* remove more CRUFT
* added comments, change plurality
* add tsconfig file
* add extends path
* fixed typo
* serveral PR review fixes
* fixed lodash type version
* “fix” types by applying a lot of any
* add details page, re-configure routes
* move tag crud to new route stuff
* update tag create/edit component api
* tags creation now working
* bunch of stuff I should have split up better…
* fixed perf bug, selected items that are removed are no longer phantom selected
* fix rendering of assignments
* remove assign to beats, the UX was too poor
* [Beats Management] Move to Ingest UI arch and initial TS effort (#20039)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* Ported over base types and arch structure
* move management of installIndexTemplate into the framework adapter
* ts-lint fix
* tslint fixes
* more ts tweaks
* fix paths
* added several working endpoints
* add more routes and bug fixes
* fix linting
* fix type remove CRUFT
* remove more cruft
* remove more CRUFT
* added comments, change plurality
* add tsconfig file
* add extends path
* fixed typo
* serveral PR review fixes
* fixed lodash type version
* “fix” types by applying a lot of any
* Beats/update (#21702)
* [ML] Fixing issue with historical job audit messages (#21718)
* Add proper aria-label for close inspector (#21719)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* [Beats Management] Move to Ingest UI arch and initial TS effort (#20039)
* [Beats Management] Initial scaffolding for plugin (#18977)
* Initial scaffolding for Beats plugin
* Removing bits not (yet) necessary in initial scaffolding
* [Beats Management] Install Beats index template on plugin init (#19072)
* Install Beats index template on plugin init
* Adding missing files
* [Beats Management] APIs: Create enrollment tokens (#19018)
* WIP checkin
* Register API routes
* Fixing typo in index name
* Adding TODOs
* Removing commented out license checking code that isn't yet implemented
* Remove unnecessary async/await
* Don't return until indices have been refreshed
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Adding TODO
* Fixing variable name
* Using a single index
* Adding expiration date field
* Adding test for expiration date field
* Ignore non-existent index
* Fixing logic in test
* Creating constant for default enrollment tokens TTL value
* Updating test
* Fixing name of test file (#19100)
* [Beats Management] APIs: Enroll beat (#19056)
* WIP checkin
* Add API integration test
* Converting to Jest test
* Create API for enrolling a beat
* Handle invalid or expired enrollment tokens
* Use create instead of index to prevent same beat from being enrolled twice
* Adding unit test for duplicate beat enrollment
* Do not persist enrollment token with beat once token has been checked and used
* Fix datatype of host_ip field
* Make Kibana API guess host IP instead of requiring it in payload
* Fixing error introduced in rebase conflict resolution
* [Beats Management] APIs: List beats (#19086)
* WIP checkin
* Add API integration test
* Converting to Jest test
* WIP checkin
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Updating mapping
* [Beats Management] APIs: Verify beats (#19103)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Fleshing out remaining tests
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Moving TODO comment to right file
* Rename determine* helper functions to find*
* Fixing assertions (#19194)
* [Beats Management] APIs: Update beat (#19148)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Add API tests
* Update template to allow version field for beat
* Implement PUT /api/beats/agent/{beat ID} API
* Make enroll beat code consistent with update beat code
* Fixing minor typo in TODO comment
* Allow version in request payload
* Make sure beat is not updated in ES in error scenarios
* Adding version as required field in Enroll Beat API payload
* Using destructuring
* Fixing rename that was accidentally reversed in conflict fixing
* [Beats Management] APIs: take auth tokens via headers (#19210)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Make "Enroll Beat" API take enrollment token via header instead of request body
* Make "Update Beat" API take access token via header instead of request body
* [Beats Management] APIs: Create configuration block (#19270)
* WIP checkin
* WIP checkin
* Add API integration test
* Converting to Jest test
* Fixing API for default case + adding test for it
* Fixing copy pasta typos
* Fixing variable name
* Using a single index
* Implementing GET /api/beats/agents API
* Creating POST /api/beats/agents/verify API
* Refactoring: extracting out helper functions
* Expanding TODO note so I won't forget :)
* Fixing file name
* Updating mapping
* Fixing minor typo in TODO comment
* Implementing POST /api/beats/configuration_blocks API
* Removing unnecessary escaping
* Fleshing out types + adding validation for them
* Making output singular (was outputs)
* Removing metricbeat.inputs
* Revert implementation of `POST /api/beats/configuration_blocks` API (#19340)
This API allowed the user to operate at a level of abstraction that is unnecessarily and dangerously too low. A better API would be at one level higher, where users can create, update, and delete tags (where a tag can contain multiple configuration blocks).
* [Beats Management] APIs: Create or update tag (#19342)
* Updating mappings
* Implementing PUT /api/beats/tag/{tag} API
* [Beats Management] Prevent timing attacks when checking auth tokens (#19363)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Prevent subtler timing attack in token comparison function
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* [Beats Management] APIs: Assign tag(s) to beat(s) (#19431)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Rename "determine" to "find"
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Updating ES archive
* Renaming
* Use destructuring
* Moving start of script to own line to increase readability
* Using destructuring
* [Beats Management] APIs: Remove tag(s) from beat(s) (#19440)
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Remove random delay
* Starting to implement POST /api/beats/beats_tags API
* Changing API
* Updating tests for changes to API
* Renaming
* Use destructuring
* Using crypto.timingSafeEqual() for comparing auth tokens
* Introduce random delay after we try to find token in ES to mitigate timing attack
* Implementing `POST /api/beats/agents_tags/removals` API
* Updating ES archive
* Use destructuring
* Moving start of script to own line to increase readability
* Nothing to remove if there are no existing tags!
* Updating tests to match changes in bulk update painless script
* Use destructuring
* Ported over base types and arch structure
* move management of installIndexTemplate into the framework adapter
* ts-lint fix
* tslint fixes
* more ts tweaks
* fix paths
* added several working endpoints
* add more routes and bug fixes
* fix linting
* fix type remove CRUFT
* remove more cruft
* remove more CRUFT
* added comments, change plurality
* add tsconfig file
* add extends path
* fixed typo
* serveral PR review fixes
* fixed lodash type version
* “fix” types by applying a lot of any
* [Beats Management] Move tokens to use JWT, add more complete test suite (#20317)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* fix broken test, this is beats CM not logstash 😊
* added readme
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* [Beats Management] add more tests, update types, break out ES into it's own adapter (#20566)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* Add initial efforts for backend framework adapter testing
* move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES
* re-typed
* renamed types to match pattern
* aditional renames
* adapter tests should always just use adapterSetup();
* database now uses InternalRequest
* corrected spelling of framework
* fix typings
* remove CRUFT
* RequestOrInternal
* Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible
* fix tests, add test, removed extra comment
* fix auth
* updated lock file
* [Beats Management] add get beat endpoint (#20603)
* [Beats Management] Move tokens to use JWT, add more complete test suite (#20317)
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* fix broken test, this is beats CM not logstash 😊
* added readme
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* inital effort to move to JWT and added jest based tests on libs
* assign beats tests all passing
* token tests now pass
* add more tests
* all tests now green
* move enrollment token back to a hash
* remove un-needed comment
* alias lodash get to avoid confusion
* isolated hash creation
* Add initial efforts for backend framework adapter testing
* move ES code to a DatabaseAdapter from BackendAdapter and add a TON of types for ES
* re-typed
* renamed types to match pattern
* aditional renames
* adapter tests should always just use adapterSetup();
* database now uses InternalRequest
* corrected spelling of framework
* fix typings
* remove CRUFT
* RequestOrInternal
* Dont pass around request objects everywhere, just pass the user. Also, removed hapi types as they were not compatible
* fix tests, add test, removed extra comment
* Moved critical path code from route, to more easeley tested domain
* fix auth
* remove beat verification, added get beat endpoint to return configs
* fix type
* update createGetBeatConfigurationRoute URL
* rename method
* update to match PR #20566
* updated lock file
* fix bad merge
* update TSLinting
* fix bad rebase
* [Beats Management] [WIP] Create public resources for management plugin (#20864)
* Init plugin public resources.
* rename beats to beats_management
* rendering react now
* Beats/initial ui (#20994)
* initial layout and main nav
* modal UI and pattern for UI established
* fix path
* wire up in-memroy adapters
* tweak adapters
* add getAll method to tags adapter (#21287)
* Beats/real adapters (#21481)
* add initial real adapters, and nulled data where we need endpoints
* UI adapters and needed endpoints added (though not tested)
* prep for route tests and some cleanup
* move files
* [Beats Management] Add BeatsTable/Bulk Action Search Component (#21182)
* Add BeatsTable and control bar components.
* Clean yarn.lock.
* Move raw numbers/strings to constants. Remove obsolete state/props.
* Update/add tests.
* Change prop name from "items" to "beats".
* Rename some variables.
* Move search bar filter definitions to table render.
* Update table to support assignment options.
* Update action control position.
* Refactor split render function into custom components.
* Beats/basic use cases (#21660)
* tweak adapter responses / types. re-add enroll ui
* routes enabled, enroll now pings the server
* full enrollment path now working
* improved pinging for beat enrollment
* fix location of history call
* reload beats list on beat enrollment completion
* add update on client side, expand update on server to allow for partial data, and user auth
* remove double beat lookup
* fix tests
* only return active beats
* disenroll now working
* fig getAll query
* re-enrolling a beat will now work
* fix types
* fix types
* update deps
* update kibana API for version
* progress on config forms
* config view inital input types working
* ts fixes
* fix more ts
* code now errors on invalid yaml
* remove un-needed include
* fix bad rebase
* saving config blocks as yaml to db is now working
* propperly formatted YAML
* loading tags back on edit screen in-progress
* fix types
* vis name validation for tag
* update EUI style
* tweak design
* fixed tag assignments (still has a ui glitch)
* fix form validation on select
* fix deps
* update deps
* attached beats now works in the edit tag screen, edit now disables changing the tag id
* better un-parsing of yaml, some elements now rendering to edit config blocks
* delete config block now works
* fix ability to edit config
* fix deps
* fix another rebase issue
* tweaks and fixes
* fix several bugs
* Beats/security (#22500)
* client side security working
* check license on route
* forgot a file
* [Beats CM] Add beats details pages (#22455)
* Move edits from previous details branch.
* Add tag view to beat details.
* Added notifications for add/remove tag from beat.
* Fix dependencies upgrade/downgrade add/removes.
* Create new page files for each sub-section of details view.
* Move page functionality from dedicated components to pages.
* [Beats CM] move to json for configs (#22693)
* move to json for configs, fix some tests
* tweaks
* add fixes (#22711)
* add fixes
* return only one config
* much improved validation for hosts, no longer require not required fields, no longer have JS in the config form config
* fix condition argument
* [Beats CM] Improve UX for assignment of tag to list of beats (#22687)
* Improve UX for assignment of tag to list of beats.
* Revert a change.
* [Beats CM] Sort beat list tags (#22729)
* Add logic to sort tags by ID when rendering in Beats List.
* Prefer lodash sortBy over inline sort implementation.
* [Beats CM] Remove key warnings from beat list components (#22772)
* Add logic to sort tags by ID when rendering in Beats List.
* Remove key warnings from beats list and associated components.
* Prefer lodash sortBy over inline sort implementation.
* [Beats CM] Add check for BeatsPage component to avoid setState when unmounted (#22836)
* Add check if component is unmounted when loading Beats List page.
* Move call to loadBeats() to componentDidMount lifecycle function.
* Update field name in table type config. (#23228)
* Utilize TagBadge functional component to standardize display of tag names. (#23253)
* [Beats CM] Add beat name to update endpoint (#23291)
* Update beat index template and update endpoints to expect name type. Add test file for update.
* Update enroll script and endpoint to create a 'name' field for beats.
* Add name field to CMBeat domain type.
* Update functional tests to include name field.
* Fix broken tag assignment functional test.
* Edit beats list table config to display most recently-updated tag time. (#23337)
* Update beat detail view info. (#23369)
* Update beat detail view info.
* Add period to end of update field.
* [Beat CM] Show only tags associated with selected beat (#23398)
* Reenable output config.
* Make beat detail tag page only show tags associated with that tag.
* Display beat name instead of ID on details screen. (#23410)
* [Beat CM] Display config name instead of beat type in beat detail view (#23411)
* Update config table in beat detail view to show config type instead of beat type.
* Modify Beat Detail view to display human-friendly names for config block types.
* [Beats CM] Add password input and re-enable output config schema (#23417)
* wip defining controls
* Complete adding formsy password field.
* Re-enable output config schema definition.
* Simplify import/export for formsy components.
* Add full list of current Filebeat & Metricbeat module (#23258)
This change also renames `Metricbeat input` to `Metricbeat module`
* Beats/ui tweaks (#23655)
* add initial breadcrumbs
* prevent errors
* New routing in place for supporting URLState needed for Kuery bar
* beats table kuery bar “working” (but not submitting)
* pulling activity view from current phase’s spec
* setup link now at correct URL
* kuery bar
* autocomplete bar now working
* Add walkthrough (#23785)
* add test system, framework, and example tests
* adding walkthrough
* clicking walkthrough step should take you to that step
* try/catch beats loading to prevent unhandeled error when there are no beats
* fix walkthrough not redirecting to main beats page
* update yarn lock files
* fix prettier
* fix x-pack package.json formatting
* update kbn server creation
* remove types from old table implementation
* move to Boom.boomify
* fix TS errors
* fix type
* rollback dep version
* fix more conflicting dep issues
* deps
* force correct node type resolution
* cleanup and skip tests (kbn test util is broken)
* added formsy and basePath
* update yarn lock file
* add beats management icon
* rename beats management main section
* Tags now called Configuration Tags in the tabs
* tokens must expire at most after 2 weeks
* fix bad auto-import
* beat details now shows the extra data needed
* tweak package.json deps for continuity
* update yarn lock for new yarn version
* [Beats CM] Re-arch table assignment control definitions (#23663)
* Patch changes to latest feature branch.
* Update table controls to offer standard search bar if kuery bar props not provided.
* Tweaked enrollment UI to inlcude beat type selection
* Reduce badge list gutter size, change logic for tag assignment to unassign all if some selected. (#24091)
* dont use genaric search bar
* removed unused import
* fix es-lint error
* fix outputs
* fix types from infra merge
* remove dupe dep
* update lock file
* push another lock file
* Add unfilteredBeats field to beat list state to prevent inappropriate redirect. (#24146)
* fix yarn lock because yarn is dumb
* Fix broken tests
2018-10-18 04:06:43 +02:00
|
|
|
beats(kibana),
|
2018-04-20 21:13:37 +02:00
|
|
|
apm(kibana),
|
2019-02-01 18:07:05 +01:00
|
|
|
maps(kibana),
|
2018-09-06 01:29:58 +02:00
|
|
|
canvas(kibana),
|
2018-04-20 21:13:37 +02:00
|
|
|
indexManagement(kibana),
|
2018-10-17 12:07:50 +02:00
|
|
|
infra(kibana),
|
Core task manager (#24356)
* Core task manager (#23555)
* Initial stab at core task manager logic
* Update task_manager readme
* Update task manager readme
* Add cancelable helper package, OSS, and general purpose, but
will be useful for writing cancelable x-pack tasks.
* Make the cancellable package promise-compliant
* Update task manager readme w/ reference to cancellable
* Change pool from lazy to eager, add support for sub-pools per task type.
* Move cancellable to task_manager, and typescriptify it.
* Working proof of concept for task manager. Still have lots to do:
clean up, tests, comments, validations, assertions, etc.
* Add pagination support to task manager fetch
* Move task manager to OSS
* Remove task manager reference from x-pack
* Make task_manager a valid core plugin
* Modify how task resource allocation works
* Remove the special case 'max' workers occupied value
* Remove x-pack package.json changes
* Make taskDefinitions a part of uiExports
* Make task docs saved-object compliant.
* Add kbnServer to the task context.
* Allow tasks to have a void / undefined return type
* revert x-pack change
* move cancellable to src/utils
* move to src/server
* use afterPluginsInit hook
* task_manager.ts rename
* add a wrapper with a setClient method
* Add tests for task runner
* Break task_pool into smaller, testable pieces
* return raw task doc fields for calling code
* remove todo comment
* helper module for default client
- setClient takes a callback fn
* fix misidentified field name
* fix rest args warning
* flatten task_pool
* remove cancellable
* return raw task doc fields for calling code
* remove todo comment
* helper module for default client
- setClient takes a callback fn
* fix rest args warning
* typescript fixes
* roll back setClient takes a callback
* createTaskRunner returns an object with run/cancel functions
* Test task store, tweak task poller tests
* Rename num_workers to max_workers
* Tweak task pool method names
* Fix cancellable test, and ts errors
* Rename doc to _doc
* Fix the doc type in the task manager index mappings
* Make task costs configurable in kibana.yml
* fix a tslint warning
* TaskManager.afterPluginsInit replaces circuitous stuff
* addMiddleware, implement beforeSave
* wip
* comment
* run context stuff
* pretty fix
* comments
* lint/pretty fixes
* safeties in case they don't define both beforeRun and beforeSave
* task runner test constructor opts fix
* Add task_manager integration tests
* FIx readme mistakes, fix schedule signature
* Fix integration tests
* Add blurb about middleware
* make a libs directory for the small things
* test for extractTaskDefinitions
* unit test for middleware
* Comment, refactor, and test task_manager
* Touch up comments
* Make task store throw error if given an unknown task type
* Fix sample task definition in readme
* Make task manager toggle on / off based on Elasticsearch connectivity
* Prevent task manager from crashing Kibana if
elasticsearch plugin is disabled
* Fix task manager signature
* Move the task manager into the task manager mixin,
fix tests. It's an uber integration object anyway, so
it seemed to make sense to keep it in one place.
* Fix task manager integration tests
* Update the task manager index creation to use auto_expand_replicas
* Fix task manager integration tests
* Fix task manager integration tests
* Fix the overzealous deletion of .kibana during tests
* Core task manager
* Allow hardcoded ID passed in
* comments
* don't deconstruct type and task for payload
* remove uiExport support
* move task_manager to x-pack/plugins
* expose as client on the plugin namespace
* fix tests
* move plugin api integration tests
* roll back readme sample task definition example
* fix sample plugin link
* server.taskManager
* sanitizeTaskDefinitions
* fix integration tests
* sanitize rename
* assert unintialized and check for duplicates in registerTaskDefinitions
* Remove unnecessary defaults from task_manager.test.ts
* Remove task manager from OSS
* Remove task manager from src
* Tidy up task_manager unit tests around elasticsearch status changes
* Integration test for specifying ID in scheduling
* Task_manager init -> preInit to allow other plugins to consume
taskManager in their init.
* Remove task manager integration tests from OSS
* spelling
* readme fix
* fix test code impacted by hapi upgrade
* Task Manager index creation changes (#24542)
* Uses putTemplate for task manager mappings and index settings.
* Removes create option.
* Opposite day, the test catches up to the code changes.
* Ignores index unavailable during searches.
* Improve taskRunner's processResult (#24880)
* Set task state to idle after run, and add failed state
* fix tests
* Test alerting demo (#25136)
* Ensures that task store init gets called before schedule.
* Removes unused option for debugging purposes.
* Fix unit tests because a second callCluster was made.
* Task manager starts sanely.
Does not preInit
Will not allow definitions after initialization
Creates store immediately.
Modifies store after all plugins have initialized
Adds static tasks that will be defined by plugins.
* Task manager should not allow operations before initialization.
* Attempts to fix runner tests.
* Fixes unit test contract with APIs.
* Removes unused type definitions.
* Removes unused package json.
* Removes unused import type.
* Removes unnecessary async applied to a function.
* Returns diferently if task store has already initialized.
* Explains how to add static task instances to task definitions. (#25880)
* Tasks cannot be scheduled until task manager is initialized.
* Adds task manager api integration tests to ci group4.
* Context of describe test must be the test framework object.
* Update src/es_archiver/lib/indices/kibana_index.js
regex update to actually disallow non-kibana system indices
Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>
* verify fillPool exception passing
* readme update about max_workers/numWorkers
* change forEach to reduce
* use public interfaces in internal method
* replace getMaxAttempts with public readonly maxAttempts
* Update x-pack/plugins/task_manager/task_store.ts
`throw new Error` and initializattion spell fix
Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>
* min = 1 for max_workers
Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>
* timeOut => timeout
* min 1
* scope as an array
* no retries
Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>
* ConcreteTaskInstance is a TaskInstance
* remove guard per joi logic
* more return types for functions
* more comments around incremental back-off
Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>
* throw error instead of return undefined
* poll_interval min 1000
* avoid handling err.stack directly
* break up processResult
* fix a few runtime issues
* only fetch idle tasks
* remove check for status idle
* always return a state, and when there is an error return the state we had at beforeRun
* check isStarted before attemptWork
Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>
* ts fix
2019-01-04 00:32:09 +01:00
|
|
|
taskManager(kibana),
|
2018-10-25 02:41:45 +02:00
|
|
|
rollup(kibana),
|
2019-04-02 20:47:15 +02:00
|
|
|
siem(kibana),
|
2018-12-19 02:59:10 +01:00
|
|
|
remoteClusters(kibana),
|
|
|
|
crossClusterReplication(kibana),
|
2018-12-19 00:08:06 +01:00
|
|
|
upgradeAssistant(kibana),
|
2019-01-04 20:29:15 +01:00
|
|
|
uptime(kibana),
|
2019-05-03 17:44:02 +02:00
|
|
|
encryptedSavedObjects(kibana),
|
2019-09-17 20:57:53 +02:00
|
|
|
lens(kibana),
|
Introduce basic alerting and actions plugin (#37042)
* Create actions plugin (#35679)
* Basic alerting plugin with actions
* Remove relative imports
* Code cleanup
* Split service into 3 parts, change connector structure
* Ability to disable plugin, ability to get actions
* Add slack connector
* Add email connector
* Ability to validate params and connector options
* Remove connectorOptionsSecrets for now
* Fix plugin config validation
* Add tests for slack connector
* Default connectors register on plugin init, console renamed to log, slack to message_slack
* Add remaining API endpoints for action CRUD
* Add list connectors API
* Change actions CRUD APIs to be closer with saved objects structure
* WIP
* Fix broken tests
* Add encrypted attribute support
* Add params and connectorOptions for email
* WIP
* Remove action's ability to have custom ids
* Remove ts-ignore
* Fix broken test
* Remove default connectors from this branch
* Fix API integration tests to use fixture connector
* Rename connector terminology to action type
* Rename actionTypeOptions to actionTypeConfig
* Code cleanup
* Fix broken tests
* Rename alerting plugin to actions
* Some code cleanup and add API unit tests
* Change signature of action type service execute function
* Add some plugin api integration tests
* Fix type check failure
* Code cleanup
* Create an actions client instead of an action service
* Apply Bill's PR feedback
* Fix broken test
* Find function to have destructured params
* Add tests to ensure encrypted attributes are not returned
* Fix broken test
* Add tests for validation
* Ensure actions can be updated without re-passing the config
* Remove dead code
* Test cleanup
* Fix eslint issue
* Apply Peter's PR feedback
* Code cleanup and fix broken tests
* Apply Brandon's PR feedback
* Add namespace support
* Fix broken test
* Pass services to action executors (#37194)
* Pass services to action executors
* Fix tests
* Apply PR feedback
* Apply PR feedback pt2
* Cleanup actions plugin (#37250)
* Cleanup actions, move code from alerting plugin PR
* Rename service terminology to registry
* Use static encryption key for encrypted attributes plugin inside of tests
* Empty data after create test is done running
* Fix type checks
* Fix inconsistent naming
* add server log action for alerting (#37530)
adds the first "builtin" alertType for performing a `server.log()`
* Create alerting plugin (#37043)
* WIP
* Rename fire function and remove @ts-ignore in all places
* Change naming in alerting service
* Remove alert instance class for now, support interval configuration
* Cleanup TS
* Split alerting between registry and client
* Use saved object alongside task manager instance
* Add remaining alerting APIs
* Change create structure
* Rename some variables, change actionGroups structure
* Use handlebars for templating strings at fire time
* Fix params given to alert type execute function
* Use alert instance class
* Alert instances support meta attributes
* Move alert instances deserialization
* Change interval to be ms
* Rename actions es archive
* Fix tests to use encrypted esArchive for action record
* Add create alert test to demo end to end flow
* Fix type check issue
* Alerts to use references to action objects
* Only update task manager tasks after saved objects are fully updated
* Use scope in task manager
* Fix type check
* Use task manager to execute actions
* Convert ids into references and back
* Apply PR feedback
* Fix broken test
* Fix some bugs
* Fix test errors
* Alert interval to be previous runAt + interval instead of now + interval
* Add range support
* Remove extra line
* Cleanup
* Add alert_instance.test.ts
* Add alert_type_registry.test.ts
* Move tests around
* Create generic task manager mock
* Add note about saved objects client mock
* Create alert_type_registry.mock.ts
* Add alerts_client.test.ts
* Add create_alert_instance_factory.test.ts
* Add create_fire_handler.test.ts
* WIP
* Fix get_create_task_runner_function.test.ts and make test pass
* Make get_create_task_runner_function.test.ts 100% coverage
* Add unit tests for routes
* Move files around
* Created transform_action_params.ts
* Add get_next_run_at.ts
* Add comment explaining why we copy nextRunAt
* Re-use state within alert instance
* Finalize code coverage in unit tests
* Create base api integration tests
* Add a test that ensures end to end functionality of an alert
* Fix ui capabilities test
* Fix broken plugin api integration test
* Fix jest tests with new saved objects client
* Fix broken integration tests
* Change api integration test fixture to make more sense, add functions for future tests
* Move alerts integration testing into own file, prep to add more tests
* Add tests to ensure failed task instances get retried
* Add get_create_task_runner_function.test.ts for actions, create encrypted saved objects mock
* Add action validation tests
* Ensure action type validation occurs on update
* Test 400 on unregistered alert types
* Ensure alertTypeId can't be updated
* Add validation test for alert create / update
* Fix broken checks / tests
* Skip failing test for now
* Cleanup jest tests
* Ensure action objects can be updated while keeping encrypted attributes readable
* Remove partial update sopport, remove ability to change actionTypeId, require config
* Ensure actionTypeConfig is validated on create and update
* Add alertTypeParams validation support
* Fix failing tests
* Ensure alert cleanup errors don't replace the original error
* Pass callCluster as a service to alerts and actions
* Only pass log to alerts client
* Pass savedObjectsClient as a service to alerting and actions
* Fix failing tests
* Remove range support, provide when current and previous task got scheduled
* Ensure Joi validation happens before every execute
* Remove skipped tests, to be done in future PR
* Apply self feedback pt1
* Apply self feedback pt2
* Fix broken tests
* Apply PR feedback
* PR feedback pt1
* Apply security team PR feedback
* PR feedback pt1
* PR feedback pt2
* PR feedback pt3
* Fix broken tests
* Fix callCluster to have signature
* Revert f11a6aeb06c65e5ab54bf3cc36694845c812b6a7
* PR feedback pt4
* Remove __jest__ folders
* PR feedback pt5
* Fix Joi from leaking secrets
* Fire instance actions in parallel instead of series
* Fix failing jest tests
* Accept core api changes
* Fix saved objects client mock
* PR feedback pt1
* Fix eslint issues
* Throw error when alert instance already fired (#39251)
* Throw error when alert instance already fired
* shouldFire doesn't need its own boolean value
* Actions & alerting getting started user guides (#39093)
* Initial user guides
* Cleanup
* Typos, example changes
* Switch to tables, use ordered list for usage
* Start docs around alert instances and templating
* Documentation changes
* Some adjustments
* Apply PR feedback
* Apply suggestions from code review
Co-Authored-By: gchaps <33642766+gchaps@users.noreply.github.com>
* PR feedback pt2
* Provide better examples for alert types
* Apply PR feedback
* Update README locations
2019-06-21 01:37:17 +02:00
|
|
|
actions(kibana),
|
|
|
|
alerting(kibana),
|
Add Ingest Manager plugin to master (#56567)
* [Fleet] Add permissions checks (#48143)
* Add permissions check to API
* Add permissions check to UI
* Undo changes
* Update index.tsx
* Update enroll.ts
* add API tests for permissions
* add fleet to trial license check
* Agent enrollment flyout (#48173)
* Fix agent table empty prompt
* First pass at agent enrollment flyout with placeholders
* Move enrollment commands into separate file and enable templated vars
* Use default double braces for templating
* [Fleet] unenroll agent from the details page (#48286)
* [Fleet] add API to unenroll agents
* [Fleet] return a 403 for inactive agent
* [Fleet] UI to unenroll an agent from the detail page
* [Fleet] Filter inactive agent from the listing (#48466)
* [Fleet] Expose policy change method from fleet plugins (#48562)
* [IM] add screenshot detail view and image endpoint (#48149)
* add api endpoint to handle images
* #47978 add screenshots component with single image
* update padding around screenshot
* import existing API_ROOT
* move ImageRequestParams interface and fix type
* pass the content-type through and add test
* fix ie11 issues with nested flex items, change radius to use eui variable
* use eui variables for padding
* add aria label and image description
* [IM] Use EPM in variables & types (#48453)
* IntegrationsManager -> EPM
* integrationsManager -> epm
* [iI]ntegration -> [pP]ackage. Update tests.
* Don't rename integrations registry URL.
* Update i18n key in x-pack/.i18nrc.json
* Update path to functional test config
* Add epm to recently re-enabled privileges test
* Update two values recently added in main feature branch
* Move/update screenshot tests
* [Fleet] Allow to edit metadata (#48682)
* [Fleet] Add a toggle to show Inactive users (#48917)
* Multiselect to unenroll agents (#48852)
* Initial pass at multiselect to unenroll agents
* Adjust loading state button text
* Fix types; use unenroll provider in agent details
* Update select all agents kuery
* Prevent inactive agents from being selected
* [Fleet] Fix privilege tests after merge of master (#49118)
* [Fleet] Temporary use the elastic user as kibana user to be able to create API keys (#49037)
* [EPM] Fix package version requirement structure (#49172)
* [EPM] Fix package version requirement structure
In https://github.com/elastic/integrations-registry/pull/134 the API structure changes. This PR should adjust to the new structure.
* Update x-pack/legacy/plugins/integrations_manager/common/types.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/integrations_manager/common/types.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/integrations_manager/common/types.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/integrations_manager/common/types.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* [EPM] 48799 markdown into readme (#49180)
* add markdown component and fetch markdown
* add package.json file with react-markdown dependency
* add markdown renderers, use readme path
* remove description and rename to readme
* remove dropshadow on images
* add loading component for readme request
* comment on loading components
* Add command to spin up a mock server for ingest endpoints (#49259)
* 48654 files from registry (#49580)
* Replace image path/handler with generic file one
* Incorporate tests from #48696
* [EPM] Make screenshot image captions optional (#48912)
* Make screenshot image captions optional
* fix typos, type, rename variable
* [EPM] update to layout and and spacing (#49413)
* fix height and color issues
* make container full height
* remove panels per design update and make white bg color
* adjust spacing
* check for empty response
* fix eslint issue
* fix layout in safari
* remove unused component
* [EPM] Rename registry url to new path (#49598)
The integrations-registry repository was renamed to package-registry and with it also all the assets inside. The url under which the service is served will also be changed (not done yet). As soon as this is done, this PR should be merged.
* [EPM] rewrite relative image paths (#49637)
* rewrite relative image paths
* remove EuiImage, use transformImageUri, clean up
* [EPM] Use NP registerFeature (#49625)
* Use NP registerFeature
* Added features plugin to deps. Used it in setup.
Moved `registerFeature` from `init` to plugin setup.
Moved rest of init into `createSetupShim`. `init` is now
```ts
init(server: Legacy.Server) {
const { initializerContext, coreSetup, pluginsSetup } = createSetupShim(server);
new Plugin(initializerContext).setup(coreSetup, pluginsSetup);
},
```
Moved feature (argument for `registerFeature`) to separate file.
Renamed plugin-specific `CoreSetup` to `EPMCoreSetup`
* [Fleet] Filter agents list table by policy name (#49968)
* Add policies lib and adapters
* Fix mock server headers
* Initial pass at policy filter dropdown by manipulating kuery string
* Adjust spec return values
* Use separate state for policy filter; fix typings and disable some eslint rules
* Fix react-related eslint errors
/Users/jfsiii/work/kibana/x-pack/legacy/plugins/integrations_manager/public/hooks/use_links.tsx
26:20 error React Hook "useCore" is called in function "addBasePath" which is neither a React function component or a custom React Hook function react-hooks/rules-of-hooks
/Users/jfsiii/work/kibana/x-pack/legacy/plugins/integrations_manager/public/screens/detail/readme.tsx
26:35 error React Hook "useLinks" cannot be called inside a callback. React Hooks must be called in a React function component or a custom React Hook function react-hooks/rules-of-hooks
39:6 error React Hook useEffect has a missing dependency: 'readmePath'. Either include it or remove the dependency array react-hooks/exhaustive-deps
/Users/jfsiii/work/kibana/x-pack/legacy/plugins/integrations_manager/public/screens/home/search_results.tsx
27:42 error `"` can be escaped with `"`, `“`, `"`, `”` react/no-unescaped-entities
27:49 error `"` can be escaped with `"`, `“`, `"`, `”` react/no-unescaped-entities
✖ 5 problems (5 errors, 0 warnings)
1 error and 0 warnings potentially fixable with the `--fix` option.
* Remove ui/* imports (#50094)
* [Fleet] Use ES api keys for agent authentication (#49639)
* add collapsible read me component (#49990)
* add collapsible read me component
* Update x-pack/legacy/plugins/integrations_manager/public/components/content_collapse.tsx
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* add empty array as second argument to useLayoutEffect
* [EPM] Make API consistent for package installation and removal (#48745)
* Adjust type names
* Install package in one go.
* Integration -> Package
* Really install all known assets
* Remove ingest pipelines on package deletion.
* [EPM] Replace image paths/handlers with generic ones for file (#48688)
* Replace image path/handler with generic file one
* Incorporate tests from #48696
* Fix merge error.
* Type tuning
* Adjust types
* Remove asset types from routes altogether
* Add timelion-sheet back to types.
* Respond with full package info to install/delete
* Be specific in return type.
* Keep installAssets() as separate step
* [EPM] Add unit tests for pathParts (#50279)
* [EPM] Add unit tests for pathParts
This adds unit tests for the pathParts function. I initially wanted to enhance the function to also support paths like `iptables-1.0.4/dataset/log/elasticsearch/ingest-pipeline/pipeline.json`. But without unit tests it was hard for me to make the changes.
This adds a unit test driven by a table so we can extend it later. Adding these tests also helps me to better understand the code.
As a note for myself, the command to run these tests is:
```
node scripts/jest --watch ./legacy/plugins/integrations_manager/server/registry/index.test.ts
```
* Update docs for merging master into feature branch (#50396)
* [Fleet] Remove in memory repository (#50431)
* [EPM] Documentation of HTTP routes & TS types for Ingest (#48798)
* Add beginning models and two routes for Ingest
* Update types & models per discussion w/Ruflin
Also reviewed data structures listed at https://docs.google.com/document/d/1IBR3f9dpHqJmXYEdg06WV34KSMd3g5k4aMGa4jde_Eg/edit#
* Update: /policies always returns array. /policy returns single policy
* Add pagination for /policy & /datasources. Uses per_page & page params
* Add API metadata. Standardize policy_id param name.
* Update descriptions to match Google Doc. Move use case to Policy.
Disabled the '@typescript-eslint/array-type' rule because it was going around in circles. It didn't like Datasource[] or Array<Datasource>
* Return to initial TS annotation for Arrays
Remove the line disabling @typescript-eslint/array-type now that it's behaving normally again :shrug:
* [EPM] Add directory structure for server/lib. (#50469)
* Add directory structure for server/lib.
* 'tests' seems to be more common than 'test'
* Make CI happy
* [EPM] Add basic documentation directory (#50478)
* [EPM] Add basic documentation directory
Having the doc directory around allows us to easily add docs from here on to document how EPM works.
To run the docs build, use the following command from the kibana directory:
```
../docs/build_docs --doc docs/epm/index.asciidoc --open
```
The above assumes that docs (https://github.com/elastic/docs) are checked out in the same directory as Kibana.
With this change, the EPM docs build is not included yet in the overall docs build. For this adjustments to https://github.com/elastic/docs/blob/master/conf.yaml must be made.
* [EPM] Add basic index template (#50471)
This PR adds the very basic index template we will use for the packages. It contains all the basic settings and some examples. The examples will be remove as soon as we have an actual implementation with packages but for now is convenient to see if it is a valid package.
This code is put into the lib directory as it does not tie directly into any handlers.
It also adds an functional tests for loading a template. This means we have a way to check if a template is valid in Elasticsearch. Based on this we can check in the future all our generated templates for validity with Elasticsearch.
To run the functional test, go to the Kibana x-pack directory. Start the first command:
```
node scripts/functional_tests_server.js --config test/epm_api_integration/config.ts
```
Keep the above running and switch to an other Terminal. Now run:
```
node scripts/functional_test_runner.js --config x-pack/test/epm_api_integration/config.ts
```
* 40752 rewrite ingest pipeline (#50627)
* Add directory structure for server/lib.
* 'tests' seems to be more common than 'test'
* Make CI happy
* Implement pipeline rewriting.
* Add more testcases
* For posterity (comment change)
* Allow beats-style template delimiters
* Be more succinct
* Document better
* Replace AssetType enum with union type (#50696)
See https://github.com/elastic/kibana/pull/50609#discussion_r346080439
Discussed in Slack and agree to revert for now. Can track down the issues & restore later
* Remove unnecessary await if we can return the promise (#50329)
* Fix whitespace per figma comments. Closes #47348 (#47350)
* Add fix & comment for TS 3.7.2 regression
* [EPM] cleanup assets, filter assets for those currently supported (#50609)
* cleanup assets, filter assets for those currently supported
* removed unused type
* fix type
* add comment for better type
* change type name to be more descriptive
* hardcode image width for ie11 (#49796)
* hardcode image width for ie11
* eslint
* improve comment
* add maxWidth
* [EPM] useKibana hook & render in plugin.start (#50110)
* plugin.start now does reactdom.render vs returning react element
export plugin function from public/index
* Move setClient call from plugin.start to plugin.setup
* Use `useUiSetting$` from `useKibana` hooks
* Fix broken app due to bad hooks usage
Can't use useKibana outside a React component.
Reverting to prior approach since it's still NP. Can revisit context usage in a followup PR
* [EPM] Install package from detail view on button click (#50735)
* Support basic "click button -> show spinner -> installed" install flow
* Remove incorrect comments. Add TS return types to data functions.
* [EPM] Use NP feature_catalogue.register (#50108)
* Use NP feature_catalogue.register
* Use type from NP plugin
* fix linting
* fix types
* fix headers in Fleet
* skipping test due to ES param change
* Revert "skipping test due to ES param change"
This reverts commit d05f20decfbfc4d91069816a6f8dfde26b5bd6bc.
* remove type field
* remove unused import
* [EPM] Final(?) update from integrations_manager -> EPM (#50976)
* Update (all remaining?) references from integrations_manager to EPM
* Update path in i18n file
* [EPM] update compatibility section (#50975)
* change min max version to single range value
* add elastic stack icon and change text
* remove badge from version and use code font
* remove euistyled
* add back version component lost in merge
* remove euiStyled
* remove old file
* Restore RequirementVersionRange type
* Disable test for elasticsearch username.
Temporary work around until we know how the stack wants to add the permissions we need. Either adding to the kibana user or creating a new user.
* Revert "Disable test for elasticsearch username."
This reverts commit f1020e4eab2ada5d854eacc44cdb6d5bd23c267f.
* Disable test for elasticsearch username.
Temporary work around until we know how the stack wants to add the permissions we need. Either adding to the kibana user or creating a new user.
* Fix EPM typing issues in register feature
* Fix typings after master merge
* [EPM] CI fixes (#51284)
* Initialize es in test.
* Add it(), no es.init()
* Clean up.
* [EPM] add confirmation modal (#51172)
* add confirmation modal, move install state to Header
* update callout to use title
* move components only used in detail view to detail dir
* use better variable names
* update to more descriptive variable names
* Restore prior response vaulues for install & delete package (#51252)
Discussed this with @skh https://github.com/elastic/kibana/pull/51112#commitcomment-35961413 & https://github.com/elastic/kibana/pull/51112#commitcomment-35970664 as well as in a video call
Also added some TS type annotations for data fetching functions to make the contracts more explicit
* [EPM] /package API only lists installable assets. Restore enums. (#51414)
* API only shows installable assets. Server types to own file. Restore enums
* Fix type imports
* Only return installable asset types (kibana for now)
* server/types now only has code from hapi which shouldn't go to client
* Add more restricted TS types to DisplayAssets object
* Flip order of arguments to Extract
In these cases it still works the same, but looking at https://www.typescriptlang.org/docs/handbook/advanced-types.html the signature is
`Extract<T, U>` - Extract from `T` those types that are assignable to `U`
so the larger set should be first
* [Fleet] Enrollment api key UI (#51495)
* Make button pretty in dark mode as well. (#51610)
* [EPM] Add docs entry about registryUrl config (#51697)
This documentation is at the moment mainly for internal use. I found myself searching for this URL several times in the code or PRs so I thought I rather add it to the docs for now.
* [EPM] Remove encoding of Kibana objects as not needed anymore
* [Fleet] Move agent status server side and API to get aggregated status for a policy (#51673)
* [EPM] Add basic docs around install/delete API endpoint (#51728)
This is mainly for internal usage at the moment to look up.
* Ingest/policy (#51741)
* wip policy
* tweaks
* tweaks
* FIX TYPOS
* WIP move policy => agent config conversion to fleet, WIP policy changed method
* fix tests and bugs
* updates tests and snaps
* more fixes
* use AGENT_POLLING_INTERVAL
* cleanup and fix some formatting
* Update x-pack/legacy/plugins/ingest/server/libs/datasources.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/ingest/server/libs/datasources.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/ingest/server/libs/outputs.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* Update x-pack/legacy/plugins/fleet/server/libs/policy.ts
Co-Authored-By: John Schulz <github.com@jfsiii.org>
* fix things broken by PR review suggestions
* remove unused field
* fix types
* fix mappings
* add datasource mappings
* Fix mappings and remove get full policy from checkin
* Fix ingest api integration tests
* run es-lint to fix fleet
* Fix typescript issues
* [EPM] Track package install state and add toast notification (#51734)
* add notifications from core to plugin
* add package install state hook
* fix type error
* use toMountPoint helper to add jsx to notification
* add warning notification to failed install
* make notifications dependency explicit prop
* move PackageInstall provider lower
* add comment about InstallStatus type overlapping InstallationStatus
* use InstallStatus type in InstallationButton component
* fix type
* [Ingest] Adds support for a working default output (#51841)
* aadding config
* add working settings to the default output
* remove default username and password
* update libs
* [EPM] Add basics for creating the ILM setup (#50474)
This contains the basic objects to setup ILM
* Create index and alias with a write index
* Get the policy
The code does not contain any functional tests yet as it is still open on how to do it best. I suggest to get this in as a foundation and then iterate on top of it.
* [EPM] Add datasource (ingest pipeline) from package (#51851)
## Summary
This mixes a few concerns but I think it's worth it to show the parts working together.
Take a look at the individual commits for a better separation of features.
This adds
- the `/datasource/install/{pkgkey}` endpoint which installs ingest pipelines from a package into ES and saves a reference to them in the EPM state Saved Object
- Connects the "Add datasource" button in the successful installation Toast to the new API
- Adds a toast notification to inform the user the datasource was added correctly
- Adds a "Delete Package" button on the details page so we can uninstall a package while we're waiting for the separate view which allows deletes
- b99eda6 Pushes logic that was in the detail view into `InstallationButton`. This consolidates the logic in one component (or one component & the existing hook) and, iiic, means we can put `<InstallationButton package={...} />` on any view and get the same behavior
I'm marking this as a normal PR so people can merge if they wish
![add-datasource-delete-package-small](https://user-images.githubusercontent.com/57655/69775686-7fb39280-1167-11ea-8d41-e2b8a02252a1.gif)
* [EPM] Add basic processing of fields.yml file (#51148)
The fields.yml is used to generate the Elasticsearch template and Kibana index pattern. This PR adds a very basic implementation of processing the fields.yml and then create an Elasticsearch template out of it. The only fields that are supported at the moment are keyword fields, more will be added as a follow up.
The testing was implemented with a golden file. The output from the method is compared to a json file. If the input is changed or the method is changed, it is possible to regenerate the files with the `-generate` flag as following:
```
node scripts/jest ./legacy/plugins/epm/server/lib/template/template.test.ts -generate
```
This will allow us to quickly test many inputs / outputs in the future, make adjustments to the existing files and generate the new outputs. We then can compare it in the diff it the changes make sense.
* [EPM] Create basic implementation to merge input template and dataset manifest (#51803)
* [EPM] Create basic implementation to merge input template and dataset manifest
With this code it is possible to take an input template for the agent and merge it with the config variables from the dataset manifest file. Currently only the name and the default value are merged. Later on we must implement to be able to pass user configured variables to it and make decision based on OS selection.
Closing https://github.com/elastic/kibana/issues/51794
* [EPM] Refactoring of lib structure (#51885)
This refactors the structure of lib. As so far all the lib parts are related to assets in the package, it is organised the same way with the same structure. Each directory has its own tests directory if it needs one. This makes it possible to (almost) not need relative paths for tests.
* [EPM] Allow to read files from fields directory (#51958)
This change allows to also extract files from the `fields` directory. Previously this was not possible because it always assumed a service must be there.
* [EPM] Install Elasticsearch Index Template for data source (#51878)
This installs the Elasticsearch index template for each dataset in a package. For now the names are hardcoded based on package key and dataset name but will be more dynamic later on when we pass the full dataset information.
The dataset extractions is a bit "hacky" at the moment and we should get a full implementation of dataset at a later stage and replace this code.
* [Fleet] Policy list, details, create, edit UIs (#51950)
* Set up simple policies list view
* Adjust spec to return single policy
* Set up simple policy details page
* Add demo stats/chart to policy details
* Add description string
* Initial setup of policy form and create policy UI
* Policy create/edit form; integrate policy list api
* Integrate create policy api
* Integrate policy detail, agent status, and policy edit APIs; adjust policy list api integration in agent enrollment
* Fix edit policy mock meta
* Fix policy list search bar
* PR and linting fixes; use typings from ingest plugin
* Fix i18n
* [EPM] Add datasource saved object type (#51871)
## Summary
This PR makes a few assumptions, and contains a lot of refactoring. It might be beneficial to look at the resulting directory structure under `server` first to get the (new) big picture.
Assumptions:
- our API deals with several concerns, for now these are packages and datasources
- we manage our own HTTP API endpoints for these concerns (in particular, don't use the ingest plugin for that)
- we manage (for now) the Kibana saved object in which datasources are saved. importing and calling methods from the ingest plugin to do that down the road will (hopefully) be a manageable change
This led to the following decisions:
- the code is separated into subdirectories by concern, containing all the route handlers and tightly coupled code
- for now, these directories are in `server/packages` and `server/datasources`. I'm tempted to move them into `server/api/{packages,datasources}` but wanted to limit the amount of refactoring in one PR
- shared code lives in `server/lib`
- some code from `server/packages` has been almost duplicated to handle saving to Datasource saved objects, some has been refactored and is used from both places. The deduplication needs further improvement
- maybe `server/registry` should also move under `server/lib` (but see above, I'm trying to not move everything around all at once)
Testing:
* Please note that this is a breaking change because the saved object type for package information
has also been renamed. You'll need to start with a fresh `.kibana-*` index. Restarting `yarn es snapshot` (withouth specifying a data directory) should do the trick.
* Package installation should still work, e.g. with a GET request to `http://localhost:5601/api/epm/package/coredns-1.0.1`. The saved objects for packages can be inspected with a GET request to `http://localhost:5601/api/saved_objects/epm-package/$PKG_KEY`, e.g. `http://localhost:5601/api/saved_objects/epm-package/coredns-1.0.1`
* Datasource creation should still work, e.g. with a GET request to `http://localhost:5601/api/epm/datasource/install/coredns-1.0.1`. The saved objects for datasources can be inspected with a GET to `http://localhost:5601/api/saved_objects/epm-datasource/$PKG_KEY`, e.g. `http://localhost:5601/api/saved_objects/epm-datasource/coredns-1.0.1`
* [Fleet] Expose policy during agent checkin (#51968)
* [EPM] Add /epr prefix to the tar.gz download path (#51881)
The registry slightly changed the .tar.gz path because of download stats reason. This adjusts for it. See https://github.com/elastic/package-registry/pull/169
* [EPM] Move template installation to lib and add asset helper (#52049)
* [EPM] Move template installation to lib and add asset helper
All the logic related to the installation of the templates for a package should be inside the template library folder. This moves the logic into this folder.
A few refactorings were made to simplify installation:
* Introduction of DataSet interface: This interface is needed to extract the data sets inside a package and install one template per data set.
* Pass package instead of package key to installation process: Passing the package instead of the package key means fetching of package information is decoupled from the installation process and abstracted. This separates the two concerns and should simplify testing.
* getAsssets method: The getAssets methods works on top of the package object to extract asset paths. It is inspired by get_objects methods but supports passing a package and a dataset.
Currently one problem with testing that exists is that to fetch the content of an asset is not decoupled yet.
* [EPM] Reduce data source to one type (#52061)
Between Fleet / Ingest / EPM there had been several interface definitions of Datasource and the related types. This reduces it to one place for the definition. The same applies to the policy definition.
The goal of this is that from now on we all rely on the same definition. If we make changes, we make them in all parts of the code.
In this PR is only the minimal change needed to get us all on one interface. Further changes will be needed that we all rely on the same saved objects etc.
* add export command
* revert 2 more files to rely on export
* revert imports
* Fix types for Datasource Saved Object
* merge in master
* fix type check
* Run VSCode's organize imports on EPM files (#52234)
Learned about it on Slack from https://twitter.com/ryanchenkie/status/1201883268527927301
Blog at https://code.visualstudio.com/updates/v1_23#_run-code-actions-on-save
Basically does the order we've been loosely following (3rd party, then relative) & alphabetic by location and variable name.
It's not customizable but it's reasonable and, afaict, consistent.
* [EPM] More realistic datasource SO. Error if package not installed. (#52229)
* Move cache 'hack' into getAssetsData
* p -> pkg. package is reserved. pkgkey is used in many places
* Remove unnecessary type cast
* Clarify reasons behind asset path manipulation
* Return the Datasource; not the Saved Object.
* Use real values from package in fake datasource SO
* Error if /datasource/install before /package/install
```
> curl --user elastic:changeme localhost:5601/api/epm/datasource/install/coredns-1.0.1
{
"statusCode": 403,
"error": "Forbidden",
"message": "coredns-1.0.1 is not installed"
}
> curl --user elastic:changeme localhost:5601/api/epm/install/coredns-1.0.1
[
{
"id": "53aa1f70-443e-11e9-8548-ab7fbe04f038",
"type": "dashboard"
},
{
"id": "Metricbeat-CoreDNS-Dashboard-ecs",
"type": "dashboard"
},
{
"id": "75743f70-443c-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "36e08510-53c4-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "277fc650-67a9-11e9-a534-715561d0bf42",
"type": "visualization"
},
{
"id": "cfde7fb0-443d-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "a19df590-53c4-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "a58345f0-7298-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "9dc640e0-4432-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "3ad75810-4429-11e9-8548-ab7fbe04f038",
"type": "visualization"
},
{
"id": "57c74300-7308-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "27da53f0-53d5-11e9-b466-9be470bbd327-ecs",
"type": "visualization"
},
{
"id": "86177430-728d-11e9-b0d0-414c3011ddbb",
"type": "visualization"
},
{
"id": "4804eaa0-7315-11e9-b0d0-414c3011ddbb",
"type": "visualization"
}
]
> curl --user elastic:changeme localhost:5601/api/epm/datasource/install/coredns-1.0.1
[
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_plaintext_json",
"type": "ingest-pipeline"
},
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_json_json",
"type": "ingest-pipeline"
},
{
"id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_entry_json",
"type": "ingest-pipeline"
}
]
```
* fix duplicated imports
* [EPM] Move golden files generation over to jest snapshot (#52203)
* [EPM] Move golden files generation over to jest snapshot
I initially used my own implementation to write the generated files. It runs out jest has a feature to write snapshots which simplifies the code a lot.
I added a loop with an additional test file so in the future we can just keep adding test files without having to modify the test code.
To updated the snapshots, the param `-u` has to be used:
```
node scripts/jest legacy/plugins/epm/server/lib/fields/field.test.ts -u
```
* [EPM] Create metrics-* and logs-* Kibana index pattern (#52277)
This creates the very basic Kibana index patterns metrics-* and logs-* for Kibana. At the moment it is overwritten every time. We need to change this in the future to take the fields from all installed data sources and regenerate it.
* [EPM] Create helper for elasticsearch asset names (#52265)
Most of the Elasticsearch assets have the same base name. This creates a helper to get the base name for the assets. In case we decide to change the base name in the future, we can change it in one place.
* fix tests and destructing
* [EPM] Update Registry types. Prevent errors installing certain datasources. (#52285)
* Update RegistryPackage type
* Use download key from EPR to fetch archive
* Fix errors caused by correcting the Registry types.
The issues were largely that some Registry types like `title, `datasets` and `assets` where marked as required, but are actually optional. This highlighted area in the code were we relied on them always being present.
We added to the issue by wrapping Registry types in `Required` which made those items which were correctly listed as optional, required for EPM code. Updated EPM types to reflect the largely pass-through nature of the EPM types.
There are two properties which we ensure are in every EPM response, those were put into their own (unexported) type.
Confirm by trying to add a datasource to a package which has no datasources, like apache-1.0.1 or system-2.0.1. In `feature-ingest` and the earlier version of this PR, the `/datasource/install` call returns a 500. In this PR it succeeds.
* [EPM] Add setup of default ILM policies (#52272)
This creates two ILM policies: logs-default and metrics-default. These are the default ILM policies used. Currently the policy content is hardcoded in the code but should be fetched from the base package in the future. The setup happens as part of the datasource installation. When a data source is installed it is a good time to check if the assets are there but we might extract this to a better place in the future.
* [EPM] 52075 add data source first page (#52320)
* Update RegistryPackage type
* add first page of add data source
* fix for ie11 flex min width bug
* remove toDetailViewRelative
* remove unneeded spread
* Update TS type names for EPR search results (#52512)
* `RegistryList -> RegistrySearchResults`
* `RegistryListItem -> RegistrySearchResult`
* Restore import sort order from #52234 (#52548)
Many of the changes from #52234 were lost. Presumably due to PR(s) merging which were based on branches which had the previous unsorted order.
* [EPM] Replace wildcard export (#52554)
* PackageNotInstalledError -> packages/index.ts
* pkgToPkgKey -> registry/index.ts (will convert existing `${name}-${version}` instances later)
* Replace export * from packages.
There's an argument that the import sites should be updated to import from `packages/get`, `packages/install`, etc but that can wait for a later PR.
* [EPM] Reduce usage of epm-package SavedObject (#52576)
* Delete existing Installation type. Rename InstallationAttributes to Installation
* Reduce usage of EPM SO. Add getInstallation().
Replaced two calls of getInstallationObject() with getInstallation().
Two less places with knowledge of SO internals.
Lots of potential improvements for EPM TS types remain (refactoring/removing Installable, etc), but this is a good incremental step, IMO
* [EPM] Fix missing export link (#52628)
Without it, things break. I am surprised CI did not catch this.
* [EPM] Cleanup ILM loading (#52632)
Before the check for the ILM policy to exist triggered an exception. With this change it is a normal response also if the policy does not exist yet.
A follow up issue will be created in Elasticsearch to get a HEAD request for this available.
* [EPM] Switch to staging URL for registry (#52626)
The old cluster with the registry will be removed as soon as this is merged.
* [EPM] Use Dataset interface to generate template (#52255)
This will make sure we have to pass much feature params and can fully rely on the datasource object to create names for assets.
* [Fleet] Use agent events to compute agent health (#52513)
* [EPM] Data source integration tests (#52542)
* Add fixtures for data source integration test.
* Move test setup to beforeEach
* Add test for datasource creation
* Handle pipelines in yml format.
* Make integration test for adding a data source pass.
* Use EPR staging URL with CDN. (#52776)
See https://github.com/elastic/kibana/pull/52626#pullrequestreview-330622868
* [EPM] Add Data Source page updates (#52705)
* remove dupe type RegistryPackage
* change switches to checkboxes, use datasets to create checkboxes, add some local form state
* update types
* [EPM] redirect after package install (#52771)
* add callback after successful installation and redirect
* add temp data sources tab content to access add data source page
* remove assets tab for mvp
* hide data sources link and redirect from data sources tab if package not installed
* change callback name
* remove commented out assets logic
* add redirect to hook
* fix type
* Use ingest datasource api (#52964)
Incremental change. Uses HTTP API for datasource creation. Will do follow-up PR which uses JS function instead
* Remove duplicate fetchInfo & installTemplates
I think this was from a bad merge, but pretty sure we don't want these functions called twice in the same function
* WIP. Pushing so others can see
* Improve correctness/flexibility of absolute URL
* Disable datasource test & template installation
* [Ingest] Data source APIs (#52448)
* Clean up ingest imports and remove unneeded mock_spec files
* Initial pass at datasources lib and API endpoints
* Add add/remove datasource to/from policy API endpoints
* Add datasource contract tests and related policy contract tests; update snapshots
* Fix tests
* Fix tests again
* Fix tests 3
* Adjust routes, PR feedback
* modify epm createDatasource endpoint to use user data (#52971)
* change epm/datasource/install/{pkg} to POST, send user data to endpoint, install pipelines and templates based on user selected datasets
* change test to post for installing a datasource
* change some names and types around
* delete request.headers['transfer-encoding'] being passed through from epm request
* [EPM] Don't share CreateFakeDatasource type (#53068)
It's not shared between client & server so it doesn't need to be in common. Also, it imports server code which would try to bring server types to the client. It's types so they're compiled away but it's important to keep common to what's truly common. Breaking this separation is why we thought enums broke the client. A lint rule just landed in master to prevent this.
* [EPM] Index template generation fixes (#53104)
* Only add keyword type field to mappings.
* Index template installation
* Handle empty fields definition files
* Re-enable index template installation
* [Fleet] Assign/Unassign data source from policy UI (#53058)
* Add index files to export various modules; normalize imports
* Clean up unused files; extract datasources table component from policy details page
* Expose http client to frontend libs; remove unused types; import ES UI's useRequest lib
* Adjust shape of rest api adapter interface to better match with rest of kibana; remove unused node adapter; change per_page param to perPage in agent events route
* Initial pass at assign data sources flyout
* Initial pass at unassigning data sources from policy
* Make data sources table searchable by package values
* Fix enrollment key lib for rest adapter param changes
* Fix imports and types
* `yarn.lock` changes after bootstrapping
* [EPM] Implement getConfig for dataset (#53261)
* [EPM] Implement getConfig for dataset
* Implements a getConfig method on a dataset object.
* Build the configuration for each dataset in a package.
* construct and save streams into datasource saved object
* [EPM] Fix template installation (#53272)
As dataset.package was not set, the installed templates contained undefined in the template name. This changes fixes this.
* [EM] Refactor ingest pipeline installation (#53309)
* Refactor ingest pipeline installation
* Only install index templates for requested datasets
* Add index.default_pipeline to index template
* Hook up pipeline rewriting
* Add correct types.
* change POST create datasources path (#53165)
* change POST create datasources path
* remove pkgkey from params
* Fix creation of a data source with a custom ID (#53537)
* [Ingest] Return associated policy IDs in data source info (#53350)
* Return number of policies from data source, surface in assign data source UI
* Update snapshots
* [EPM]: Assign data source to policy in UI (#53597)
* Let ES generate source ids. Refactor along the way.
* Datasource.id isn't optional. It's just missing before we send to Ingest
* Delete EPM's mapping of datasources saved object. Ingest handles that.
* Keep datasource object-related work in constructDatasource
* Move asset installation into own function. Keep entry point high-level.
* More descriptive (less ambiguous) names for these two functions
* Use enum values from Ingest instead of plain strings
* Limit the 'type' key of references to known asset types.
* Update variable names to clarify that we're merging arrays of references
* Use [].flat instead .reduce + .concat to avoid error on empty arrays.
* Pass PackageInfo value directly to component vs pulling off n properties
* Name handlers/options based on the data, not the UI element
* Populate policy combo box based on values from Ingest policy API
* Mark Dataset.vars as optional.
* Add TODOs
* Add commands to run API tests to README (#53847)
* Limit functions to 3 params max. Update those which used more (#53848)
* [EPM] Code in 'common' directories shouldn't import server code (#53854)
* [Fleet] Code in 'common' directories shouldn't import server code (#53938)
* [Fleet] Remove server code from common folder in fleet
* [Fleet] Fix typescript issues after master merge
* [EPM] Fix typescript issues after master merge
* Fix eslint issues
* Fix typescript issues after merge
* Fix merge master missing line
* Fix merge conflict
* [Fleet] Fix registration of Ingest management section (#54065)
* Fix registration of Ingest management section
* Fix i18n key
* [Fleet] Remove server code from common folder in ingest (#53969)
* [Fleet] Connect fleet to policy change update (#53201)
* [Fleet] Send created event when a policy is created
* [Fleet] updated created event when a policy is created
* [Fleet] Send deleted event when a policy is deleted
* [Fleet] Rename output.url => output.hosts (#54258)
* [Ingest] Remove policies UI (#54308)
* Remove meta field UI from policy add/edit form
* Initial pass at policy bulk+single delete UI and API
* Adjust policy links from agent list and detail pages so that links are only active if policy exists
* Add delete policy UI to policy detail page
* Disable policy delete button for default policy
* Commit updated kbn-pm artifact.
CI is failing with messages like
14:52:28 ERROR: 'yarn kbn run build -i @kbn/pm' caused changes to the following files:
14:52:28
14:52:28 packages/kbn-pm/dist/index.js
Following advice from https://elastic.slack.com/archives/C0D8P2XK5/p1570032166063400 and running/committing build
* Update kbn/pm package
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* [EPM] create logs metrics index patterns (#54037)
* fixes bug in for loop returning too early and not looping through all yaml files creating incomplete index template, move loading yaml files to own function, other cleanup
* use reduce in place of for loop
* basic functionality for creating index patterns
* separate logs and metrics index patterns
* dedupe fields
* adjust flattenFields to rename nested fields with parent name path
* some tests
* use yml files for tests
* add awaits as part of installing the package
* optimize loading of yaml files
* fix typo
* change type packageName to package
* update tests to use all files from the beginning
* fix type errors
* fix test
* Use dataset.package from registry
https://github.com/elastic/kibana/pull/54037#pullrequestreview-340362812
* Form validation on add datasource page. (#53920)
* [Ingest] Add support for policy `label` field (#54413)
* Allow `label` field in policy APIs, update UIs to support `label` field
* PR review changes, typo fixes, update tests
* [Fleet] Fix api key creation (#54498)
* [Fleet] Move agent acks to his own endpoint (#54401)
* [Ingest] Fix MaxListenersExceededWarning during kibana boot (#54745)
* [Fleet] Create a default api key for the default output (#54658)
* [EPM] update index pattern fields (#54862)
* add DatasetType type
* move loadFieldsFromYaml to fields
* add logic to determine field values
* update tests
* add back accidentally removed readFromDocValues
* remove DatasetType and add IndexPatternType
* rename dedup to dedupe
* group tests
* use null coalescing operator
* Fix typing issues
* [Fleet] Support agent event subtype STOPPING (#55231)
* [EPM] Prevent double submit when creating data sources
Update installationRequested state after submit (#55100)
* [EPM] handle alias fields when creating kibana index pattern (#55254)
* add alias support, update flattenFields to handle alias copying, update and add tests
* update snapshot
* update findFieldByPath to return undefined if not leaf node
* remove temporary alias type from map
* [EPM] handle multi fields when creating index patterns (#55554)
* handle mult_fields
* move nested function out
* [Fleet] Fleet/spec docs (#55619)
* [EPM] Start to document definitions (#55361)
This is a first stab at creating a place where we define the terms we use across ingest management. This is not complete but defines a place where we can add all the future defintions.
* [EPM] Indexing strategy docs (#55301)
* [EPM] Indexing strategy docs
This documentation is to start documenting our new indexing strategy in a public place and have it versioned. This will allow us to share the current state of the indexing strategy more easily in a single place, track it when updated and also already have it ready for our future users to look it up and understand the benefits of it.
* update typos
* fix one more typo
* apply review feedback
* skip fields that are disabled (#55735)
* [Fleet] Fix fleet typing issues after merging master
* [EPM] Create fieldFormatMap in kibana index pattern (#55892)
* add support for fieldFormatMap
* output params must be camel case
* fix case
* add url_template param
* [Fleet] Use user from saved object to create apiKeys (#55458)
* [EPM] Document package upgrade behaviour (#56138)
This PR adds more detailed documentation on what should happen when a package is upgraded.
* Remove some explicit typing to pass type checks.
This abstraction will be removed in the single plugin going into master.
* remove readFromDocValues (#56227)
* Add symlink to yarn.lock to fix(?) CI
https://github.com/elastic/kibana/pull/56443/checks?check_run_id=418123781 failed saying EPM directory "MUST have a 'yarn.lock' symlink"
Seems to have originated with https://github.com/elastic/kibana/pull/55440
Following example from other legacy/plugins/* in that PR
* Like 441d9ed, but correct
* [Ingest] Convert `ingest` plugin to new platform `ingest_manager` plugin (#56262)
* Seed Ingest Manager as a new NP plugin
* Add contexts for core, deps, and config. Begin routing and nav UI
* Export NP ready request from top-level es_ui_shared/public/
* Add nav styling w/ theming, add useRequest hook
* Set up license and config server-side services; add test routes
* Move most types and constants into /common
* Initial pass at:
* data stream and agent config models
* data stream routes and schemas
* Initial pass at agent config api route handlers
* Change plugin id to camel case
* Fix circular schema dependency, add security as optional plugin
* Create appContext service, use request user info in agent config routes + libs
* Create default agent config
* Add default output host config, output typings, and create default output and its api key
* Move saved object mapping to new plugin
* Change data streams -> datasources
* Add legacy plugin to bootstrap mappings
* Adjust fleet's ingest dependencies
* Disable policies UI in Fleet
* Adjust EPM's ingest dependencies
* Adjust ingest manager base API route
* Adjust fleet's client side ingest dependencies
* Remove more ingest dependencies from fleet
* REMOVE MOST OF LEGACY INGEST PLUGIN
* Add section for agent configs in UI nav
* Allow useRequest and sendRequest consumers to specify typing for response
* Initial pass at porting over agent config list UI
* Port over agent config creation
* Port over delete agent config functionality
* Fix app routing
* Port over fleet setup routes
* Adjust fleet's ingest dependencies
* Make fleet happy path work, skip some tests (MESSY! :))
* Remove policy list UI code from fleet
* Change useRequestResponse error type
* Add missing agent config schemas and hooks
* Fix type check issues
* Register IM under management
* Fix type issues as a result of changes to use/sendRequest interfaces
* Make all ingest saved objects *not* space-aware
* Fix i18n path
* Fix app categories import
* Fix datasource package assets schema (array of asset objects)
* Seed Ingest Manager privileges to fix tests
* Change `features` to optional plugin instead of required
* Fix security privileges tests
* Fix feature test
* Fix duplicate enrollment key created for default agent config
* Fix fleet agent enrollment by catching agent config 404
* PR feedback
* First pass at deleting everything related to EPM & Fleet plugins
* Remove EPM-specific react-markdown
* Remove some unwanted differences from master.
Removed more EPM and Fleet changes
* React.FC -> React.FunctionComponent to match the pattern in master
* Selectively enable features based on config values
* Fix typing and(?) CI
* Enable xpack.ingestManger during tests
Follow example from xpack.endpoint flags
* Don't create BASE_URL value based on PLUGIN_ID
* Don't add optional routes. Use config for registry url
* Add README with some skeleton docs on tests, etc
* Undo changes to some files based on PR feedback
* Commit the result of 'yarn kbn run build -i @kbn/pm'
* Commit the result of 'yarn kbn run build -i @kbn/pm'
* WIP. But add some passing api integration tests
└-: apis
└-> "before all" hook
└-: Ingest Manager plugin
└-> "before all" hook
└-: /agent_configs
└-> "before all" hook
└-> should get agent configs
└-> "before each" hook: global before each
└- ✓ pass (42ms) "apis Ingest Manager plugin /agent_configs should get agent configs"
└-> should create agent config
└-> "before each" hook: global before each
└- ✓ pass (63ms) "apis Ingest Manager plugin /agent_configs should create agent config"
└-> should have new agent config
└-> "before each" hook: global before each
└- ✓ pass (41ms) "apis Ingest Manager plugin /agent_configs should have new agent config"
└-> should delete that new config
└-> "before each" hook: global before each
└- ✓ pass (1.0s) "apis Ingest Manager plugin /agent_configs should delete that new config"
└-> should get datasources
└-> "before each" hook: global before each
└- ✓ pass (26ms) "apis Ingest Manager plugin /agent_configs should get datasources"
└-> should 404 from EPM package api
└-> "before each" hook: global before each
└- ✓ pass (41ms) "apis Ingest Manager plugin /agent_configs should 404 from EPM package api"
└-> "after all" hook
└-> "after all" hook
└-> "after all" hook
6 passing (1.2s)
* WIP
* Fix TS type errors.
Types are created tests. Should they come from test project?
* Add route to delete datasources (mirrors agent_configs api)
* Remove old ESArchiver files
* Add integration tests for each /agent_config & /datasources route
* Update README
Remove bash integration test scripts
* More updates to README
* Remove xpack.kueryAutocomplete from x-pack/.i18n.json
* Use default appRoute. We can't use /app/ingest (at least for now).
Looking at the app_router it seems like any route starting with `/app` must be `/app/:appId`.
I also don't think appRoute is fully supported. I'll follow up with Platform.
* Use appBasePath from app mount params for React Router basename
* Remove out-of-date comments
* Use the ES client from core
* Move License & Config services WIP. Pushing to share
* Move config & license services into App Context service.
* Remove FTR tests for Ingest Manager
* Remove license-related propperties
* Remove clusterClient properties
* Don't set this.security if not in dependencies
* WIP (failing) integration_test
* No more unknown key errors. Timeouts now
* Test routes with default (disabled) and manager-only flags
* Add tests to cover all permutations of flags (A, AB, AC, ABC)
* Use TS types vs config-schema in common & public
Move all @kbn/config-schema usage to server
Co-authored-by: Matt Apperson <me@mattapperson.com>
Co-authored-by: Jen Huang <its.jenetic@gmail.com>
Co-authored-by: Nicolas Chaulet <n.chaulet@gmail.com>
Co-authored-by: Sandra Gonzales <neptunian@users.noreply.github.com>
Co-authored-by: Nicolas Ruflin <spam@ruflin.com>
Co-authored-by: Sonja Krause-Harder <krauseha@gmail.com>
Co-authored-by: Brian Seeders <seeders@gmail.com>
Co-authored-by: Tyler Smalley <tylersmalley@me.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-02-19 03:30:38 +01:00
|
|
|
ingestManager(kibana),
|
2020-01-14 01:16:20 +01:00
|
|
|
triggersActionsUI(kibana),
|
2018-04-20 21:13:37 +02:00
|
|
|
];
|
|
|
|
};
|