* Use client from branch
* Get type checking working in core
* Fix types in other plugins
* Update client types + remove type errors from core
* migrate Task Manager Elasticsearch typing from legacy library to client library
* use SortOrder instead o string in alerts
* Update client types + fix core type issues
* fix maps ts errors
* Update Lens types
* Convert Search Profiler body from a string to an object to conform to SearchRequest type.
* Fix SOT types
* Fix/mute Security/Spaces plugins type errors.
* Fix bootstrap types
* Fix painless_lab
* corrected es typing in Event Log
* Use new types from client for inferred search responses
* Latest type defs
* Integrate latest type defs for APM/UX
* fix core errors
* fix telemetry errors
* fix canvas errors
* fix data_enhanced errors
* fix event_log errors
* mute lens errors
* fix or mute maps errors
* fix reporting errors
* fix security errors
* mute errors in task_manager
* fix errors in telemetry_collection_xpack
* fix errors in data plugins
* fix errors in alerts
* mute errors in index_management
* fix task_manager errors
* mute or fix lens errors
* fix upgrade_assistant errors
* fix or mute errors in index_lifecycle_management
* fix discover errors
* fix core tests
* ML changes
* fix core type errors
* mute error in kbn-es-archiver
* fix error in data plugin
* fix error in telemetry plugin
* fix error in discover
* fix discover errors
* fix errors in task_manager
* fix security errors
* fix wrong conflict resolution
* address errors with upstream code
* update deps to the last commit
* remove outdated comments
* fix core errors
* fix errors after update
* adding more expect errors to ML
* pull the lastest changes
* fix core errors
* fix errors in infra plugin
* fix errors in uptime plugin
* fix errors in ml
* fix errors in xpack telemetry
* fix or mute errors in transform
* fix errors in upgrade assistant
* fix or mute fleet errors
* start fixing apm errors
* fix errors in osquery
* fix telemetry tests
* core cleanup
* fix asMutableArray imports
* cleanup
* data_enhanced cleanup
* cleanup events_log
* cleaup
* fix error in kbn-es-archiver
* fix errors in kbn-es-archiver
* fix errors in kbn-es-archiver
* fix ES typings for Hit
* fix SO
* fix actions plugin
* fix fleet
* fix maps
* fix stack_alerts
* fix eslint problems
* fix event_log unit tests
* fix failures in data_enhanced tests
* fix test failure in kbn-es-archiver
* fix test failures in index_pattern_management
* fixing ML test
* remove outdated comment in kbn-es-archiver
* fix error type in ml
* fix eslint errors in osquery plugin
* fix runtime error in infra plugin
* revert changes to event_log cluser exist check
* fix eslint error in osquery
* fixing ML endpoint argument types
* fx types
* Update api-extractor docs
* attempt fix for ese test
* Fix lint error
* Fix types for ts refs
* Fix data_enhanced unit test
* fix lens types
* generate docs
* Fix a number of type issues in monitoring and ml
* fix triggers_actions_ui
* Fix ILM functional test
* Put search.d.ts typings back
* fix data plugin
* Update typings in typings/elasticsearch
* Update snapshots
* mute errors in task_manager
* mute fleet errors
* lens. remove unnecessary ts-expect-errors
* fix errors in stack_alerts
* mute errors in osquery
* fix errors in security_solution
* fix errors in lists
* fix errors in cases
* mute errors in search_examples
* use KibanaClient to enforce promise-based API
* fix errors in test/ folder
* update comment
* fix errors in x-pack/test folder
* fix errors in ml plugin
* fix optional fields in ml api_integartoon tests
* fix another casting problem in ml tests
* fix another ml test failure
* fix fleet problem after conflict resolution
* rollback changes in security_solution. trying to fix test
* Update type for discover rows
* uncomment runtime_mappings as its outdated
* address comments from Wylie
* remove eslint error due to any
* mute error due to incompatibility
* Apply suggestions from code review
Co-authored-by: John Schulz <github.com@jfsiii.org>
* fix type error in lens tests
* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* update deps
* fix errors in core types
* fix errors for the new elastic/elasticsearch version
* remove unused type
* remove unnecessary manual type cast and put optional chaining back
* ML: mute Datafeed is missing indices_options
* Apply suggestions from code review
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
* use canary pacakge instead of git commit
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Nathan Reese <reese.nathan@gmail.com>
Co-authored-by: Wylie Conlon <wylieconlon@gmail.com>
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
Co-authored-by: restrry <restrry@gmail.com>
Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: John Schulz <github.com@jfsiii.org>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* Add force option to DELETE package endpoint.
* Add integration test.
* Adjust openapi spec.
* Run EPM tests before fleet setup tests.
* Run package delete tests first in EPM tests
* Add data plugin to server app context
* First attempt at switching to indexPatternService for EPM index pattern creation & deletion, instead of interacting directly with index pattern SOs
* Simplify bulk install package, remove upgradePackage() method in favor of consolidating with installPackage(), use installPackage() for bulk install instead
* Update tests
* Change cache logging of objects to trace level
* Install index patterns as a post-package installation operation, for bulk package installation, install index pattern only if one or more packages are actually new installs, add debug logging
* Allow getAsSavedObjectBody to return non-scripted fields when allowNoIndex is true
* Allow `getFieldsForWildcard` to return fields saved on index pattern when allowNoIndices is true
* Bring back passing custom ID for index pattern SO
* Fix tests
* Revert "Merge branch 'index-pattern/allow-no-index' into epm/missing-index-patterns"
This reverts commit 8e712e9c00, reversing
changes made to af0fb0eaa8.
* Allow getAsSavedObjectBody to return non-scripted fields when allowNoIndex is true
(cherry picked from commit 69b93da180)
* Update API docs
* Run post-install ops for install by upload too
* Remove allowedInstallTypes param
* Adjust force install conditions
* Revert "Update API docs"
This reverts commit b9770fdc56.
* Revert "Allow getAsSavedObjectBody to return non-scripted fields when allowNoIndex is true"
This reverts commit afc91ce32f.
* Go back to using SO client for index patterns :(
* Stringify attributes again for SO client saving
* Fix condition for reinstall same pkg version
* Remove unused type
* Adjust comment
* Update snapshot
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Refactoring in progress towards https://github.com/elastic/kibana/issues/90437
Collapse `forceUninstallAgents` into `uninstallAgents` into one function with an option. The pseudocode diff is
```diff
- function uninstallAgents() {
- // filtering logic A
- // side effects A
- }
- function forceUninstallAgents() {
- // filtering logic A
- // side effects B
- }
+ function uninstallAgents({ flag: boolean}) {
+ // filtering logic A
+ // if flag: side effects B
+ // else: side effects A
+ }
```
actually, there is [_one difference_](5564f25383 (diff-ecc3c625f2366949f1723e56b8477f6afb552ccfbcf3a71e0c28b2062fd05195)) in the filtering logic
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Problem
While working on changes for bulk reassign https://github.com/elastic/kibana/issues/90437, I found that the server has a runtime error and returns a 500 if given an invalid or missing id.
<details><summary>server error stack trace</summary>
```
│ proc [kibana] server log [12:21:48.953] [error][fleet][plugins] TypeError: Cannot read property 'policy_revision_idx' of undefined
│ proc [kibana] at map (/Users/jfsiii/work/kibana/x-pack/plugins/fleet/server/services/agents/helpers.ts:15:34)
│ proc [kibana] at Array.map (<anonymous>)
│ proc [kibana] at getAgents (/Users/jfsiii/work/kibana/x-pack/plugins/fleet/server/services/agents/crud.ts:191:32)
│ proc [kibana] at runMicrotasks (<anonymous>)
│ proc [kibana] at processTicksAndRejections (internal/process/task_queues.js:93:5)
│ proc [kibana] at Object.reassignAgents (/Users/jfsiii/work/kibana/x-pack/plugins/fleet/server/services/agents/reassign.ts:91:9)
│ proc [kibana] at postBulkAgentsReassignHandler (/Users/jfsiii/work/kibana/x-pack/plugins/fleet/server/routes/agent/handlers.ts:314:21)
│ proc [kibana] at Router.handle (/Users/jfsiii/work/kibana/src/core/server/http/router/router.ts:272:30)
│ proc [kibana] at handler (/Users/jfsiii/work/kibana/src/core/server/http/router/router.ts:227:11)
│ proc [kibana] at exports.Manager.execute (/Users/jfsiii/work/kibana/node_modules/@hapi/hapi/lib/toolkit.js:60:28)
│ proc [kibana] at Object.internals.handler (/Users/jfsiii/work/kibana/node_modules/@hapi/hapi/lib/handler.js:46:20)
│ proc [kibana] at exports.execute (/Users/jfsiii/work/kibana/node_modules/@hapi/hapi/lib/handler.js:31:20)
│ proc [kibana] at Request._lifecycle (/Users/jfsiii/work/kibana/node_modules/@hapi/hapi/lib/request.js:370:32)
│ proc [kibana] at Request._execute (/Users/jfsiii/work/kibana/node_modules/@hapi/hapi/lib/request.js:279:9)
```
</details>
<details><summary>see test added in this PR fail on master</summary>
```
1) Fleet Endpoints
reassign agent(s)
bulk reassign agents
should allow to reassign multiple agents by id -- some invalid:
Error: expected 200 "OK", got 500 "Internal Server Error"
```
</details>
## Root cause
Debugging runtime error in `searchHitToAgent` found some TS type mismatches for the ES values being returned. Perhaps from one or more of the recent changes to ES client & Fleet Server. Based on `test:jest` and `test:ftr`, it appears the possible types are `GetResponse` or `SearchResponse`, instead of only an `ESSearchHit`.
https://github.com/elastic/kibana/pull/94632/files#diff-254d0f427979efc3b442f78762302eb28fb9c8857df68ea04f8d411e052f939cL11
While a `.search` result will include return matched values, a `.get` or `.mget` will return a row for each input and a `found: boolean`. e.g. `{ _id: "does-not-exist", found: false }`. The error occurs when [`searchHitToAgent`](1702cf98f0/x-pack/plugins/fleet/server/services/agents/helpers.ts (L11)) is run on a get miss instead of a search hit.
## PR Changes
* Added a test to ensure it doesn't fail if invalid or missing IDs are given
* Moved the `bulk_reassign` tests to their own test section
* Filter out any missing results before calling `searchHitToAgent`, to match current behavior
* Consolidate repeated arguments into and code for getting agents into single [function](https://github.com/elastic/kibana/pull/94632/files#diff-f7377ed9ad56eaa8ea188b64e957e771ccc7a7652fd1eaf44251c25b930f8448R70-R87): and [TS type](https://github.com/elastic/kibana/pull/94632/files#diff-f7377ed9ad56eaa8ea188b64e957e771ccc7a7652fd1eaf44251c25b930f8448R61-R68)
* Rename some agent service functions to be more explicit (IMO) but behavior maintained. Same API names exported.
This moves toward the "one result (success or error) per given id" approach for https://github.com/elastic/kibana/issues/90437
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* merge all the typings at root level
* merge x-pack/tsconfig into tsconfig.json
* fix tsconfig after changes in master
* remove unnecessary typings
* update paths to the global typings
* update paths to the global elaticsearch typings
* fix import
* fix path to typings/elasticsearch in fleet plugin
* remove file deleted from master
* fix lint errors
* When xpack.securitySolution.fleetServerEnabled is true, then Endpoint artifact manifest will use a fleet-server relative url for the artifacts generated (note: this flag is temporary until we ship v7.13)
* Refactors the security solution fleet integration extension point callbacks so that some action handlers can be executed in parallel (the creation of detection engine prepackaged rules can sometime take some time to complete
## Problem
There's a circular dependency https://github.com/elastic/kibana/issues/91111 between the `fleet` and `security_solution` plugins
* `security_solution` depends on `fleet`, but
* `fleet` has (_had_ with this PR) an `import` from `security_solution` (migrations for the 7.11 and 7.12 package policy objects)
## Proposed solution
### (A) This PR
Move the two imported functions from `security` into `fleet`.
### (B) Follow up issue
Putting integration-specific code into `fleet` doesn't scale (technically or cognitively). Discuss if this use case (specifying saved object migrations, etc) applies to other plugins. e.g. can `apm` do this? `nginx`? If so, should we find a way to move this out of `fleet`?
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
closes https://github.com/elastic/kibana/issues/91111
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fleet: adds new service for Artifact storage management
* Fleet: Expose new `createArtifactsClient()` from Fleet `Plugin#start` interface
* Endpoint: Change Endpoint to use FleetArtifactClient and initial implementation of EndpointArtifactClient
* Endpoint: Add `fleetServerEnabled` feature flag to security solution plugin (will be used in next PR)
* Endpoint: Artifact download api adjusted to get artifact from fleet index
* Endpoint: Added new esArchive for artifacts stored in .fleet-artifacts index for API integration tests
* Make package validation be based on types
* Add package info to cache after it is generated from ES storage
* Add logging around package info retrieval and when cache is set
* Add snapshot api integration test for uploaded package info
* Use the apache package for snapshot test instead
* Remove date field from snapshot
* Update docs
* Fix streams getting overridden
* Add back package field to data streams
* PR fixes
* Add UI validation for string YAML values in policies.
* Do not quote YAML strings containing special characters.
* Add test case for wildcards in the middle of strings.
* Add multiline test case.
* Polish test case.
* Update API docs
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Set all keyword and text fields for `index.query.default_field` setting
* Update tests and snapshots
* Fix test
* Add default field limit safeguard
* Add logging when beyond limit
* Update tests to mock app context (because I added logger usage)
* Update api integration test
* Rename consts
## Problem
Blocks of 10-15 `import`s are common in the plugin and there a few places which have ~50 lines of `import`s. It makes it more difficult to understand the where/why of what's being imported.
We've had instances while files import from the same module in different lines. i.e.
```ts
import { a } from './file';
... 5-10 lines later
import { b } from './file';
```
## Proposed solution
Add a lint rule to enforce a convention on the module `import` order. This can help in the same way Prettier & ESLint help to format type signatures or other code. It makes it easier to understand or notice any changes in the code. It's also able to be fixed automatically (`node scripts/eslint.js --fix` or any existing "format on save" in an editor).
## This PR
replaces #92980 (based on https://github.com/elastic/kibana/pull/92980#pullrequestreview-601070556)
### Lint rule
f9be98d Add eslint rule to enforce/autofix import group order. Use the same rule as a few other plugins. Groups `import` statements by type as shown in the [lint rule docs](https://github.com/benmosher/eslint-plugin-import/blob/master/docs/rules/order.md#importorder-enforce-a-convention-in-module-import-order
). The order is:
1. node "builtin" modules
2. "external" modules
3. "internal" modules
4. modules from a "parent" directory
5. "sibling" modules from the same or a sibling's directory, "index" of the current directory, everything else
e.g.
```typescript
import fs from 'fs';
import path from 'path';
import _ from 'lodash';
import chalk from 'chalk';
import foo from 'src/foo';
import foo from '../foo';
import qux from '../../foo/qux';
import bar from './bar';
import baz from './bar/baz';
import main from './';
```
The [lint rule](https://github.com/benmosher/eslint-plugin-import/blob/master/docs/rules/order.md#importorder-enforce-a-convention-in-module-import-order) is relatively light handed. It only ensures the `imports` are groups together in the given order. It doesn't alphabetize or otherwise sort the order of the files.
e.g. imports aren't rewritten to be in alphabetical order. This is fine
```ts
import from './c';
import from './a';
import from './b';
```
The [docs show other options](https://github.com/benmosher/eslint-plugin-import/blob/master/docs/rules/order.md#options) and 2831f02bc7/.eslintrc.js (L1138-L1168) uses many of them
### Newlines option
The newlines settings means a change from something like
```typescript
import fs from 'fs';
import path from 'path';
import _ from 'lodash';
import chalk from 'chalk';
import foo from 'src/foo';
import foo from '../foo';
import qux from '../../foo/qux';
import bar from './bar';
import baz from './bar/baz';
import main from './';
```
to
```typescript
import fs from 'fs';
import path from 'path';
import _ from 'lodash';
import chalk from 'chalk';
import foo from 'src/foo';
import foo from '../foo';
import qux from '../../foo/qux';
import bar from './bar';
import baz from './bar/baz';
import main from './';
```
Added it as a separate commit 2831f02 in case we want to avoid it, but I believe it's an improvement overall. Especially on the files with 25+ lines of imports. Even the "worst case" of something like this isn't bad (IMO). Especially since it's an automatic reformat like anything else in prettier
```typescript
import fs from 'fs';
import _ from 'lodash';
import foo from '../foo';
import main from './';
```
