* ✨ First pass with visualization validation + error messages
* 🔥 Remove indexpattern error handling for now
* 🏷️ Fix type issues
* ✅ Add getErrorMessage test for data table
* ✅ Add tests for pie and metric error messages
* 🌐 Fix i18n checks issues
* 🐛 Fix last issue
* ✅ Add more tests for the XY visualization validation code
* 👌 Included all feedback from first review
* ✏️ Off by one message
* 🌐 Fix i18n duplicate id
* 🌐 Fix last i18n issue
* 🐛 Fixed a hook reflow issue
* ♻️+✅ Reworked validation flow + tests
* 🏷️ Fix type issue
* 🐛 Improved XY corner cases validation logic
* 🐛 Fix empty datatable scenario
* ✨ + ✅ Improved error messages for invalid datasources + tests
* 🌐 Add missing i18n translation
* 🏷️ Fix type issues
* 🌐 Fix i18n issues
* ✨ Filter out suggestions which fail to build
* 🚚 Migrate datatable validation logic to the building phase, handling it as building state
* 🏷️ Fix type issue
* ✏️ Add comment for future enhancements
* ✏️ Updated comment
* :world_with_meridians: Refactor axis labels
* 🌐 Reworked few validation messages
* 🐛 Fix break down validation + percentage charts
* ✅ Align tests with new validation logic
* ♻️ Fix suggestion panel validation to match main panel
* 🌐 Fix i18n issues
* 🔧 Fix some refs for validation checks in suggestions
* 🐛 Fix missing key prop in multiple errors scenario
* 🐛 Fix swtich issue from XY to partition
* 🌐 Fix i18n messages and aligned tests
* 🐛 Fix suggestions switching bug
* :refactor: Add more validation + refactored validation logic in a single place
* ✏️ Add note about lint hooks disable rule
* 🚨 Fix linting issue
* 🏗️ Add infra API for datasource advanced validation
* ✅ Align tests with new API
* ✅ Fix type issues in tests
* 👌 Early exists added
* ✨ Add layers groups to the API
* ✅ Fix some broken test after the validation change
* 👌 Move to disctionary shape
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* resolves https://github.com/elastic/kibana/issues/78148
Adds a `defaultActionMessage` to the index threshold alert, so that the `message` parameter for actions will be pre-filled with a useful message
* remove use of legacy state system and legacy serialization
* remove legacy min_age input component and re-add missing import
* rename shared -> shared_fields for more clarity
* some more cleanup and fixing regressions on policy name for creating new policy from existing policy
* move extract policy static code to lib folder and remove "policies" dir from services
* fix jest tests and minor policy flyout inconsistency
* remove legacy helper
* fix client integration tests
* fix min for set index priority
* moved save policy function into edit policy section
* remove unused translations
* refactor form files to own edit_policy/form folder
* remove "fix errors" badge to fix UX - users can see errors in line before pressing save so the value of this badge has diminished
* fix i18n after removing phase error badge
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Adds a `RunsWhen` field to actions in the Alerts Flyout when creating / editing an Alert which allows the user to assign specific actions to a certain Action Groups
## Summary
* Add `installPackage` with `installSource` param, to provide a single interface the `install*` functions.
```diff
- const res = await installPackageFromRegistry({
+ const res = await installPackage({
+ installSource: 'registry',
```
and
```diff
- const res = await installPackageByUpload({
+ const res = await installPackage({
+ installSource: 'upload',
```
* Push some repeated work (`install`, `removable`) from `install*` into `_installPackage`. Which also simplifies its interface.
### installPackage
For now `installPackage` checks the `installSource` and calls the same `install*` functions to prevent any change in behavior but there's still a lot of overlap between `installPackageFromRegistry` & `installPackageByUpload`. I think we can bring them together into `installPackage` using the same branching on `installSource`.
### local checks with curl
<details><summary>curl request/responses for happy path:</summary>
```
## zip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"response":[{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"}]}
## Uploaded packages can be deleted as expected:
curl -X DELETE -u elastic:changeme http://localhost:5601/api/fleet/epm/packages/apache-0.1.4 -H 'kbn-xsrf: xxx'
{"response":[{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"}]}
## Now upload
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.tar.gz -H 'kbn-xsrf: xyz' -H 'Content-Type: application/gzip'
{"response":[{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"logs-apache.error-0.1.4","type":"ingest_pipeline"},{"id":"logs-apache.access-0.1.4","type":"ingest_pipeline"},{"id":"logs-apache.error","type":"index_template"},{"id":"metrics-apache.status","type":"index_template"},{"id":"logs-apache.access","type":"index_template"}]}
```
</details>
<details><summary>curl request/responses for archive errors:</summary>
```
## Wrong content type:
### tar.gz with application/zip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.tar.gz -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Error during extraction of package: Error: end of central directory record signature not found. Assumed content type was application/zip, check if this matches the archive type."}
### zip with application/gzip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/gzip'
{"statusCode":400,"error":"Bad Request","message":"Archive seems empty. Assumed content type was application/gzip, check if this matches the archive type."}
## Invalid packages
### Two top-level directories:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_two_toplevels_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Package contains more than one top-level directory."}
### No manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_no_manifest_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Package must contain a top-level manifest.yml file."}
### Invalid YAML in manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_manifest_invalid_yaml_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Could not parse top-level package manifest: YAMLException: bad indentation of a mapping entry at line 2, column 7:\n name: apache\n ^."}
### Mandatory field missing in manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_manifest_missing_field_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Invalid top-level package manifest: one or more fields missing of name, version, description, type, categories, format_version"}
### Top-level directory doesn't match name and version from manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_toplevel_mismatch_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Name thisIsATypo and version 0.1.4 do not match top-level directory apache-0.1.4"}
```
</details>
#### TS type check examples on `installPackage`
<details><summary>screenshots</summary>
<img width="379" alt="Screen Shot 2020-11-08 at 4 00 14 PM" src="https://user-images.githubusercontent.com/57655/98484251-1d1e9f80-21dc-11eb-93f8-601036b45355.png">
<img width="890" alt="Screen Shot 2020-11-08 at 4 00 21 PM" src="https://user-images.githubusercontent.com/57655/98484252-1db73600-21dc-11eb-88d1-5faa498f94fc.png">
<img width="396" alt="Screen Shot 2020-11-08 at 4 01 06 PM" src="https://user-images.githubusercontent.com/57655/98484253-1db73600-21dc-11eb-8e2a-10a5762f4a95.png">
<img width="441" alt="Screen Shot 2020-11-08 at 4 01 25 PM" src="https://user-images.githubusercontent.com/57655/98484254-1db73600-21dc-11eb-9d9a-c1620dcad11e.png">
<img width="879" alt="Screen Shot 2020-11-08 at 4 02 54 PM" src="https://user-images.githubusercontent.com/57655/98484255-1db73600-21dc-11eb-8f36-7da3e9256feb.png">
</details>
### Checklist
- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
Observability was importing `dateAsStringRt` from APM, which creates an implicit circular dependency between the two plugins.
Copy that function into where it was being used in observability to remove the dependency.
Related to #80508.
## Summary
* Adds logging output for trouble shooting
* Adds an API to be able to configure how many concurrent searches and how many items per search to use
API additions are these two switches:
```
concurrent_searches
items_per_search
```
When you create a rule. You can use the following example to post one or to change the settings to see the performance impact:
```ts
./post_rule.sh ./rules/queries/query_with_threat_mapping_perf.json
```
Without using these two experimental API settings, the functionality is the same as the existing algorithm and only advanced users will be able to set the additional REST settings through this API. If you use the front end after setting the settings, the settings will be reset as that's how the forms code currently works and this will not preserve the settings if afterwards a rule is edited/changed.
Both these API settings should be considered experimental and potentially breakable as we figure out the best performance strategies for indicator matching.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* Implemented Alerting health status pusher by using task manager and status pooler for Kibana status plugins 'kibanahost/api/status'
* Exposed health task registration to alerts plugin
* Fixed type error
* Extended health API endpoint with info about decryption failures, added correct health task implementation
* adjusted query
* Tested locally and got it working as expected, fixed tests and type check
* Added unit tests
* Changed AlertExecutionStatusErrorReasons to be enum
* Uppercase the enum
* Replaced string values to enum
* Fixed types
* Extended AlertsClient with getHealth method
* added return type to healthStatus$
* Added configurable health check interval and timestamps
* Extended update core status interval to 5mins
* Fixed failing tests
* Registered alerts config
* Fixed date for ok health state
* fixed jest test
* fixed task state
* Fixed due to comments, moved getHealth to a plugin level
* fixed type checks
* Added sorting to the latest Ok state last update
* adjusted error queries
* Fixed jest tests
* removed unused
* fixed type check
* Closes#77695.
Adds new configuration 'xpack.apm.ui.maxServiceEnvironments' to set the
max number of service environments visible in APM UI.
* renamed config 'xpack.apm.ui.maxServiceEnvironments' -> 'xpack.apm.maxServiceEnvironments'
* Renames 'xpack.apm.ui.maxServiceEnvironments' -> 'xpack.apm.maxServiceEnvironments' in the docs.
* removed incorrect size param on the composite terms sub-agg
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
* Function rename to follow existing get/fetch convention
```diff
- export async function loadRegistryPackage(
+ export async function getRegistryPackage(
```
* Moved `unpackRegistryPackageToCache` into `getRegistryPackage`
* In my opinion, those three/four statements are more clear than the previous name
* It isn't used anywhere else
## Summary
This PR addresses https://github.com/elastic/kibana/issues/63865 . Please read the issue for more detail, but essentially, stale data on the tables and use of relative date format leads to confusion as to whether the table was auto refreshing or not.
* Fixes apm e2e runner script by updating the yarn commands to match the
script names defined in the package.json
* removes the yarn install step for cypress dependencies, since it's no
longer necessary.
* Removed apm/e2e/package.json
* simplified paths for binary dependencies
## Summary
Moving the memory store functions to `archive/cache.ts` to better express their role.
They are archive-related functions. The registry is just one possible source of an archive/assets. Also considered moving to `assets`, but can always come back to that.
`deletePackageCache` undoes side-effects from `unpackArchiveToCache` so put them in the same file
## Summary
Basic cut-and-paste of `parseAndVerify*` functions from `archive/index.ts` to `archive/validation.ts`.
Should be easier to mock now, replace later, etc.
* Update renderer handlers when element is changed
* Update handlers before render
* Add canvas functional test for filters
* Update snapshot and remove log
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Update `unpackRegistryPackageToCache` to call `unpackArchiveToCache` instead of duplicating much of it.
Now an archive is iterated & put in cache via the same function regardless of its initial source.
* [Visualizations] New vis wizard
* Update functional tests
* Create oss plugins for maps and lens and unregister alias function
* Add new plugins to .i18nrc.json
* Add readme and codeowners to the new plugins
* update docs
* fix tests
* fix types
* fixes
* Update development docs
* fix oss functional tests
* Fix jest and x-pack functional tests
* Fix functional test
* changes on the layout
* Cleanup and responsiveness
* cleanup unecessary code
* add common folder to the new OSS plugins
* remove unecessary translations
* Update limits.yml file
* Fix basic label
* Add experimental badge on controls vis
* Nice improvements
* fixes
* Improving styles
* Making modal go full height on smaller screens
* Fixing sass lint warning
* fix lint error
* fix internationalization error
* PR fixes
* PR changes
* Use useCallback where possible
* Remove translations that need to be translated again
* Lazy Load wizard modal
* Remove legacyMapVisualizationWarning
* Import the OSS plugins constants from the plugins
* Export constant from lensOss
* Change the new oss plugins from OSS to Oss
* Add a new line to the kibana.json files of the new plugins
* New nit fix
* Fix spaces
* Change the texts for the first step of the modal
* Fix test
* Fixes some of the PR comments
* Add onClick funtionality to the entire aggregation based card
* Cards description changes, introduce a copyFromRoot method to solve the problem of when disabling the x-pack plugic, to also disable the oss
* Create new FTR for testing the functionality of the wizard when both maps and lens apps are disabled
* fix eslint error
* Change groupTitles and descriptions
* Change input vis description
* Remove the copyFromRoot from the signature of the ConfigDeprecationFactory and export it from the main entrypoint
* Make the disabled cards badge clickable
* Changes from code review
* Fix functional tests failures
* Rename groupTitle to titleInWizard to be more specific
* Change vega vis note
* minor design changes
* fix problem with plugins list docs
* Retrieve maps and lens landing page from docs service and add tracking url param
* Fix funtional test for the new dashboard flow
* Fix logic in alias registry for removing the discardOnRegister alias
* no need to remove the alias entry from the discardOnRegister array
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: miukimiu <elizabet.oliveira@elastic.co>
## Summary
Fixes: https://github.com/elastic/kibana/issues/82148
We have errors and do not generate a signal when a source index already has utilized and reserved the "signal" field for their own data purposes. This fix is a bit tricky and has one medium sized risk which is we also support "signals generated on top of existing signals". Therefore we have to be careful and do a small runtime detection of the "data shape" of the signal's data type. If it looks like the user is using the "signal" field within their mapping instead of us, we move the customer's signal into "original_signal" inside our "signal" structure we create when we copy their data set when creating a signal.
To help mitigate the risks associated with this critical bug with regards to breaking signals on top of signals I have:
* This adds unit tests
* This adds end to end tests for testing generating signals including signals on signals to help mitigate risk
The key test for this shape in the PR are in the file:
```
detection_engine/signals/build_event_type_signal.ts
```
like so:
```ts
export const isEventTypeSignal = (doc: BaseSignalHit): boolean => {
return doc._source.signal?.rule?.id != null && typeof doc._source.signal?.rule?.id === 'string';
};
```
Example of what happens when it does a "move" of an existing numeric signal keyword type:
```ts
# This causes a clash with us using the name signal as a numeric.
PUT clashing-index/_doc/1
{
"@timestamp": "2020-10-28T05:08:53.000Z",
"signal": 1
}
```
Before, this was an error. With this PR it now will restructure this data like so when creating a signal along with additional signal ancestor information, meta data. I omitted some of the data from the output signal for this example.
```ts
{
... Other data copied ...
"signal":
{
"original_signal": 1 <--- We "move it" here now
"parents":
[
{
"id": "BhbXBmkBR346wHgn4PeZ",
"type": "event",
"index": "your-index-name",
"depth": 0
},
],
"ancestors":
[
{
"id": "BhbXBmkBR346wHgn4PeZ",
"type": "event",
"index": "your-index-name",
"depth": 0
},
],
"status": "open",
"depth": 1,
"parent":
{
"id": "BhbXBmkBR346wHgn4PeZ",
type: "event",
"index": "your-index-name",
"depth": 0
},
"original_time": "2019-02-19T17:40:03.790Z",
"original_event":
{
"action": "socket_closed",
"dataset": "socket",
"kind": "event",
"module": "system"
},
}
```
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* Add ILM url generator and use in IM for cross linking to policy edit page
* Fix policy name in the link
* Add review suggestions
* Fix import
* Fix eslint error
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>