* plugged Task Manager lifecycle into status reactively
* fixed tests
* Revert "fixed tests"
This reverts commit e9f2cd05bd.
* made action group fields optional
* revert deletion
* again
* extracted action type for mto its own component
* extracted more sections of the action form to their own components
* updated icon
* added docs
* fixed always firing alert
* fixed export of components
* fixed react warning
* Adding flag for notifying on state change
* Updating logic in task runner
* Starting to update tests
* Adding tests
* Fixing types check
* Tests and types
* Tests
* Tests
* Tests
* Tests
* Tests
* Renaming field to a more descriptive name. Adding migrations
* Renaming field to a more descriptive name. Adding migrations
* Fixing tests
* Type check and tests
* Moving schedule and notify interval to bottom of flyout. Implementing dropdown from mockup in new component
* Changing boolean flag to enum type and updating in triggers_actions_ui
* Changing boolean flag to enum type and updating in alerts plugin
* Fixing types check
* Fixing monitoring jest tests
* Changing last references to old variable names
* Moving form inputs back to the top
* Renaming to alert_notify_when
* Updating functional tests
* Adding new functional test for notifyWhen onActionGroupChange
* Updating wording
* Incorporating action subgroups into logic
* PR fixes
* Updating functional test
* Fixing types check
* Changing default throttle interval to hour
* Fixing types check
Co-authored-by: Gidi Meir Morris <github@gidi.io>
* Added accessibility test for ingest node pipelines empty state.
* Added accessibility test for detail view and list view and added helpers to keep the test class clean.
* Added the rest of the tests for a11y for ingest node pipelines and updated some test subjects to aid the testing.
* Fixed the spacing issues.
* Update config.ts
Uncommitted other test files.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Reporting/PDF] Custom layout option for Canvas
* fix snapshots
* --wip-- [skip ci]
* check pdf data
* add test
* functional tests work
* add fixme comment
* read strings from pdf for test
* Update reports.ts
* function name / comment improvment
* Add Canvas toggle to choose pdf layout type
* Fix Canvas pdf panel storybook test
* Update style for new Canvas report type switch
* Update canvas share menu snapshot
* Fix tests for validating Canvas PDF using inline snapshots
Run test server with:
node scripts/functional_tests_server.js --config x-pack/test/functional/config.js
Run test suite with:
node scripts/functional_test_runner.js --config x-pack/test/functional/config.js --grep 'Canvas PDF Report'
* Fix i18n and typo
* Add a test for removing borders
* Fix i18n
* Update snapshot
Co-authored-by: Timothy Sullivan <tsullivan@elastic.co>
Co-authored-by: Tim Sullivan <tsullivan@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* WIP: basic reindexing works, lots of edge cases and TODOs to tackle
* Add note
* Add version metadata to signals documents
* WIP: Starting over from the ground up
* Removes obsolete endpoints/functions
* Adds endpoint for checking the migration status of signals indices
* Adds helper functions to represent the logical pieces of answering
that question
* Fleshing out upgrade of signals
* triggers reindex for each index
* starts implementing followup endpoint to "finalize" after reindexing
is finished
* Fleshing out more of the upgrade path
Still moving logic around a bunch.
* Pad the version number of our destination migration index
Instead of e.g. `.siem-signals-default-000001-r5`, this will generate
`.siem-signals-default-000001-r000005`.
This shouldn't matter much, but it may make it easier for users at a
glance to see the story of each index.
* Fleshing out more upgrade finalization
* Verifies that task matches the specified parameters
* Verifies that document counts are the same
* updates aliases
* finalization endpoint requires both source/dest indexes since we can't
determine that from the task itself.
* Ensure that new signals are generated with an appropriate schema_version
* Apply migration cleanup policy to obsolete signals indexes
After upgrading a particular signals index, we're left with both the old
and new copies of the index. While the former is unlinked, it's still
taking up disk space; this ensures that it will eventually be deleted,
but gives users enough time to recover data if necessary.
This also ensures that, as with the normal signals ILM policy, it is
present during our normal sanity checks.
* Move more logic into component functions
* Fix type errors
* Refactor to make things a little more organized
* Moves migration-related routes under signals/ to match their routing
* Generalizes migration-agnostic helpers, moves them to appropriate
folders (namely index/)
* Inlined getMigrationStatusInRange, a hyper-specific function with
limited utility elsewhere
* Add some JSDoc comments around our new functions
This is as much to get my thoughts in order as it is for posterity.
Next: tests!
* Adds integration tests around migration status route
* Adds io-ts schema for route params
* Adds es_archiver data to represent an outdated signals index
* Adds API integration tests for our signals upgrade endpoint
* Adds io-ts schema for route params
* Adds second signals index archive, updates docs
* Adds test helper to wait for a given index to have documents
* Adds test helper to retrieve the relevant index name from a call to
esArchive.load
* WIP: Fleshing out finalization tests
* Consolidate terminalogy around a migration
We're no longer making a distinction between an upgrade vs. an update
vs. a migration vs. a reindex: a migration is the concept that
encompasses this work. Both an index and individual documents can
require a migration, but both follow the same code path to migrate.
* Implement encoding of migration details
This will be a slightly better API: rather than having to pass all three
fields to finalize the migration, API users can instead send the token.
* Better transformation of errors thrown from the elasticsearch client
These often contain detailed information that we were previously
dropping. This will give better info on the migration finalization
endpoint, but should give more information across all detection_engine
endpoints in the case of an es client error.
* Finishing integration tests around finalization endpoint
This lead to a few changes in the responses from our different
endpoints; mainly, we pass both the migration token AND its constituent
parts to aid in debugging.
* Test an error case due to a reindexing failure
This would be really hard to reproduce with an integration test since
we'd need to generate a specific reindex failure. Much easier to stub
some ES calls to exercise that code in a unit test.
* Remove unnecessary version info from signals documents
We now record a single document-level version field. This represents the
version of the document's _source, which is generated by our rule
execution.
When either a mapping _or_ a transformation is added, this version will
be bumped such that new signals will contain the newest version, while
the index itself may still contain the old mappings.
The transformation pipeline will use the signal version to short-circuit
unnecessary transformations.
* Migrate an index relative to the ACTUAL template version
This handles the case where a user is attempting to migrate, but has not
yet rolled over to the newest template. Running rules may insert "new"
signals into an "old" index, but from the perspective of the app no
migration is necessary in that case.
If/when they roll over, the aforementioned index (and possibly older
ones) will be qualified as outdated, and can be migrated.
* Enrich our migration_status endpoint with an is_outdated qualification
This can be determined programatically, but for users manually
interpreting this response, the qualification will help.
* Update migration scripts
* More uniform version checking
* getIndexVersion always returns a number
* version comparisons use isOutdated
* Fix signal generation unit tests
We now generate a version field to indicate the version under which the
signal was created/migrated.
* Support reindex options to be sent to create_migration endpoint
Rather than having to perform a manual reindex, this should give API
users some control over the performance of their automated migration.
* Fix signal generation integration tests
These were failing on our new signal field.
* Add unit tests for getMigrationStatus
* Add a basic test for getSignalsIndicesInRange
Since this is ultimately just an aggregation query there's not much else
to test.
* Add unit test for the naming of our destination migration index
* Handle write indices in our migration logic
* Treat write indices as any other index in migration status endpoint
* Migration API rejects requests containing write indices
* Migration API rejects requests containing unknown/non-signals indices
* Add original hot phase to migration cleanup policy
Without this phase, ILM gets confused as it tries to move to the delete
phase and fails.
* Update old comment
The referenced field has changed.
* Delete task document as part of finalization
* Accurately report recoverable errors on create_signals_migration route
If we have a recoverable error: e.g. the destination index already
exists, or a specified index is a write index, we now report those
errors as part of the normal 200 response as these do not preclude other
specified indices from being migrated.
However, if non-signals indices are specified, we do continue to reject
the entire request, as that's indicative of misuse of the endpoint.
* initial commit
refactored multi-index, multi-pipeline jobs for 7.11. These are new modules that will live alongside the existing jobs.
* Update ml_modules.tsx
added new module names to the list
* Update get_module.ts
added new module names
* Linter fixes
* Order matters
* manifest fixes
added colon char to the module name and shortened the description
* additon to description
after talking with the security team today, adding this suggested text to the beginning of the description so it will tend to be visible to the user:
"This is a new refactored job which works on ECS compatible events across multiple indices."
* Adjust module recognizer test for auditbeat dataset
* influencers
changes to the metadata jobs to make influencers identical to the originals
* change for security app
changes to two datafeeds needed for logic in the Security app - added the suffix "_ecs" to two ids.
Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
Co-authored-by: Robert Oskamp <robert.oskamp@elastic.co>
## Summary
- Move registry URL details from handler to Registry service
- rename `archive/save_to_es` to `archive/storage` to better reflect its role
- Move SO handling from http into new `getAsset` method in `archive/storage`
- Rearrange code in HTTP handler to be more easily moved to one or two service method(s) which deal with the logic for getting the asset.
### Checklist
- [ ] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios
## Summary
1. cf0d567 - Export function which maps file path to saved object id
2. e33d7d4 - "make it work" ™️ quality version of "use cached or saved object or registry" file in HTTP handler. It's doing too much and should eventually move the logic to service method(s).
* Port @tsg's work on task manager.
Remove 2nd var to track telemetry opt in.
Add ES client to start querying index.
Use query to get docs from a dummy index.
Change how index is queried.
Get diagnostic alerts to send to staging cluster.
Record last timestamp.
PoC on telemetry opt in via 2 processes.
Revert to original solution
* Update on agreed method. Fixes race condition.
* Expand wildcards.
* stage.
* Add rule.ruleset collection.
* Update telemetry sender with correct query for loading diag alerts.
* Add similar task tests to endpont artifact work.
* Fix broken import statement.
* Create sender mocks.
* Update test to check for func call.
* Update unused reference.
* record last run.
* Update index.
* fix import
* Fix test.
* test fix.
* Pass unit to time diff calc.
* Tests should pass now hopefully.
* Add additional process fields to allowlist.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This PR introduces a new concept of an _Action Subgroup_ (naming is open for discussion) which can be used by an Alert Type when scheduling actions.
An Action Subgroup can be dynamically specified, unlike Action Groups which have to be specified on the AlertType definition.
When scheduling actions, and AlertType can specify an _Action Subgroup_ along side the scheduled _Action Group_, which denotes that the alert instance falls into some kind of narrower grouping in the action group.
* add settings
* update abab package to version with types
* add test case for CORS
* add tests for cors config
* fix jest tests
* add deprecation message
* tweak deprecation
* make test runable on Cloud
* add docs
* fix type error
* add test to throw on invalid URL
* address comments
* Update src/core/server/http/http_config.test.ts
Co-authored-by: Larry Gregory <lgregorydev@gmail.com>
* Update docs/setup/settings.asciidoc
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
* allow kbn-xsrf headers to be set on CORS request
Co-authored-by: Larry Gregory <lgregorydev@gmail.com>
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
* moved error and loading notices for data allocation field to below description
* removed test code
* expect controls to be showing, only render notice after network request has finished
* added loading spinner for field inputs
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* moved shrink field to shared_fields and added it to the hot phase
* updated test
* update legacy jest test
* removed configuration context for now
* remove unused i18n and remove duplicated isRolloverEnabled check;
* fixed shrink field to use new described field component
* added test for removing shrink field in serialization and re-ordered fields in hot phase advanced
