* CI Reporter for saved objects field count (#70580)
* CI Reporter for saved objects field count
* Metrics needs to be an array
* Fix type failures
* Link to field count issue
* Revert "Link to field count issue"
This reverts commit 8c0126b838.
* Break down field count per type
* Don't log total metric as metrics report already calculates this
* Add saved objects field count ci metrics test to codeowners
* Address review comments
* Add field count CI metrics for disabled plugins
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
# Conflicts:
# .github/CODEOWNERS
* Don't backport CODEOWNERS file to 7.x
* Add functional test for Kibana embedded in iframe (#68544)
* convert kbn test config into TS
* add test for Kibana embedded in iframe
* run embedded tests in functional suite
* ignore tls errors in functional tests by default
* switch test to https
* remove env vars mutation
* allow to pass ssl config to Kibana
* pass ssl config to axios
* adopt KbnClient interfaces
* adopt KibanaServer
* use KbnRequester in security service
* set sameSiteCookies:None in test
* acceptInsecureCerts in chrome
* remove leftovers
* fix type error
* remove unnecessary field
* address comments
* refactor plugin
* refactor test
* make acceptInsecureCerts configurable
* run firefox tests on ci
* up TS version
* fix firefox.sh script
* fix path
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
# Conflicts:
# test/functional/services/remote/remote.ts
# test/functional/services/remote/webdriver.ts
# x-pack/scripts/functional_tests.js
* create browser config. lost during conflict resolution
* rename siem to security_solution
* rename siem to security solution inside of code
* rename translation keys
* fix snapshot
* replace siem for security solution in tutorial
* missing translation to be renamed
* fix types for api test integration
* updates runner file to match the new path
* change category for kibana settings
* miss renaming in advance settings
* fixes cypress tests
* fix api integration test
* fix new translation
* fix unit test
* update translation i18n
* update translation i18n II
Co-authored-by: Gloria Hornero <snootchie.boochies@gmail.com>
Co-authored-by: Gloria Hornero <snootchie.boochies@gmail.com>
* [SIEM][Lists] Adds circular dependency checker for lists plugin
## Summary
* Added dependency checker for the public and common folders for lists
# Conflicts:
# test/scripts/jenkins_xpack.sh
* Updated from conflict issue
* move core provier to NP. allows to run tests on every page
* remove no-base-path
* pass whole KbnTestServer config
* fix build path
* another id
* np
* build kibana tests plugins. it is used in xpack tests
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR migrates the vast majority of Alerting legacy code to the Kibana Platform.
This includes:
1. Removed legacy Task Manager
2. Migrates Fixture plugins in Alerting, Triggers UI and Task Manager Perf
This does not includes:
1. The PagerDuty simulator due to a lack of support for custom responses in the platform. issue opened. https://github.com/elastic/kibana/issues/65045
2. The Webhooks simulator due to a lack of support for custom authorisation. Requires investigation.
* bring testing on Firefox back
* disable some tests
* skip more tests
* cut more suites for Firefox
* skip more tests for Firefox
* replace smoke tag with includeFirefox
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Move server code into NP folder
* NP config is not yet used
* Relative imports are somewhat broken
* Move common folder into NP
* Move cypress folder into NP
* Move scripts folder into NP
* Move misc. config into NP folder
A few of these were moved into the cypress folder as they're
cypress-specific.
I tried to update all the relative paths but some are likely broken. I'm
not going to know until other stuff is fixed, though.
* Move value for siem index pattern into common/constants
The other default values live in there, this is no different.
* Update paths following file move
If this was referencing the full project, it now references both paths
(legacy for UI, and NP for server).
* Fix typescript errors related to module resolution
These are mostly updating imports to the common/ folder on the UI side
(since things changed relative to those files).
* Replace Legacy Config with NP Config
* Updates plugin to use NP config
* defines new config previously coming from savedObjects config
* cleans up legacy types
Conflicts:
x-pack/plugins/siem/server/lib/detection_engine/routes/rules/export_rules_route.ts
x-pack/plugins/siem/server/lib/detection_engine/routes/rules/import_rules_route.ts
x-pack/plugins/siem/server/lib/detection_engine/rules/types.ts
x-pack/plugins/siem/server/plugin.ts
x-pack/plugins/siem/server/routes/index.ts
x-pack/plugins/siem/server/types.ts
* Remove local SIEM tsconfig
This was originally added to address an issue with tsserver, but that
issue is no longer relevant. The presence of this file confuses
typescript into thinking that siem is a separate TS project.
* Update kibana.json to declare our dependencies
These are not necessarily correct in terms of what's required/optional,
but this is what's declared in our types.
* Remove legacy plugin instantiation
* Removes legacy instantiation of server plugin, which is now handled by
NP
* Loosens legacy config spec so we no longer have to duplicate config
types
* Update tests with NP config
These were written against the old Hapi config function; now, we just
have a POJO.
* Update es_archiver helpers' paths
I'm not quite sure if these are working yet, but they're no longer
throwing errors.
* Ignore restricted path on script
This was cribbed from infra, who has made a similar change.
* Ignore restricted path on temporary savedObject mappings import
This will be changed subsequently when we switch to the NP form of
savedObject type registration.
* Add symlink to lockfile
* Fix paths on circular deps script
* Add separate config for Rule and Timeline saved objects
We had previously used the savedObjects' config, but those are not
currently exposed to us on New Platform. For now, we're going to split
this into two sets of values for the SOs we deal with
importing/exporting within the SIEM app, with the same defaults as
savedObjects.
* Fixing relative paths within cypress
These are strings that wouldn't be caught by typescript.
* Migrated sample_action to NP. Panel action tests returned to the test flow.
* fixed names
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [ci/es] always pull snapshots, break early when they're not available
* move to setup.sh after deps are installed
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: spalger <spalger@users.noreply.github.com>
Adds Navigation APIs to Alerting.
Parts to this PR:
Adds a client side (Public) plugin to Alerting, including two APIs: registerNavigation & registerDefaultNavigation. These allow a plugin to register navigation handlers for any alerts which it is the consumer of- one for specific AlertTypes and one for a default handler for all AlertTypes created by the plugin.
The Alert Details page now uses these navigation handlers for the View In App button. If there's an AlertType specific handler it uses that, otherwise it uses a default one and if the consumer has not registered a handler - it remains disabled.
A generic Alerting Example plugin that demonstrates usage of these APIs including two AlertTypes - one that always fires, and another that checks how many people are in Outer Space and allows you to trigger based on that. 😉 To enable the plugin run yarn start --ssl --run-examples
* run jest with `--detectOpenHandles` on CI to figure out what is happening with pauses
* focus tests on jest integration
* force kill child processes in config reload test
* skip flaky suite
* increase timeout for looking for installed packages
* run all tests again
* Refactor test entry by runner (#44679)
Over the last few years we've increased the number of test runners.
Entry points by test type have become too lage of a category for unique
names, so this moves top level test scripts under yarn and replaces test
types with the runner name.
e.g. `yarn test:browser` -> `yarn test:karma`
Closes#41133
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* fix merge
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* build immutable bundles for new platform plugins (#53976)
* build immutable bundles for new platform plugins
* only inspect workers if configured to do so
* [navigation] use an index.scss file
* add yarn.lock symlink
* set pluginScanDirs in test so fixtures stay consistent
* cleanup helpers a little
* fix type error
* support KBN_OPTIMIZER_MAX_WORKERS for limiting workers via env
* test support for KBN_OPTIMIZER_MAX_WORKERS
* expand the available memory for workers when only running one or two
* add docs about KBN_OPTIMIZER_MAX_WORKERS environment variable
* fix README link
* update kbn/pm dist
* implement bundle caching/reuse
* update kbn/pm dist
* don't check for cache if --no-cache is passed
* update renovate config
* standardize on index.scss, move console styles over
* add support for --no-cache to cli
* include worker config vars in optimizer version
* ignore concatenated modules
* update integration test
* add safari to browserslist to avoid user-agent warnings in dev
* update docs, clean up optimizer message/misc naming
* always handle initialized messages, don't ignore states that are attached to specific events
* reword caching docs, add environment var to disable caching
* tweak logging and don't use optimizer.useBundleCache as that's disabled in dev
* handle change notifications
* batch changes for 1 second
* rename CompilerState type to CompilerMsg
* getChanges() no longer needs to assign changes to dirs
* remove unused deps
* split up run_worker.ts and share cacheKey generation logic
* add a couple docs
* update tests and remove unused imports
* specify files when creating bundle cache key
* remove one more unused import
* match existing dev cli output more closely
* update kbn/pm dist
* set KBN_NP_PLUGINS_BUILT to avoid warning in CI
* avoid extending global window type
* add note to keep pluginScanDirs in sync
* pass browserslistEnv in workerConfig so it is used for cache key
* load commons.bundle.js in parallel too
* emit initialized+success states if all bundles are cached
* load bootstraps as quickly as possible
* skip flaky suite
* bump
* update jest snapshots
* remove hashing from cache key generation
* remove unnecessary non-null assertion
* improve docs and break up Optimizer#run()
* remove unused import
* refactor kbn/optimizer to break up observable logic, implement more helpful cache invalidation logic with logging
* fix tests
* add initializing phase
* avoid rxjs observable constructor
* remove unnecessary rxjs helper, add tests for bundle cache
* update consumers of optimizer
* update readme with new call style
* replace "new platform" with "kibana platform"
* fix a couple more renames
* add support for several plain-text file formats
* fix naming of OptimizerMsg => OptimizerUpdate, use "store" naming too
* one more OptimizerMsg update
* ensure bundles are not cached when cache config is false
* test for initializing states and bundle cache events
* remove unnecessary timeout change
* Remove unnecessary helpers
* Add tests for BundleCache class
* Add tests for Bundle class
* test summarizeEvent$
* missing paths are no longer listed in mtimes map
* add tests for optimizer/cache_keys
* Add some extra docs
* Remove labeled loop
* add integration test for kbn-optimizer watcher components
* querystring-browser removed
* tweak logging a smidge, improve info and final message
* remove unused imports
* remove duplication of getModuleCount() method
* move type annotation that validates things
* clear up the build completion message
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [kbn/optimizer] Fix windows support (#57592)
* [kbn/optimizer] simplify run_workers.ts a smidge
* use Path.resolve() to create windows paths from normalized ones
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit 343bc9c303)
* remove istanbul/code coverage references
* fix webpack config syntax
* removal of querystring-browser was backported to 7.x
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Allocate multiple flaky test runner agents as executions grow
* WIP support for deleting kibana install dir during CI
* Add setup script for testing scripts
* Add REMOVE_KIBANA_INSTALL_DIR=1 to flaky test runner
* Change flaky test runner worker processes from 8 to 12
* Add a new platform embeddable example plugin
* Remove extra hello world test impl.
* cleanup
* code review updates
* Change example to highlight and have parent filter out children
* Fix deep comparison of embeddable prop
* adjust help text
* run mocha tests from x-pack with root mocha script
* Only run Karma tests in xpack intake job
* disable failing suites
* fix typo
* skip correct suite (there are multiple root suites)
* support disabling junit reporting with $DISABLE_JUNIT_REPORTER
* don't generate junit in ispec_plugin tests
# Conflicts:
# x-pack/tasks/test.ts
* [a11y] add initial accessibility functional tests
* add accessibility jobs
* fix config path
* remove percy setup from scripts
* disable color-contrast rule
* apply changes from @myasonik
* define aria-controls/owns props even when suggestions aren't visible
* [ftr/a11y] only throw error when there are errors
* adding tests for management page
* add a11y test for management page
* adding ignore rules' to a11y
* accessibility test for kibana home
* 7 passing tests, 0 failures
* jest snapshot update
* support a11y test in pipeline job
* update a11y test script for pipelines
* use oss compatible ci setup
* fix exclude syntax
* add default exclusion syntax
* Revert "Revert "Add pipeline for flaky test runner job (#46740)""
This reverts commit 7d96a13fad.
Also reconcile changes to Jenkinsfile since original revert happened
* Fix param parsing and add missed change
* Add missing variable
* refactor failed_tests_reporter to use TS, no octokit
* update renovate config
* ensure that all kbn-test files are in ts project
* fix some type errors
* add some more tests
* [kbn-test/githubapi] cleanup and document
* collect log messages as strings instead of message objects
* ensure issue is open when updating body
* improve readability of getKibanaIssues
* expose axios helpers from dev-utils
* fix request params for fetching github issues and validate locally
* include a README for failed_tests_reporter
* improve axios error helpers
# Conflicts:
# packages/kbn-test/src/index.ts
# renovate.json5
* Re-split ciGroups after pipeline rollout
Revert "Revert "Revert "Revert "Revert "[ci] compress jobs for CI stab… (#45454)"
This reverts commit 9a109f2170.
Revert "set IS_PIPELINE_JOB in intake jobs (#45850)"
This reverts commit b1a01effa8.
* Split one of the slow test suites up to try to make overall CI faster
* Disable visualRegression groups, they are being handled in other work
* Revert "Split one of the slow test suites up to try to make overall CI faster"
This reverts commit 1213239545.
* Move some different xpack ciGroup8 suites around
* Revert hiding Kibana icon, test Percy rendering
* Update jobs.yml
* Use percy token for kibana dev
* Update Percy token to read from kibana-issues
* Read correct field
* increase timeout
* Run each visual job 10 times
* Revert last commit
* Increase timeout
* Disable visual tests and revert percy token
* Fix jest tests
* Another jest test fix
* set IS_PIPELINE_JOB in intake jobs
* firefox smoke doesn't get an initial build
* firefox smoke doesn't get an initial build (oss edition)
* re-enable all jobs
* Revert "Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"""
This reverts commit 148b8c0f90.
* sync changes with Jenkinsfile
(cherry picked from commit 9a109f2170)
* Pipeline
* WIP some work for parallelization with ciGroups
* Fix xpack kibana install dir, and add some debugging
* Attempt to quick fix a few tests
* Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)""
This reverts commit 078ac2897f.
* Recombine test groups, and try runbld again
* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline
* Fix typo in shell script
* Remove some debug code
* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT
* Fix test that uses hard-coded es transport port and add it back in to parallel groups
* Disable checks reporter again for now
* Set env var for TEST_ES_TRANSPORT_PORT in pipeline
* Update Jenkinsfile for shorter testrunner labels
* Fix another hard-coded transport port
* Fix a new test with hard-coded URLs
* Jenkinsfile cleanup and fix one of the groups
* Fix double slash
* Testing vault credentials on jenkins server
* Add a non-existent credential
* Revert "Add a non-existent credential"
This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc.
* Try github-checks-reporter again
* github-checks-reporter should only run for elastic/kibana, forks won't work
* Clean up some debug code
* Changing names around to try to make BlueOcean UI a little better
* Add more stages
* Make some changes to stage structure to mirror a nested example from CloudBees
* Handle TODOs, and some cleanup in Jenkinsfile
* Pass GIT_BRANCH when started without GHPRB, fix branch check
* Fix mailer problem and add code that ensures all tests are in cigroups back in
* Test adding worker/job name to junit report paths
* Remove some duplication from ci_setup scripts
* Fix unit test that uses junit path
* Don't reinstall node every time setup_env is run
* Fix yarn install logic
* Fix another unit test that uses junit output dir
* Download latest ES snapshot after kibana builds
* Make sure junit reports are always processed
* Add two failing tests for testing purposes
* Add support to Jenkinsfile for kibana build e-mails
* Remove some debug code for email sending
* Change JOB env handling in junit paths and move it to a sub-directory
* Revert "Add two failing tests for testing purposes"
This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c.
* Fix junit report path in test
* Don't send kibana emails on build abort
* Address PR feedback, formatting and use built-in url formatting library
* Fix path formatting for functional test
* Add email sending back in to Jenkinsfile
* Fix another unit test with path problem
(cherry picked from commit 27d23c4184)
# Conflicts:
# src/dev/ci_setup/setup.sh
# src/dev/failed_tests/cli.js
* [ftr/percy] integrate percy with functional test runner
* execute ftr within `percy exec`
* share PERCY_BIN and GRUNT_BIN vars in setup
* [license] whitelist `(Unlicense OR Apache-2.0)`
* [percy] include pr/parallel info in env
* echo the actual variable value
* destructure required modules
* [ci] calculate percy env in a script
* remove outdated eslint overrides
* oops, nonce should be the same for each build
* take snapshots in context tests
* add percy snapshots to some dashboard tests
* account for extra execution of PERCY_BIN
* remove usage of percy service
* rename service to `visualTesting`
* write a simple test
* switch left-over percy test to visualTesting
* set log level for percy agent
* trigger ci
* try using the system chrome install instead of downloading one
* fix export definition
* Don't skip chromium download
* Add Spencer's fix to point to chrome installation
* Attempting to split tests into own ci job and test files
* Renumber job
* Add tag to decorations file
* Try new files with existing CI job
* Try again to create a new CI job
* Fix eslint problem with space
* eslint updates
* fix console test
* make test names unique
* Update ci group names
* Try old name again
* save
* add separate visual_regression ci job
* add visual regression ci job
* add new job to jobs.yml
* update path to take_percy_snapshot script
* use fixed percy job total
* fix common services import
* move visual_create_and_add_embeddables to visual_regression project
* finish moving create_and_add_embeddables
* add x-pack visual regression job
* migrate dashboard_snapshots tests to visual_regression
* remove references to removed test files
* restore some unnecessary changes
* reimplement the login page tests
* yarn.lock update
* fix test
* remove old CI_GROUP 100 block
* update failed_tests trap
* reduce yarn.lock changes
* disable debug logging
* disable visual-regression tests for now, we can enable them in specific prs
* skip and adjust functional tests for Firefox
* downgrade geckodriver to 0.22.0
* [ftr] add firefox specific config file
* remove hard coded firefox adjustments
* remove firefox specific screenshot directory
* run functional tests in firefox in x-pack
* pass the logger to readConfigFile()
* rename local var
* skip xpack firefox config
* run xpack firefox functional tests in separate command
* update report name for XPack firefox results
* adjust viz shared item test
* skip shared item test
* [test/feature_controls] increase timeout to 20 sec & use forceLogout
* FF timeout on TSVB, refresh page on failure for url navigation
* [feature_controls/visualize_security] increase timeout to 20 sec
* skip dev_tools_spaces tests on FF
* run all groups 3 times
* skip more func tests
* skip more tests
* skip all feature controls tests for Firefox
* revert back changes in feature controls tests
* rename xpack report for chrome
* skip tile map for FF, wait for render before saving
* Revert "run all groups 3 times"
This reverts commit 12b4f69627.
* revert refresh on login
* apply feedback
* Split ciGroup3 and ciGroup5 (#36928)
* split large x-pack ciGroup5 into 3 groups
* split large x-pack ciGroup3 into 2 groups
* Add groups to the jenkins xpack ci group file
* Re-level some more work
* move es_search_source test to new describe block
* move es_search_source to first in ciGroup7
* move to x-pack-ciGroup6
* Typescript sample panel action
* Update EUI version to match main cabana version
* update yarn.lock
* add back typings include
* use correct relative path
* [ci/es] base default es version on pkg.branch value
* [ci/env] initialize $TEST_ES_FROM in checkout_sibling_es.sh
* [UA] remove TEST_ES_SNAPSHOT_VERSION override
* [ci/es] call checkout_sibling_es.sh from one location
* [ci] remove unused `--from` param
* [ci/env] always default to snapshots, switch to source when necessary
* [kbn/test] default esFrom to $TEST_ES_FROM
* [ci/setup] fix define order
* [ci/grunt] don't pass --esFrom let env pass through
* [ci/env] use branch, not version
* [ci] use same indent style as following lines
* [kbn/test] apply default values when processing args
* [kbn/test] simplify defaults, read default on each process
* Use hosts instead of url for elasticsearch.
* Adds jest integration tests to run in xpack intake.
* Reverting the exclusion of throwing errors in tests.
* Strange test that seems to fail without console error exception being thrown.
* Fix issues with wrong props for EuiInMemoryTable.
* Removes the masking of console errors in tests.
Since this is no longer part of a reporting task in Jenkins, we no
longer need to re-run kbn bootstrap when generating a report.
We use the PR_SOURCE_BRANCH environment variable to prevent genererating
Github issues when we are on a PR.
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* Reporting phantom and chromium tests should run in parallel.
* Chromium tests should be done in group 9.
* Attempting to use group 2 in ci for chromium tests.
* X-pack CI jobs should have 7 groups.
* Phantom tests should be in group 7.
* [ci][ftr][kbn/test] split up CI jobs
* [ci] run --assert-none-excluded in ci groups, before builds
* [ftr] improve error message when excluded tests found
* [ci] document other places the ciGroups live
* chore(19674): added dev flag when calling jenkins:unit grunt task so we can check dev licenses on ours ci jobs.
* chore(NA): update yarn lock files with integrity info.
* Replace xvfb with headless browser
* Update karma to run in headless chrome
Update Karma tests for an option to run chrome in headless.
* Fix lint issues
* Fix spaces
Closes#22840
In the functional tests we want a better way to include/exclude certain tests, especially as we move forward with #22359. This PR allows us to decorate suite objects with "tags", which won't clutter up the test names and can be used to filter out specific tests within a single test config. The functional test runner supports defining `--include-tag` and `--exclude-tag` CLI arguments, and multiple can be defined.
The method of filtering out tests for running against cloud has been updated to use this approach and I plan to do the same to #22359 once this is merged.
For instances where we would otherwise be running from source in a tracked elastic/elasticsearch branch, let's use a snapshot instead. This will eliminate some gradle issues we are experiencing in master.
