* initial implementation of configurable test users
* user superuser by default to match master
* referenced the configs in reporting and api integration
* setting the minimum number of default roles
* looking for x-pack tests with users and roles
* add testUserService in dashboard mode tests
* running only ciGroup7
* uncommenting - addign visualization
* re-enabling all CI groups to run on CI
* reinstating Jenkinsfile
* disable Test user for OIDC config
* improved logging and added Roles for OSS tests to get better info on the runs.
* disable test_user for auth tests
* don't fetch enabledPlugins when testuser disabled
* fix es-lint
* running oss tests with x-pack enabled
* [revertme] build default dist for oss tests
* updating NOTICE.txt file as it complained in the kibana intake tests
* changed to pick OSS builds
* trying a license change to trial
* switch back to xpack builds
* created a new sample data role and used it in homepage tests
* revert test/scripts/jenkins_ci_group.sh
* only refresh browser and wait for chrome if we are already on Kibana page
* fix large_string test to use minimum set of roles and privileges
* fix for date nanos custom timestamp with a configured role
* changes to the files with addition of new roles for the test_user
* reverting to OSS changes and few additions to the time_zone test to run as a test_user
* changes to security
* changes to the x-pack test to use elastic superuser
* fix for chart_types test
* fixes to area chart , input control test
* fix for dashboard filtering test and a new config role
* changes to handle the x-pack tests
* additional role for date nanos mixed
* added the logstash role to the accessibility tests
* removed telemetry setting
* docs+few changes to the tests
* removed Page navigation
* removed pageNavigation which was unused
* test/accessibility/apps/management.ts
* update management.ts
* aria label, and other changes
* accidentally checked in a piped file with results.
* accidentally checked in a piped file with results.
* accidentally checked in a piped file with results.
* accidentally checked in a piped file with results.
* accidentally checked in a piped file with results.
* accidentally checked in a piped file with results.
* accidentally checked in a piped file with results.
* accidentally checked in a piped file with results.
* reverted
* unloading of logstash data, fixing aria label
* aria-label
* added the required role
* fix for tsvb chart
* fix for sample data test reverted home_page pageobject file
* changes to sample data test and visualize index file to incorporate OSS changes
* changes to describe() and some more changes to incorporate in settings_page
* re-adding the after()
* removed unwanted roles
* replaced kibana_user with kibana_admin
* added the check of deprecated kibana_user
* testing with kibana_admin role
* fix for discover test
* incorporated the review comments
* incorporated the review comments
* incorporate review comments and added restoreDefaults()
* removed describe.only
* reverted the OSS logic change I had here- pulled into seperate PR
* incorporated the review comments
* incorporated review changes
* adding hidden=true to find hidden kibanaChrome
* change field.test.tsx to be same as that of master branch
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Move defaultAppId config param into kibanaLegacy
* Move disableWelcomeScreen config param into Home plugin
* Update api and docs with silent option for renameFromRoot
* Initial role mappings UI
* apply design edits
* address PR feedback
* fix type cast for number field
* Update x-pack/legacy/plugins/security/public/views/management/role_mappings/edit_role_mapping/components/mapping_info_panel/mapping_info_panel.tsx
Co-Authored-By: Joe Portner <5295965+jportner@users.noreply.github.com>
* Cleanup FTR configuration, and handle role mapping 404 errors properly
* align naming of role mappings feature check
* Apply suggestions from code review
Co-Authored-By: Brandon Kobel <brandon.kobel@gmail.com>
* add missing test assertions
* inlining feature check logic
* switch to using snapshot
* use href instead of onClick
* adding delete unit test
* consolidate href building
* unify page load error handling
* simplify initial loading state
* documenting unconditional catch blocks
* use nodes.info instead of transport.request
* Apply suggestions from code review
Co-Authored-By: Brandon Kobel <brandon.kobel@gmail.com>
* move model out of LP into NP
* convert except_field_rule to except_any_rule
* docs, take 1
* update gif
Co-authored-by: Joe Portner <5295965+jportner@users.noreply.github.com>
Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Additional validation for elasticsearch username
If "elastic" user is set in config:
* In dev mode, throws an error
* In prod mode, logs a deprecation warning
* Fix user for functional tests
* Revert last two commits
Revert "Fix user for functional tests" and
"Fix user for plugin functional tests in Jenkinsfile"
* Change elasticsearch creds for test server
Now uses "kibana" user instead of "elastic" user
* Fix plugin API functional tests
* Fix PKI API integration test
* Change log messages, now conditional on `dist: false` not `dev: true`
* Added base folder structure for Newsfeed plugin
* Added base folders for lib and component
* Added newsfeed button to navigation controls on the right side
* add getApi() to return api data observable (#49581)
* Added flyout base body and provided EuiHeaderAlert component inside the newsfeed plugin
* Moved newsfeed plugin to OSS and added for the styles purpose new folder for legacy plugin 'newsfeed' with the same id to support this
* Added subscribe on fetch newsfeed change
* Add NewsfeedApiDriver class (#49710)
* add NewsfeedApiDriver class
* fix xpack prefix
* add corner case handling
* Added data binding to the ui
* added EuiHeaderAlert style overrides (#49739)
* Fixed due to comments on PR
* add missing fields to NewsfeedItem and FetchResult
* fix templating of service url
* gracefully handle temporary request failure
* Mapped missing fields for data and badge
* Fixed typos issues
* integrate i18n.getLocale()
* allow service url root to be changed in dev mode
* replace a lot of consts with config
* fix flyout height (#49809)
* Add "error" field to FetchResult: Error | null
* simplify fetch error handling
* Do not store hash for items that are filtered out
* add expireOn in case it is useful to UI
* always use staging url for dev config
* unit test for newsfeed api driver
* simplify modelItems
* Fixed eslint errors
* Fixed label translations
* Add unit test for concatenating the stored hashes with the new
* add newsfeed to i18n.json
* Fixed expression error
* --wip-- [skip ci]
* fix parse error
* fix test
* test(newsfeed): Added testing endpoint which simulates the Elastic Newsfeed for consumption in functional tests
* add tests for getApi()
* add tests for getApi
* Added no news page
* fix fetch not happening after page refresh with sessionStorage primed
* test(newsfeed): Added testing endpoint which simulates the Elastic Newsfeed for consumption in functional tests
* Added loading screen
* Small fixes due to comments
* Fixed issue with stop fetching news on error catch
* test(newsfeed): Configure FTS to point newsfeed to the simulated newsfeed endpoit
* Fixed browser error message: Invariant Violation: [React Intl] Could not find required `intl` object. <IntlProvider> needs to exist in the component ancestry.
* Fixed typo issue in label name
* polish the code changes
* Add simple jest/enzyme tests for the components
* honor utc format
* Filter pre-published items
* Fall back to en
* retry tests
* comment clarfication
* Setup newsfeed service fixture from test/common/config
* Added base functional tests for newsfeed functionality
* valid urlroot is for prod
* add documentation for the supported enabled setting
* more urlRoot
* --wip-- [skip ci]
* add the before for fn
* add ui_capabilties test
* update jest snapshot
* Fixed failing test
* finish newsfeed error functional test
* include ui_capability config
* error case testing in ci group 6
* refactor(newsfeed): moved newsfeed api call so that it is done before its use
* code polish
* enabled newsfeed_err test in CI
* initial push
* self code review
* ignore node-fetch type
* usageFetcher api
* user agent metric
* telemetry plugin collector
* remove extra unused method
* remove unused import
* type check
* fix collections tests
* pass kfetch as dep
* add ui metrics integration test for user agent
* dont start ui metrics when not authenticated
* user agent count always 1
* fix broken ui-metric integration tests
* try using config.get
* avoid fetching configs if sending
* type unknown -> string
* check if fetcher is causing the issue
* disable ui_metric from functional tests
* enable ui_metric back again
* ignore keyword above 256
* check requesting app first
* clean up after all the debugging :)
* fix tests
* always return 200 for ui metric reporting
* remove boom import
* logout after removing role/user
* undo some changes in tests
* inside try catch
* prevent potential race conditions in priorities with =
* use snake_case for telemetry plugin collection
* usageFetcher -> sendUsageFrom
* more replacements
* remove extra unused route
* config() -> config
* Update src/legacy/core_plugins/telemetry/index.ts
Co-Authored-By: Mike Côté <mikecote@users.noreply.github.com>
* Update src/legacy/core_plugins/ui_metric/server/routes/api/ui_metric.ts
Co-Authored-By: Mike Côté <mikecote@users.noreply.github.com>
* config() -> config
* fix SO update logic given the current changes
* fix opt in check
* triple check
* check for non boolean
* take into account older settings
* import TelemetryOptInProvider
* update test case
* [dev-utils] implement basic KbnClient util for talking to Kibana server
* update KbnClient to expose full KibanaServerService API
* expose request() function and uriencode helper
* [uiSettings] retry read on conflicts auto upgrading
* expose function for resolving a Kibana server url
* only use apis in test hooks
* run x-pack-ciGroup2 60 times
* log retries as errors so they are included in console output for job
* bump
* Revert "run x-pack-ciGroup2 60 times"
This reverts commit 6b6f392edf.
* refactor urlencode tag to be a little clearer
* support customizing maxAttempts in request method
* [ftr/savedObjects] add simple saved object api client to ftr services
* fix typo
* use consistent spacing
* fix types and TS-ify the rest of kibanaServer service
* expose server urls with better API
* tweak status api response types
* fix http body param name
* second arg to axios post is the post data
* use standardized error handling behavior
* Revert "use standardized error handling behavior"
This reverts commit 7e9a7f8dc5.
* revert unnecessary changes
* [services/es_archiver] retry uiSettings update
* run x-pack-ciGroup7 30x times
* Revert "run x-pack-ciGroup7 30x times"
This reverts commit 80e199c3aa.
* [saved_object_api_integration/common/services] add retry service to the set
* add retry service for x-pack api tests
In https://github.com/elastic/kibana/pull/31234 there were some extra changes that I've reverted, like use of the `tsconfig-paths` package to magically rewrite import statements to defy the standard node module resolution algorithm, the inclusion of several unnecessary options in the `test/tsconfig.json` file, and changes of the line-endings in the config files. This also brings a few enhancements from https://github.com/elastic/kibana/pull/30190 including a modularized version of the expect.js types, and options for explicit mappings for the PageObjects and services used in ftr tests.
As suggested by @legrego in #28792, this PR adds a config flag to completely disable the welcome screen, which is enabled by the functional test runner to ensure that the welcome screen doesn't pop up.
Right now the functional tests are run against a distributable of Kibana in CI, but that distributable is running with `--env.name=development`. That causes the optimizer to run again before the tests can start and prevents the functional tests from running against the actual version of the application users will end up getting. This seems necessary for some tests, but not all of them, but I would like to get all of the tests running against the production version of Kibana soon.
This PR implements a second ftr config, `test/functional_production` that uses basically a copy of the `test/functional` config but with a few minor adjustments, removing the `--env.name=development` kbnServerArg and using a unique junit report name. To accomplish this I needed to modify the `@kbn/test` module to only pass the `--dev` flag to the Kibana server if it is being run in development mode, which it currently does by testing the args for `--env.name=development` or two args next to each other: `'--env.name'` and `'development'`. It does this by converting the `extraKbnOpts` option into an `addExtraKbnArgs` function, which is called with the final args just before passing them to the proc runner and given a chance to modify then after all other args are resolved (pulling from different places in config based on the build type, etc.)
Over the next couple weeks I'll push up PRs for individual test suites, migrating them over to the new production config, hopefully in a short period of time we will have all the function test suites back under that `test/function` config and can run them all against the Kibana server in production mode.
This PR implements the `uiSettings.overrides` setting which [when stored in kibana.yml or passed as config args when starting Kibana] allows forcing some uiSettings to always have a specific value. This setting accepts a map of uiSetting keys to values that will always be used to override whatever is stored in the config saved object.
When users view the settings in the advanced settings UI they are disabled and describe why they can't be changed.
Attempting to change these values from the uiSettings client/service/api is also prevented, causing a 400 error to be thrown and/or sent as the response.
We currently use the `retry` service to call a function over and over in a loop, waiting for it to run without throwing an error, and ultimately failing if it does not succeed before a timeout is exceeded. This is the easiest way to get certain interactions to work, either because we don't know when we should be able to execute the interaction successfully, or because the timing is just too tricky to plan out correctly. Another place where we are using the `retry` service, which I don't think is appropriate, is when we need to wait for a certain condition to be met. This is where `retry.waitFor()` comes in:
```js
await retry.waitFor('dashboard search to be enabled', async () => {
const searchInput = await testSubjects.find('savedObjectFinderSearchInput');
return await searchInput.isEnabled();
})
```
The `retry.waitFor()` method behaves much like the `retry.try()` method behind the scenes, calling a function over and over, but instead of waiting for it to run without throwing an error it waits for it to return a "truthy" value. It also requires a description string that is used to make rather nice log output and a more descriptive error message than something like https://github.com/elastic/kibana/blob/master/test/functional/apps/dashboard/_data_shared_attributes.js#L61-L67
* [kbn-test] convert kibana-install-dir flag to installDir option
* [kbn-test] replicate kibana-install-dir handling to startServers
* [ftr] try running functional tests in production in CI
* Revert "[ftr] try running functional tests in production in CI"
This reverts commit e5b94aa024.
* [core/public/legacyPlatform] exclude ui/test_harness from the distributable
* [optimizer] fix `process.env.IS_KIBANA_DISTRIBUTABLE` definition
* [optimizer] only define `process.env.IS_KIBANA_DISTRIBUTABLE` when needed
Adding a `webpack.DefinePlugin` slows down the optimizer a small amount,
so only apply it when it is necessary, and skip it if it is going to
be defined as "false".
* [kbn-test/startServer] don't run in --dev mode if running from dist
* [ftr/kibanaServer/version] attach `-SNAPSHOT` suffix to version if running build_snapshot
* Replace test:api with @kbn/test runTests
* Improve CLI help menu 🆘
* Use --es-from
* Replace jenkins:selenium with kbn-test
* Validate cli args, fixing test in the process
* Clean up some stuff
* Code review fixes
* Explanation for collectCliArgs
* Remove exit codes, they're useless anyway.
* Make markdown vis test pass with dev_mode setting
* Tests
* Remove unneeded export
* Code review: move console logging up to cli.js
* Code review: refactor startServers and runTests to take single options collection
* Code review: Remove all things I am sure we do not use
* Improve tests
* Code review fixes
* Pass created log to runFtr, runElasticsearch, runKibanaServer
* Update --es-from option to --esFrom
In order to make the license that applies to each file as clear as possible, and to be consistent with elasticsearch, we are adding Apache 2.0 license headers to the top of each file.
Existence of this header is enforced by eslint and tslint and missing headers were automatically added in the last commit by running:
```
node scripts/eslint --fix && node scripts/tslint --fix
```
* Revert "[DOCS] Removes redundant index.asciidoc files (#19192)"
This reverts commit d11b5aae9a.
* Revert "[typescript] add typescript support for the server and browser (#19104)"
This reverts commit c6112067fc.
* Revert "Option to run kibana from build for CI (#19125)"
This reverts commit 5969860303.
Restructure testing with kbn-test package
- Run with multiple configs, move cli options to config
- Package-ify kbn-test
- Eventually we'll have functional_test_runner live in a package
of its own, and then this kbn-test will use that as a dependency,
probably still as a devDependency.
- Implement functional_tests_server
- Collapse single and multiple config apis into one command
Use kbn-es
Replace es_test_cluster + es_test_config with kbn/test utils
Implement new createEsTestCluster
Improve scripts, jsdocs, cli top-level tools
Lift error handling to the top level
