Right now the functional tests are run against a distributable of Kibana in CI, but that distributable is running with `--env.name=development`. That causes the optimizer to run again before the tests can start and prevents the functional tests from running against the actual version of the application users will end up getting. This seems necessary for some tests, but not all of them, but I would like to get all of the tests running against the production version of Kibana soon.
This PR implements a second ftr config, `test/functional_production` that uses basically a copy of the `test/functional` config but with a few minor adjustments, removing the `--env.name=development` kbnServerArg and using a unique junit report name. To accomplish this I needed to modify the `@kbn/test` module to only pass the `--dev` flag to the Kibana server if it is being run in development mode, which it currently does by testing the args for `--env.name=development` or two args next to each other: `'--env.name'` and `'development'`. It does this by converting the `extraKbnOpts` option into an `addExtraKbnArgs` function, which is called with the final args just before passing them to the proc runner and given a chance to modify then after all other args are resolved (pulling from different places in config based on the build type, etc.)
Over the next couple weeks I'll push up PRs for individual test suites, migrating them over to the new production config, hopefully in a short period of time we will have all the function test suites back under that `test/function` config and can run them all against the Kibana server in production mode.
* [kbn-test] convert kibana-install-dir flag to installDir option
* [kbn-test] replicate kibana-install-dir handling to startServers
* [ftr] try running functional tests in production in CI
* Revert "[ftr] try running functional tests in production in CI"
This reverts commit e5b94aa024.
* [core/public/legacyPlatform] exclude ui/test_harness from the distributable
* [optimizer] fix `process.env.IS_KIBANA_DISTRIBUTABLE` definition
* [optimizer] only define `process.env.IS_KIBANA_DISTRIBUTABLE` when needed
Adding a `webpack.DefinePlugin` slows down the optimizer a small amount,
so only apply it when it is necessary, and skip it if it is going to
be defined as "false".
* [kbn-test/startServer] don't run in --dev mode if running from dist
* [ftr/kibanaServer/version] attach `-SNAPSHOT` suffix to version if running build_snapshot
In order to make the license that applies to each file as clear as possible, and to be consistent with elasticsearch, we are adding Apache 2.0 license headers to the top of each file.
Existence of this header is enforced by eslint and tslint and missing headers were automatically added in the last commit by running:
```
node scripts/eslint --fix && node scripts/tslint --fix
```
* feat(12976): node version validation at runtime.
* refact(12976): move the code into a static utilities class.
* test(12976): added first test case using jest.
* test(12976): added test cases for node_version.
* feat(12976): create setup env node to bootstrap babel, ts-node and node version validator.
* refact(12976): migrated node version code from es6 to es5.
* feat(12976): node version validation at runtime.
* refact(12976): move the code into a static utilities class.
* test(12976): added first test case using jest.
* test(12976): added test cases for node_version.
* feat(12976): create setup env node to bootstrap babel, ts-node and node version validator.
* refact(12976): migrated node version code from es6 to es5.
* fix(12976): remove one level from ts node register cache directory link.
* chore(12976): added caret to semver dependecy in order to support minor versions.
* refact(12976): small change from named import to default import on node version validator.
* refact(12976): removed ts_node_register and add the code to babel_register.
* feat(12976): split eslint config in order to properly support files built to run before and after node version validator. refact(12976): convert script files to es5 code. refact(12976): delete inline eslint configs from node version check related files.
* refact(12976): remove ts node register file.
* refact(12976): completely port setup_node_env to es5.
* refact(12976): remove babel_register invokation from external dependencies in scripts.
* refact(12976): move node_version code directly into node_version_validator inside setup_node_env folder.
* refact(12976): only node version validator for kbn script.
* [tslint] lint typescript code
* [tslint] filter projects when running specific project
* [dev/ts] use more explicit types
* [dev/ts] add note about why using glob
* [dev/ts] rely on ts, use fewer getters
* [mocha] run tests with mocha script, remove grunt-simple-mocha
* [ci] force colored output
* [ci] disable color when reading the yarn bin
* [dev/build/exec] support tooling log having its own chalk
* [dev/mocha] avoid changing the cwd of the process
* [dev/mocha] only match files, speed up negative matches
* [cli/serve/test] strip ansi control characters from logs
* [ci] disable color in all `yarn bin` invocations
* [dev/mocha] cleanup old runInBand check
* [dev/mocha] enable gloabl leak checks
Restructure testing with kbn-test package
- Run with multiple configs, move cli options to config
- Package-ify kbn-test
- Eventually we'll have functional_test_runner live in a package
of its own, and then this kbn-test will use that as a dependency,
probably still as a devDependency.
- Implement functional_tests_server
- Collapse single and multiple config apis into one command
Use kbn-es
Replace es_test_cluster + es_test_config with kbn/test utils
Implement new createEsTestCluster
Improve scripts, jsdocs, cli top-level tools
Lift error handling to the top level
* [dev/notice] add scripts for generating NOTICE.txt file
* [notice] react-resize-detector@0.6.0 was removed in b445389b79
* [notice] move notice text into relevant source
* [dev/notice] Generate NOTICE.txt file
* [jenkins] verify that notice.txt is up to date in CI
* [tasks/notice] update test to use new NOTICE.txt file
* [dev/notice] update company name in NOTICE.txt
* [notice/cli] exit with 0 when --help requested
* [notice/cli] add helpful logging
* [notice/cli] use --validate flag name instead
* [notice/cli] simplify NEWLINE_RE, ignore obscure line endings
* [utils/decode_geo_hash] fixup comment
* [utils/decode_geo_hash] remove useless comment
* [kbn-plugin-generator] add plugin generator to the repo
* [plugin-generator] use snake_case plugin name for directory name
* [plugin-generator] fix typo
* [plugin-generator] remove translation support until we resume i18n efforts
* [yarn] update lockfile
* [mocha] remove plugin-generator selector from mocha tests
* [plugin-generator] update generated readme to recommend yarn
* [plugin-generator] add readme to generator pacakge
* [plugin-generator] link from plugin-resource docs
* [plugin-generator] mention very important `kbn bootstrap` script
* [plugin-generator] rework some parts of the README
* [plugin-generator] log actual directory name with system separators
* [plugin-generator] include bootstrap/yarn preinstall check script
* Introduce `kbn`, the Kibana build tool
* yarn kbn
* Make all deps devDeps
* Exclude __fixtures__ folder from Jest to avoid warnings
* Review fixes
* Update readme
* Use 'yarn kbn'
* Consistent rootPath
* Link to kbn tool
* Unsupported URL 'debug help' in contributing guide
* [mocha] use custom reporter for legible results in jenkins
* [jest] use custom result processor for legible results in jenkins
* [karma] enable junit output on CI
* [mocha/junitReporter] accept rootDirectory as configuration
* [jest/reporter] use reporters option added in jest 20
* [toolingLog] remove black/white specific colors
* [dev/mocha/junit] no reason for junit to be a "reporter"
* typos
* [dev/mocha/junit] use else if
* [karma/junit] use string#replace for explicitness
* [junit] use test file path as "classname"
* [ftr/mocha] no longer a "console" specific reporter
* Use separate startup scripts for development and production
* build kibana directly
* [build] Use downloaded node when pre-optimizing
* clearer variable name
* Add breaking changes docs
* [precommitHook] move to dev script
* [eslint] fix lint error
* [dev/eslint] do simply boolean checks first
* [dev] use shared REPO_ROOT constant
* [dev/File] precompute relative/ext of path
* [dev/eslint] fix relative import
* [dev/eslint] fix typos
* [grunt] remove unused run:eslintStaged config
* [dev/run] only create log if we are going to use it
* [dev/run/isFailError] ensure return value is Boolean
* [dev/precommitHook] use less intermediate vars
* [babel] create babel-preset and babel-register modules
* [babel-preset] add comments to babel-preset-env.target choices
* [babel-preset] use more normal BUILT_WITH_BABEL signal
* [babel-register] extended comment about filtering ./src in dist
* [babel-preset] stage of class properties has changed, sorta
* Shortcut to build asciidocs
The command to build the Kibana docs using the elastic/docs repo can be
easy to forget, so this script is an easy to way to do it right every
time.
* Configurable docs repo location
For folks that clone the elastic docs repo to a different location than
as a sibling of kibana.
* [scripts] convert `npm run makelogs` to script
* [scripts] convert `npm run mocha` to script
* [scripts] add warning to `test:ui:runner` npm script
* [script] fix location of scripts in warnings
* [functional_test_runner] replace functional testing tools with custom/pluggable solution
* [functional_test_runner] Convert unit tests to commonjs format
* [functional_test_runner] Fix dashboard test in wrong mode
* [functional_test_runner] Add dashboardLandingPage test subject
* [functional_test_runner] Get Visualize page object
* [functional_test_runner] Fix outdated references
* [functional_test_runner] Fix more outdated refs
* [functional_test_runner] Remove duplicate tests
* [functional_test_runner] Improve test readability
* [functional_test_runner] 😞 So many duplicate methods
* [functional_test_runner] Move mgmt `before` outside toplevel describe
* [functional_test_runner] Settings page obj missing methods
* [functional_test_runner] Add improvements from @gammon
* [functional_test_runner] Fix return statements in async funcs
* [functional_test_runner] Move before() to correct scope
* [functional_test_runner] Add after() hooks to remove index patterns
* [functional_test_runner] Attempt to fix vertical bar chart tests
* [functional_test_runner] Clean up
* [functional_test_runner] Reinstate unit tests
* [functional_test_runner] Set default loglevel back to info
* [functional_test_runner] Replace `context`s with `describe`s
* [functional_test_runner] Better error handling
* [functional_test_runner] Add in new Tile Map tests
* Incorporate changes from master
* [functional_test_runner] validate that every test file has a single top-level suite
* Update contributing doc with link to full doc
* [docs] Spelling and grammar fixes
* docs: writing and running functional tests
* [docs] Move plugin doc to plugin area
* [docs] Housekeeping. Doc in wrong place
* [docs] Remove dup doc file
* [grunt] Only run mocha_setup when running tests, not every grunt task
* As a part of bringing functional testing to plugins, esArchiver gives these plugins a way to capture and reload es indexes without needing to write a bunch of custom code. It works similarly to the elasticDump and ScenarioManager tools that it replaces.
Differences:
- Streaming implementation allows for much larger archives
- CLI for creating and using archives
- Configurable archive location
- Stores the data in gzipped files (better for source control, searching, large archives)
- Automatically identifies and upgrades Kibana config documents
Methods:
- `#load(name)`: import an archive
- `#loadIfNeeded(name)`: import an archive, but skip the documents what belong to any existing index
- `#unload(name)`: delete the indexes stored in an archive
CLI operations:
- `./bin/es_archiver save <name> [index patterns...]`: save the mapping and documents in one or more indexes that match the wild-card patterns into an the `<name>` archive
- `./bin/es_archiver load <name>`: load the mapping and documents from the `<name>` archive
* [functional_tests/common/nagivate] check for statusPage
* [es_archiver] move bins into new scripts dir
* [functional_tests/apps/context] use esArchiver
* [esArchiver] general improvements after showing to a few folks
- remove auto-upgrading config doc logic (until we have better access to kibana version info)
- export unload command
- remove preemptive checks in favor of reacting to errors
- use type "doc" vs "hit" for doc records (consistency)
- wrote a bunch of pending tests to think though and plan
* [esArchiver] make log a stream that writes to itself
* [esArchiver] fill in stats and archive format tests
* [esArchiver] splitup action logic
* [esArchiver/cli] fix cli --help output and comment
* [esArchiver] remove type-based param coercion
* [esArchiver/log] use strings for log levels
* [esArchvier] remove unused var
* [esArchiver/indexDocRecordsStream] add tests
* [esArchive] fill in remaining tests
* [esArchiver] fix dem tests
* [eslint] remove unused vars
* [esArchiver/loadIfNeeded] fix call to load()
* [esArchiver] remove loadDumpData helpers