* Add doc building for data plugin
* Added data server md file
* index file for suggestion component
* Clear out internal imports from public API
* Remove FunctionalComponent usage
* comment out working docs
* Added fixes following @octogonz feedback
* new docs
* Clean up
* Clean up 2
* fix jest
* rename docs script file
* update tasks
* tasks
* update docs
Over the last few years we've increased the number of test runners.
Entry points by test type have become too lage of a category for unique
names, so this moves top level test scripts under yarn and replaces test
types with the runner name.
e.g. `yarn test:browser` -> `yarn test:karma`
Closes#41133
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Generate core API docs from TSDoc comments
Uses api-extractor and api-documenter to generate documentation for
the Kibana core API from TSDoc comments in the source code.
Documentation can be generated using `npm run docs:api`.
I used --no-verify to ignore the following pre-commit hook errors:
1. Filenames MUST use snake_case - api-extractor.json
It's possible to specify a different config file, but I prefer to keep the "standard" config file name.
2. UNHANDLED ERROR: Unable to find tsconfig.json file selecting "common/core_api_review/kibana.api.ts". Ensure one exists and it is listed in "src/dev/typescript/projects.ts"
This is not a source file, so safe to ignore.
* Flesh out API docs a little bit
* Ignore snake_case check for api-extractor.json
* Ignore api-extractor's review file from pre-commit check
* Try to fix build failing by using masters yarn.lock
* I'm being stupid
* Found a better home for ignoring common/core_api_review/kibana.api.ts
* Node script for detecting core API changes
I initially wanted to include this as a precommit hook, but it takes
quite long to execute (~12s) so might be better suited as a test or
as part of the release process.
The script currently fails because api-extractor uses an older version
of typescript.
* Fix tslint precommit hook ignore condition
* Write tsdoc-metadata.json into ./build
* Add LogMeta and ElasticSearch to exported types & docs
* Suppress logging when running api-extractor from script
* Improve check_core_api_changes script and run as test
* Inline api-extractor.json config
* Fix check_core_api_changes --help flag
* LogMeta TSDoc comments
* check_core_api_changes: fail if api-extractor produces warnings or errors
And print more useful messages to the console
* Move ignored ts files list into dev/file
* Add back build:types since api-exporter cannot operate on source files
* Upgrade api-exporter/documenter
* api-extractor: independantly analyze core/public and core/server
Becasue of https://github.com/Microsoft/web-build-tools/issues/1029
api-extractor can't use core/index.ts as a single entry point for
analyzing the public and server API's as isolated namespaces.
Instead we analyze these projects separately. This introduces other
problems like the api review files and documentation always being
called "kibana." from the package.json filename.
* Build types as part of build task
* Include types in typescript browser compilation
* Force inclusion of core/public for building types
* Fix api review filename in api-exporter errors
* Update docs and API review files
* Fix api-extractor warnings
* Remove ts file ignored list since it's no longer necessary
* Rename exported api package name
* Review comments
* Export other missing types
* Upgrade api-documenter to latest beta
* Export more missing types
* Fix warnings and add api-exporter to Jenkins tests
* Correctly handle runBuildTypes() exceptions
* Fix another swallowed exception
* Fix api-extractor warnings after master merge
Based on #32705
We currently have TypeScript code that was backported to 7.0, which was backported without issue because it falls outside of any TypeScript projects in 7.0. This means that the pre-commit hooks break on changes to these files, and that they are not getting type checked by the type_check script. To fix this we need to verify that every typescript file in the repository is covered by a tsconfig.json file as part of CI.
Since this is no longer part of a reporting task in Jenkins, we no
longer need to re-run kbn bootstrap when generating a report.
We use the PR_SOURCE_BRANCH environment variable to prevent genererating
Github issues when we are on a PR.
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
* [ci][ftr][kbn/test] split up CI jobs
* [ci] run --assert-none-excluded in ci groups, before builds
* [ftr] improve error message when excluded tests found
* [ci] document other places the ciGroups live
Right now the functional tests are run against a distributable of Kibana in CI, but that distributable is running with `--env.name=development`. That causes the optimizer to run again before the tests can start and prevents the functional tests from running against the actual version of the application users will end up getting. This seems necessary for some tests, but not all of them, but I would like to get all of the tests running against the production version of Kibana soon.
This PR implements a second ftr config, `test/functional_production` that uses basically a copy of the `test/functional` config but with a few minor adjustments, removing the `--env.name=development` kbnServerArg and using a unique junit report name. To accomplish this I needed to modify the `@kbn/test` module to only pass the `--dev` flag to the Kibana server if it is being run in development mode, which it currently does by testing the args for `--env.name=development` or two args next to each other: `'--env.name'` and `'development'`. It does this by converting the `extraKbnOpts` option into an `addExtraKbnArgs` function, which is called with the final args just before passing them to the proc runner and given a chance to modify then after all other args are resolved (pulling from different places in config based on the build type, etc.)
Over the next couple weeks I'll push up PRs for individual test suites, migrating them over to the new production config, hopefully in a short period of time we will have all the function test suites back under that `test/function` config and can run them all against the Kibana server in production mode.
* Replace test:api with @kbn/test runTests
* Improve CLI help menu 🆘
* Use --es-from
* Replace jenkins:selenium with kbn-test
* Validate cli args, fixing test in the process
* Clean up some stuff
* Code review fixes
* Explanation for collectCliArgs
* Remove exit codes, they're useless anyway.
* Make markdown vis test pass with dev_mode setting
* Tests
* Remove unneeded export
* Code review: move console logging up to cli.js
* Code review: refactor startServers and runTests to take single options collection
* Code review: Remove all things I am sure we do not use
* Improve tests
* Code review fixes
* Pass created log to runFtr, runElasticsearch, runKibanaServer
* Update --es-from option to --esFrom
In order to make the license that applies to each file as clear as possible, and to be consistent with elasticsearch, we are adding Apache 2.0 license headers to the top of each file.
Existence of this header is enforced by eslint and tslint and missing headers were automatically added in the last commit by running:
```
node scripts/eslint --fix && node scripts/tslint --fix
```
* [tslint] lint typescript code
* [tslint] filter projects when running specific project
* [dev/ts] use more explicit types
* [dev/ts] add note about why using glob
* [dev/ts] rely on ts, use fewer getters
The old backport tool we used, Jasper, would create PRs with .rej files
when it couldn't merge a backport cleanly. That lead to this task which
would skip the tests if .rej files were found. Now we don't use Jasper
and this check takes about 1 minute to run on CI, so we can remove it
and save some time.
* [dev/notice] add scripts for generating NOTICE.txt file
* [notice] react-resize-detector@0.6.0 was removed in b445389b79
* [notice] move notice text into relevant source
* [dev/notice] Generate NOTICE.txt file
* [jenkins] verify that notice.txt is up to date in CI
* [tasks/notice] update test to use new NOTICE.txt file
* [dev/notice] update company name in NOTICE.txt
* [notice/cli] exit with 0 when --help requested
* [notice/cli] add helpful logging
* [notice/cli] use --validate flag name instead
* [notice/cli] simplify NEWLINE_RE, ignore obscure line endings
* [utils/decode_geo_hash] fixup comment
* [utils/decode_geo_hash] remove useless comment
* [build] don't copy unwanted files
* [build] condense mkdir/touch tasks
* [i18n] verifyTranslations task has nothing to do with the build, just run in tests
* Build packages before running ESLint on CI
* Add production task to kbn-build
* Ensure packages are bootstrapped before running the build
* Run ESLint on kbn-build
* build before running selenium
* change directory into built snapshot
* WIP: what is in the build dir
* WIP: skip unit tests
* run selenium on build
* [functional tests on build] Derive version
* [functional tests on build] Missing folder name prefix
* WIP: debug on ubuntu VM
* WIP: double kibana stabilize timeout
* WIP: remove ubuntu VM debug change
* WIP: add logs for docParams
* WIP: add logs for es
* build release version without -SNAPSHOT-
* [functional test release] Cleanup
* Add low level i18n plugin
Manages languages that are available and is responsible for loading translated
content at the granularity of a plugin.
To be done:
- APIs for store and retrieval
* Use Kibana install as root for the translation store directory
Setting the path for storing the bundled language translation files to
<KIBANA_INSTALL>/data/store_translations/<PLUGIN_NAME>
* Updated i18n core plugin APIs to be asynchronous
To be done:
- Better error handling in APIs
- Fix threading issue with storePluginLanguageTranslations API
* Fix thread synchroization issue in storePluginLanguageTranslations
* Update error handling in i18n core plugin
* Change to use NodeJS mkdirp function for creating directories recursively
Updates with review comments from @srl295. Changed export syntax to show the
exported functions at end of file.
* Add REST API for getting translations of a language for a plugin
To be done:
- Add algorithm to decide on the language for a plugin by comparing the accept languages
from the REST call and the plugin supported languages
- Add REST API tests
* Add algorithm for determining plugin language when retrieving translations
Client would pass languages used in the 'accept-language' header. These
languages would then be compared against the plugin supported languages
and best compared language would be selected.
To be done:
- Add REST API tests
* Add API to return all registered plugin language translations
* Add HAPI API to get all plugins translation files
* Update register translations API to be independent of plugin name and language
The register API is updated to be independent of plugin name and language. The API will now
traverse the path given and create language bundles as per language files it traverses.
The translations files structure has also been simplified to be just key/value objects.
To be done:
- Add hapi API to get translations
- Extend the API tests to test responses
* Update API test
* Add eslint fix for API test
* Update with review comments
From review https://github.com/elastic/kibana/pull/7545#issuecomment-231147435
following comments updated:
- README, .gitignore, and .eslintrc are not needed in a core plugin
- package.json only needs name and version
- unit tests need to go in a tests directory otherwise they won't get picked up
by the grunt tasks. Also our convention is to name the test file with the same
name as the module it's testing (so i18n_tests.js should just be i18n.js)
- For consistency with the rest of the code base, rename the data directory to fixtures.
- Prefer const (or let if necessary). Don't use var.
- Use ES6 imports/exports rather than commonjs style
- Only export the i18n module's public API. For instance, I don't think getPluginTranslationDetails is used outside of the i18n module, so it shouldn't be exposed publicly. If you want to expose it for testing purposes, I would recommend creating an i18n directory with an index.js file that exports the module's public API, and a separate i18n.js file with the "private" API. index.js will be for public use, i18n.js will be for private internal use.
* Update after review comments
From review (https://github.com/elastic/kibana/pull/7545#issuecomment-231884490):
- i18n module API should return promises for async operations instead of using
callbacks
- All filesystem access should be async
- Unit tests need to be updated based on new proposed plugin structure
(single language file, not split by view)
From design (https://github.com/elastic/kibana/issues/6515#issuecomment-231400097):
- Removed API as will consider in later phase
TODO:
- Make write function async
* Update after review comments
Updated write function to be asynchronous
* Update registerTranslations API to take absolute translation file as argument
The API originally took the directory as the argument but following reviews it
was decided to change to absolute file because it will be less brittle
since it is more explicit.
* Translate the Kibana welcome message
Translates the start-up message (“Kibana is loading ...”)in the Jade template.
To be done:
- Means to register the core plugin translations. They are currently added
in the fixtures directory as static files. Need to be generated on the fly.
* Add build task to generate core plugin translations
Task which calls registerTranslations API and then a task which copies the
regsitered translations to <kibana_root>/build/kibana
* Add hook to optimize module to add registration during dev startup
Registration of the core plugin translations during development start of
Kibana server. The translations include the welcome message and server error
startup message.
* Handle scenario when the user preferred language header is not passed
The UI when loading asks i18n plugin which language translation to use
depending on the user preferred language header 'accept-language'.
This commit is to handle scenario where header is not passed. The algorithm
then chooses the default language.
* Replace registering of translations at plugin install time to the plugin init phase
This change follows review comments in:
https://github.com/elastic/kibana/issues/6515#issuecomment-236237218
* Update after review comments
Comments:
- https://github.com/elastic/kibana/pull/7545#discussion-diff-72890673
- https://github.com/elastic/kibana/pull/7545#discussion-diff-72894762
* Update after plugin folder layout changes in Kibana
This require to use <kibana_root>/data for registered translations
and i18n plugin moved to core_plugins from plugins.
Refer to PR for more details:
https://github.com/elastic/kibana/pull/7562
* Update translation registration to file path rather than bundling
After review discussions it was agreed to just register the absolute paths
to translation files rather than bundling each file into one central file
at registration.
* Update review comments
This commit contains the following review comments:
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74661282
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74661392
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74662271
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74663235
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74669201
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74669269
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74669419
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74669628
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74799382
* Update review comments
The following review comments are included in the commit:
- https://github.com/elastic/kibana/pull/7545#discussion_r74663515
- https://github.com/elastic/kibana/pull/7545#discussion_r74666995
- https://github.com/elastic/kibana/pull/7545#discussion_r74805552
* Expose the i18n APIs in the server object for plugin access
Plugins should call the i18n plugin APIs through the server object
and not directly from the module.
This closes he following comments:
- https://github.com/elastic/kibana/pull/7545#discussion_r74662598
- https://github.com/elastic/kibana/pull/7545#discussion_r74669327
- https://github.com/elastic/kibana/pull/7545#discussion_r74669765
* Update accept-language-parser module to 1.2.0
Module version 1.2.0 fixes issue:
https://github.com/opentable/accept-language-parser/issues/8
This commit updates review comments:
https://github.com/elastic/kibana/pull/7545#discussion-diff-75525214https://github.com/elastic/kibana/pull/7545#issuecomment-240290461
* Add i18n default locale as a configurable item
Adds 'defaultLocale' configurable item to the i18n plugin configuration.
The default locale is used for translations if the locale specified by user
is not supported.
This commit satisfies the review comment:
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74669970
* Move UI i18n wrapper functionality into a module
This commit better structures the i18n capability so that it can be called
in UI code in a clearly defined fashion with minimum code. It also fixes
potential race conditions.
This commit updates review comments:
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74804791
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74801802
- https://github.com/elastic/kibana/pull/7545#discussion-diff-74670457
* Fill any missing translations using translations from default locale
The default language translations are loaded and are compared against the selected
language translations. The comparison can then highlight any missing translation
keys and can load the default translations keys as needed. This helps to unsure
where possible that a translation string is available in most scenarios even if not
in the locale requested.
This commit resolves review comments:
- https://github.com/elastic/kibana/pull/7545#issuecomment-239202583
- https://github.com/elastic/kibana/pull/7545#issuecomment-239203734
* Add unit tests for the i18n UI wrapper functions
* Fix issues after rebase with master
* Add translation keys verification tool
This tool helps to check that translation keys are translated. This tool can be
used for non-angular translation constructs like the Jade templates.
* Updates after review comments
Updates for review comments:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-3748114
* Update after review comments
Update for review comments:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-3937958
To be done:
- Update of unit tests for UI and server
- Call of verify translations
* Update unit tests after review changes
There was a number of changes to the i18n module and the ui i18n wrapper
following review comments. This commit is to update the unit tests with
respect.
* Add build task for verify translations
* Update the kibana i18n IDs to be prefixed with kibana
* Update verify translations to test registered translations
It was testing the static translation files. It is now updated to
test the translations registered when Kibana server is started and
the plugins have initialized.
* Update after review comments
Updates following review comments:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-5529711
* Update after review
This commit contain updates after the following review:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-5707951
* Updates after review
Updates for review comments:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-6656571
* Update after review
Updates for the following review comments:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-6911265
* Update after review
Updates after the following review comments:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-7084765
* Update unit tests to use expect throwError
* Update after rebase with master
Loading message changed following merge of commit
26c53e8a8d (diff-e25d7fee746a4f249e17f87c02fd95f8R55)
This required update to the welcome message and how it is called.
* Update following review
Updated the following review comments:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-9297662
* Update the algorithm to return the locale
The algorithm to return which locale to use for translations based on the user
locale list and the regsitered locales is updated in this commit. The algorithm
previously did an exact match on all the user locales first before (by priority)
then checking for best case match. The algorithm is now modified to check each
user locale starting with the highest priority first for an exact match and then
for best case match. If no match it then moves to the next user locale with
the next highest priority. This is to follow the priority list that a user
browser is configured for where there maybe a locale translation available
but might not be the exact match with regard to the locale code and/or script.
An example of this is that the highest priority locale of the user is 'en-US'
but the locale translation available is 'en'. It is better select the 'en'
locale rather than select the next highest locale which is an exact match.
* Update after review comments
Updates after the following reviews:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-9785665https://github.com/elastic/kibana/pull/7545#pullrequestreview-9786404
* Fix after merge with master
Change in the flo and layout of ui index meant that acceptLanguages were not
being passed. This commit is an update to fix this so that the welcome
messages are loaded.
* Update after review comments
This commit is for updates after the following review:
https://github.com/elastic/kibana/pull/7545#pullrequestreview-10435175
* Fix issue when unit test run in CI as core translations are registered
When unit tests are run on a test server (like in the CI), it will start
Kibana server and register the core translations. This means that the i18n
unit tests need to be able to store the existing registration prior to
testing and replace after testing.
* [server/ui] move i18n into ui module
* [server/ui] restore renderApp() method signature
* [server/ui] unify i18n logic in UiI18n class
* [server] move translation files into "translations" dir
* Update i18n module to loaded by multiple server instances within the one process
* Update i18n module to a class
Moving the i18n module into a class so as to encapsulate the registered
translations which means there can be different and distinct instances per process.
This is to accomodate the user case where there might be multiple Kibana server
instances in a process and the localization should be at the server level.
* Identify private members in a class with underscore-prefix convention
* Remove redundant translation from core translation file
Message starting with 'Give me a moment...' is no longer part of loading
message folowing a rebase with master.
* [ui/i18n] reject translations files that do not use absolute paths
* Update config item locale to defaultLocale
* Update after review comments
- Update after following review: https://github.com/elastic/kibana/pull/7545#pullrequestreview-12775161
- Also, fix syntax mess following rebase with master of src/optimize/index.js
* Fix rebase with master error
* Add task for verifying translations in CI
* Fix lint errors
The tests in master are currently failing regularly because our current browser tests are serious memory hogs. Investigation reveals that nearly every test is retaining all of the memory it causes to be allocated. We have made some progress to being able to diagnose the problems, but we expect that problem to take some serious work to fix. We need a short-term solution though, and this is it.
Rather than modify the bundling process, we will shard the top-level test suites by name. For now, we've created 4 shards, but adding new shards is trivial if we need to.
Sharding is accomplished by creating a murmur3 hash of the top level suite names, then bucketing based on the hash output. If a test suite resolves to shard2, but we are running shard1, we simply never pass the function to `mocha.describe()`. Rather than redefine every describe statement, we have shimmed the global `window.describe()` function to accomplish this.
