Commit graph

35 commits

Author SHA1 Message Date
Spencer 4e8ab56497
build immutable bundles for new platform plugins (#53976)
* build immutable bundles for new platform plugins

* only inspect workers if configured to do so

* [navigation] use an index.scss file

* add yarn.lock symlink

* set pluginScanDirs in test so fixtures stay consistent

* cleanup helpers a little

* fix type error

* support KBN_OPTIMIZER_MAX_WORKERS for limiting workers via env

* test support for KBN_OPTIMIZER_MAX_WORKERS

* expand the available memory for workers when only running one or two

* add docs about KBN_OPTIMIZER_MAX_WORKERS environment variable

* fix README link

* update kbn/pm dist

* implement bundle caching/reuse

* update kbn/pm dist

* don't check for cache if --no-cache is passed

* update renovate config

* standardize on index.scss, move console styles over

* add support for --no-cache to cli

* include worker config vars in optimizer version

* ignore concatenated modules

* update integration test

* add safari to browserslist to avoid user-agent warnings in dev

* update docs, clean up optimizer message/misc naming

* always handle initialized messages, don't ignore states that are attached to specific events

* reword caching docs, add environment var to disable caching

* tweak logging and don't use optimizer.useBundleCache as that's disabled in dev

* handle change notifications

* batch changes for 1 second

* rename CompilerState type to CompilerMsg

* getChanges() no longer needs to assign changes to dirs

* remove unused deps

* split up run_worker.ts and share cacheKey generation logic

* add a couple docs

* update tests and remove unused imports

* specify files when creating bundle cache key

* remove one more unused import

* match existing dev cli output more closely

* update kbn/pm dist

* set KBN_NP_PLUGINS_BUILT to avoid warning in CI

* avoid extending global window type

* add note to keep pluginScanDirs in sync

* pass browserslistEnv in workerConfig so it is used for cache key

* load commons.bundle.js in parallel too

* emit initialized+success states if all bundles are cached

* load bootstraps as quickly as possible

* skip flaky suite

* bump

* update jest snapshots

* remove hashing from cache key generation

* remove unnecessary non-null assertion

* improve docs and break up Optimizer#run()

* remove unused import

* refactor kbn/optimizer to break up observable logic, implement more helpful cache invalidation logic with logging

* fix tests

* add initializing phase

* avoid rxjs observable constructor

* remove unnecessary rxjs helper, add tests for bundle cache

* update consumers of optimizer

* update readme with new call style

* replace "new platform" with "kibana platform"

* fix a couple more renames

* add support for several plain-text file formats

* fix naming of OptimizerMsg => OptimizerUpdate, use "store" naming too

* one more OptimizerMsg update

* ensure bundles are not cached when cache config is false

* test for initializing states and bundle cache events

* remove unnecessary timeout change

* Remove unnecessary helpers

* Add tests for BundleCache class

* Add tests for Bundle class

* test summarizeEvent$

* missing paths are no longer listed in mtimes map

* add tests for optimizer/cache_keys

* Add some extra docs

* Remove labeled loop

* add integration test for kbn-optimizer watcher components

* querystring-browser removed

* tweak logging a smidge, improve info and final message

* remove unused imports

* remove duplication of getModuleCount() method

* move type annotation that validates things

* clear up the build completion message

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-02-12 19:42:42 -07:00
Alexey Antonov deda49e9f0
[ui/utils/query_string]: Remove unused methods & migrate apps to querystring lib (#56957)
* replace querystring (querystring-browser) -> query-string

* QueryString remove encode/decode methods

* remove query_string file

* remove querystring-browser from package.json

* add kibana_utils\url module

* cleanup

* update notice.txt

* fix merge conflict

* fix CI

* fix wrong import

* fix CI

* fix X-Pack firefox smoke test

* remove urlUtils.parseUrlQuery

* remove url.stringifyUrlQuery

* use url.encodeQuery

* Record<string, any> -> ParsedQuery

* Update src/plugins/console/public/application/containers/editor/legacy/console_editor/editor.tsx

Co-Authored-By: Luke Elmers <lukeelmers@gmail.com>

* add more tests for APM

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Luke Elmers <lukeelmers@gmail.com>
2020-02-12 19:51:03 +03:00
Joe Portner 13593344a0
Add lockfile symlinks (#55440)
This is to enable dependency scanning tools to correctly resolve
the dependencies in each package.json file that is used in
production.
2020-01-27 11:38:20 -05:00
Spencer 357be5970d
share specific instances of some ui packages (#54079)
* share specific instances of some ui packages

* remove unnecessary eslint changes, every package will define deps anyway

* remove mentions of moment webpackShims in eslint resolver

* remove use of lodash

* list angular as dep for x-pack

* add operations as codeowner of shared-deps pkg
2020-01-10 15:22:18 -07:00
spalger 8b7aaa4849 Revert "Update webpack related packages (#53082)"
This reverts commit 8b36eb46ea.
2019-12-17 11:32:49 -07:00
renovate[bot] 8b36eb46ea Update webpack related packages (#53082)
* Update webpack related packages

* fix updated config flag

* update other changes css-loader config options
2019-12-16 12:14:20 -07:00
renovate[bot] 1da0469521 Update webpack related packages (#47402) 2019-10-05 14:36:02 -07:00
renovate[bot] 25b0e2caeb Update dependency lru-cache to ^4.1.5 (#45157) 2019-09-09 13:33:31 -07:00
renovate[bot] f051184d5e Update dependency debug to ^2.6.9 (#45155) 2019-09-09 13:29:02 -07:00
Tre ef4a73a491
[Plugin Generator] Add integration test (#43219)
Addresses https://github.com/elastic/kibana/issues/17061

### Features Covered 
 - generating a plugin

#### Then from within the generated plugin's directory
 - running `yarn build`
 - running `yarn test:browser`
 - running `yarn test:server`
 - running `yarn start`
 - running `yarn preinstall`
 - running `yarn lint`
 - running `yarn kbn --help`
 - running `yarn es --help`
2019-09-06 13:10:47 -06:00
renovate[bot] 76d8dbde72 Update webpack related packages (#43903)
* Update webpack related packages

* update kbn-pm dist

* update kbn/pm dist
2019-08-26 08:17:52 -07:00
renovate[bot] b2340d5cae Pin dependencies (#37454) 2019-05-29 22:07:52 -07:00
renovate[bot] fa597135bc Pin dependencies (#37404)
* Pin dependencies

* [eslint] ignore large files that take a long time to parse

* [eslint-plugin-import] force `querystring` to be considered a core module

* [eslint-plugin-import] auto-fix import order issues
2019-05-29 20:34:52 -07:00
Eli Perelman 78cbe42982
Prefer third-party plugin development in ./plugins instead of ../kibana-extra (#31748)
* Prefer third-party plugin development in plugins instead of kibana-extra

* Fix failing recursive directory creation and removal

* Add new built version of kbn-pm
2019-03-07 17:04:29 -06:00
Court Ewing 2ce51a5be5 Update paths to ui, server, deprecation, plugin_discovery in src/legacy
This commit accompanies the four that precede it. Rather than squash
them altogether, the four previous commits all do nothing except move
files to help avoid conflicts.
2019-02-11 10:41:37 -05:00
Larry Gregory 22ba11eb52
Prevents dynamic import/require statements (#27515)
## Summary

This PR enables the `import/no-dynamic-require` eslint rule, which requires that all require/import statements be statically defined.

This mitigates certain attack vectors, such as those which enabled [`ESA-2018-18`](https://www.elastic.co/community/security)/[`CVE-2018-17246`](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-17246)
2018-12-21 11:15:20 -05:00
Tiago Costa d5b2c8eaf2
Create vendor dll for the client modules (#22618)
* feat(NA): first dll bundler code.

* chore(NA): upgrade to webpack 4.

* chore(NA): updated package.json

* chore(NA): removed old code.

* chore(NA): add parallel option.

* chore(NA): removed console log and old var about  node modules.

* chore(NA): turn off unsafe cache.

* chore(NA): update lock files.

* chore(NA): new config for dll generation.

* chore(NA): update stats emit.

* chore(NA): update dependencies.

* chore(NA): right cache loaders.

* chore(NA): remove ui_bundles alias.

* feat(20749): init implementation on bridge plugin for dll bundler.

* feat(20749): init implementation for dll compiler.

* feat(20749): dll bundler init from other process and webpack wrapper..

* feat(20749): optimizer changes to integrate with  dll bundler.

* chore(20749): split into different processes.

* refact(20749): change to single running process.

* refact(NA): improvements on dll bundler plugin.

* refact(NA): removing including loop on plugins.

* refact(20749): only run dllReference once.

* chore(20749): todo on result.request path building.

* chore(NA): lock files updated.

* chore(NA): avoiding dll paths being removed.

* chore(NA): tests on sync generation.

* chore(NA): changes on entry paths compiler.

* chore(NA): different hooks tests.

* chore(20749): first working version, single process, for dynamic building dll.

* feat(20749): last gross features for the dynamicdllplugin.

* refact(20749): string interpolation when creating the path to delete in every optimizer cycle. feat(20749): creating the DynamicDllPlugin foundations.

* chore(NA): updated base optimizer run function args.

* chore(20749): first working solution with vendor dll both for prod and dev.

* chore(20749): useful todos on client dlls.

* feat(NA): auto built blacklist for server node_modules.

* refact(NA): removed unused code.

* chore(NA): update all webpack and loaders stuff to last versions.

* refact(NA): first refacts on clean taks related with dll. feat(NA): added clean empty folders task.

* refact(NA): removed support for console logs during the build.

* refact(NA): removed extra space.

* refact(NA): removed extra space.

* refact(NA): getDllModules function.

* chore(NA): removed unsafeCache option.

* feat(NA): removed unsafeCache option.

* refact(NA): apply general inheritance principles to the optimizer: hooks, and init. refact(NA): merge dynamic dll plugin into the common config. refact(NA): restore old template structure - vendors.dll style is always emitted.

* fix(NA): fs_optimizer run function by not returning inside async func.

* fix(NA): change template anchor name from vendor to vendors.

* docs(NA): added info about files were keeping when cleaning the bundles.

* fix(NA): filtering elible dll modules to delete.

* refact(NA): removed old dll bundler in favor on new dynamic dll plugin.

* refact(NA): initial split for clean modules on dll task.

* refact(NA): update new dll config model. refact(NA): update config on dynamicdllplugin.

* refact(NA): major work refactor for dynamic dll plugin.

* refact(NA): extract clean node modules on dll task to its own folder.

* refact(NA): organize imports.

* docs(NA): add docs to registerCompilerHooks function for the optimizer.

* refact(NA): finished refactor for dynamic dll plugin with correct error handling for runWebpack function.

* refact(NA): basic structure for clean client modules on dll task.

* fix(NA): resolve path for dll manifest during cclean build tasks.

* refact(NA): split webpack dll related functions to their own file for clean client modules on dll task.

* refact(NA): added error handling for the clean client modules on dll task - webpack dll related functions.

* docs(NA): added license header.

* refact(NA): complete split out the functions from the clean modules on dll task to the code_parser file.

* refact(NA): main task entries compose.

* docs(NA): extend docs for the getDependenciesFromFile function.

* refact(NA): final structure split for clean client node modules dll task.

* fix(NA): added missing param to calculate top level dependencies.

* docs(NA): completed todo description about dll location.

* fix(NA): add production option to webpack config on kbn-pm.

* docs(NA): extended documentation about style extraction.

* refact(NA): removed extra comments.

* feat(NA): env variable to force dll creation.

* feat(NA): include option to define folders to keep on delete empty folders task.

* refact(NA): remove unused method from dll compiler.

* feat(NA): move dlls outside bundles folder and support request for /dlls from a browser perspective.

* chore(NA): gitignore updated to include new dlls folder.

* chore(NA): eslintignore updated.

* chore(NA): removed strange file from repo.

* test(NA): fix failing tests for bundles_route.

* fix(NA): change paths array to path string in a server route config.

* fix(NA): remove infinite loop calls on register hooks functions.

* fix(NA): readFile should only override the file in case it not exists.

* refact(NA): removed dll compiler finish log on success with stats.

* fix(NA): dll compiler alias.

* fix(NA): dynamic dll plugin flow on needs compile.

* fix(NA): raw alias config.

* Revert "fix(NA): raw alias config."

This reverts commit ebb245a786.

* feat(NA): raw alias for moment on dll config model.

* fix(NA): removed uiBundles from base_optimizer call on dynamicdllplugin.

* chore(NA): decrease moment versions.

* chore(NA): temporary changes on dll compiler.

* fix(NA): majority of problems between dll approach, webpackshims and browser tests.

* fix(NA): webpackShims integration with client vendors dll. fix(NA): enable esm modules mutability for development. fix(NA): only clean dll contents from build when they belong to node_modules.

* fix(NA): fix for endless dll compilation loop.

* fix(NA): removed esm plugin and skipped test using wrong stub strategy.

* docs(NA): added some comments for the skipped test.

* feat(NA): considering requires inside webpackShims valid entry paths to add to the dll entry file.

* fix(NA): small fix for the max compilation logic.

* docs(NA): add small explanatory note.

* fix(NA): building requires results with relative requires found onn webpackShims.

* docs(NA): add small note for error handling on dll compiler.

* fix(NA): move precinct to prod dependencies.

* test(NA): testing running functional tests on production.

* fix(NA): restore tests run config for development flag. feat(NA): force dll creation with uiBundles is Dev flag.

* chore(NA): update dependencies.

* feat(NA): test update dll to completely match base optimizer one.

* fix(NA): removed ts.

* refact(NA): removed unused consts.

* fix(NA): set back transpile sacss task in place.

* fix(NA): fix resolve remoing ts and tsx.

* fix(NA): set back transpile sacss task in place.

* fix(NA): removing isDevmode from mustCompileDll.

* fix(NA): add search for import statements into the dependencies visitor.

* fix(NA): add dll suffix to vendors resource on ui bootstrap template.

* fix(NA): some configs for unrelated dll work projects.

* chore(NA): upgrade canvas plugins webpack build to webpack4.

* chore(NA): add shim for moment-duration-format.

* chore(NA): stup moment-duration-format into the moment webpackShim.

* chore(NA): setup moment-duration-format into the moment-timezone webpackShim.

* fix(NA): moment and moment-timezone webpackShims

* chore(NA): added moment and moment-timezone webpackShims to x-pack. fix(NA): revert changes on webpackShims for oss kibana.

* fix(NA): xpack webpackshims for moment.

* fix(NA): remove xpack webpakcshims for moment.

* fix(NA): webpakcshims for moment.

* fix(NA): remove every changes from webpackShims and xpack webpackShims.

* fix(NA): fix visitors to gather server dependencies resulting from export * from and export x, 'x' from.

* chore(NA): update some webpack related dependencies.

* fix(NA): in the dll the plugins need to have their own dependencies. It is the same for the ones into the tests relying on test against distributable.

* feat(NA): including test/plugin_functional plugins into the kbn-pm bootstrap tasks.

* fix(NA): wrong built yarn lock files.

* chore(NA): updated webpack related dependencies.

* feat(NA): add missing color for dynamic_dll_plugin logging tag.

* chore(NA): removed forgotten console.log.

* chore(NA): removed forgotten dead code.

* chore(NA): removed missing old comment.

* docs(NA): added missing notice for 2 tools we have relied on.

* refact(NA): added is to a boolean variable to keep the consistency inside the code parser strategies.

* fix(NA): update notice cli to exclude search inside dlls bundles. chore(NA): update notice file.

* feat(NA): use lodash matches in the code parser visitors logic.

* docs(NA): updated notice file related with the code parser visitors logic..

* docs(NA): added explanation for the process that decides if we should build a new dll or not.

* test(NA): added missing tests for some usefull parts of the code.

* refact(NA): split registerCompileHook function into small ones.

* chore(NA): uncomment scripts from tests.

* feat(NA): isolate code-parser in a kbn package

* fix(NA): missing relative dot into the cwd function.

* chore(NA): update all inter deps to match.

* fix(NA): rebuild the yarn locks and the package jsons based on master ones.

* fix(NA): move babel-code-parser to the prod deps.

* chore(NA): include build path instead of the base root path.

* refact(NA): integrate plugin_functional test plugins with workspaces.

* fix(NA): include missing license for plugin functional test plugins.

* fix(NA): always write posix paths into the dll entry file in order to make the dlls compilation working on windows too. chore(NA): improve error handling into dll compiler.

* fix(NA): revert wrong moved line from canvas.

* fix(NA): match ts-loader version between kibana and x-pack.

* fix(NA): sync dll compiler with base_optimizer.

* fix(NA): exclude kbn-interpreter from the dll.

* refact(NA): remove exclusion of kbn-interpretor from base_optimizer.

* chore(NA): add dlls folder to the yarn kbn clean script.

* fix(NA): missing utf8 input format encoding when reading a file to create the hash into the watch optimizer cahce.

* refact(NA): re-engineering to the dynamic_dll_plugin logs and lifecycle.

* fix(NA): update clean node modules task to search under legacy/core_plugins.

* fix(NA): fix build on windows with globby on cleaning dlls for the watch optimizer cache.

* docs(NA): update documentation for the clean client node modules build task.

* docs(NA): added extra comment to clarify the purpose for the built entrypoints.

* chore(NA): update clean client node_modules code to use posix path.

* feat(NA): add support for discovering server entries over the legacy plugins and the new plugins.
2018-12-05 15:45:19 +00:00
Aleh Zasypkin 5d2601d602 [packages] fix paths. 2018-12-03 13:03:14 +01:00
Josh Dover 77522a7bd2
[kbn-pm] Use yarn workspaces for dependencies (#24095) 2018-11-12 12:38:11 -06:00
Spencer d22bdfec46
[yarn] Upgrade to 1.10.1 (#23971)
* [yarn] upgrade required version

* [yarn] update lock files to include integrity

* [yarn] coalesce locked readable-stream versions to avoid yarn bug
2018-10-15 14:34:30 -07:00
Aleh Zasypkin 73f955db19
Upgrade eslint/tslint/prettier plugin versions. (#23470) 2018-09-27 11:54:09 +02:00
Joe Fleming 1eaed69548
fix: name in import resolver readme (#21316)
use @kbn instead of @elastic to match the name in the package.json
2018-07-27 11:04:55 -07:00
Spencer 8662475834
[core/ui] bootstrap the legacy platform within the new platform (#20699)
Fixes #20694

Implements super basic new platform `core` system, which includes two services: `core.injectedMetadata` and `core.legacyPlatform`. The `core` currently has two responsibilities:

 1. read the metadata from the DOM and initialize the `ui/metadata` module with legacy metadata, proving out how we plan to expose data from the new platform through the existing APIs/modules to the legacy platform.
 2. bootstrap the legacy platform by loading either `ui/chrome` or `ui/test_harness`

Because `core` mutates the `ui/metadata` module before bootstrapping the legacy platform all existing consumers of `ui/metadata` won't be impacted by the fact that metadata loading was moved into the new platform. We plan to do this for many other services that will need to exist in both the legacy and new platforms, like `ui/chrome` (see #20696).
2018-07-17 22:07:13 -07:00
Tim Roes cb5ee01c6a
Increase prettier line width to 100 (#20535)
* Increase prettier line width to 100

* Fix packages JS prettier

* Change style guide to 100 width

* Fix line-width in latest master changes
2018-07-09 22:50:37 +02:00
Josh Soref 2b27fb1c5f Fix misspellings (#19981) 2018-06-26 20:17:41 -07:00
Spencer f0cc432faf
Apache 2.0 license headers (#19383)
In order to make the license that applies to each file as clear as possible, and to be consistent with elasticsearch, we are adding Apache 2.0 license headers to the top of each file.

Existence of this header is enforced by eslint and tslint and missing headers were automatically added in the last commit by running:

```
node scripts/eslint --fix && node scripts/tslint --fix
```
2018-05-28 20:06:30 -07:00
Spencer c6112067fc
[typescript] add typescript support for the server and browser (#19104)
* [typescript] add typescript support for the server and browser

* [ts-jest] upgrade to latest version

* [jest] support test.tsx files

* [jest/ts] modify `ts-jest.tsConfigFile` config based on filePath

* [types] use correct major version of minimatch types

* [jest] add ts support to x-pack jest config

* [ts/projects] fix tsconfig.json not found error message

* [optimizer/ts] use lowercase jsx option

* [tsconfig] remove ui/* alias

* [plguin-helpers] remove mention of `buildSourcePatterns`

* [plugin-helpers] expect typescript to be a devDep

* [dev/build] place transpile tasks next to each other

* [ts/x-pack] add common and server directories to ts project

* [dev/ts/project] use a limited set of globs to find tsconfig files
2018-05-18 10:54:57 -07:00
spalger 0ad665f216 [eslint/import-resolver] ensure compat with slightly older versions of node for IDE support 2018-05-17 08:04:52 -07:00
Spencer 0b03166d2d
[eslint] unify resolver configs (#19102)
* [eslint] unify resolver configs

Our eslint resolver settings currently rely on the fact that we define
our resolver with a string globally, and an object in the overrides.
This causes the override value to completely override/replace the global
setting, which is desired, but when the global setting is converted to
an object they are merged, causing both resolvers to run.

This is a problem because some dependencies in the UI side of things
will resolve with the node resolver, but will resolve incorrectly
because they are intended to use some webpack specific override.

While trying to add TypeScript I needed to pass argument to the node
resolver which uncovered this issue. The change here moves us away from
using the node resolver directly and instead uses the kibana resolver
with `forceNode: true` set when linting server code and `forceNode:
false` when resolving imports that will be handled by webpack.

* [import-resolver] use object spread operator
2018-05-16 10:45:41 -07:00
Spencer e1a2fcbd96
[uiExports] migrate uiApp "uses" to explicit imports in apps (#17828)
* [uiExports] migrate uiApp "uses" to explicit imports in apps

* [uiApp] update tests for getModules() method

* [optimize/uiExports] improve naming and comments

* [uiExports] sort imports so they load in the same order as before

* [testHarness] load hacks when testing in the browser

* [x-pack/uiExports] use new uiExports modules

* [testHarness] describe why we import uiExports/hacks

* [optimize] remove needless [].concat()

* [optimize/createUiExportsModule] string.includes > string.indexOf

* [uiExports/createUiExportsModule] remove needless capture of module exports
2018-05-03 12:18:50 -07:00
Jenkins CI fe4609647d Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
Tim Roes 07cf4b6b08
Move ng_mock wrapper to test_utils (#17712) 2018-04-17 08:14:54 +02:00
Spencer 6db8fea28e
[eslint-import-resolver] support mapping plugins to directories (#17667)
* [eslint-import-resolver] support mapping plugins to directories

* use shorthand syntax
2018-04-12 15:28:30 -07:00
Spencer 22d70fcb22
[eslint-import-resolver] Add installation docs 2018-04-11 16:12:14 -07:00
Spencer 4f2a19bd9b
[eslint-import-resolver-kibana] bring in repo as package (#17665)
* [eslint-import-resolver-kibana] bring in repo as package

* [eslint-import-resolver] enable prettier
2018-04-11 15:21:29 -07:00