6c62c686cf
* chore(NA): upgrade oss to lodash4 chore(NA): migrate cli, cli_plugin, cli_keystore, dev, test_utils and apm src script to lodash4 chore(NA): missing file for cli plugin chore(NA): add src core chore(NA): es archiver and fixtures chore(NA): try to fix functional test failure chore(NA): migrate src/legacy entirely to lodash4 except src/legacy/core_plugins chore(NA): move legacy core plugins to lodash4 chore(NA): upgrade optimize to lodash4 chore(NA): upgrade to lodash4 on advanced_settings, charts, console and dashboard chore(NA): migrate to lodash4 on dev_tools, discover, embeddable, es_ui)shared, expressions, home plugins chore(NA): upgrade data plugin to lodash4 chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4 chore(NA): missing data upgrades to lodash4 Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4" This reverts commit 137055c5fed2fc52bb26547e0bc1ad2e3d4fe309. Revert "Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4"" This reverts commit f7e73688782998513d9fb6d7e8f0765e9beb28d1. Revert "chore(NA): missing data upgrades to lodash4" This reverts commit 92b85bf947a89bfc70cc4052738a6b2128ffb076. Revert "chore(NA): upgrade data plugin to lodash4" This reverts commit 88fdb075ee1e26c4ac979b6681d8a2b002df74c6. chore(NA): upgrade idx_pattern_mgt, input_control_vis, inspector, kbn_legacy, kbn_react, kbn_usage_collections, kbn_utils, management and maps_legacy to lodash4 chore(NA): map src plugin data to lodash3 chore(NA): missing lodash.clonedeep dep chore(NA): change packages kbn-config-schema deps chore(NA): update renovate config chore(NA): upgrade vis_type plugins to lodash4 chore(NA): move vis_type_vislib to lodash3 chore(NA): update visualizations and visualize to lodash4 chore(NA): remove lodash 3 types from src and move test to lodash4 chore(NA): move home, usage_collection and management to lodash 3 Revert "chore(NA): move home, usage_collection and management to lodash 3" This reverts commit f86e8585f02d21550746569af54215b076a79a3d. chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3 chore(NA): update x-pack test to mock lodash4 Revert "chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3" This reverts commit 2d10fe450533e1b36db21d99cfae3ce996a244e0. * chore(NA): move x-pack and packages to lodash 4 * chore(NA): remove mention to lodash from main package.json * chore(NA): remove helper alias for lodash4 and make it the default lodash * chore(NA): fix last failing types in the repo * chore(NA): fix public api * chore(NA): fix types for agg_row.tsx * chore(NA): fix increment of optimizer modules in the rollup plugin * chore(NA): migrate `src/core/public/http/fetch.ts` (#5) * omit undefined query props * just remove merge usage * fix types * chore(NA): fixes for feedback from apm team * chore(NA): recover old behaviour on apm LoadingIndeicatorContext.tsx * chore(NA): fixes for feedback from watson * Platform lodash4 tweaks (#6) * chore(NA): fix types and behaviour on src/core/server/elasticsearch/errors.ts * Canvas fixes for lodash upgrade * [APM] Adds unit test for APM service maps transform (#7) * Adds a snapshot unit test for getConnections and rearranges some code to make testing easier * reverts `ArrayList` back to `String[]` in the painless script within `fetch_service_paths_from_trace_ids.ts` * chore(NA): update yarn.lock * chore(NA): remove any and use a real type for alerts task runner Co-authored-by: Gidi Meir Morris <github@gidi.io> * chore(NA): used named import for triggers_actions_ui file * chore(NA): fix eslint * chore(NA): fix types * Delete most uptime lodash references. * Simplify. Clean up types. * [Uptime] Delete most uptime lodash references (#8) * Delete most uptime lodash references. * Simplify. Clean up types. * chore(NA): add eslint rule to avoid using lodash3 * chore(NA): apply changes on feedback from es-ui team * fix some types (#9) * Clean up some expressions types. * chore(NA): missing ts-expect-error statements * Upgrade lodash 4 vislib (#11) * replace lodash 3 with lodash 4 on vislib plugin * Further changes * further replacement of lodash3 to 4 * further work on upgrading to lodash 4 * final changes to update lodash * chore(NA): upgrade data plugin to lodash4 chore(NA): upgrade data plugin public to lodash4 chore(NA): fix typecheck task chore(NA): fix agg_config with hasIn chore(NA): assign to assignIn and has to hasIn chore(NA): upgrade data plugin server to lodash4 chore(NA): new signature for core api fix(NA): match behaviour between lodash3 and lodash4 for set in search_source * chore(NA): remove lodash3 completely from the repo * chore(NA): fix x-pack/test/api_integration/apis/metrics_ui/snapshot.ts missing content * chore(NA): fix lodash usage on apm * chore(NA): fix typecheck for maps * Patch lodash template (#12) * Applying changes from https://github.com/elastic/kibana/pull/64985 * Using isIterateeCall, because it seems less brittle * Also patching `lodash/template` and `lodash/fp/template` * Reorganizing some files... * Revising comment * Ends up `_` is a function also... I hate JavaScript Co-authored-by: Pierre Gayvallet <pierre.gayvallet@gmail.com> Co-authored-by: Josh Dover <me@joshdover.com> Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co> Co-authored-by: Oliver Gupte <ogupte@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Gidi Meir Morris <github@gidi.io> Co-authored-by: Justin Kambic <justin.kambic@elastic.co> Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com> Co-authored-by: Luke Elmers <luke.elmers@elastic.co> Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> Co-authored-by: kobelb <brandon.kobel@elastic.co> |
||
---|---|---|
.. | ||
plugins/task_manager_performance | ||
test_suites/task_manager | ||
config.js | ||
ftr_provider_context.d.ts | ||
README.md | ||
services.ts |
Task Manager Performance Integration Test
This test provides a test framework for spawning multiple concurrent tasks in Task Manager and measuring how well Task Manager performs in claiming, processing and finalising these tasks.
We keep this test disabled as it is used on an ad hoc basis and we feel it is worth keeping around for future use, rather than being rewritten time and time again.
How To Run The Tests
Setup
In the ./test_suites/task_manager/task_manager_perf_integration.ts
file you see the following configuration:
{ tasksToSpawn: 10, durationInSeconds: 60 }
tasksToSpawn
tells the test runner how many tasks to spawn. Each task has a 1s interval so it will try to rerun it every second.
durationInSeconds
tells the test running how many seconds you'd like the test to run for.
running
- Enable the test in
./test_suites/task_manager/index.ts
by removing the.skip
from thedescribe.skip
. - Run the test server from within the
x-pack
folder:node scripts/functional_tests_server.js --config=test/plugin_api_perf/config.js
- Run the test runner from within the
x-pack
folder:node scripts/functional_test_runner.js --config=test/plugin_api_perf/config.js
The Results
After the test runs you should get the following output:
└-: task_manager_perf
└-> "before all" hook
└-: stressing task manager
└-> "before all" hook
└-> should run 10 tasks every second for a minute
└-> "before each" hook: global before each
└-> "before each" hook
│ debg Stress Test Result:
│ debg Average number of tasks executed per second: 4.846153846153846
│ debg Average time it took from the moment a task's scheduled time was reached, until Task Manager picked it up: 8220.473076923077
└- ✓ pass (1.0m) "task_manager_perf stressing task manager should run 10 tasks every second for a minute"
└-> "after all" hook
└-> "after all" hook
1 passing (1.0m)
If you look at the debug output you'll see a summary of how the test went: You'll see the average number of tasks executed per second, over a period of each 5 second window (meaning we calculate the running average based on a sliding window of 5 seconds). You'll also see the average time it takes from the moment a task's scheduled time was reached, until Task Manager picked it up for execution.
Running the test against multiple Kibana and a single Elasticsearch
This is not a clean and ideal way of running this test, but it is a workaround that's worth understanding.
Test Description
The idea is that we would like to test performance of running multiple Kibana instances side by side, both pointing at the same Elasticsearch cluster.
This is needed in order to verify that the distributed nature of Kibana doesn't introduce issues or break assumptions in our developed solutions.
The challenge is that the plugin used to create the Perf test is exposed in the FTS, but not in a standard Kibana build.
Running two Kibana in FTS side by side is actually very tricky, so below is a step by step method for achieving this. Ideally we can clean this up and make it easier and less hacky in the future, but for now, this documents how this can be achieved.
Method
- You need two cloned repos of Kibana, so clone a second Kibana of your personal form along side your existing clone. Personally I have two co-located Kibana folders (
./elastic/kibana
and./elastic/_kibana
, where the first is my working clone and the other is never used for actual dev work, but that's just me -GM). - You can run the FTS in the main clone of your fork by running
node scripts/functional_tests_server.js --config=test/plugin_api_perf/config.js
in thex-pack
folder. - Once you've began running the default FTS, you want your second FTS to run such that it is referencing the Elasticsearch instance started by that first FTS. You achieve this by exporting a
TEST_ES_URL
Environment variable that points at it. By default, you should be able to run this:export TEST_ES_URL=http://elastic:changeme@localhost:9220
. Do this in a terminal window opened in your second clone of Kibana (in my case, the./elastic/_kibana
folder). - One issue I encountered with FTS is that I can't tell it not to start its own ES instance at all. To achieve this, in
packages/kbn-test/src/functional_tests/tasks.js
you need to comment out the line that starts up its own ES (const es = await runElasticsearch({ config, options: opts });
[line 85] andawait es.cleanup();
shortly after) - Next you want each instance of Kibana to run with its own UUID as that is used to identify each Kibana's owned tasks. In the file
x-pack/test/functional/config.js
simple change the uuid on the line--server.uuid=
into any random UUID. - Now that you've made these changes you can kick off your second Kibana FTS by running ths following in the second clone's
x-pack
folder:TEST_KIBANA_PORT=5621 node scripts/functional_tests_server.js --config=test/plugin_api_perf/config.js
. This runs Kibana on a different port than the first FTS (5621
instead of5620
). - With two FTS Kibana running and both pointing at the same Elasticsearch. Now, you can run the actual perf test by running
node scripts/functional_test_runner.js --config=test/plugin_api_perf/config.js
in a third terminal