83259d3fb9
* [WIP] Index lifecycle (#25071) * Index lifecycle management wizard * Adding index lifecycle management files * Updates * Fix errors and add flyout for node details * New diff tool * Scroll to change for review diff * Some feedback on copy * Updating copy, moving components around and fixing bugs with the diff view * Add logic to auto enable a phase when something is set * redesign ilm * Adding server api tests * Removing debug and some tweaks from dave's work * Conditionally show this message * Policy selection cleanup * Updates for better UX * [COPYEDIT | ILM] Copyedit of text in index lifecycle management UI * Use better default text * Remove debug * Adding readme and comments * Update readme * Do not need this anymore * Remove debug or commented out code * Remove these - they are in the tests PR * Toggle system indices * Aliases are not defined here anymore * Handle rollover better in warm phase and remove from cold,delete * adding learn more link component and switching over to using that * fixing UI issue when no policies exist * various fixes * some cleanup * moving number_of_replicas due to API change * modifying some messaging * fixing typo * fixing some diff issues and not adding sattr_name if none chosen * making write alias required for template step and making necessary API changes * removing alias definition from template patching as it needs to be per index * some copy edits for clarity * fixing issue with editing existing policy when rollover starts the warm phase * addressing PR feedback on server side code * addressing PR feedback * removing additional spaces from findMatchingNodes call * changing template to index template in one more place * fixing issue with error message showing when bootstrap is successful * fixing node options for warm and cold phase * adding seconds to duration fields to match what ES supports * changing icon for enabled steps so it does not look like an error indicator * adjusting icon color for enabled lifecycle steps * fixing issue with editing an existing policy with warm phase on rollover enabled * fixing issue with default unit for age dropdowns * fixing issues with shrink action serialization and deserialization * fixing issue with deserialization of ES policy for shrink * removing shrink option from UI when primary shard count is not greater than 1 for hot phase * going straight to create policy when no policies exist * improving lifecycle policy selection * adding active badge instead of checkmark for active lifecycle policy phases * some cleanup of unneeded properties and only showing save as new when it is appropriate * removing stray fullWidth attribute * adding missing minimum for replica count for warm phase * adding scroll to top for review step * fixing issue with start warm phase after rollover introduced by time representaiton change from ES * making shrink options not show for primary shard count of 1 as you can't shrink in that situation * fixing issue with editing existing policy and saveAsNew * bare bones policy table implementation * implementing delete policy behavior * fixing sorting and paging * fixing policy table title * rudimentary navigation flow * fixing delete * Index lifecycle management wizard (#21925) * Index lifecycle management wizard * Adding index lifecycle management files * Updates * Fix errors and add flyout for node details * New diff tool * Scroll to change for review diff * Some feedback on copy * Updating copy, moving components around and fixing bugs with the diff view * Add logic to auto enable a phase when something is set * redesign ilm * Adding server api tests * Removing debug and some tweaks from dave's work * Conditionally show this message * Policy selection cleanup * Updates for better UX * [COPYEDIT | ILM] Copyedit of text in index lifecycle management UI * Use better default text * Remove debug * Adding readme and comments * Update readme * Do not need this anymore * Remove debug or commented out code * Remove these - they are in the tests PR * Toggle system indices * Aliases are not defined here anymore * Handle rollover better in warm phase and remove from cold,delete * adding learn more link component and switching over to using that * fixing UI issue when no policies exist * various fixes * some cleanup * moving number_of_replicas due to API change * modifying some messaging * fixing typo * fixing some diff issues and not adding sattr_name if none chosen * making write alias required for template step and making necessary API changes * removing alias definition from template patching as it needs to be per index * some copy edits for clarity * fixing issue with editing existing policy when rollover starts the warm phase * addressing PR feedback on server side code * addressing PR feedback * removing additional spaces from findMatchingNodes call * changing template to index template in one more place * fixing issue with error message showing when bootstrap is successful * fixing node options for warm and cold phase * adding seconds to duration fields to match what ES supports * changing icon for enabled steps so it does not look like an error indicator * adjusting icon color for enabled lifecycle steps * fixing issue with editing an existing policy with warm phase on rollover enabled * fixing issue with default unit for age dropdowns * fixing issues with shrink action serialization and deserialization * fixing issue with deserialization of ES policy for shrink * removing shrink option from UI when primary shard count is not greater than 1 for hot phase * going straight to create policy when no policies exist * improving lifecycle policy selection * adding active badge instead of checkmark for active lifecycle policy phases * some cleanup of unneeded properties and only showing save as new when it is appropriate * removing stray fullWidth attribute * adding missing minimum for replica count for warm phase * adding scroll to top for review step * fixing issue with start warm phase after rollover introduced by time representaiton change from ES * making shrink options not show for primary shard count of 1 as you can't shrink in that situation * fixing issue with editing existing policy and saveAsNew * adjusting to changes in ES API * adding version and modified date to policies table * implementing new CRUD approach * simplified delete * cleanup edit_policy * removed wizard code * fixing issue with edit policy * fixing issue with closing delete confirmation modal * making max age and max size not mutually exclusive * removing names of covered indices from policy table * changing minimum_age to min_age * first pass at index lifecycle extensions * adding retry button for ilm covered index that is in error * first pass at index lifecycle banner * i18n work * more i18n * fixing issue with node attributes * removing console.log statements * fixing issue with deserializing number_of_shards for edit policy * defaulting shrink to false and fixing ui spacing issue * removing hot phase shard count from warm phase * scrolling to first error when user submits form for edit policy * disabling UI for index management when enabled is false in kibana.yml * disabling index lifecycle management when enabled is false in kibana.yml * extending index management filter to allow for searching fields * add support for filtering to indices with errors for index lifecycle management banner * i18n work * fixing error wrappers * fixing tests * adding view JSON for index lifecycle policy on edit screen * fixing label for i18n on policy JSON flyout * removing console.log statements * fixing tests * removing console.log statements * adding key for banner extensions * fixing bad import for FormattedMessage * add link to edit index lifecycle policy from index management index summary * adding key for map of summary extensions * adding proper icon for retry lifecycle action * factoring out common min age component * factoring out common NodeAllocation component * add copy to clipboard for json policy flyout * adding validation rules for policy names to match ES rules * fixing issues with policy names with non-alphabetic characters like & ^ % ? * moving create policy button to top right and adding fill * adding better empty state for policy list * moving shrink and force merge titles and descriptions to the left * moving show JSON button to right and limiting width of JSON flyout * fixing warning about lifecycle prop type for JSON flyout * fixing issue with warning on prop isShowingErrors * removing outdated README * simplifying constants * moving components to more logical places * moving lib and api files to services * renaming provider to enricher * factoring out common data enricher behavior * consolidating index management extension code * removing unnecessary colon * Revert "removing unnecessary colon" This reverts commit |
||
|---|---|---|
| .. | ||
| .github | ||
| build_chromium | ||
| common | ||
| dev-tools | ||
| plugins | ||
| scripts | ||
| server/lib | ||
| tasks | ||
| test | ||
| test_utils | ||
| webpackShims | ||
| .gitignore | ||
| .kibana-plugin-helpers.json | ||
| gulpfile.js | ||
| index.js | ||
| package.json | ||
| README.md | ||
| tsconfig.json | ||
| tslint.yaml | ||
Elastic License Functionality
This directory tree contains files subject to the Elastic License. The files subject to the Elastic License are grouped in this directory to clearly separate them from files licensed under the Apache License 2.0.
Development
By default, Kibana will run with X-Pack installed as mentioned in the contributing guide.
Elasticsearch will run with a basic license. To run with a trial license, including security, you can specifying that with the yarn es command.
Example: yarn es snapshot --license trial --password changeme
Testing
Running unit tests_bundle
You can run unit tests by running:
yarn test
If you want to run tests only for a specific plugin (to save some time), you can run:
yarn test --plugins <plugin>[,<plugin>]* # where <plugin> is "reporting", etc.
Running single test file
Edit test file, changing top level describe to describe.only. Run tests with normal commands.
Running Jest Unit Tests
# from x-pack folder
node scripts/jest
Debugging browser tests
yarn test:browser:dev
Initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.
Run single tests by appending grep parameter to the end of the URL. For example http://localhost:9876/debug.html?grep=ML%20-%20Explorer%20Controller will only run tests with 'ML - Explorer Controller' in the describe block.
Running server unit tests
You can run server-side unit tests by running:
yarn test:server
Running functional tests
The functional UI tests, the API integration tests, and the SAML API integration tests are all run against a live browser, Kibana, and Elasticsearch install. Each set of tests is specified with a unique config that describes how to start the Elasticsearch server, the Kibana server, and what tests to run against them. The sets of tests that exist today are functional UI tests (specified by this config), API integration tests (specified by this config), and SAML API integration tests (specified by this config).
The script runs all sets of tests sequentially like so:
- builds Elasticsearch and X-Pack
- runs Elasticsearch with X-Pack
- starts up the Kibana server with X-Pack
- runs the functional UI tests against those servers
- tears down the servers
- repeats the same process for the API and SAML API integration test configs.
To do all of this in a single command run:
node scripts/functional_tests
Running UI tests
To run only the functional UI tests:
node scripts/functional_tests --config test/functional/config
It does the same as the previous command, except that it only does setup/test/teardown for the UI tests.
Running API integration tests
API integration tests are run with a unique setup usually without UI assets built for the Kibana server.
API integration tests are intended to test only programmatic API exposed by Kibana. There is no need to run browser and simulate user actions, which significantly reduces execution time. In addition, the configuration for API integration tests typically sets optimize.enabled=false for Kibana because UI assets are usually not needed for these tests.
To run only the API integration tests:
node scripts/functional_tests --config test/api_integration/config
Running SAML API integration tests
We also have SAML API integration tests which set up Elasticsearch and Kibana with SAML support. Run only API integration tests with SAML enabled like so:
node scripts/functional_tests --config test/saml_api_integration/config
Running Reporting functional tests
See here for more information on running reporting tests.
Developing functional tests
If you are developing functional tests then you probably don't want to rebuild Elasticsearch and wait for all that setup on every test run, so instead use this command to build and start just the Elasticsearch and Kibana servers:
node scripts/functional_tests_server
After the servers are started, open a new terminal and run this command to run just the tests (without tearing down Elasticsearch or Kibana):
# Make sure you are in the x-pack directory
cd x-pack
# Invoke the functional_test_runner from Kibana. Try sending --help to learn more
node ../scripts/functional_test_runner
For both of the above commands, it's crucial that you pass in --config to specify the same config file to both commands. This makes sure that the right tests will run against the right servers. Typically a set of tests and server configuration go together.
Read more about how the scripts work here.
For a deeper dive, read more about the way functional tests and servers work here.
Issues starting dev more of creating builds
You may see an error like this when you are getting started:
[14:08:15] Error: Linux x86 checksum failed
at download_phantom.js:42:15
at process._tickDomainCallback (node.js:407:9)
That's thanks to the binary Phantom downloads that have to happen, and Bitbucket being annoying with throttling and redirecting or... something. The real issue eludes me, but you have 2 options to resolve it.
- Just keep re-running the command until it passes. Eventually the downloads will work, and since they are cached, it won't ever be an issue again.
- Download them by hand from Bitbucket and copy them into the
.phantompath. We're currently using 1.9.8, and you'll need the Window, Mac, and Linux builds.