Go to file
Tim Sullivan d568e65e78
Core task manager (#24356)
* Core task manager (#23555)

* Initial stab at core task manager logic

* Update task_manager readme

* Update task manager readme

* Add cancelable helper package, OSS, and general purpose, but
will be useful for writing cancelable x-pack tasks.

* Make the cancellable package promise-compliant

* Update task manager readme w/ reference to cancellable

* Change pool from lazy to eager, add support for sub-pools per task type.

* Move cancellable to task_manager, and typescriptify it.

* Working proof of concept for task manager. Still have lots to do:
clean up, tests, comments, validations, assertions, etc.

* Add pagination support to task manager fetch

* Move task manager to OSS

* Remove task manager reference from x-pack

* Make task_manager a valid core plugin

* Modify how task resource allocation works

* Remove the special case 'max' workers occupied value

* Remove x-pack package.json changes

* Make taskDefinitions a part of uiExports

* Make task docs saved-object compliant.

* Add kbnServer to the task context.

* Allow tasks to have a void / undefined return type

* revert x-pack change

* move cancellable to src/utils

* move to src/server

* use afterPluginsInit hook

* task_manager.ts rename

* add a wrapper with a setClient method

* Add tests for task runner

* Break task_pool into smaller, testable pieces

* return raw task doc fields for calling code

* remove todo comment

* helper module for default client
 - setClient takes a callback fn

* fix misidentified field name

* fix rest args warning

* flatten task_pool

* remove cancellable

* return raw task doc fields for calling code

* remove todo comment

* helper module for default client
 - setClient takes a callback fn

* fix rest args warning

* typescript fixes

* roll back setClient takes a callback

* createTaskRunner returns an object with run/cancel functions

* Test task store, tweak task poller tests

* Rename num_workers to max_workers

* Tweak task pool method names

* Fix cancellable test, and ts errors

* Rename doc to _doc

* Fix the doc type in the task manager index mappings

* Make task costs configurable in kibana.yml

* fix a tslint warning

* TaskManager.afterPluginsInit replaces circuitous stuff

* addMiddleware, implement beforeSave

* wip

* comment

* run context stuff

* pretty fix

* comments

* lint/pretty fixes

* safeties in case they don't define both beforeRun and beforeSave

* task runner test constructor opts fix

* Add task_manager integration tests

* FIx readme mistakes, fix schedule signature

* Fix integration tests

* Add blurb about middleware

* make a libs directory for the small things

* test for extractTaskDefinitions

* unit test for middleware

* Comment, refactor, and test task_manager

* Touch up comments

* Make task store throw error if given an unknown task type

* Fix sample task definition in readme

* Make task manager toggle on / off based on Elasticsearch connectivity

* Prevent task manager from crashing Kibana if
elasticsearch plugin is disabled

* Fix task manager signature

* Move the task manager into the task manager mixin,
fix tests. It's an uber integration object anyway, so
it seemed to make sense to keep it in one place.

* Fix task manager integration tests

* Update the task manager index creation to use auto_expand_replicas

* Fix task manager integration tests

* Fix task manager integration tests

* Fix the overzealous deletion of .kibana during tests

* Core task manager

* Allow hardcoded ID passed in

* comments

* don't deconstruct type and task for payload

* remove uiExport support

* move task_manager to x-pack/plugins

* expose as client on the plugin namespace

* fix tests

* move plugin api integration tests

* roll back readme sample task definition example

* fix sample plugin link

* server.taskManager

* sanitizeTaskDefinitions

* fix integration tests

* sanitize rename

* assert unintialized and check for duplicates in registerTaskDefinitions

* Remove unnecessary defaults from task_manager.test.ts

* Remove task manager from OSS

* Remove task manager from src

* Tidy up task_manager unit tests around elasticsearch status changes

* Integration test for specifying ID in scheduling

* Task_manager init -> preInit to allow other plugins to consume
taskManager in their init.

* Remove task manager integration tests from OSS

* spelling

* readme fix

* fix test code impacted by hapi upgrade

* Task Manager index creation changes (#24542)

* Uses putTemplate for task manager mappings and index settings.

* Removes create option.

* Opposite day, the test catches up to the code changes.

* Ignores index unavailable during searches.

* Improve taskRunner's processResult (#24880)

* Set task state to idle after run, and add failed state

* fix tests

* Test alerting demo (#25136)

* Ensures that task store init gets called before schedule.

* Removes unused option for debugging purposes.

* Fix unit tests because a second callCluster was made.

* Task manager starts sanely.

Does not preInit
Will not allow definitions after initialization
Creates store immediately.
Modifies store after all plugins have initialized
Adds static tasks that will be defined by plugins.

* Task manager should not allow operations before initialization.

* Attempts to fix runner tests.

* Fixes unit test contract with APIs.

* Removes unused type definitions.

* Removes unused package json.

* Removes unused import type.

* Removes unnecessary async applied to a function.

* Returns diferently if task store has already initialized.

* Explains how to add static task instances to task definitions. (#25880)

* Tasks cannot be scheduled until task manager is initialized.

* Adds task manager api integration tests to ci group4.

* Context of describe test must be the test framework object.

* Update src/es_archiver/lib/indices/kibana_index.js

regex update to actually disallow non-kibana system indices

Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>

* verify fillPool exception passing

* readme update about max_workers/numWorkers

* change forEach to reduce

* use public interfaces in internal method

* replace getMaxAttempts with public readonly maxAttempts

* Update x-pack/plugins/task_manager/task_store.ts

`throw new Error` and initializattion spell fix

Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>

* min = 1 for max_workers

Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>

* timeOut => timeout

* min 1

* scope as an array

* no retries

Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>

* ConcreteTaskInstance is a TaskInstance

* remove guard per joi logic

* more return types for functions

* more comments around incremental back-off

Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>

* throw error instead of return undefined

* poll_interval min 1000

* avoid handling err.stack directly

* break up processResult

* fix a few runtime issues

* only fetch idle tasks

* remove check for status idle

* always return a state, and when there is an error return the state we had at beforeRun

* check isStarted before attemptWork

Co-Authored-By: tsullivan <tsullivan@users.noreply.github.com>

* ts fix
2019-01-03 16:32:09 -07:00
.ci Reporting phantom and chromium tests should run in parallel. (#26566) 2018-12-06 13:45:40 -05:00
.github Creating Accessibility Issue Template (#25515) 2018-12-19 18:38:25 -05:00
bin [bin/windows] Remove title (#21567) 2018-08-03 06:53:54 -05:00
config Support for multiple Elasticsearch nodes and sniffing (#21928) 2018-12-18 13:22:43 -06:00
data [folder structure] plugins/.data -> data 2016-07-05 14:06:16 -05:00
docs update docs for reporting move to share menu (#27683) 2019-01-03 15:06:12 -07:00
licenses Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
packages Management: EUI Navigation sidebar (#25905) 2019-01-03 17:09:42 -06:00
plugins [folder structure] plugins/.data -> data 2016-07-05 14:06:16 -05:00
scripts Add SCSS linting (#27328) 2019-01-03 09:01:43 -07:00
src Core task manager (#24356) 2019-01-03 16:32:09 -07:00
style_guides Point to EUI for SASS/CSS guidelines (#27415) 2018-12-18 14:39:38 -05:00
tasks Add SCSS linting (#27328) 2019-01-03 09:01:43 -07:00
test Management: EUI Navigation sidebar (#25905) 2019-01-03 17:09:42 -06:00
typings Management: EUI Navigation sidebar (#25905) 2019-01-03 17:09:42 -06:00
utilities Only using known handlers with handlebars (#27061) 2018-12-17 08:29:37 -08:00
webpackShims add polyfill for ChildNode remove (#21797) 2018-08-10 15:21:11 -06:00
x-pack Core task manager (#24356) 2019-01-03 16:32:09 -07:00
.backportrc.json Add 6.6 to .backportrc.json 2018-12-19 14:51:13 -06:00
.browserslistrc Convert all Less files to Sass in Dashboard, using EUI variable scope (#21374) 2018-08-14 19:05:33 -04:00
.editorconfig Apply the same editorconfig rules to both markdown and asciidoc files 2016-09-07 11:41:41 -04:00
.eslintignore [GIS] Add Maps Plugin (#24804) 2018-12-19 16:14:41 -05:00
.eslintrc.js [GIS] Add Maps Plugin (#24804) 2018-12-19 16:14:41 -05:00
.gitignore chore(.gitignore): ignore sublime workspace files (#26516) 2018-12-07 16:11:11 +02:00
.i18nrc.json [CCR] Remote Clusters and Cross-cluster Replication apps (#26777) 2018-12-18 17:59:10 -08:00
.node-version Upgrade to NodeJS 10 (#25157) 2018-12-10 17:41:51 +00:00
.nvmrc Upgrade to NodeJS 10 (#25157) 2018-12-10 17:41:51 +00:00
.prettierrc Increase prettier line width to 100 (#20535) 2018-07-09 22:50:37 +02:00
.sass-lint.yml Add SCSS linting (#27328) 2019-01-03 09:01:43 -07:00
.yarnrc [kbn-pm] Use yarn workspaces for dependencies (#24095) 2018-11-12 12:38:11 -06:00
CONTRIBUTING.md Create vendor dll for the client modules (#22618) 2018-12-05 15:45:19 +00:00
FAQ.md propose language changes (#10709) 2017-03-05 12:10:32 -05:00
Gruntfile.js Apache 2.0 license headers (#19383) 2018-05-28 20:06:30 -07:00
kibana.d.ts EUI 5.8.1 comes with a brand color refresh. Also removes lots of KUI (#27009) 2018-12-18 10:59:31 -08:00
LICENSE.txt Migrate x-pack-kibana source to kibana 2018-04-24 13:48:10 -07:00
NOTICE.txt NOTICE (#27875) 2019-01-02 12:46:09 +01:00
package.json Add SCSS linting (#27328) 2019-01-03 09:01:43 -07:00
preinstall_check.js Apache 2.0 license headers (#19383) 2018-05-28 20:06:30 -07:00
README.md readme: remove erroneous snapshot builds link 2018-11-25 14:34:00 -05:00
STYLEGUIDE.md Point to EUI for SASS/CSS guidelines (#27415) 2018-12-18 14:39:38 -05:00
tsconfig.browser.json [typescript] continue to use the default config in development (#21966) 2018-08-14 15:27:07 -07:00
tsconfig.json Typescript: stub browser typings (#26914) 2018-12-18 17:44:39 -05:00
tsconfig.types.json Add Kibana bootstrap step to generate types exposed by the core and its plugins. (#23686) 2018-10-05 21:00:26 +02:00
tslint.yaml Adding a11y rules from tslint-microsoft-contrib to tslint (#26888) 2018-12-18 06:25:31 -05:00
TYPESCRIPT.md docs(Typescript): Update doc to declare @elastic/eui typings in global typings file (#26160) 2018-11-28 10:46:07 +01:00
yarn.lock [Reporting] Remove Phantom (#27142) 2019-01-03 14:35:15 -07:00

Kibana

Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.

Getting Started

If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.

If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.

Using a Kibana Release

If you want to use a Kibana release in production, give it a test run, or just play around:

Building and Running Kibana, and/or Contributing Code

You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:

Documentation

Visit Elastic.co for the full Kibana documentation.

For information about building the documentation, see the README in elastic/docs.

Version Compatibility with Elasticsearch

Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.

Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.

Situation Example Kibana version Example ES version Outcome
Versions are the same. 5.1.2 5.1.2 💚 OK
ES patch number is newer. 5.1.2 5.1.5 ⚠️ Logged warning
ES minor number is newer. 5.1.2 5.5.0 ⚠️ Logged warning
ES major number is newer. 5.1.2 6.0.0 🚫 Fatal error
ES patch number is older. 5.1.2 5.1.0 ⚠️ Logged warning
ES minor number is older. 5.1.2 5.0.0 🚫 Fatal error
ES major number is older. 5.1.2 4.0.0 🚫 Fatal error

Questions? Problems? Suggestions?

  • If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
  • Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.