maxmustermann/kibana
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
kibana/x-pack/plugins/ingest_manager
History
Mikhail Shustov 2a82ff9566
[KP] use new ES client in SO service (#72289) 
* adapt retryCallCluster for new ES client

* review comments

* retry on 408 ResponseError

* remove legacy retry functions

* use Migrator Es client in SO migration

* update migration tests

* improve ES typings and mocks

* migrate decorate ES errors

* add repository es client

* use new es client in so repository

* update repository tests

* fix migrator integration tests

* declare _seq_no & _primary_term on get response. _source expect to be a string

* make _sourceIncludes and refresh compatible with the client

* add test for repository_es_client

* move ApiResponse to es client mocks

* TEMP: handle wait_for as true for deleteByNamespace

* add tests for migration_es_client

* TEMP: skip test for deleteByNamespace refresh

* pass ignore as transport option in mget

* log both es client and response errors

* fix update method test failures

* update deleteByNamespace refresh settings

es doesn't support 'refresh: wait_for' for `updateByQuery` endpoint

* update repository tests. we do not allow customising wait_for

* do not delegate retry logic to es client

* fix type errors after master merged

* fix repository tests

* fix security solutions code

SO doesn't throw Error with status code anymore. Always use SO error helpers

* switch error conditions to use SO error helpers

* cleanup

* address comments about mocks

* use isResponseError helper

* address comments

* fix type errors

Co-authored-by: pgayvallet <pierre.gayvallet@elastic.co>
2020-07-25 11:59:56 +02:00
..
common [Ingest Manager] Support DEGRADED state in fleet agent event (#73104) 2020-07-24 08:40:41 -04:00
dev_docs [Ingest Manager] Rename data sources to package configs (#70259) 2020-07-01 16:14:21 -07:00
public [INGEST_MANAGER] Make package config name blank for endpoint on Package Config create (#73082) 2020-07-24 09:02:23 -04:00
scripts
server [KP] use new ES client in SO service (#72289) 2020-07-25 11:59:56 +02:00
CHANGELOG.md
kibana.json [kbn/optimizer] implement "requiredBundles" property of KP plugins (#70911) 2020-07-09 18:43:17 -07:00
package.json [Ingest Manager] Use long polling for agent checkin (#68922) 2020-06-18 14:42:27 -04:00
README.md [Ingest Manager] Use docker registry for fleet api integration tests (#72621) 2020-07-22 13:29:44 -04:00
yarn.lock [Ingest Manager] Use long polling for agent checkin (#68922) 2020-06-18 14:42:27 -04:00

README.md

Ingest Manager

Plugin

  • The plugin is enabled by default. See the TypeScript type for the the available plugin configuration options
  • Adding xpack.ingestManager.enabled=false will disable the plugin including the EPM and Fleet features. It will also remove the PACKAGE_CONFIG_API_ROUTES and AGENT_CONFIG_API_ROUTES values in common/constants/routes.ts
  • Adding --xpack.ingestManager.fleet.enabled=false will disable the Fleet API & UI
  • Both EPM and Fleet require ingestManager be enabled. They are not standalone features.
  • For Gold+ license, a custom package registry URL can be used by setting xpack.ingestManager.registryUrl=http://localhost:8080

Fleet Requirements

Fleet needs to have Elasticsearch API keys enabled, and also to have TLS enabled on kibana, (if you want to run Kibana without TLS you can provide the following config flag --xpack.ingestManager.fleet.tlsCheckDisabled=false)

Also you need to configure the hosts your agent is going to use to comunication with Elasticsearch and Kibana (Not needed if you use Elastic cloud). You can use the following flags:

--xpack.ingestManager.fleet.elasticsearch.host=http://localhost:9200
--xpack.ingestManager.fleet.kibana.host=http://localhost:5601

Development

Getting started

See the Kibana docs for how to set up your dev environment, run Elasticsearch, and start Kibana

One common development workflow is:

  • Bootstrap Kibana 
    yarn kbn bootstrap
  • Start Elasticsearch in one shell 
    yarn es snapshot -E xpack.security.authc.api_key.enabled=true
  • Start Kibana in another shell 
    yarn start --xpack.ingestManager.enabled=true --no-base-path

This plugin follows the common, server, public structure from the Architecture Style Guide . We also follow the pattern of developing feature branches under your personal fork of Kibana.

Tests

API integration tests

You need to have docker to run ingest manager api integration tests

  1. In one terminal, run the tests from the Kibana root directory with

    INGEST_MANAGEMENT_PACKAGE_REGISTRY_PORT=12345 yarn test:ftr:server --config x-pack/test/ingest_manager_api_integration/config.ts

  2. in a second terminal, run the tests from the Kibana root directory with

    INGEST_MANAGEMENT_PACKAGE_REGISTRY_PORT=12345 yarn test:ftr:runner --config x-pack/test/ingest_manager_api_integration/config.ts

    Optionally you can filter which tests you want to run using --grep

    INGEST_MANAGEMENT_PACKAGE_REGISTRY_PORT=12345 yarn test:ftr:runner --config x-pack/test/ingest_manager_api_integration/config.ts --grep='fleet'