kibana/x-pack/plugins/apm
Mikhail Shustov a15178a1b0
[7.x] ES client : use the new type definitions (#83808) (#95391)
* ES client : use the new type definitions (#83808)

* Use client from branch

* Get type checking working in core

* Fix types in other plugins

* Update client types + remove type errors from core

* migrate Task Manager Elasticsearch typing from legacy library to client library

* use SortOrder instead o string in alerts

* Update client types + fix core type issues

* fix maps ts errors

* Update Lens types

* Convert Search Profiler body from a string to an object to conform to SearchRequest type.

* Fix SOT types

* Fix/mute Security/Spaces plugins type errors.

* Fix bootstrap types

* Fix painless_lab

* corrected es typing in Event Log

* Use new types from client for inferred search responses

* Latest type defs

* Integrate latest type defs for APM/UX

* fix core errors

* fix telemetry errors

* fix canvas errors

* fix data_enhanced errors

* fix event_log errors

* mute lens errors

* fix or mute maps errors

* fix reporting errors

* fix security errors

* mute errors in task_manager

* fix errors in telemetry_collection_xpack

* fix errors in data plugins

* fix errors in alerts

* mute errors in index_management

* fix task_manager errors

* mute or fix lens errors

* fix upgrade_assistant errors

* fix or mute errors in index_lifecycle_management

* fix discover errors

* fix core tests

* ML changes

* fix core type errors

* mute error in kbn-es-archiver

* fix error in data plugin

* fix error in telemetry plugin

* fix error in discover

* fix discover errors

* fix errors in task_manager

* fix security errors

* fix wrong conflict resolution

* address errors with upstream code

* update deps to the last commit

* remove outdated comments

* fix core errors

* fix errors after update

* adding more expect errors to ML

* pull the lastest changes

* fix core errors

* fix errors in infra plugin

* fix errors in uptime plugin

* fix errors in ml

* fix errors in xpack telemetry

* fix or mute errors in transform

* fix errors in upgrade assistant

* fix or mute fleet errors

* start fixing apm errors

* fix errors in osquery

* fix telemetry tests

* core cleanup

* fix asMutableArray imports

* cleanup

* data_enhanced cleanup

* cleanup events_log

* cleaup

* fix error in kbn-es-archiver

* fix errors in kbn-es-archiver

* fix errors in kbn-es-archiver

* fix ES typings for Hit

* fix SO

* fix actions plugin

* fix fleet

* fix maps

* fix stack_alerts

* fix eslint problems

* fix event_log unit tests

* fix failures in data_enhanced tests

* fix test failure in kbn-es-archiver

* fix test failures in index_pattern_management

* fixing ML test

* remove outdated comment in kbn-es-archiver

* fix error type in ml

* fix eslint errors in osquery plugin

* fix runtime error in infra plugin

* revert changes to event_log cluser exist check

* fix eslint error in osquery

* fixing ML endpoint argument types

* fx types

* Update api-extractor docs

* attempt fix for ese test

* Fix lint error

* Fix types for ts refs

* Fix data_enhanced unit test

* fix lens types

* generate docs

* Fix a number of type issues in monitoring and ml

* fix triggers_actions_ui

* Fix ILM functional test

* Put search.d.ts typings back

* fix data plugin

* Update typings in typings/elasticsearch

* Update snapshots

* mute errors in task_manager

* mute fleet errors

* lens. remove unnecessary ts-expect-errors

* fix errors in stack_alerts

* mute errors in osquery

* fix errors in security_solution

* fix errors in lists

* fix errors in cases

* mute errors in search_examples

* use KibanaClient to enforce promise-based API

* fix errors in test/ folder

* update comment

* fix errors in x-pack/test folder

* fix errors in ml plugin

* fix optional fields in ml api_integartoon tests

* fix another casting problem in ml tests

* fix another ml test failure

* fix fleet problem after conflict resolution

* rollback changes in security_solution. trying to fix test

* Update type for discover rows

* uncomment runtime_mappings as its outdated

* address comments from Wylie

* remove eslint error due to any

* mute error due to incompatibility

* Apply suggestions from code review

Co-authored-by: John Schulz <github.com@jfsiii.org>

* fix type error in lens tests

* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>

* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts

Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>

* update deps

* fix errors in core types

* fix errors for the new elastic/elasticsearch version

* remove unused type

* remove unnecessary manual type cast and put optional chaining back

* ML: mute Datafeed is missing indices_options

* Apply suggestions from code review

Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>

* use canary pacakge instead of git commit

Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Nathan Reese <reese.nathan@gmail.com>
Co-authored-by: Wylie Conlon <wylieconlon@gmail.com>
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
Co-authored-by: restrry <restrry@gmail.com>
Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: John Schulz <github.com@jfsiii.org>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
# Conflicts:
#	package.json
#	src/core/server/saved_objects/migrationsv2/integration_tests/migration.test.ts
#	src/plugins/data/server/autocomplete/value_suggestions_route.ts
#	src/plugins/discover/public/application/angular/doc_table/create_doc_table_react.tsx
#	x-pack/plugins/apm/server/lib/services/get_service_dependencies/get_destination_map.ts
#	x-pack/plugins/infra/server/lib/alerting/inventory_metric_threshold/evaluate_condition.ts
#	x-pack/plugins/security/server/authentication/api_keys/api_keys.ts
#	yarn.lock

* fix file broken during conflict resolution

* fix lint errors and test failure

Co-authored-by: Tomas Della Vedova <delvedor@users.noreply.github.com>
2021-03-25 10:33:44 -04:00
..
.storybook [7.x] Elastic License 2.0 (#90192) 2021-02-03 18:39:13 -08:00
common Rename alerts plugin to alerting (#92898) (#93827) 2021-03-05 21:38:33 -05:00
dev_docs Converts usage collection README to .mdx (#92982) (#93186) 2021-03-02 06:54:12 -05:00
e2e chore(NA): ensure scripts run with --preserve-symlinks (#94046) (#94636) 2021-03-15 18:57:30 -04:00
ftr_e2e Initial version of adding artifacts per policy to the manifest. (#89130) (#90968) 2021-02-10 18:42:53 +01:00
public [7.x] ES client : use the new type definitions (#83808) (#95391) 2021-03-25 10:33:44 -04:00
scripts [7.x] ES client : use the new type definitions (#83808) (#95391) 2021-03-25 10:33:44 -04:00
server [7.x] ES client : use the new type definitions (#83808) (#95391) 2021-03-25 10:33:44 -04:00
typings [APM] Profiling (#91818) (#92250) 2021-02-22 15:13:05 -05:00
.prettierrc
CONTRIBUTING.md
jest.config.js [7.x] Elastic License 2.0 (#90192) 2021-02-03 18:39:13 -08:00
kibana.json Rename alerts plugin to alerting (#92898) (#93827) 2021-03-05 21:38:33 -05:00
readme.md [APM] Fix readme for running tsc (#95074) (#95119) 2021-03-22 20:29:52 -04:00
tsconfig.json Merge tsconfig and x-pack/tsconfig files (#94519) (#94703) 2021-03-16 12:33:47 -04:00

Documentation for APM UI developers

Local environment setup

Kibana

git clone git@github.com:elastic/kibana.git
cd kibana/
yarn kbn bootstrap
yarn start --no-base-path

APM Server, Elasticsearch and data

To access an elasticsearch instance that has live data you have two options:

A. Connect to Elasticsearch on Cloud (internal devs only)

Find the credentials for the cluster here

B. Start Elastic Stack and APM data generators

git clone git@github.com:elastic/apm-integration-testing.git
cd apm-integration-testing/
./scripts/compose.py start master --all --no-kibana

Docker Compose is required

Testing

E2E (Cypress) tests

x-pack/plugins/apm/e2e/run-e2e.sh

Starts Kibana (:5701), APM Server (:8201) and Elasticsearch (:9201). Ingests sample data into Elasticsearch via APM Server and runs the Cypress tests

Unit testing

Note: Run the following commands from kibana/x-pack/plugins/apm.

Run unit tests

npx jest --watch

Update snapshots

npx jest --updateSnapshot

Coverage

HTML coverage report can be found in target/coverage/jest after tests have run.

open target/coverage/jest/index.html

Functional tests

Start server

node scripts/functional_tests_server --config x-pack/test/functional/config.js

Run tests

node scripts/functional_test_runner --config x-pack/test/functional/config.js --grep='APM specs'

APM tests are located in x-pack/test/functional/apps/apm. For debugging access Elasticsearch on http://localhost:9220` (elastic/changeme)

API integration tests

Our tests are separated in two suites: one suite runs with a basic license, and the other with a trial license (the equivalent of gold+). This requires separate test servers and test runners.

Basic

# Start server
node scripts/functional_tests_server --config x-pack/test/apm_api_integration/basic/config.ts

# Run tests
node scripts/functional_test_runner --config x-pack/test/apm_api_integration/basic/config.ts

The API tests for "basic" are located in x-pack/test/apm_api_integration/basic/tests.

Trial

# Start server
node scripts/functional_tests_server --config x-pack/test/apm_api_integration/trial/config.ts

# Run tests
node scripts/functional_test_runner --config x-pack/test/apm_api_integration/trial/config.ts

The API tests for "trial" are located in x-pack/test/apm_api_integration/trial/tests.

For debugging access Elasticsearch on http://localhost:9220` (elastic/changeme)

Linting

Note: Run the following commands from kibana/.

Typescript

yarn tsc --noEmit --emitDeclarationOnly false --project x-pack/plugins/apm/tsconfig.json --skipLibCheck

Prettier

yarn prettier  "./x-pack/plugins/apm/**/*.{tsx,ts,js}" --write

ESLint

yarn eslint ./x-pack/plugins/apm --fix

Setup default APM users

APM behaves differently depending on which the role and permissions a logged in user has. For testing purposes APM uses 3 custom users:

apm_read_user: Apps: read. Indices: read (apm-*)

apm_write_user: Apps: read/write. Indices: read (apm-*)

kibana_write_user Apps: read/write. Indices: None

To create the users with the correct roles run the following script:

node x-pack/plugins/apm/scripts/setup-kibana-security.js --role-suffix <github-username-or-something-unique>

The users will be created with the password specified in kibana.dev.yml for elasticsearch.password

Debugging Elasticsearch queries

All APM api endpoints accept _debug=true as a query param that will result in the underlying ES query being outputted in the Kibana backend process.

Example: /api/apm/services/my_service?_debug=true

Storybook

Start the Storybook development environment with yarn storybook apm. All files with a .stories.tsx extension will be loaded. You can access the development environment at http://localhost:9001.

Experimental features settings

To set up a flagged feature, add the name of the feature key (apm:myFeature) to commmon/ui_settings_keys.ts and the feature parameters to server/ui_settings.ts.

Test for the feature like:

import { myFeatureEnabled } from '../ui_settings_keys';
if (core.uiSettings.get(myFeatureEnabled)) {
  doStuff();
}

Settings can be managed in Kibana under Stack Management > Advanced Settings > Observability.

Further resources