735d3bae8c
Make some changes to how we deal with data telemetry in APM and reduce the number of fields we're storing in Saved Objects in the .kibana index.
Add a telemetry doc in dev_docs explaining how telemetry is collected and how to make updates. (In this PR the docs only cover data telemetry, but there's a space for the behavioral telemetry docs.)
Stop storing the mapping for the data telemetry in the Saved Object but instead use `{ dynamic: false }`.
This reduces the number of fields used by APM in the .kibana index (as requested in #43673.)
Before:
```bash
> curl -s -X GET "admin:changeme@localhost:9200/.kibana/_field_caps?fields=*&pretty=true" | jq '.fields|length'
653
```
After:
```bash
> curl -s -X GET "admin:changeme@localhost:9200/.kibana/_field_caps?fields=*&pretty=true" | jq '.fields|length'
415
```
We don't need the mapping anymore for storing the saved object, but we still do need to update the telemetry repository when the mapping changes, and the `upload-telemetry-data` script uses that mapping when generating data.
For these purposes the mapping in now defined in TypeScript in a function in common/apm_telemetry.ts.
It's broken down into some variables that and put together as the same mapping object that was there before, but having it in this form should make it easier to update.
A new script, `merge-telemetry-mapping`, takes the telemetry repository's xpack-phone-home.json mapping, merges in the result of our mapping and replaces the file. The result can be committed to the telemetry repo, making it easier to make changes to the mapping.
References #61583
Fixes #67032
|
||
|---|---|---|
| .. | ||
| common | ||
| dev_docs | ||
| e2e | ||
| public | ||
| scripts | ||
| server | ||
| typings | ||
| .prettierrc | ||
| CONTRIBUTING.md | ||
| jest.config.js | ||
| kibana.json | ||
| readme.md | ||
Documentation for APM UI developers
Setup local environment
Kibana
git clone git@github.com:elastic/kibana.git
cd kibana/
yarn kbn bootstrap
yarn start --no-base-path
APM Server, Elasticsearch and data
To access an elasticsearch instance that has live data you have two options:
A. Connect to Elasticsearch on Cloud (internal devs only)
Find the credentials for the cluster here
B. Start Elastic Stack and APM data generators
git clone git@github.com:elastic/apm-integration-testing.git
cd apm-integration-testing/
./scripts/compose.py start master --all --no-kibana
Docker Compose is required
E2E (Cypress) tests
x-pack/plugins/apm/e2e/run-e2e.sh
Starts Kibana (:5701), APM Server (:8201) and Elasticsearch (:9201). Ingests sample data into Elasticsearch via APM Server and runs the Cypress tests
Unit testing
Note: Run the following commands from kibana/x-pack/plugins/apm.
Run unit tests
npx jest --watch
Update snapshots
npx jest --updateSnapshot
Coverage
HTML coverage report can be found in target/coverage/jest after tests have run.
open target/coverage/jest/index.html
Functional tests
Start server
node scripts/functional_tests_server --config x-pack/test/functional/config.js
Run tests
node scripts/functional_test_runner --config x-pack/test/functional/config.js --grep='APM specs'
APM tests are located in x-pack/test/functional/apps/apm.
For debugging access Elasticsearch on http://localhost:9220` (elastic/changeme)
API integration tests
Our tests are separated in two suites: one suite runs with a basic license, and the other with a trial license (the equivalent of gold+). This requires separate test servers and test runners.
Basic
# Start server
node scripts/functional_tests_server --config x-pack/test/apm_api_integration/basic/config.ts
# Run tests
node scripts/functional_test_runner --config x-pack/test/apm_api_integration/basic/config.ts
The API tests for "basic" are located in x-pack/test/apm_api_integration/basic/tests.
Trial
# Start server
node scripts/functional_tests_server --config x-pack/test/apm_api_integration/trial/config.ts
# Run tests
node scripts/functional_test_runner --config x-pack/test/apm_api_integration/trial/config.ts
The API tests for "trial" are located in x-pack/test/apm_api_integration/trial/tests.
For debugging access Elasticsearch on http://localhost:9220` (elastic/changeme)
Linting
Note: Run the following commands from kibana/.
Prettier
yarn prettier "./x-pack/plugins/apm/**/*.{tsx,ts,js}" --write
ESLint
yarn eslint ./x-pack/plugins/apm --fix
Setup default APM users
APM behaves differently depending on which the role and permissions a logged in user has. For testing purposes APM uses 3 custom users:
apm_read_user: Apps: read. Indices: read (apm-*)
apm_write_user: Apps: read/write. Indices: read (apm-*)
kibana_write_user Apps: read/write. Indices: None
To create the users with the correct roles run the following script:
node x-pack/plugins/apm/scripts/setup-kibana-security.js --role-suffix <github-username-or-something-unique>
The users will be created with the password specified in kibana.dev.yml for elasticsearch.password
Debugging Elasticsearch queries
All APM api endpoints accept _debug=true as a query param that will result in the underlying ES query being outputted in the Kibana backend process.
Example:
/api/apm/services/my_service?_debug=true
Storybook
Start the Storybook development environment with
yarn storybook apm. All files with a .stories.tsx extension will be loaded.
You can access the development environment at http://localhost:9001.