kibana/x-pack/plugins/apm/server/routes/service_map.ts

96 lines
2.7 KiB
TypeScript
Raw Normal View History

Service Map Data API at Runtime (#54027) * [APM] Runtime service maps * Make nodes interactive * Don't use smaller range query on initial request * Address feedback from Ron * Get all services separately * Get single service as well * Query both transactions/spans for initial request * Optimize 'top' query for service maps * Use agent.name from scripted metric * adds basic loading overlay * filter out service map node self reference edges from being rendered * Make service map initial load time range configurable with `xpack.apm.serviceMapInitialTimeRange` default to last 1 hour in milliseconds * ensure destination.address is not missing in the composite agg when fetching sample trace ids * wip: added incremental data fetch & progress bar * implement progressive loading design while blocking service map interaction during loading * adds filter that destination.address exists before fetching sample trace ids * reduce pairs of connections to 1 bi-directional connection with arrows on both ends of the edge * Optimize query; add update button * Allow user interaction after 5s, auto update in that time, otherwise show toast for user to update the map with button * Correctly reduce nodes/connections * - remove non-interactive state while loading - use cytoscape element definition types * - readability improvements to the ServiceMap component - only show the update map button toast after last request loads * addresses feedback for changes to the Cytoscape component * Add span.type/span.subtype do external nodes * PR feedback Co-authored-by: Dario Gieselaar <d.gieselaar@gmail.com>
2020-01-13 22:25:14 +01:00
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import Boom from 'boom';
import * as t from 'io-ts';
import {
invalidLicenseMessage,
isActivePlatinumLicense,
} from '../../common/service_map';
Service Map Data API at Runtime (#54027) * [APM] Runtime service maps * Make nodes interactive * Don't use smaller range query on initial request * Address feedback from Ron * Get all services separately * Get single service as well * Query both transactions/spans for initial request * Optimize 'top' query for service maps * Use agent.name from scripted metric * adds basic loading overlay * filter out service map node self reference edges from being rendered * Make service map initial load time range configurable with `xpack.apm.serviceMapInitialTimeRange` default to last 1 hour in milliseconds * ensure destination.address is not missing in the composite agg when fetching sample trace ids * wip: added incremental data fetch & progress bar * implement progressive loading design while blocking service map interaction during loading * adds filter that destination.address exists before fetching sample trace ids * reduce pairs of connections to 1 bi-directional connection with arrows on both ends of the edge * Optimize query; add update button * Allow user interaction after 5s, auto update in that time, otherwise show toast for user to update the map with button * Correctly reduce nodes/connections * - remove non-interactive state while loading - use cytoscape element definition types * - readability improvements to the ServiceMap component - only show the update map button toast after last request loads * addresses feedback for changes to the Cytoscape component * Add span.type/span.subtype do external nodes * PR feedback Co-authored-by: Dario Gieselaar <d.gieselaar@gmail.com>
2020-01-13 22:25:14 +01:00
import { setupRequest } from '../lib/helpers/setup_request';
import { getServiceMap } from '../lib/service_map/get_service_map';
import { getServiceMapServiceNodeInfo } from '../lib/service_map/get_service_map_service_node_info';
import { createRoute } from './create_route';
import { rangeRt, uiFiltersRt } from './default_api_types';
import { notifyFeatureUsage } from '../feature';
[APM] Metrics-powered UI (#73953) * [APM] Optimize service overview queries * Review feedback * Use correct indices/filters for service overview metrics * [APM] Optimize traces overview * Separate queries into separate file * Support union types for aggregations * [APM] Don't fetch dynamic index pattern in setupRequest We don't need a dynamic index pattern for parsing the filters from the query bar. Additionally, instead of fetching uiIndices in `getParamsForSearchRequest`, we can use `indices` that we already fetched in `setupRequest`. * use join utility function to merge requests * Correct path for UI_SETTINGS import * Query transaction indices in getEnvironments() * [APM] Introduce apm.types Instead of using a combination of index + terms filters on processor.event, add a top-level setting that allows you to define a type, which can be a processor event type, agent configuration or custom link. This allows us to more easily compose queries. * Set size of terms agg on error rate aggregation * Metrics * Use separate clients for apm events and other uses * Separate function for calculating relative impact * use UIProcessorEvent type in IURLParams type * Remove unused import * Split out strategy in helper functions * Use cloneDeep in apmEventClient.search * Consistent usage of getUseAggregatedTransactions * Update traces functional test * Update API tests * Update responses for functional tests * Review feedback * Fix type for filter/filters aggregation * Review feedback * Rename useAggregatedTransactions > searchAggregatedTransactions * Use correct route name for transaction redirect page * Closes #67744. * Review feedback; offer other strategies than 'auto' * Fix functional tests * [APM] Always load esarchives from common Instead of requiring every test suite to store its archives under {suite}/fixtures/es_archiver, always load them from common/fixtures/es_archiver. * Update script * Make sure tests pass * Update snapshots for API tests * Filter for transaction.root when fetching top traces * Make sure must_not clause is formatted correctly Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-09-16 19:20:08 +02:00
import { getSearchAggregatedTransactions } from '../lib/helpers/aggregated_transactions';
Service Map Data API at Runtime (#54027) * [APM] Runtime service maps * Make nodes interactive * Don't use smaller range query on initial request * Address feedback from Ron * Get all services separately * Get single service as well * Query both transactions/spans for initial request * Optimize 'top' query for service maps * Use agent.name from scripted metric * adds basic loading overlay * filter out service map node self reference edges from being rendered * Make service map initial load time range configurable with `xpack.apm.serviceMapInitialTimeRange` default to last 1 hour in milliseconds * ensure destination.address is not missing in the composite agg when fetching sample trace ids * wip: added incremental data fetch & progress bar * implement progressive loading design while blocking service map interaction during loading * adds filter that destination.address exists before fetching sample trace ids * reduce pairs of connections to 1 bi-directional connection with arrows on both ends of the edge * Optimize query; add update button * Allow user interaction after 5s, auto update in that time, otherwise show toast for user to update the map with button * Correctly reduce nodes/connections * - remove non-interactive state while loading - use cytoscape element definition types * - readability improvements to the ServiceMap component - only show the update map button toast after last request loads * addresses feedback for changes to the Cytoscape component * Add span.type/span.subtype do external nodes * PR feedback Co-authored-by: Dario Gieselaar <d.gieselaar@gmail.com>
2020-01-13 22:25:14 +01:00
export const serviceMapRoute = createRoute(() => ({
path: '/api/apm/service-map',
params: {
query: t.intersection([
t.partial({
environment: t.string,
2020-05-22 09:08:58 +02:00
serviceName: t.string,
}),
2020-05-22 09:08:58 +02:00
rangeRt,
]),
Service Map Data API at Runtime (#54027) * [APM] Runtime service maps * Make nodes interactive * Don't use smaller range query on initial request * Address feedback from Ron * Get all services separately * Get single service as well * Query both transactions/spans for initial request * Optimize 'top' query for service maps * Use agent.name from scripted metric * adds basic loading overlay * filter out service map node self reference edges from being rendered * Make service map initial load time range configurable with `xpack.apm.serviceMapInitialTimeRange` default to last 1 hour in milliseconds * ensure destination.address is not missing in the composite agg when fetching sample trace ids * wip: added incremental data fetch & progress bar * implement progressive loading design while blocking service map interaction during loading * adds filter that destination.address exists before fetching sample trace ids * reduce pairs of connections to 1 bi-directional connection with arrows on both ends of the edge * Optimize query; add update button * Allow user interaction after 5s, auto update in that time, otherwise show toast for user to update the map with button * Correctly reduce nodes/connections * - remove non-interactive state while loading - use cytoscape element definition types * - readability improvements to the ServiceMap component - only show the update map button toast after last request loads * addresses feedback for changes to the Cytoscape component * Add span.type/span.subtype do external nodes * PR feedback Co-authored-by: Dario Gieselaar <d.gieselaar@gmail.com>
2020-01-13 22:25:14 +01:00
},
handler: async ({ context, request }) => {
if (!context.config['xpack.apm.serviceMapEnabled']) {
throw Boom.notFound();
}
if (!isActivePlatinumLicense(context.licensing.license)) {
throw Boom.forbidden(invalidLicenseMessage);
}
notifyFeatureUsage({
licensingPlugin: context.licensing,
featureName: 'serviceMaps',
});
[APM] Service maps anomaly detection integration by environment (#70932) * Closes #69480 & #70419. - Adds anomaly detection integration to service maps backed by apm ML jobs per environment - Loads transaction stats and anomalies for each transaction types - Renders a selector in the popop to choose a transaction type to view stats * - implements original anomaly detection integration design for service maps popover - only aggregates transaction KPIs and anomaly scores for transaction.type = "request" or "page-load" - supports environment filter 'All' option to display data from all APM anomaly detection jobs - handle case where popover metrics don't exist for services outside the current environment filter * fixes some CI errors * Simplified messaging for service popop with not data in the current environment * PR feedback, renamed max anomalies -> service anomalies including the file name * - defines custom_settings.job_tags.apm_ml_version in ML job creation, then filters for it when returing valid APM ML jobs * changes shape of of service anomalies from an array to a object keyed by serviceName * removes the url encoding from ML job link href to how it was previously. * PR feedback * Popover no data state simplified: - renders the "no data" message as plain text instead of in a callout - hides the 'Anomaly detection' section if there is not anomaly data. * Fixes filtering bug when user selects 'Environment: Not defined'. Now filters properly by filtering for docs where service.environment does not exist * filters jobs fetched in the settings page by `job.custom_settings.job_tags.apm_ml_version` * Fixed bad import from last upstream merge Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-09 14:22:34 +02:00
const logger = context.logger;
Service Map Data API at Runtime (#54027) * [APM] Runtime service maps * Make nodes interactive * Don't use smaller range query on initial request * Address feedback from Ron * Get all services separately * Get single service as well * Query both transactions/spans for initial request * Optimize 'top' query for service maps * Use agent.name from scripted metric * adds basic loading overlay * filter out service map node self reference edges from being rendered * Make service map initial load time range configurable with `xpack.apm.serviceMapInitialTimeRange` default to last 1 hour in milliseconds * ensure destination.address is not missing in the composite agg when fetching sample trace ids * wip: added incremental data fetch & progress bar * implement progressive loading design while blocking service map interaction during loading * adds filter that destination.address exists before fetching sample trace ids * reduce pairs of connections to 1 bi-directional connection with arrows on both ends of the edge * Optimize query; add update button * Allow user interaction after 5s, auto update in that time, otherwise show toast for user to update the map with button * Correctly reduce nodes/connections * - remove non-interactive state while loading - use cytoscape element definition types * - readability improvements to the ServiceMap component - only show the update map button toast after last request loads * addresses feedback for changes to the Cytoscape component * Add span.type/span.subtype do external nodes * PR feedback Co-authored-by: Dario Gieselaar <d.gieselaar@gmail.com>
2020-01-13 22:25:14 +01:00
const setup = await setupRequest(context, request);
const {
2020-05-22 09:08:58 +02:00
query: { serviceName, environment },
Service Map Data API at Runtime (#54027) * [APM] Runtime service maps * Make nodes interactive * Don't use smaller range query on initial request * Address feedback from Ron * Get all services separately * Get single service as well * Query both transactions/spans for initial request * Optimize 'top' query for service maps * Use agent.name from scripted metric * adds basic loading overlay * filter out service map node self reference edges from being rendered * Make service map initial load time range configurable with `xpack.apm.serviceMapInitialTimeRange` default to last 1 hour in milliseconds * ensure destination.address is not missing in the composite agg when fetching sample trace ids * wip: added incremental data fetch & progress bar * implement progressive loading design while blocking service map interaction during loading * adds filter that destination.address exists before fetching sample trace ids * reduce pairs of connections to 1 bi-directional connection with arrows on both ends of the edge * Optimize query; add update button * Allow user interaction after 5s, auto update in that time, otherwise show toast for user to update the map with button * Correctly reduce nodes/connections * - remove non-interactive state while loading - use cytoscape element definition types * - readability improvements to the ServiceMap component - only show the update map button toast after last request loads * addresses feedback for changes to the Cytoscape component * Add span.type/span.subtype do external nodes * PR feedback Co-authored-by: Dario Gieselaar <d.gieselaar@gmail.com>
2020-01-13 22:25:14 +01:00
} = context.params;
[APM] Metrics-powered UI (#73953) * [APM] Optimize service overview queries * Review feedback * Use correct indices/filters for service overview metrics * [APM] Optimize traces overview * Separate queries into separate file * Support union types for aggregations * [APM] Don't fetch dynamic index pattern in setupRequest We don't need a dynamic index pattern for parsing the filters from the query bar. Additionally, instead of fetching uiIndices in `getParamsForSearchRequest`, we can use `indices` that we already fetched in `setupRequest`. * use join utility function to merge requests * Correct path for UI_SETTINGS import * Query transaction indices in getEnvironments() * [APM] Introduce apm.types Instead of using a combination of index + terms filters on processor.event, add a top-level setting that allows you to define a type, which can be a processor event type, agent configuration or custom link. This allows us to more easily compose queries. * Set size of terms agg on error rate aggregation * Metrics * Use separate clients for apm events and other uses * Separate function for calculating relative impact * use UIProcessorEvent type in IURLParams type * Remove unused import * Split out strategy in helper functions * Use cloneDeep in apmEventClient.search * Consistent usage of getUseAggregatedTransactions * Update traces functional test * Update API tests * Update responses for functional tests * Review feedback * Fix type for filter/filters aggregation * Review feedback * Rename useAggregatedTransactions > searchAggregatedTransactions * Use correct route name for transaction redirect page * Closes #67744. * Review feedback; offer other strategies than 'auto' * Fix functional tests * [APM] Always load esarchives from common Instead of requiring every test suite to store its archives under {suite}/fixtures/es_archiver, always load them from common/fixtures/es_archiver. * Update script * Make sure tests pass * Update snapshots for API tests * Filter for transaction.root when fetching top traces * Make sure must_not clause is formatted correctly Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-09-16 19:20:08 +02:00
const searchAggregatedTransactions = await getSearchAggregatedTransactions(
setup
);
return getServiceMap({
setup,
serviceName,
environment,
searchAggregatedTransactions,
logger,
});
2020-05-22 09:08:58 +02:00
},
Service Map Data API at Runtime (#54027) * [APM] Runtime service maps * Make nodes interactive * Don't use smaller range query on initial request * Address feedback from Ron * Get all services separately * Get single service as well * Query both transactions/spans for initial request * Optimize 'top' query for service maps * Use agent.name from scripted metric * adds basic loading overlay * filter out service map node self reference edges from being rendered * Make service map initial load time range configurable with `xpack.apm.serviceMapInitialTimeRange` default to last 1 hour in milliseconds * ensure destination.address is not missing in the composite agg when fetching sample trace ids * wip: added incremental data fetch & progress bar * implement progressive loading design while blocking service map interaction during loading * adds filter that destination.address exists before fetching sample trace ids * reduce pairs of connections to 1 bi-directional connection with arrows on both ends of the edge * Optimize query; add update button * Allow user interaction after 5s, auto update in that time, otherwise show toast for user to update the map with button * Correctly reduce nodes/connections * - remove non-interactive state while loading - use cytoscape element definition types * - readability improvements to the ServiceMap component - only show the update map button toast after last request loads * addresses feedback for changes to the Cytoscape component * Add span.type/span.subtype do external nodes * PR feedback Co-authored-by: Dario Gieselaar <d.gieselaar@gmail.com>
2020-01-13 22:25:14 +01:00
}));
export const serviceMapServiceNodeRoute = createRoute(() => ({
path: `/api/apm/service-map/service/{serviceName}`,
params: {
path: t.type({
2020-05-22 09:08:58 +02:00
serviceName: t.string,
}),
query: t.intersection([rangeRt, uiFiltersRt]),
},
handler: async ({ context, request }) => {
if (!context.config['xpack.apm.serviceMapEnabled']) {
throw Boom.notFound();
}
if (!isActivePlatinumLicense(context.licensing.license)) {
throw Boom.forbidden(invalidLicenseMessage);
}
const setup = await setupRequest(context, request);
const {
2020-05-22 09:08:58 +02:00
path: { serviceName },
} = context.params;
[APM] Metrics-powered UI (#73953) * [APM] Optimize service overview queries * Review feedback * Use correct indices/filters for service overview metrics * [APM] Optimize traces overview * Separate queries into separate file * Support union types for aggregations * [APM] Don't fetch dynamic index pattern in setupRequest We don't need a dynamic index pattern for parsing the filters from the query bar. Additionally, instead of fetching uiIndices in `getParamsForSearchRequest`, we can use `indices` that we already fetched in `setupRequest`. * use join utility function to merge requests * Correct path for UI_SETTINGS import * Query transaction indices in getEnvironments() * [APM] Introduce apm.types Instead of using a combination of index + terms filters on processor.event, add a top-level setting that allows you to define a type, which can be a processor event type, agent configuration or custom link. This allows us to more easily compose queries. * Set size of terms agg on error rate aggregation * Metrics * Use separate clients for apm events and other uses * Separate function for calculating relative impact * use UIProcessorEvent type in IURLParams type * Remove unused import * Split out strategy in helper functions * Use cloneDeep in apmEventClient.search * Consistent usage of getUseAggregatedTransactions * Update traces functional test * Update API tests * Update responses for functional tests * Review feedback * Fix type for filter/filters aggregation * Review feedback * Rename useAggregatedTransactions > searchAggregatedTransactions * Use correct route name for transaction redirect page * Closes #67744. * Review feedback; offer other strategies than 'auto' * Fix functional tests * [APM] Always load esarchives from common Instead of requiring every test suite to store its archives under {suite}/fixtures/es_archiver, always load them from common/fixtures/es_archiver. * Update script * Make sure tests pass * Update snapshots for API tests * Filter for transaction.root when fetching top traces * Make sure must_not clause is formatted correctly Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-09-16 19:20:08 +02:00
const searchAggregatedTransactions = await getSearchAggregatedTransactions(
setup
);
return getServiceMapServiceNodeInfo({
setup,
serviceName,
[APM] Metrics-powered UI (#73953) * [APM] Optimize service overview queries * Review feedback * Use correct indices/filters for service overview metrics * [APM] Optimize traces overview * Separate queries into separate file * Support union types for aggregations * [APM] Don't fetch dynamic index pattern in setupRequest We don't need a dynamic index pattern for parsing the filters from the query bar. Additionally, instead of fetching uiIndices in `getParamsForSearchRequest`, we can use `indices` that we already fetched in `setupRequest`. * use join utility function to merge requests * Correct path for UI_SETTINGS import * Query transaction indices in getEnvironments() * [APM] Introduce apm.types Instead of using a combination of index + terms filters on processor.event, add a top-level setting that allows you to define a type, which can be a processor event type, agent configuration or custom link. This allows us to more easily compose queries. * Set size of terms agg on error rate aggregation * Metrics * Use separate clients for apm events and other uses * Separate function for calculating relative impact * use UIProcessorEvent type in IURLParams type * Remove unused import * Split out strategy in helper functions * Use cloneDeep in apmEventClient.search * Consistent usage of getUseAggregatedTransactions * Update traces functional test * Update API tests * Update responses for functional tests * Review feedback * Fix type for filter/filters aggregation * Review feedback * Rename useAggregatedTransactions > searchAggregatedTransactions * Use correct route name for transaction redirect page * Closes #67744. * Review feedback; offer other strategies than 'auto' * Fix functional tests * [APM] Always load esarchives from common Instead of requiring every test suite to store its archives under {suite}/fixtures/es_archiver, always load them from common/fixtures/es_archiver. * Update script * Make sure tests pass * Update snapshots for API tests * Filter for transaction.root when fetching top traces * Make sure must_not clause is formatted correctly Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-09-16 19:20:08 +02:00
searchAggregatedTransactions,
});
2020-05-22 09:08:58 +02:00
},
}));