maxmustermann/kibana
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[Security Solution][Detections] Display callout if users have new ML jobs installed (#94393)

Browse source 
* WIP: Add basic structure of our ML Job callout

* Tests are not implemented
* logic is questionable
* Detections now makes redundant ML API calls

* Fix JSDoc reference

* Move ML Jobs callout to Rules page

As opposed to the more general Detections page.

* Extends callout logic to include installation of any affected jobs

* If old jobs are used with new ECS data, you'll be missing
  anomalies/alerts
* If new jobs are used with old ECS data, you'll be missing
  anomalies/alerts

* Flesh out our link to ML Job compatibility docs

This page doesn't exist yet; the URI/copy is subject to change.

* ML Job Upgrade -> ML Job Compatibility

This is a more accurate name for the concept since the problem is more
general than presence/absence of an upgrade.

* Add some placeholder copy to get the ball rolling

* Test callout behavior with different API responses

* Prevent fetching ML data when ML popover is opened/closed

We already fetch this data when the component is initially rendered. In
the normal workflow of page load -> open popover, we perform six (6) ML
API calls, 3 of which are redundant.

The one downside of this is that opening/closing the popover will not
refresh data; however, this workflow would previously have resulted in 6
API calls as well.

* Revert "Prevent fetching ML data when ML popover is opened/closed"

This reverts commit 810b78d2b9.

* Update link to relevant documentation

We're going to add a new section to this existing page, and link
directly to that heading. We should be able to generate whatever anchor
we need here, so choosing one arbitrarily on the assumption that docs
can make it work.

* Update copy from product

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Ryland Herrick 2021-03-15 19:44:18 -05:00 committed by GitHub
parent 7dc9fce0c8
commit f14ac90e7d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 235 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
x-pack/plugins/security_solution/public/common/components/links_to_docs/links_components.tsx
View file

@ -22,3 +22,10 @@ export const DetectionsRequirementsLink = () => (
linkText={i18n.DETECTIONS_REQUIREMENTS_LINK_TEXT}
/>
);
export const MlJobCompatibilityLink = () => (
<DocLink
docPath={i18n.ML_JOB_COMPATIBILITY_LINK_PATH}
linkText={i18n.ML_JOB_COMPATIBILITY_LINK_TEXT}
/>
);

8
x-pack/plugins/security_solution/public/common/components/links_to_docs/links_translations.ts
View file

@ -33,3 +33,11 @@ export const DETECTIONS_REQUIREMENTS_LINK_TEXT = i18n.translate(
defaultMessage: 'Detections prerequisites and requirements',
}
);
export const ML_JOB_COMPATIBILITY_LINK_PATH = 'alerts-ui-monitor.html#ml-job-compatibility';
export const ML_JOB_COMPATIBILITY_LINK_TEXT = i18n.translate(
'xpack.securitySolution.documentationLinks.mlJobCompatibility.text',
{
defaultMessage: 'ML job compatibility',
}
);

2
x-pack/plugins/security_solution/public/common/components/ml/hooks/use_installed_security_jobs.ts
View file

@ -31,7 +31,7 @@ export interface UseInstalledSecurityJobsReturn {
* necessary (running jobs, etc).
*
* NOTE: If you need to include jobs that are not currently installed, try the
* {@link useInstalledSecurityJobs} hook.
* {@link useSecurityJobs} hook.
*
*/
export const useInstalledSecurityJobs = (): UseInstalledSecurityJobsReturn => {

63
x-pack/plugins/security_solution/public/detections/components/callouts/ml_job_compatibility_callout/affected_job_ids.ts Normal file
View file

@ -0,0 +1,63 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
// These are the job IDs of ML jobs that are dependent on specific ECS data. If
// any of them is installed, we want to notify the user that they potentially
// have incompatibility between their beats, rules, and jobs.
// There are four modules of jobs that are affected. However, because the API
// that returns installed jobs does not include those jobs' modules, hardcoding
// the IDs from those modules (as found in e.g.
// x-pack/plugins/ml/server/models/data_recognizer/modules/security_windows/manifest.json)
// allows us to make this determination from a single API call.
export const affectedJobIds: string[] = [
// security_linux module
'v2_rare_process_by_host_linux_ecs',
'v2_linux_rare_metadata_user',
'v2_linux_rare_metadata_process',
'v2_linux_anomalous_user_name_ecs',
'v2_linux_anomalous_process_all_hosts_ecs',
'v2_linux_anomalous_network_port_activity_ecs',
// security_windows module
'v2_rare_process_by_host_windows_ecs',
'v2_windows_anomalous_network_activity_ecs',
'v2_windows_anomalous_path_activity_ecs',
'v2_windows_anomalous_process_all_hosts_ecs',
'v2_windows_anomalous_process_creation',
'v2_windows_anomalous_user_name_ecs',
'v2_windows_rare_metadata_process',
'v2_windows_rare_metadata_user',
// siem_auditbeat module
'rare_process_by_host_linux_ecs',
'linux_anomalous_network_activity_ecs',
'linux_anomalous_network_port_activity_ecs',
'linux_anomalous_network_service',
'linux_anomalous_network_url_activity_ecs',
'linux_anomalous_process_all_hosts_ecs',
'linux_anomalous_user_name_ecs',
'linux_rare_metadata_process',
'linux_rare_metadata_user',
'linux_rare_user_compiler',
'linux_rare_kernel_module_arguments',
'linux_rare_sudo_user',
'linux_system_user_discovery',
'linux_system_information_discovery',
'linux_system_process_discovery',
'linux_network_connection_discovery',
'linux_network_configuration_discovery',
// siem_winlogbeat module
'rare_process_by_host_windows_ecs',
'windows_anomalous_network_activity_ecs',
'windows_anomalous_path_activity_ecs',
'windows_anomalous_process_all_hosts_ecs',
'windows_anomalous_process_creation',
'windows_anomalous_script',
'windows_anomalous_service',
'windows_anomalous_user_name_ecs',
'windows_rare_user_runas_event',
'windows_rare_metadata_process',
'windows_rare_metadata_user',
];

69
x-pack/plugins/security_solution/public/detections/components/callouts/ml_job_compatibility_callout/index.test.tsx Normal file
View file

@ -0,0 +1,69 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { mount } from 'enzyme';
import React from 'react';
import { TestProviders } from '../../../../common/mock';
import { useInstalledSecurityJobs } from '../../../../common/components/ml/hooks/use_installed_security_jobs';
import { MlJobCompatibilityCallout } from './index';
jest.mock('../../../../common/components/ml/hooks/use_installed_security_jobs');
describe('MlJobCompatibilityCallout', () => {
it('renders when new affected jobs are installed', () => {
(useInstalledSecurityJobs as jest.Mock).mockReturnValue({
loading: false,
jobs: [{ id: 'v2_linux_rare_metadata_process' }],
});
const wrapper = mount(
<TestProviders>
<MlJobCompatibilityCallout />
</TestProviders>
);
expect(wrapper.exists('[data-test-subj="callout-ml-job-compatibility"]')).toEqual(true);
});
it('renders when old affected jobs are installed', () => {
(useInstalledSecurityJobs as jest.Mock).mockReturnValue({
loading: false,
jobs: [{ id: 'linux_rare_metadata_process' }],
});
const wrapper = mount(
<TestProviders>
<MlJobCompatibilityCallout />
</TestProviders>
);
expect(wrapper.exists('[data-test-subj="callout-ml-job-compatibility"]')).toEqual(true);
});
it('does not render if no affected jobs are installed', () => {
(useInstalledSecurityJobs as jest.Mock).mockReturnValue({
loading: false,
jobs: [{ id: 'windows_rare_user_type10_remote_login' }],
});
const wrapper = mount(
<TestProviders>
<MlJobCompatibilityCallout />
</TestProviders>
);
expect(wrapper.exists('[data-test-subj="callout-ml-job-compatibility"]')).toEqual(false);
});
it('does not render while jobs are loading', () => {
(useInstalledSecurityJobs as jest.Mock).mockReturnValue({
loading: true,
jobs: [],
});
const wrapper = mount(
<TestProviders>
<MlJobCompatibilityCallout />
</TestProviders>
);
expect(wrapper.exists('[data-test-subj="callout-ml-job-compatibility"]')).toEqual(false);
});
});

35
x-pack/plugins/security_solution/public/detections/components/callouts/ml_job_compatibility_callout/index.tsx Normal file
View file

@ -0,0 +1,35 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import React, { memo } from 'react';
import { CallOutMessage, CallOutSwitcher } from '../../../../common/components/callouts';
import { useInstalledSecurityJobs } from '../../../../common/components/ml/hooks/use_installed_security_jobs';
import { affectedJobIds } from './affected_job_ids';
import * as i18n from './translations';
const mlJobCompatibilityCalloutMessage: CallOutMessage = {
type: 'primary',
id: 'ml-job-compatibility',
title: i18n.ML_JOB_COMPATIBILITY_CALLOUT_TITLE,
description: <i18n.MlJobCompatibilityCalloutBody />,
};
const MlJobCompatibilityCalloutComponent = () => {
const { loading, jobs } = useInstalledSecurityJobs();
const newJobsInstalled = jobs.some((job) => affectedJobIds.includes(job.id));
return (
<CallOutSwitcher
namespace="detections"
condition={!loading && newJobsInstalled}
message={mlJobCompatibilityCalloutMessage}
/>
);
};
export const MlJobCompatibilityCallout = memo(MlJobCompatibilityCalloutComponent);

50
x-pack/plugins/security_solution/public/detections/components/callouts/ml_job_compatibility_callout/translations.tsx Normal file
View file

@ -0,0 +1,50 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import React from 'react';
import { i18n } from '@kbn/i18n';
import { FormattedMessage } from '@kbn/i18n/react';
import { MlJobCompatibilityLink } from '../../../../common/components/links_to_docs';
export const ML_JOB_COMPATIBILITY_CALLOUT_TITLE = i18n.translate(
'xpack.securitySolution.detectionEngine.mlJobCompatibilityCallout.messageTitle',
{
defaultMessage: 'Your ML jobs may be incompatible with your data sources and/or ML rules',
}
);
export const MlJobCompatibilityCalloutBody = () => (
<FormattedMessage
id="xpack.securitySolution.detectionEngine.mlJobCompatibilityCallout.messageBody"
defaultMessage="{summary} Related documentation: {docs}"
values={{
summary: (
<p>
<FormattedMessage
id="xpack.securitySolution.detectionEngine.mlJobCompatibilityCallout.messageBody.summary"
defaultMessage="Machine learning rules specify ML jobs that in
turn have dependencies on data fields populated by the Elastic
beats and agent integrations that were current when the ML job
was created. New ML jobs, prefixed with V2, have been updated to
operate on now-current ECS fields. If you are using multiple
versions of beats and agents, you need to create new machine
learning rules that specify the new ML (V2) jobs, and enable them
to run alongside your existing machine learning rules, in order
to ensure continued rule coverage."
/>
</p>
),
docs: (
<ul>
<li>
<MlJobCompatibilityLink />
</li>
</ul>
),
}}
/>
);

2
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/index.tsx
View file

@ -36,6 +36,7 @@ import { SecurityPageName } from '../../../../app/types';
import { LinkButton } from '../../../../common/components/links';
import { useFormatUrl } from '../../../../common/components/link_to';
import { NeedAdminForUpdateRulesCallOut } from '../../../components/callouts/need_admin_for_update_callout';
import { MlJobCompatibilityCallout } from '../../../components/callouts/ml_job_compatibility_callout';
type Func = () => Promise<void>;
@ -161,6 +162,7 @@ const RulesPageComponent: React.FC = () => {
<>
<NeedAdminForUpdateRulesCallOut />
<ReadOnlyRulesCallOut />
<MlJobCompatibilityCallout />
<ValueListsModal
showModal={showValueListsModal}
onClose={() => setShowValueListsModal(false)}