Commit graph

56 commits

Author SHA1 Message Date
Brian Seeders 14b2157990
[CI] Disable tracked branch jobs in Jenkins, enable reporting in Buildkite (#112604) 2021-09-21 11:31:15 -04:00
Tiago Costa 0ac3d8568c
chore(NA): remove KBN_ES_SNAPSHOT_USE_UNVERIFIED var from Jenkinsfile 2021-07-14 17:16:57 +01:00
Tyler Smalley 50a8a28974
[test] Reverts tests skipped to promote snapshot (#104571)
* temp use unverified snapshot

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* Revert "skip failing es promotion suite (#104469)"

This reverts commit 87971e74e1.

* Revert "skip failing es promotion suite (#104467)"

This reverts commit c72ad3edcb.

* Revert "skip failing es promotion suite (#104466)"

This reverts commit a0b36c75f5.

* Revert "skip failing es promotion suite (#104413)"

This reverts commit 9773e3f678.

* Revert "skip failing es promotion suite (#104409)"

This reverts commit 79608dcc9e.

* Revert "skip failing es promotion suite (#104366)"

This reverts commit dfc5dbb31b.

* Revert "skip failing es promotion suite (#104365)"

This reverts commit 269577703a.

* Revert "skip failing es promotion suite (#104364)"

This reverts commit 16f69d24b6.

* Revert "skip failing es promotion suite (#104362)"

This reverts commit 2e492c2083.

* Revert another commit to promote ES snapshot

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-07-14 17:13:51 +01:00
Brian Seeders 7262ac53c2
[CI] Don't do CI stats reporting/failures for feature branch PRs (#99668) 2021-05-10 16:38:32 -04:00
Spencer 246f4be00e
require successful ci-stats for build success in PRs (#98846)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-04-30 13:49:00 -07:00
Tyler Smalley b9bccf33aa
[CI] Increase pipeline timeouts (#91587)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2021-02-16 15:23:06 -08:00
Tiago Costa 085698ed62
chore(NA): remove usage of unverified es snapshots (#83589)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-18 00:18:31 +00:00
Robert Oskamp 62436e3f03
Functional tests - fix esArchive mappings with runtime fields (#83530)
* Functional tests - fix esArchive mappings with runtime fields

* Modify jenkinsfile to run with unverified ES snapshot

* Adjust security index fields test for the new mapping structure

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2020-11-17 20:15:41 +00:00
Brian Seeders 0ca8e99a9f
[CI] Add pipeline task queue framework and merge workers into one (#71268) 2020-08-04 13:13:51 -04:00
Brian Seeders 329a98c2ac
[CI] In-progress Slack notifications (#74012) 2020-07-31 20:56:39 -04:00
MadameSheema 12d5b8d2f9
executes cypress tests when there is a change in parts of alerting team code we use (#73256) 2020-07-28 13:13:01 +02:00
Spencer a0f7dced13
[kbn/optimizer] report sizes of assets produced by optimizer (#71319)
* Revert "Report page load asset size (#66224)"

This reverts commit 6f57fa0b2d.

* [kbn/optimizer] report sizes of assets produced by optimizer

* coalese the fast-glob versions we're using to prevent additional installs

* update kbn/pm dist

* Revert "update kbn/pm dist"

This reverts commit 68e24f0fad.

* Revert "coalese the fast-glob versions we're using to prevent additional installs"

This reverts commit 4201fb60b6.

* remove fast-glob, just recursivly call readdirSync()

* update integration tests to use new chunk filename

Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-14 17:23:14 -07:00
Spencer b6a06dfb61
Revert #64011 and subsequent fixes (#71137)
* Revert "temporarily disable firefox functional tests in PRs (#71116)"

This reverts commit 54bd07f81b.

* Revert "[savedObjects field count] run in baseline job (#70999)"

This reverts commit 53ee7a762d.

* Revert "[CI] Add pipeline task queue framework and merge workers into one (#64011)"

This reverts commit 465ed21194.

* Revert revert of change to jenkins_xpack_visual_regression.sh

Co-authored-by: spalger <spalger@users.noreply.github.com>
2020-07-08 13:05:12 -07:00
Brian Seeders 465ed21194
[CI] Add pipeline task queue framework and merge workers into one (#64011) 2020-07-07 14:14:30 -04:00
Rudolf Meijering 2f905e7d86
CI Reporter for saved objects field count (#70580)
* CI Reporter for saved objects field count

* Metrics needs to be an array

* Fix type failures

* Link to field count issue

* Revert "Link to field count issue"

This reverts commit 8c0126b838.

* Break down field count per type

* Don't log total metric as metrics report already calculates this

* Add saved objects field count ci metrics test to codeowners

* Address review comments

* Add field count CI metrics for disabled plugins

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 16:53:47 +02:00
Brian Seeders 1ad18794ef
[CI] Record Github commit statuses outside of PRs (#69432) 2020-06-23 13:32:03 -04:00
spalger 02a3800c93 disable pageLoadMetrics job, it's gotten really flaky 2020-06-20 00:26:56 -07:00
Spencer 0577f9bdcf
include ci-stats metrics in pr comment (#68563)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-06-12 16:37:38 -07:00
Dmitry Lemeshko 6f57fa0b2d
Report page load asset size (#66224)
* performance_tests: draft version

* update cli runner and script

* ingest metrics

* save asset type for plugins

* Update src/dev/performance/ingest_metrics.ts

Co-authored-by: Spencer <email@spalger.com>

* follow review comments

* fix size calc, add FTR config, move src to kbn/test

* fix import, remove unused interface

* Update packages/kbn-test/src/page_load_metrics/capture_page_load_metrics.ts

Co-authored-by: Spencer <email@spalger.com>

* start chromium with no-sandbox

* add logging

* check page contains expected element, cut apps to 5

* fix locator & typo

* Update packages/kbn-test/src/page_load_metrics/navigation.ts

Co-authored-by: Spencer <email@spalger.com>

* Update navigation.ts

* Update navigation.ts

* bump puppeteer version

* fix typo

* update navigation script

* update config file

* update yarn.lock

* fix

* take screenshot on failure

* update screenshot title

* fix screenshot saving and error

* invalid locator

* Revert "invalid locator"

This reverts commit 3007539a69.

* run script in a loop 10 times

* Revert "run script in a loop 10 times"

This reverts commit 6cfa219140.

* path config value directly

* fix screenshots directory setup

* update maps locator, common for landing and new map

Co-authored-by: Spencer <email@spalger.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Mikhail Shustov <restrry@gmail.com>
2020-06-05 16:42:55 +02:00
Xavier Mouligneau 1216b0f7cd
[SECURITY] Rename siem plugin to security_solution (#67902)
* rename siem to security_solution

* rename siem to security solution inside of code

* rename translation keys

* fix snapshot

* replace siem for security solution in tutorial

* missing translation to be renamed

* fix types for api test integration

* updates runner file to match the new path

* change category for kibana settings

* miss renaming in advance settings

* fixes cypress tests

* fix api integration test

* fix new translation

* fix unit test

* update translation i18n

* update translation i18n II

Co-authored-by: Gloria Hornero <snootchie.boochies@gmail.com>
2020-06-04 05:35:13 -04:00
Brian Seeders 02f2e80fce
[CI] Add 20 minutes to overall build timeout 2020-05-15 12:15:19 -04:00
Brian Seeders 58b77fcb34
[CI] Add slack alerts to tracked branch jobs, change default channel, change formatting (#66580) 2020-05-15 10:39:58 -04:00
Ryland Herrick 4cc5b3a4d3
[SIEM] Client NP Cutover (#64251)
* Move SIEM public/ folder to NP plugin

This is solely renames; fixes come next.

* Update relative imports in our API tests

* Fix linter errors following move to NP folder

These paths got a little shorter, so some lines could be collapsed.

* Move client dependencies to NP package.json

I'm removing the @types/js-yaml for now because I'm not sure we need it;
I'll add it back later if we do.

* Fix relative imports to other plugins

* Fix errant uses of ui/chrome

* Remove legacy plugin shim

* Move feature registration into plugin

This previously had to be part of legacy bootstrapping due to an order
of operations issue.

* Disconnect legacy plugin

The index file should now be redundant with what's in the plugin:

* app registration
* feature registration

* Move public gitattributes

* Remove references to legacy embeddables

We can now use the NP API. Maps embeddable will not work here until
their work is merged, but this should prevent us from importing legacy
code and thus breaking the build.

* Add our frontend dependencies to kibana.json

These are all required for now, because that's how they're typed. If
they _should_ be optional (and I think several should), we need to
update the type and handle the null case within the app.

* Replace use of ui/new_platform mocks in embeddable utils

* Fix remaining jest tests

* Replace build-breaking ui/new_platform mocks with equivalents in core
proper
* Remove unnecessary mocks of ui/new_platform

* Remove references to legacy SIEM folder

* I left the reference in CODEOWNERS in case someone tries to sneak
something back
* I left the .gitignore reference for the same reason

* Fix mocks of relative paths

These were not caught by typescript and were causing test failures.

* Export our client plugin contracts

They're empty for now.

* Move from deprecated appmount API

The new one dropped a param we weren't using.

* Add missing mock causing test failures

* Don't re-export core types from our plugin

Import them from core where we need them, instead

* Move Actions UI registry outside of mount

This is already imported, there's no benefit (and potential timing
issues) with doing this inside the mount.

* Add security's setup contract to our StartServices

This doesn't change what's used, only how we're typing it. The types are now a
little more truthful as:

* our StartPlugins don't include setup contracts
* our StartServices includes everything we use at Start time, including
the one setup plugin.

* Add order and icon back to the sidebar link

* Replace plugin class properties with constants

These are shared, and should be consistent.

* Enable our UI on NP

* Add missed plugin dependencies

We're not using their contracts, but we are importing code from them.

* Revert use of constant in translation

Can't do that, whoops

* i18n our feature catalogue entry

* Remove unnecessary array from single element

* Remove unused keys

These were the legacy translations used... well, I don't know where they
were used.

* Ignore circular dependencies in external plugins

* Normalize exclusions

* Add undeclared dependencies to kibana.json

We import our maps embeddable from maps, and we pass inspector to the
embeddable. I just missed these in my audit. This was causing errors in
the map embeddable.

* Await our call to setLayerList

This is an async call that we need to complete before we can render.

* Reduce siem plugin size

When we load our initial plugin (before our app is loaded), were were
implicitly importing all of kibana_react with this import. While a
global module prevents this from affecting our bundle size currently,
that could change in the future. Since we only need a reference to our
class, we just import that instead.
2020-04-27 17:59:21 -05:00
Spencer 00ec971b6e
Implement basic CI metric reporting (#64263) 2020-04-27 13:24:52 -07:00
Dmitry Lemeshko c0bf910a70
Re-enable Firefox functional testing on CI (#64159)
* bring testing on Firefox back

* disable some tests

* skip more tests

* cut more suites for Firefox

* skip more tests for Firefox

* replace smoke tag with includeFirefox

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-04-24 23:59:14 +03:00
Ryland Herrick 40f82229f0
[SIEM] Server cutover to New Platform (#63430)
* Move server code into NP folder

* NP config is not yet used
* Relative imports are somewhat broken

* Move common folder into NP

* Move cypress folder into NP

* Move scripts folder into NP

* Move misc. config into NP folder

A few of these were moved into the cypress folder as they're
cypress-specific.

I tried to update all the relative paths but some are likely broken. I'm
not going to know until other stuff is fixed, though.

* Move value for siem index pattern into common/constants

The other default values live in there, this is no different.

* Update paths following file move

If this was referencing the full project, it now references both paths
(legacy for UI, and NP for server).

* Fix typescript errors related to module resolution

These are mostly updating imports to the common/ folder on the UI side
(since things changed relative to those files).

* Replace Legacy Config with NP Config

* Updates plugin to use NP config
  * defines new config previously coming from savedObjects config
* cleans up legacy types

 Conflicts:
	x-pack/plugins/siem/server/lib/detection_engine/routes/rules/export_rules_route.ts
	x-pack/plugins/siem/server/lib/detection_engine/routes/rules/import_rules_route.ts
	x-pack/plugins/siem/server/lib/detection_engine/rules/types.ts
	x-pack/plugins/siem/server/plugin.ts
	x-pack/plugins/siem/server/routes/index.ts
	x-pack/plugins/siem/server/types.ts

* Remove local SIEM tsconfig

This was originally added to address an issue with tsserver, but that
issue is no longer relevant. The presence of this file confuses
typescript into thinking that siem is a separate TS project.

* Update kibana.json to declare our dependencies

These are not necessarily correct in terms of what's required/optional,
but this is what's declared in our types.

* Remove legacy plugin instantiation

* Removes legacy instantiation of server plugin, which is now handled by
NP
* Loosens legacy config spec so we no longer have to duplicate config
types

* Update tests with NP config

These were written against the old Hapi config function; now, we just
have a POJO.

* Update es_archiver helpers' paths

I'm not quite sure if these are working yet, but they're no longer
throwing errors.

* Ignore restricted path on script

This was cribbed from infra, who has made a similar change.

* Ignore restricted path on temporary savedObject mappings import

This will be changed subsequently when we switch to the NP form of
savedObject type registration.

* Add symlink to lockfile

* Fix paths on circular deps script

* Add separate config for Rule and Timeline saved objects

We had previously used the savedObjects' config, but those are not
currently exposed to us on New Platform. For now, we're going to split
this into two sets of values for the SOs we deal with
importing/exporting within the SIEM app, with the same defaults as
savedObjects.

* Fixing relative paths within cypress

These are strings that wouldn't be caught by typescript.
2020-04-17 12:31:54 -05:00
Brian Seeders 42539a56eb
Only run xpack siem cypress in PRs when there are siem changes (#60661) 2020-03-23 10:30:14 -04:00
Brian Seeders 8d19fb05a1
Skip CI based on changes in PR (#59939) 2020-03-12 11:06:53 -04:00
MadameSheema 3d4d03d2d1
activates SIEM Cypress tests in Jenkins (#59181)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-03-04 10:35:05 +01:00
Brian Seeders c90cfe208d
[CI] Pipeline refactoring (#56447) 2020-03-02 15:04:27 -05:00
spalger fe3864282a disable firefox smoke tests so we can fix flakiness out of band 2020-02-14 00:46:55 -07:00
Brian Seeders 99f224097c
Remove matrix build support (#54202) 2020-01-27 17:10:34 -05:00
Brian Seeders 4df1c4c9c1
[CI] Retry flaky tests (#53961) 2020-01-27 14:46:05 -05:00
Brian Seeders 5d5a4ca0ab
Add PR comments to pipeline (#51504) 2019-12-03 18:06:30 -05:00
spalger fe24b3d441 disable visual regression tests 2019-12-02 13:33:00 -07:00
Brian Seeders c8eef0458b
Experimenting with delay in test startup (#51459)
* Experimenting with delay in test startup

* Move firefox tests sooner becuase they take a long time to run

* Move functional test parallel process delay to a spot that should work better for flaky test runner
2019-12-02 14:21:38 -05:00
Brian Seeders efbf012a70
Change pipeline timeout from 3 hours to 2 hours (#51098) 2019-11-20 14:32:48 -05:00
Brian Seeders 504604a4f2
Add labels to shell scripts in Jenkins (#49657) 2019-11-15 10:59:05 -05:00
Spencer 0010447f0f
[a11y] add initial accessibility functional tests (#43584)
* [a11y] add initial accessibility functional tests

* add accessibility jobs

* fix config path

* remove percy setup from scripts

* disable color-contrast rule

* apply changes from @myasonik

* define aria-controls/owns props even when suggestions aren't visible

* [ftr/a11y] only throw error when there are errors

* adding tests for management page

* add a11y test for management page

* adding ignore rules' to a11y

* accessibility test for kibana home

* 7 passing tests, 0 failures

* jest snapshot update

* support a11y test in pipeline job

* update a11y test script for pipelines

* use oss compatible ci setup

* fix exclude syntax

* add default exclusion syntax
2019-11-07 10:07:40 -05:00
liza-mae 56c32a24fd
Enable visual tests (#47861)
* Enable visual tests

* Uncomment visual tests jobs

* More testing - use dev token and only run visual tests

* Fix dev token

* Fix token again

* Skip infra tests

* Set style of table

* Try setting discover sample size

* Fix Jenkinsfile

* Remove login_page test from config (it skipped but failing)

* Remove line

* Prep to merge, revert token and update Jenkinsfile

* Try skipping at Security level, instead of removing from config
2019-10-30 14:19:40 -06:00
Brian Seeders 9da6c07059
Re-add pipeline for flaky test runner job (#48781)
* Revert "Revert "Add pipeline for flaky test runner job (#46740)""

This reverts commit 7d96a13fad.

Also reconcile changes to Jenkinsfile since original revert happened

* Fix param parsing and add missed change

* Add missing variable
2019-10-25 12:57:46 -04:00
Brian Seeders 62b21dd421
Add ability to use ram disk for jenkins workspace (#47746)
* Add ability to use ram disk for jenkins workspace

* Re-combine ciGroup agents

* Address some PR feedback / questions

* Add --preserve-root
2019-10-18 12:13:21 -04:00
Spencer 9b34d6b76f
refactor failed_tests_reporter to use TS, no octokit (#46993)
* refactor failed_tests_reporter to use TS, no octokit

* update renovate config

* ensure that all kbn-test files are in ts project

* fix some type errors

* add some more tests

* [kbn-test/githubapi] cleanup and document

* collect log messages as strings instead of message objects

* ensure issue is open when updating body

* improve readability of getKibanaIssues

* expose axios helpers from dev-utils

* fix request params for fetching github issues and validate locally

* include a README for failed_tests_reporter

* improve axios error helpers
2019-10-08 13:56:07 -07:00
Spencer 3ea2771ce4
try running fewer jobs in parallel on the same worker (#47403) 2019-10-05 20:51:07 -07:00
Brian Seeders 7d96a13fad
Revert "Add pipeline for flaky test runner job (#46740)"
This reverts commit e89425552b.
2019-10-04 16:55:20 -04:00
Brian Seeders e89425552b
Add pipeline for flaky test runner job (#46740)
* WIP Jenkinsfile for flaky test runner

* Fix syntax

* A few more jenkinsfile fixes

* A few more fixes

* Can't round numbers in Jenkins groovy sandbox apparently

* More fixes

* Only do build_kbn_tp_sample_panel_action once during flaky testing

* Fix path and try setting a different JOB value

* Keep track of flaky test failures

* Introduce flaky test for testing

* Some flaky test pipeline cleanup

* Fix a couple of issues with flaky test failure tracking

* Update flaky test runner build name/desc with metadata

* Revert "Introduce flaky test for testing"

This reverts commit 202e9d86cf130ef37aff1817781a5b694aa2aa32.

* Try adding a local shared library

* Move local library loading logic to kibana pipeline library

* Move shared groovy code to shared library

* Add missed file

* Add ability to specify multiple agents for flaky test runner

* Update kibana-pipeline-library version

* Fix bug causing early exit for oss ciGroup1
2019-10-04 13:53:46 -04:00
Brian Seeders af4db98ccc
Use different method of determining build status in pipelines (#45814)
* Switch method of determining build status for pipeline jobs to handle aborted builds better
2019-09-27 14:45:12 -04:00
Brian Seeders cafc857aba Re-split ciGroups after pipeline rollout (#46375)
* Re-split ciGroups after pipeline rollout

Revert "Revert "Revert "Revert "Revert "[ci] compress jobs for CI stab… (#45454)"

This reverts commit 9a109f2170.

Revert "set IS_PIPELINE_JOB in intake jobs (#45850)"

This reverts commit b1a01effa8.

* Split one of the slow test suites up to try to make overall CI faster

* Disable visualRegression groups, they are being handled in other work

* Revert "Split one of the slow test suites up to try to make overall CI faster"

This reverts commit 1213239545.

* Move some different xpack ciGroup8 suites around
2019-09-25 15:18:37 -07:00
Brian Seeders 4142659079
Run pipeline scripts with runbld (#45844)
Change how runbld is invoked, and skip junit processing by default
2019-09-18 10:38:39 -04:00
Spencer b1a01effa8
set IS_PIPELINE_JOB in intake jobs (#45850)
* set IS_PIPELINE_JOB in intake jobs

* firefox smoke doesn't get an initial build

* firefox smoke doesn't get an initial build (oss edition)

* re-enable all jobs
2019-09-17 10:26:32 -07:00