Commit graph

23 commits

Author SHA1 Message Date
Brian Seeders 5d5a4ca0ab
Add PR comments to pipeline (#51504) 2019-12-03 18:06:30 -05:00
spalger fe24b3d441 disable visual regression tests 2019-12-02 13:33:00 -07:00
Brian Seeders c8eef0458b
Experimenting with delay in test startup (#51459)
* Experimenting with delay in test startup

* Move firefox tests sooner becuase they take a long time to run

* Move functional test parallel process delay to a spot that should work better for flaky test runner
2019-12-02 14:21:38 -05:00
Brian Seeders efbf012a70
Change pipeline timeout from 3 hours to 2 hours (#51098) 2019-11-20 14:32:48 -05:00
Brian Seeders 504604a4f2
Add labels to shell scripts in Jenkins (#49657) 2019-11-15 10:59:05 -05:00
Spencer 0010447f0f
[a11y] add initial accessibility functional tests (#43584)
* [a11y] add initial accessibility functional tests

* add accessibility jobs

* fix config path

* remove percy setup from scripts

* disable color-contrast rule

* apply changes from @myasonik

* define aria-controls/owns props even when suggestions aren't visible

* [ftr/a11y] only throw error when there are errors

* adding tests for management page

* add a11y test for management page

* adding ignore rules' to a11y

* accessibility test for kibana home

* 7 passing tests, 0 failures

* jest snapshot update

* support a11y test in pipeline job

* update a11y test script for pipelines

* use oss compatible ci setup

* fix exclude syntax

* add default exclusion syntax
2019-11-07 10:07:40 -05:00
liza-mae 56c32a24fd
Enable visual tests (#47861)
* Enable visual tests

* Uncomment visual tests jobs

* More testing - use dev token and only run visual tests

* Fix dev token

* Fix token again

* Skip infra tests

* Set style of table

* Try setting discover sample size

* Fix Jenkinsfile

* Remove login_page test from config (it skipped but failing)

* Remove line

* Prep to merge, revert token and update Jenkinsfile

* Try skipping at Security level, instead of removing from config
2019-10-30 14:19:40 -06:00
Brian Seeders 9da6c07059
Re-add pipeline for flaky test runner job (#48781)
* Revert "Revert "Add pipeline for flaky test runner job (#46740)""

This reverts commit 7d96a13fad.

Also reconcile changes to Jenkinsfile since original revert happened

* Fix param parsing and add missed change

* Add missing variable
2019-10-25 12:57:46 -04:00
Brian Seeders 62b21dd421
Add ability to use ram disk for jenkins workspace (#47746)
* Add ability to use ram disk for jenkins workspace

* Re-combine ciGroup agents

* Address some PR feedback / questions

* Add --preserve-root
2019-10-18 12:13:21 -04:00
Spencer 9b34d6b76f
refactor failed_tests_reporter to use TS, no octokit (#46993)
* refactor failed_tests_reporter to use TS, no octokit

* update renovate config

* ensure that all kbn-test files are in ts project

* fix some type errors

* add some more tests

* [kbn-test/githubapi] cleanup and document

* collect log messages as strings instead of message objects

* ensure issue is open when updating body

* improve readability of getKibanaIssues

* expose axios helpers from dev-utils

* fix request params for fetching github issues and validate locally

* include a README for failed_tests_reporter

* improve axios error helpers
2019-10-08 13:56:07 -07:00
Spencer 3ea2771ce4
try running fewer jobs in parallel on the same worker (#47403) 2019-10-05 20:51:07 -07:00
Brian Seeders 7d96a13fad
Revert "Add pipeline for flaky test runner job (#46740)"
This reverts commit e89425552b.
2019-10-04 16:55:20 -04:00
Brian Seeders e89425552b
Add pipeline for flaky test runner job (#46740)
* WIP Jenkinsfile for flaky test runner

* Fix syntax

* A few more jenkinsfile fixes

* A few more fixes

* Can't round numbers in Jenkins groovy sandbox apparently

* More fixes

* Only do build_kbn_tp_sample_panel_action once during flaky testing

* Fix path and try setting a different JOB value

* Keep track of flaky test failures

* Introduce flaky test for testing

* Some flaky test pipeline cleanup

* Fix a couple of issues with flaky test failure tracking

* Update flaky test runner build name/desc with metadata

* Revert "Introduce flaky test for testing"

This reverts commit 202e9d86cf130ef37aff1817781a5b694aa2aa32.

* Try adding a local shared library

* Move local library loading logic to kibana pipeline library

* Move shared groovy code to shared library

* Add missed file

* Add ability to specify multiple agents for flaky test runner

* Update kibana-pipeline-library version

* Fix bug causing early exit for oss ciGroup1
2019-10-04 13:53:46 -04:00
Brian Seeders af4db98ccc
Use different method of determining build status in pipelines (#45814)
* Switch method of determining build status for pipeline jobs to handle aborted builds better
2019-09-27 14:45:12 -04:00
Brian Seeders cafc857aba Re-split ciGroups after pipeline rollout (#46375)
* Re-split ciGroups after pipeline rollout

Revert "Revert "Revert "Revert "Revert "[ci] compress jobs for CI stab… (#45454)"

This reverts commit 9a109f2170.

Revert "set IS_PIPELINE_JOB in intake jobs (#45850)"

This reverts commit b1a01effa8.

* Split one of the slow test suites up to try to make overall CI faster

* Disable visualRegression groups, they are being handled in other work

* Revert "Split one of the slow test suites up to try to make overall CI faster"

This reverts commit 1213239545.

* Move some different xpack ciGroup8 suites around
2019-09-25 15:18:37 -07:00
Brian Seeders 4142659079
Run pipeline scripts with runbld (#45844)
Change how runbld is invoked, and skip junit processing by default
2019-09-18 10:38:39 -04:00
Spencer b1a01effa8
set IS_PIPELINE_JOB in intake jobs (#45850)
* set IS_PIPELINE_JOB in intake jobs

* firefox smoke doesn't get an initial build

* firefox smoke doesn't get an initial build (oss edition)

* re-enable all jobs
2019-09-17 10:26:32 -07:00
Brian Seeders b8469350cf
GHPRB-derived env vars should be blank when missing, not the string 'null' (#45642) 2019-09-13 17:43:10 -04:00
Brian Seeders e3a3b7fbd8
Fix a few issues related to sending emails in pipeline (#45470) 2019-09-13 15:15:49 -04:00
Spencer ef5c520826
Call failure reporter directly from Jenkinsfile (#45551)
* Run failed_tests/cli once per node

* force failures for testing

* Revert "force failures for testing"

This reverts commit 3ddd9240fe.
2019-09-13 11:15:38 -07:00
Brian Seeders e92857090b
Don't set any job properties from the Jenkinsfile (#45522) 2019-09-12 10:22:36 -04:00
Spencer 9a109f2170
Revert "Revert "Revert "Revert "[ci] compress jobs for CI stab… (#45454)
* Revert "Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"""

This reverts commit 148b8c0f90.

* sync changes with Jenkinsfile
2019-09-11 15:27:43 -07:00
Brian Seeders 27d23c4184 Jenkins pipeline with parallel cigroups (#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)""

This reverts commit 078ac2897f.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem
2019-09-11 11:58:28 -07:00