kibana/test/scripts/jenkins_visual_regression.sh

35 lines
984 B
Bash
Raw Normal View History

[WIP] FTR Percy changes (#36897) (#38996) * [ftr/percy] integrate percy with functional test runner * execute ftr within `percy exec` * share PERCY_BIN and GRUNT_BIN vars in setup * [license] whitelist `(Unlicense OR Apache-2.0)` * [percy] include pr/parallel info in env * echo the actual variable value * destructure required modules * [ci] calculate percy env in a script * remove outdated eslint overrides * oops, nonce should be the same for each build * take snapshots in context tests * add percy snapshots to some dashboard tests * account for extra execution of PERCY_BIN * remove usage of percy service * rename service to `visualTesting` * write a simple test * switch left-over percy test to visualTesting * set log level for percy agent * trigger ci * try using the system chrome install instead of downloading one * fix export definition * Don't skip chromium download * Add Spencer's fix to point to chrome installation * Attempting to split tests into own ci job and test files * Renumber job * Add tag to decorations file * Try new files with existing CI job * Try again to create a new CI job * Fix eslint problem with space * eslint updates * fix console test * make test names unique * Update ci group names * Try old name again * save * add separate visual_regression ci job * add visual regression ci job * add new job to jobs.yml * update path to take_percy_snapshot script * use fixed percy job total * fix common services import * move visual_create_and_add_embeddables to visual_regression project * finish moving create_and_add_embeddables * add x-pack visual regression job * migrate dashboard_snapshots tests to visual_regression * remove references to removed test files * restore some unnecessary changes * reimplement the login page tests * yarn.lock update * fix test * remove old CI_GROUP 100 block * update failed_tests trap * reduce yarn.lock changes * disable debug logging * disable visual-regression tests for now, we can enable them in specific prs
2019-06-14 20:16:48 +02:00
#!/usr/bin/env bash
set -e
if [[ -z "$IS_PIPELINE_JOB" ]] ; then
trap 'node "$KIBANA_DIR/src/dev/failed_tests/cli"' EXIT
else
Jenkins pipeline with parallel cigroups (#45285) * Pipeline * WIP some work for parallelization with ciGroups * Fix xpack kibana install dir, and add some debugging * Attempt to quick fix a few tests * Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"" This reverts commit 078ac2897fbb2e32db08990e3cbbd8cb65b6f536. * Recombine test groups, and try runbld again * Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline * Fix typo in shell script * Remove some debug code * Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT * Fix test that uses hard-coded es transport port and add it back in to parallel groups * Disable checks reporter again for now * Set env var for TEST_ES_TRANSPORT_PORT in pipeline * Update Jenkinsfile for shorter testrunner labels * Fix another hard-coded transport port * Fix a new test with hard-coded URLs * Jenkinsfile cleanup and fix one of the groups * Fix double slash * Testing vault credentials on jenkins server * Add a non-existent credential * Revert "Add a non-existent credential" This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc. * Try github-checks-reporter again * github-checks-reporter should only run for elastic/kibana, forks won't work * Clean up some debug code * Changing names around to try to make BlueOcean UI a little better * Add more stages * Make some changes to stage structure to mirror a nested example from CloudBees * Handle TODOs, and some cleanup in Jenkinsfile * Pass GIT_BRANCH when started without GHPRB, fix branch check * Fix mailer problem and add code that ensures all tests are in cigroups back in * Test adding worker/job name to junit report paths * Remove some duplication from ci_setup scripts * Fix unit test that uses junit path * Don't reinstall node every time setup_env is run * Fix yarn install logic * Fix another unit test that uses junit output dir * Download latest ES snapshot after kibana builds * Make sure junit reports are always processed * Add two failing tests for testing purposes * Add support to Jenkinsfile for kibana build e-mails * Remove some debug code for email sending * Change JOB env handling in junit paths and move it to a sub-directory * Revert "Add two failing tests for testing purposes" This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c. * Fix junit report path in test * Don't send kibana emails on build abort * Address PR feedback, formatting and use built-in url formatting library * Fix path formatting for functional test * Add email sending back in to Jenkinsfile * Fix another unit test with path problem (cherry picked from commit 27d23c41847e79e6e1dfaef25982d7a2165a490d) # Conflicts: # src/dev/ci_setup/setup.sh # src/dev/failed_tests/cli.js
2019-09-11 20:58:28 +02:00
source src/dev/ci_setup/setup_env.sh
fi
source "$KIBANA_DIR/src/dev/ci_setup/setup_percy.sh"
Jenkins pipeline with parallel cigroups (#45285) * Pipeline * WIP some work for parallelization with ciGroups * Fix xpack kibana install dir, and add some debugging * Attempt to quick fix a few tests * Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"" This reverts commit 078ac2897fbb2e32db08990e3cbbd8cb65b6f536. * Recombine test groups, and try runbld again * Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline * Fix typo in shell script * Remove some debug code * Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT * Fix test that uses hard-coded es transport port and add it back in to parallel groups * Disable checks reporter again for now * Set env var for TEST_ES_TRANSPORT_PORT in pipeline * Update Jenkinsfile for shorter testrunner labels * Fix another hard-coded transport port * Fix a new test with hard-coded URLs * Jenkinsfile cleanup and fix one of the groups * Fix double slash * Testing vault credentials on jenkins server * Add a non-existent credential * Revert "Add a non-existent credential" This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc. * Try github-checks-reporter again * github-checks-reporter should only run for elastic/kibana, forks won't work * Clean up some debug code * Changing names around to try to make BlueOcean UI a little better * Add more stages * Make some changes to stage structure to mirror a nested example from CloudBees * Handle TODOs, and some cleanup in Jenkinsfile * Pass GIT_BRANCH when started without GHPRB, fix branch check * Fix mailer problem and add code that ensures all tests are in cigroups back in * Test adding worker/job name to junit report paths * Remove some duplication from ci_setup scripts * Fix unit test that uses junit path * Don't reinstall node every time setup_env is run * Fix yarn install logic * Fix another unit test that uses junit output dir * Download latest ES snapshot after kibana builds * Make sure junit reports are always processed * Add two failing tests for testing purposes * Add support to Jenkinsfile for kibana build e-mails * Remove some debug code for email sending * Change JOB env handling in junit paths and move it to a sub-directory * Revert "Add two failing tests for testing purposes" This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c. * Fix junit report path in test * Don't send kibana emails on build abort * Address PR feedback, formatting and use built-in url formatting library * Fix path formatting for functional test * Add email sending back in to Jenkinsfile * Fix another unit test with path problem (cherry picked from commit 27d23c41847e79e6e1dfaef25982d7a2165a490d) # Conflicts: # src/dev/ci_setup/setup.sh # src/dev/failed_tests/cli.js
2019-09-11 20:58:28 +02:00
if [[ -z "$IS_PIPELINE_JOB" ]] ; then
node scripts/build --debug --oss;
linuxBuild="$(find "$KIBANA_DIR/target" -name 'kibana-oss-*-linux-x86_64.tar.gz')"
installDir="$PARENT_DIR/install/kibana"
mkdir -p "$installDir"
tar -xzf "$linuxBuild" -C "$installDir" --strip=1
else
installDir="$(realpath $PARENT_DIR/kibana/build/oss/kibana-*-SNAPSHOT-linux-x86_64)"
destDir=${installDir}-${CI_WORKER_NUMBER}
cp -R "$installDir" "$destDir"
export KIBANA_INSTALL_DIR="$destDir"
fi
[WIP] FTR Percy changes (#36897) (#38996) * [ftr/percy] integrate percy with functional test runner * execute ftr within `percy exec` * share PERCY_BIN and GRUNT_BIN vars in setup * [license] whitelist `(Unlicense OR Apache-2.0)` * [percy] include pr/parallel info in env * echo the actual variable value * destructure required modules * [ci] calculate percy env in a script * remove outdated eslint overrides * oops, nonce should be the same for each build * take snapshots in context tests * add percy snapshots to some dashboard tests * account for extra execution of PERCY_BIN * remove usage of percy service * rename service to `visualTesting` * write a simple test * switch left-over percy test to visualTesting * set log level for percy agent * trigger ci * try using the system chrome install instead of downloading one * fix export definition * Don't skip chromium download * Add Spencer's fix to point to chrome installation * Attempting to split tests into own ci job and test files * Renumber job * Add tag to decorations file * Try new files with existing CI job * Try again to create a new CI job * Fix eslint problem with space * eslint updates * fix console test * make test names unique * Update ci group names * Try old name again * save * add separate visual_regression ci job * add visual regression ci job * add new job to jobs.yml * update path to take_percy_snapshot script * use fixed percy job total * fix common services import * move visual_create_and_add_embeddables to visual_regression project * finish moving create_and_add_embeddables * add x-pack visual regression job * migrate dashboard_snapshots tests to visual_regression * remove references to removed test files * restore some unnecessary changes * reimplement the login page tests * yarn.lock update * fix test * remove old CI_GROUP 100 block * update failed_tests trap * reduce yarn.lock changes * disable debug logging * disable visual-regression tests for now, we can enable them in specific prs
2019-06-14 20:16:48 +02:00
export TEST_BROWSER_HEADLESS=1
checks-reporter-with-killswitch "Kibana visual regression tests" \
yarn run percy exec \
node scripts/functional_tests \
--debug --bail \
--kibana-install-dir "$installDir" \
--config test/visual_regression/config.ts;