kibana/Jenkinsfile

62 lines
3.4 KiB
Text
Raw Normal View History

Jenkins pipeline with parallel cigroups (#45285) * Pipeline * WIP some work for parallelization with ciGroups * Fix xpack kibana install dir, and add some debugging * Attempt to quick fix a few tests * Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"" This reverts commit 078ac2897fbb2e32db08990e3cbbd8cb65b6f536. * Recombine test groups, and try runbld again * Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline * Fix typo in shell script * Remove some debug code * Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT * Fix test that uses hard-coded es transport port and add it back in to parallel groups * Disable checks reporter again for now * Set env var for TEST_ES_TRANSPORT_PORT in pipeline * Update Jenkinsfile for shorter testrunner labels * Fix another hard-coded transport port * Fix a new test with hard-coded URLs * Jenkinsfile cleanup and fix one of the groups * Fix double slash * Testing vault credentials on jenkins server * Add a non-existent credential * Revert "Add a non-existent credential" This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc. * Try github-checks-reporter again * github-checks-reporter should only run for elastic/kibana, forks won't work * Clean up some debug code * Changing names around to try to make BlueOcean UI a little better * Add more stages * Make some changes to stage structure to mirror a nested example from CloudBees * Handle TODOs, and some cleanup in Jenkinsfile * Pass GIT_BRANCH when started without GHPRB, fix branch check * Fix mailer problem and add code that ensures all tests are in cigroups back in * Test adding worker/job name to junit report paths * Remove some duplication from ci_setup scripts * Fix unit test that uses junit path * Don't reinstall node every time setup_env is run * Fix yarn install logic * Fix another unit test that uses junit output dir * Download latest ES snapshot after kibana builds * Make sure junit reports are always processed * Add two failing tests for testing purposes * Add support to Jenkinsfile for kibana build e-mails * Remove some debug code for email sending * Change JOB env handling in junit paths and move it to a sub-directory * Revert "Add two failing tests for testing purposes" This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c. * Fix junit report path in test * Don't send kibana emails on build abort * Address PR feedback, formatting and use built-in url formatting library * Fix path formatting for functional test * Add email sending back in to Jenkinsfile * Fix another unit test with path problem (cherry picked from commit 27d23c41847e79e6e1dfaef25982d7a2165a490d) # Conflicts: # src/dev/ci_setup/setup.sh # src/dev/failed_tests/cli.js
2019-09-11 20:58:28 +02:00
#!/bin/groovy
library 'kibana-pipeline-library'
kibanaLibrary.load()
kibanaPipeline(timeoutMinutes: 155, checkPrChanges: true) {
ciStats.trackBuild {
githubPr.withDefaultPrComments {
catchError {
retryable.enable()
parallel([
'kibana-intake-agent': workers.intake('kibana-intake', './test/scripts/jenkins_unit.sh'),
'x-pack-intake-agent': workers.intake('x-pack-intake', './test/scripts/jenkins_xpack.sh'),
'kibana-oss-agent': workers.functional('kibana-oss-tests', { kibanaPipeline.buildOss() }, [
'oss-firefoxSmoke': kibanaPipeline.functionalTestProcess('kibana-firefoxSmoke', './test/scripts/jenkins_firefox_smoke.sh'),
'oss-ciGroup1': kibanaPipeline.ossCiGroupProcess(1),
'oss-ciGroup2': kibanaPipeline.ossCiGroupProcess(2),
'oss-ciGroup3': kibanaPipeline.ossCiGroupProcess(3),
'oss-ciGroup4': kibanaPipeline.ossCiGroupProcess(4),
'oss-ciGroup5': kibanaPipeline.ossCiGroupProcess(5),
'oss-ciGroup6': kibanaPipeline.ossCiGroupProcess(6),
'oss-ciGroup7': kibanaPipeline.ossCiGroupProcess(7),
'oss-ciGroup8': kibanaPipeline.ossCiGroupProcess(8),
'oss-ciGroup9': kibanaPipeline.ossCiGroupProcess(9),
'oss-ciGroup10': kibanaPipeline.ossCiGroupProcess(10),
'oss-ciGroup11': kibanaPipeline.ossCiGroupProcess(11),
'oss-ciGroup12': kibanaPipeline.ossCiGroupProcess(12),
'oss-accessibility': kibanaPipeline.functionalTestProcess('kibana-accessibility', './test/scripts/jenkins_accessibility.sh'),
// 'oss-visualRegression': kibanaPipeline.functionalTestProcess('visualRegression', './test/scripts/jenkins_visual_regression.sh'),
]),
'kibana-xpack-agent': workers.functional('kibana-xpack-tests', { kibanaPipeline.buildXpack() }, [
'xpack-firefoxSmoke': kibanaPipeline.functionalTestProcess('xpack-firefoxSmoke', './test/scripts/jenkins_xpack_firefox_smoke.sh'),
'xpack-ciGroup1': kibanaPipeline.xpackCiGroupProcess(1),
'xpack-ciGroup2': kibanaPipeline.xpackCiGroupProcess(2),
'xpack-ciGroup3': kibanaPipeline.xpackCiGroupProcess(3),
'xpack-ciGroup4': kibanaPipeline.xpackCiGroupProcess(4),
'xpack-ciGroup5': kibanaPipeline.xpackCiGroupProcess(5),
'xpack-ciGroup6': kibanaPipeline.xpackCiGroupProcess(6),
'xpack-ciGroup7': kibanaPipeline.xpackCiGroupProcess(7),
'xpack-ciGroup8': kibanaPipeline.xpackCiGroupProcess(8),
'xpack-ciGroup9': kibanaPipeline.xpackCiGroupProcess(9),
'xpack-ciGroup10': kibanaPipeline.xpackCiGroupProcess(10),
'xpack-accessibility': kibanaPipeline.functionalTestProcess('xpack-accessibility', './test/scripts/jenkins_xpack_accessibility.sh'),
'xpack-siemCypress': { processNumber ->
[SIEM] Client NP Cutover (#64251) (#64603) * Move SIEM public/ folder to NP plugin This is solely renames; fixes come next. * Update relative imports in our API tests * Fix linter errors following move to NP folder These paths got a little shorter, so some lines could be collapsed. * Move client dependencies to NP package.json I'm removing the @types/js-yaml for now because I'm not sure we need it; I'll add it back later if we do. * Fix relative imports to other plugins * Fix errant uses of ui/chrome * Remove legacy plugin shim * Move feature registration into plugin This previously had to be part of legacy bootstrapping due to an order of operations issue. * Disconnect legacy plugin The index file should now be redundant with what's in the plugin: * app registration * feature registration * Move public gitattributes * Remove references to legacy embeddables We can now use the NP API. Maps embeddable will not work here until their work is merged, but this should prevent us from importing legacy code and thus breaking the build. * Add our frontend dependencies to kibana.json These are all required for now, because that's how they're typed. If they _should_ be optional (and I think several should), we need to update the type and handle the null case within the app. * Replace use of ui/new_platform mocks in embeddable utils * Fix remaining jest tests * Replace build-breaking ui/new_platform mocks with equivalents in core proper * Remove unnecessary mocks of ui/new_platform * Remove references to legacy SIEM folder * I left the reference in CODEOWNERS in case someone tries to sneak something back * I left the .gitignore reference for the same reason * Fix mocks of relative paths These were not caught by typescript and were causing test failures. * Export our client plugin contracts They're empty for now. * Move from deprecated appmount API The new one dropped a param we weren't using. * Add missing mock causing test failures * Don't re-export core types from our plugin Import them from core where we need them, instead * Move Actions UI registry outside of mount This is already imported, there's no benefit (and potential timing issues) with doing this inside the mount. * Add security's setup contract to our StartServices This doesn't change what's used, only how we're typing it. The types are now a little more truthful as: * our StartPlugins don't include setup contracts * our StartServices includes everything we use at Start time, including the one setup plugin. * Add order and icon back to the sidebar link * Replace plugin class properties with constants These are shared, and should be consistent. * Enable our UI on NP * Add missed plugin dependencies We're not using their contracts, but we are importing code from them. * Revert use of constant in translation Can't do that, whoops * i18n our feature catalogue entry * Remove unnecessary array from single element * Remove unused keys These were the legacy translations used... well, I don't know where they were used. * Ignore circular dependencies in external plugins * Normalize exclusions * Add undeclared dependencies to kibana.json We import our maps embeddable from maps, and we pass inspector to the embeddable. I just missed these in my audit. This was causing errors in the map embeddable. * Await our call to setLayerList This is an async call that we need to complete before we can render. * Reduce siem plugin size When we load our initial plugin (before our app is loaded), were were implicitly importing all of kibana_react with this import. While a global module prevents this from affecting our bundle size currently, that could change in the future. Since we only need a reference to our class, we just import that instead.
2020-04-28 18:41:25 +02:00
whenChanged(['x-pack/plugins/siem/', 'x-pack/test/siem_cypress/']) {
kibanaPipeline.functionalTestProcess('xpack-siemCypress', './test/scripts/jenkins_siem_cypress.sh')(processNumber)
}
},
// 'xpack-visualRegression': kibanaPipeline.functionalTestProcess('xpack-visualRegression', './test/scripts/jenkins_xpack_visual_regression.sh'),
]),
])
}
Jenkins pipeline with parallel cigroups (#45285) * Pipeline * WIP some work for parallelization with ciGroups * Fix xpack kibana install dir, and add some debugging * Attempt to quick fix a few tests * Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"" This reverts commit 078ac2897fbb2e32db08990e3cbbd8cb65b6f536. * Recombine test groups, and try runbld again * Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline * Fix typo in shell script * Remove some debug code * Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT * Fix test that uses hard-coded es transport port and add it back in to parallel groups * Disable checks reporter again for now * Set env var for TEST_ES_TRANSPORT_PORT in pipeline * Update Jenkinsfile for shorter testrunner labels * Fix another hard-coded transport port * Fix a new test with hard-coded URLs * Jenkinsfile cleanup and fix one of the groups * Fix double slash * Testing vault credentials on jenkins server * Add a non-existent credential * Revert "Add a non-existent credential" This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc. * Try github-checks-reporter again * github-checks-reporter should only run for elastic/kibana, forks won't work * Clean up some debug code * Changing names around to try to make BlueOcean UI a little better * Add more stages * Make some changes to stage structure to mirror a nested example from CloudBees * Handle TODOs, and some cleanup in Jenkinsfile * Pass GIT_BRANCH when started without GHPRB, fix branch check * Fix mailer problem and add code that ensures all tests are in cigroups back in * Test adding worker/job name to junit report paths * Remove some duplication from ci_setup scripts * Fix unit test that uses junit path * Don't reinstall node every time setup_env is run * Fix yarn install logic * Fix another unit test that uses junit output dir * Download latest ES snapshot after kibana builds * Make sure junit reports are always processed * Add two failing tests for testing purposes * Add support to Jenkinsfile for kibana build e-mails * Remove some debug code for email sending * Change JOB env handling in junit paths and move it to a sub-directory * Revert "Add two failing tests for testing purposes" This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c. * Fix junit report path in test * Don't send kibana emails on build abort * Address PR feedback, formatting and use built-in url formatting library * Fix path formatting for functional test * Add email sending back in to Jenkinsfile * Fix another unit test with path problem (cherry picked from commit 27d23c41847e79e6e1dfaef25982d7a2165a490d) # Conflicts: # src/dev/ci_setup/setup.sh # src/dev/failed_tests/cli.js
2019-09-11 20:58:28 +02:00
}
if (params.NOTIFY_ON_FAILURE) {
slackNotifications.onFailure()
kibanaPipeline.sendMail()
}
}
Jenkins pipeline with parallel cigroups (#45285) * Pipeline * WIP some work for parallelization with ciGroups * Fix xpack kibana install dir, and add some debugging * Attempt to quick fix a few tests * Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"" This reverts commit 078ac2897fbb2e32db08990e3cbbd8cb65b6f536. * Recombine test groups, and try runbld again * Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline * Fix typo in shell script * Remove some debug code * Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT * Fix test that uses hard-coded es transport port and add it back in to parallel groups * Disable checks reporter again for now * Set env var for TEST_ES_TRANSPORT_PORT in pipeline * Update Jenkinsfile for shorter testrunner labels * Fix another hard-coded transport port * Fix a new test with hard-coded URLs * Jenkinsfile cleanup and fix one of the groups * Fix double slash * Testing vault credentials on jenkins server * Add a non-existent credential * Revert "Add a non-existent credential" This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc. * Try github-checks-reporter again * github-checks-reporter should only run for elastic/kibana, forks won't work * Clean up some debug code * Changing names around to try to make BlueOcean UI a little better * Add more stages * Make some changes to stage structure to mirror a nested example from CloudBees * Handle TODOs, and some cleanup in Jenkinsfile * Pass GIT_BRANCH when started without GHPRB, fix branch check * Fix mailer problem and add code that ensures all tests are in cigroups back in * Test adding worker/job name to junit report paths * Remove some duplication from ci_setup scripts * Fix unit test that uses junit path * Don't reinstall node every time setup_env is run * Fix yarn install logic * Fix another unit test that uses junit output dir * Download latest ES snapshot after kibana builds * Make sure junit reports are always processed * Add two failing tests for testing purposes * Add support to Jenkinsfile for kibana build e-mails * Remove some debug code for email sending * Change JOB env handling in junit paths and move it to a sub-directory * Revert "Add two failing tests for testing purposes" This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c. * Fix junit report path in test * Don't send kibana emails on build abort * Address PR feedback, formatting and use built-in url formatting library * Fix path formatting for functional test * Add email sending back in to Jenkinsfile * Fix another unit test with path problem (cherry picked from commit 27d23c41847e79e6e1dfaef25982d7a2165a490d) # Conflicts: # src/dev/ci_setup/setup.sh # src/dev/failed_tests/cli.js
2019-09-11 20:58:28 +02:00
}