Commit graph

70 commits

Author SHA1 Message Date
Tyler Smalley
7211f78ce1
Bumps Jest related packages (#78720)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2020-10-01 14:38:51 -07:00
Spencer
3c85d3d2af
[kbn/es] use a basic build process (#78090)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-09-28 11:14:17 -07:00
Spencer
62f25506d3
[kbn/es] only make one attempt in tests to avoid timeout (#69197)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-06-16 09:54:41 -07:00
restrry
bf04235dae apply prettier styles 2020-05-22 09:08:58 +02:00
Spencer
77a89c2c3b
[kbn/es] retry fetching native realm users (#66511) 2020-05-14 12:03:39 -07:00
Brian Seeders
c0c7dccac7
Throw error in kbn-es when no reserved users found (#65911) 2020-05-12 10:25:12 -04:00
Larry Gregory
6a6deef151
Deprecate kibana user in favor of kibana_system user (#63186) 2020-05-05 11:36:40 -04:00
Brian Seeders
6950c260ef
kbn-es: Support choosing the correct architecture (#61096) 2020-03-24 14:12:54 -04:00
Brian Seeders
e792292923
Fix failing snapshot artifact tests when using env var (#56063) 2020-01-27 21:44:58 -05:00
Joe Portner
719ff259fc
Add support for certificates in PKCS#12 (P12) key stores (#53810)
Kibana now supports the usage of PKCS#12 (P12) key stores and trust stores for certificates and keys.
2020-01-09 13:03:16 -05:00
Brian Seeders
4b6c2f84e1
Elasticsearch snapshots automation (#53706) 2020-01-07 11:50:08 -05:00
Jonathan Budzenski
1d6eef1f7c pin es to 82c601c0 (#51075) 2019-11-19 10:00:08 -07:00
Jonathan Budzenski
1cc2a34745
Removes references to Elasticsearch mapping types (#47610)
* rm types

* fix infra/reindex test

* rm max_adjacency_matrix_filters

* lint

* more fixes

* comment return

* override documentation links

* typo

* return unfedined

* skip highlight test

* lint
2019-11-11 16:41:08 -06:00
renovate[bot]
feceb0f98e Update eslint related packages (#48513)
* Update eslint related packages

* add errorMessage to no-restricted-paths validation

* add license to eslintrc file

* ignore broken plugin fixture

* upgrade tangled peer deps

* give name to unnamed func to avoid typescript-eslint/typescript-eslint#1078

* migrate away from removed prefer-interface

* migrate away from removed no-angle-bracket-type-assertions rule

* migrate away from removed no-triple-slash-reference rule

* old config was invalid and made this rule a noop

* fix array-type rule config

* fix readonly paramProperties that weren't being checked

* fix propType ordering

* remove references to non-existent rules

* remove unnecessary disable that is improperly formatted

* fix comment-related spacing

* fix typo/invalid namespace overrides

* rename fn to avoid hook-alike behavior

* remove reference to old removed rule

* use caret version for prettier

* reference babel-eslint with absolute path
2019-10-21 16:32:31 -07:00
Spencer
26fc5c5046
[kbn/es] use an ES_TMPDIR that is within the install path (#47901)
* [kbn/es] use an ES_TMPDIR that is within the install path

* support tests which don't pass installPath
2019-10-11 07:13:22 -07:00
Spencer
3bc803e371
[mkdirp] remove in favor of recursive fs.mkdir (#47251)
* [mkdirp] remove in favor of recursive fs.mkdir

* add eslint rule to educate future contributors

* reword the eslint error message to prevent copy-pasting callback code

* Commit updated kbn-pm dist

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* Fix typo

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-10-04 08:15:06 -07:00
Spencer
510566d66b
[kbn-es] make custom snapshot url a feature (#45750)
* [kbn-es] make custom snapshot url a feature

* download es snapshot first
2019-09-15 08:10:43 -07:00
Tyler Smalley
35d33b7fbf
[kbn-es] Support for passing regex value to ES (#42651)
We test each value to determine if it's a file, in which case we move it to the
config directory. This is required to support things like SSL certificates which
are required to be within the ES directory. path.isAbsolute evalues to true for
a regular expression. In order to resolve this, we also verify that the file
exists.

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-08-08 18:03:03 -07:00
Mike Côté
aaea5679d0
Add SSL support when starting Elasticsearch from yarn es (#42527)
* Initial work

* Add integration tests

* Use constants

* Fix broken code

* Handle scenario where esArgs is a string

* Remove || []

* Apply PR feedback

* Use const format
2019-08-07 10:34:44 -04:00
Aleh Zasypkin
1a103c02e4
[@kbn/es] Add a predefined list of Elasticsearch secure settings to be added into keystore. Re-enable OIDC tests. (#42239) 2019-08-01 09:14:51 +02:00
Aleh Zasypkin
6a29ef4105
Make Kerberos authentication provider work with Elastic anonymous access. (#40994) 2019-07-15 17:30:58 +02:00
Spencer
62127f18eb
[kbn/es] add command to build es snapshots for each license an… (#40331) 2019-07-05 16:12:41 -07:00
spalger
3300b1cdf3 [master] force snapshot artifact if KBN_ES_SNAPSHOT_URL is set (#40317)
* force snapshot artifact if `KBN_ES_SNAPSHOT_URL` is set

* override the snapshot to a specific build at a custom url

(cherry picked from commit 6fda5bf3bf)
2019-07-03 18:40:41 -07:00
Jonathan Budzenski
5f1d4cac79
[kbn-es] unset JAVA_HOME when using snapshot bundled jdk (#37316)
* [kbn-es] unset JAVA_HOME when using snapshot bundled jdk

* set explicitly
2019-06-24 12:59:18 -05:00
Spencer
eeab45df7b
[kbn/es] support minor version numbers (#39353) 2019-06-20 14:52:28 -07:00
Spencer
b5ac046314
[kbn/es] auto retry snapshot download on unexpected errors (#38944)
* [kbn/es] auto retry snapshot download on unexpected errors

* missed an await

* pass log to retry()
2019-06-14 09:38:39 -07:00
Aleh Zasypkin
580edcd1c1
Introduce Kerberos authentication provider. (#36112) 2019-05-29 10:40:19 +03:00
spalger
282c5e184f [kbn-es] retry on all error types while setting up native realm
(cherry picked from commit a578261158)
2019-05-23 07:51:18 -07:00
Spencer
83b7ea76b0
[kbn/es] auto-retry all requests in native realm setup (#36867)
* [kbn/es] auto-retry all requests in native realm setup

* fix argument to autoRetry
2019-05-22 09:43:38 -07:00
spalger
e8ffa2f3c7 [kbn-es] use bare ToolingLog rather than stub (#36862) 2019-05-21 21:57:59 -07:00
spalger
8d7e1237bb loosen check for security exception and log metadata so we can investigate "offline" 2019-05-21 20:24:33 -07:00
Tyler Smalley
caaf68742f
[kbn-es] Support ES security running from source/archive (#36733)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-21 10:37:29 -07:00
Spencer
508424ef07
[kbn/es] await native realm setup, error if there are failures (#36475)
* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (#36290)""

This reverts commit a102ca1f97.

* [kbn/es] retry setPassword call if it fails
2019-05-20 13:24:18 -07:00
spalger
a102ca1f97 Revert "[kbn-es] await native realm setup, error if there are failures (#36290)"
This reverts commit 6ef9fbe423.
2019-05-10 12:52:37 -07:00
Spencer
6ef9fbe423
[kbn-es] await native realm setup, error if there are failures (#36290)
* [kbn-es] await native realm setup, error if there are failures

* cleanup listener from started promise too

* fix typo

* log native realm setup errors on run and handle 400 when xpack isn't available

* avoid creating potentially unused promises

* avoid Promise constructor that's larger than necessary

* group similar operations, add some light comments

* set passwords in parallel so that logging is grouped

* update native realm tests

* update es_bin fixture to handle _xpack requests

* log started after server is listening

* remove unused mockEsBin args

* use more realistic http address log
2019-05-10 07:42:26 -07:00
Tyler Smalley
860977694b
[kbn-es] Security updates (#36288)
* In kbn-test, provide password to cluster.start()
* Security enabled by default, overwrite by passing `-E "xpack.security.enabled=false"`

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-08 12:25:26 -07:00
Tyler Smalley
8f782a8dbd
[kbn-es] Set password for native realm accounts (#35586)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-01 09:34:14 -07:00
Aaron Zhao
f0b47f1a68 [kbn-es] Add network archive downloading to installArchive (#31535) 2019-04-23 21:48:14 -07:00
Spencer
c982856d31
[kbn/es] print headers when requests fail (#34237) 2019-03-31 18:17:12 -07:00
Spencer
eefa2a54af
[kbn/es] print first 500 bytes of invalid content (#34133)
We added checksum verification to `@kbn/es` in https://github.com/elastic/kibana/pull/33893, which is working, and in turn proves that using the Artifact API to download snapshots wasn't enough to fix the issues we've been seeing. In https://kibana-ci.elastic.co/job/elastic+kibana+7.0/412/JOB=kibana-ciGroup8,node=immutable/console we see:

```
00:14:20.762  │ info Installing from snapshot
00:14:20.763      │ info version: 7.0.0
00:14:20.763      │ info install path: /var/lib/jenkins/workspace/elastic+kibana+7.0/JOB/kibana-ciGroup8/node/immutable/kibana/.es/test-97zzlri1nyb
00:14:20.764      │ info license: oss
00:14:20.764      │ info downloading artifact info from https://artifacts-api.elastic.co/v1/versions/7.0.0-SNAPSHOT/builds/latest/projects/elasticsearch
00:14:20.852      │ info downloading artifact from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz
00:14:22.765      │ info downloading artifact checksum from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz
00:14:22.818 
00:14:22.818 artifact downloaded from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz does not match expected checksum
00:14:22.819   expected: 1e5188c1410e0299777dd1cb9638aa12de4067ea80a77ff6723986746bbc7b1a35d580023f605e6e1ae84887e675ba10184032da80240a42289ee6c5d5361c66
00:14:22.820   received: 7630553ab83828196931917aa15f17bf1f31d541fd11e0f1c82d89f294fbe47dca7be99d96da5baa0a9545712e3c9340f9b3d989e5056a3f9471f43bb582becc
```

The new logging shows that we used the explicit URL for the artifact `https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz` and still got invalid data, which confirms our suspicion that something funky is happening with the CDN. In order to understand more about what's going on here this PR adds a buffer that caches the first 500 bytes from the artifact response and will log it in utf8 format if the checksum doesn't match. I'm hoping we're going to see some HTML or something that will help explain what's happening.
2019-03-28 19:41:07 -07:00
Spencer
bedf74faf2
[kbn/es] use artifact api to determine snapshot url (#33893)
* [kbn/es] use artifact api to determine snapshot url

* actually use AbortController

* [kbn/es] handle optional os and jdk indicators in artifact name
2019-03-28 14:29:39 -07:00
Jonathan Budzenski
61f999304b [kbn-es] Platform specific snapshots (#29673) 2019-02-01 20:59:59 -08:00
Tyler Smalley
c70aeec6fd
[kbn-es] Support platform specific distributions (#29678)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-01-31 08:41:29 -08:00
Josh Dover
45a4d3bce0
[kbn-es] Add support to kbn-es and kbn-test for data archives (#28723)
* Add support to kbn-es and kbn-test for data archives

* Fix log indent

* Decompress directly to data directory

Co-Authored-By: joshdover <me@joshdover.com>

* Improve logs
2019-01-23 16:45:19 -06:00
Josh Dover
0ba1db2d5b
Add ENV_VAR for overriding ES snapshot version (#28768) 2019-01-15 11:27:54 -06:00
Tiago Costa
45a67701f2
Upgrade to NodeJS 10 (#25157)
* feat(NA): upgrade node js version on file configs.

* chore(NA): migrate configs and 3rd party dependencies to work on node js 10.x

* fix(NA): add missing async function declaration.

* chore(NA): updated elastic/good package to work with node10

* chore(NA): update lockfiles.

* fix(NA): add missing dep.

* fix(NA): types for node 10.

* test(NA): fix error return type for node10.

* fix(NA): kbn-pm webpack config to unlazy a require using lazy-cache. fix(NA): build to work with node 10.

* test(NA): jest integration test for kbn-pluin-helpers.

* test(NA): fix jest tests for kbn-es.

* fix(NA): use ostmpdir instead of a tmp folder inside the fixtures.

* fix(NA): change afterEach on kbn es decompress test.

* fix(NA): change afterEach on kbn es decompress test.

* fix(NA): readd mock-fs for the tests that still use it on kbn-es and that works on node10.

* fix(NA): readd mock-fs for the tests that still use it on kbn-es and that works on node10.

* refact(NA): rewrite tests using mock-fs and completely remove this dependency.

* fix(NA): failing test implementation using jest mock in order to replace mock-fs.

* fix(NA): update jest snapshots to match new ones generated one node 10.

* fix(NA): cli/cluster mock to spyOn off method instead off spyOn removeListener as this was changed on Node 10.

* fix(NA): tests for cluster_manager to also spyOn off and on instead of addListener and removeListener

* test(NA): fix management advance settings image field test flow.

* fix(NA): apply missing types for src/core/server/plugins/discovery/plugins_discovery.ts.

* test(NA): updated 2 missing snapshots for KuiCodeEditor on kbn-ui-framework.

* refact(NA): fix eslint errors.

* refact(NA): fix ts code with tslint fix. chore(NA): update jest snapshots.

* chore(NA): migrate kbn config schema peer dependency to last used joi version to avoid warning on bootstrap.

* fix(NA): tslint errors.

* chore(NA): upgrade types node to the last version.

* fix(NA): missing utf8 input format encoding when reading a file.

* chore(NA): upgrade to node 10.14.1

* fix(NA): Buffer api usage to avoid deprecation warnings.
2018-12-10 17:41:51 +00:00
Jonathan Budzenski
70788a9a3c
[es] remove alpha1 suffix (#25763) 2018-11-20 15:51:49 -06:00
Spencer
df8514fa09
[kbn/es] add --download flag to snapshot command to warm the cache (#25830)
In order to pre-fetch Elasticsearch snapshots and avoid timeouts during download in images, this adds a `--download` option to the `es snapshot` command.
2018-11-16 15:11:43 -08:00
Jonathan Budzenski
fa8237d197
[build] Support version-qualifier flag (#21663)
* [build] Support version-qualifier flag

* add help description

* fix test

* is-release

* temp: re-add alpha1 to package.json

* use version qualifier in build

* fix merge

* Revert "temp: re-add alpha1 to package.json"

This reverts commit a70688542c.

* x-pack

* remove ci build flag

* remove ci build flag

* remove qualifier from canvas

* remove qualifier from infra

* fix x-pack argv

* update comment
2018-11-02 09:39:12 -05:00
Spencer
59206ec096
[kbn/es] add context to error message (#24664)
This just tweaks the kbn-es error message to provide more context than just `Not Found`
2018-10-26 16:57:08 -07:00