Commit graph

84 commits

Author SHA1 Message Date
spalger
8e39ba342e Revert "Update dependency simple-git to v1.129.0 (#53077)"
This reverts commit 9a05c75add.
2019-12-17 11:32:50 -07:00
spalger
ccdd84a00d Revert "Update dependency execa to ^3.4.0 (#53072)"
This reverts commit ad6a07583b.
2019-12-17 11:32:49 -07:00
renovate[bot]
ad6a07583b Update dependency execa to ^3.4.0 (#53072)
* Update dependency execa to ^3.4.0

* update kbn/pm dist

* update kbn/pm dist again
2019-12-16 10:20:54 -07:00
renovate[bot]
9a05c75add Update dependency simple-git to v1.129.0 (#53077) 2019-12-15 10:18:42 -07:00
Spencer
db401e40a5
add simple description of how to pin es snapshot versions (#51225) 2019-11-20 19:11:53 -07:00
Jonathan Budzenski
1d6eef1f7c pin es to 82c601c0 (#51075) 2019-11-19 10:00:08 -07:00
Jonathan Budzenski
1cc2a34745
Removes references to Elasticsearch mapping types (#47610)
* rm types

* fix infra/reindex test

* rm max_adjacency_matrix_filters

* lint

* more fixes

* comment return

* override documentation links

* typo

* return unfedined

* skip highlight test

* lint
2019-11-11 16:41:08 -06:00
Spencer
a859577b40
Remove rimraf (#48985)
* Update dependency del to v5

* unify del/rimraf usage with del v5

* update yarn.lock

* update kbn-pm distributable

* remove poorly transformed code

* force some deletes from tmp

* mock less of the fs module

* force tmp deletion in a few more suites

* please make this the last force
2019-10-24 06:29:43 -07:00
renovate[bot]
f0e9132535 Update dependency execa to v3 (#48975)
* Update dependency execa to v3

* update kbn/pm dist

* remove old @types/execa package

* remove use of removed methods/properties, specify preferLocal when needed

* [@kbn/pm] specify preferLocal when running install scripts

* update kbn/pm dist

* Joi now converts input to matching option from .valid()

* fix failed tests

* Revert "Joi now converts input to matching option from .valid()"

This reverts commit 75531b1d09.
2019-10-23 19:23:37 -07:00
renovate[bot]
feceb0f98e Update eslint related packages (#48513)
* Update eslint related packages

* add errorMessage to no-restricted-paths validation

* add license to eslintrc file

* ignore broken plugin fixture

* upgrade tangled peer deps

* give name to unnamed func to avoid typescript-eslint/typescript-eslint#1078

* migrate away from removed prefer-interface

* migrate away from removed no-angle-bracket-type-assertions rule

* migrate away from removed no-triple-slash-reference rule

* old config was invalid and made this rule a noop

* fix array-type rule config

* fix readonly paramProperties that weren't being checked

* fix propType ordering

* remove references to non-existent rules

* remove unnecessary disable that is improperly formatted

* fix comment-related spacing

* fix typo/invalid namespace overrides

* rename fn to avoid hook-alike behavior

* remove reference to old removed rule

* use caret version for prettier

* reference babel-eslint with absolute path
2019-10-21 16:32:31 -07:00
renovate[bot]
0568464815 Update dependency tree-kill to ^1.2.1 (#48025)
* Update dependency tree-kill to ^1.2.1

* refactor promisify so types are assigned correctly
2019-10-12 20:06:00 -07:00
Spencer
26fc5c5046
[kbn/es] use an ES_TMPDIR that is within the install path (#47901)
* [kbn/es] use an ES_TMPDIR that is within the install path

* support tests which don't pass installPath
2019-10-11 07:13:22 -07:00
renovate[bot]
23650a0c1f Update dependency @elastic/elasticsearch to ^7.4.0 (#47338) 2019-10-05 05:30:50 -07:00
Spencer
3bc803e371
[mkdirp] remove in favor of recursive fs.mkdir (#47251)
* [mkdirp] remove in favor of recursive fs.mkdir

* add eslint rule to educate future contributors

* reword the eslint error message to prevent copy-pasting callback code

* Commit updated kbn-pm dist

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* Fix typo

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-10-04 08:15:06 -07:00
Spencer
510566d66b
[kbn-es] make custom snapshot url a feature (#45750)
* [kbn-es] make custom snapshot url a feature

* download es snapshot first
2019-09-15 08:10:43 -07:00
renovate[bot]
0001c74370 Update dependency del to ^4.1.1 (#44806) 2019-09-04 13:15:59 -07:00
renovate[bot]
dfddf3dbdd Update dependency chalk to ^2.4.2 (#44032)
* Update dependency chalk to ^2.4.2

* update kbn-pm dist
2019-08-27 21:58:53 -07:00
renovate[bot]
63ad141b3c Update node-fetch related packages (#43948) 2019-08-26 12:37:43 -07:00
renovate[bot]
bf7e066cc9 Update dependency @elastic/elasticsearch to ^7.3.0 (#43911) 2019-08-25 13:55:03 -07:00
Tyler Smalley
35d33b7fbf
[kbn-es] Support for passing regex value to ES (#42651)
We test each value to determine if it's a file, in which case we move it to the
config directory. This is required to support things like SSL certificates which
are required to be within the ES directory. path.isAbsolute evalues to true for
a regular expression. In order to resolve this, we also verify that the file
exists.

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-08-08 18:03:03 -07:00
Mike Côté
aaea5679d0
Add SSL support when starting Elasticsearch from yarn es (#42527)
* Initial work

* Add integration tests

* Use constants

* Fix broken code

* Handle scenario where esArgs is a string

* Remove || []

* Apply PR feedback

* Use const format
2019-08-07 10:34:44 -04:00
Aleh Zasypkin
1a103c02e4
[@kbn/es] Add a predefined list of Elasticsearch secure settings to be added into keystore. Re-enable OIDC tests. (#42239) 2019-08-01 09:14:51 +02:00
Aleh Zasypkin
6a29ef4105
Make Kerberos authentication provider work with Elastic anonymous access. (#40994) 2019-07-15 17:30:58 +02:00
Spencer
62127f18eb
[kbn/es] add command to build es snapshots for each license an… (#40331) 2019-07-05 16:12:41 -07:00
spalger
3300b1cdf3 [master] force snapshot artifact if KBN_ES_SNAPSHOT_URL is set (#40317)
* force snapshot artifact if `KBN_ES_SNAPSHOT_URL` is set

* override the snapshot to a specific build at a custom url

(cherry picked from commit 6fda5bf3bf)
2019-07-03 18:40:41 -07:00
Jonathan Budzenski
caae025582
Init @kbn/es readme (#39850)
* init kbn-es readme

* formatting
2019-06-28 10:20:37 -05:00
Jonathan Budzenski
5f1d4cac79
[kbn-es] unset JAVA_HOME when using snapshot bundled jdk (#37316)
* [kbn-es] unset JAVA_HOME when using snapshot bundled jdk

* set explicitly
2019-06-24 12:59:18 -05:00
Spencer
eeab45df7b
[kbn/es] support minor version numbers (#39353) 2019-06-20 14:52:28 -07:00
Spencer
b5ac046314
[kbn/es] auto retry snapshot download on unexpected errors (#38944)
* [kbn/es] auto retry snapshot download on unexpected errors

* missed an await

* pass log to retry()
2019-06-14 09:38:39 -07:00
renovate[bot]
c60e3491a2 chore(deps): update dependency del to v4 (#37466)
* chore(deps): update dependency del to v4

* remove @types/del since they ship with the package now
2019-05-30 13:40:35 -07:00
Aleh Zasypkin
580edcd1c1
Introduce Kerberos authentication provider. (#36112) 2019-05-29 10:40:19 +03:00
spalger
282c5e184f [kbn-es] retry on all error types while setting up native realm
(cherry picked from commit a578261158)
2019-05-23 07:51:18 -07:00
Spencer
83b7ea76b0
[kbn/es] auto-retry all requests in native realm setup (#36867)
* [kbn/es] auto-retry all requests in native realm setup

* fix argument to autoRetry
2019-05-22 09:43:38 -07:00
spalger
e8ffa2f3c7 [kbn-es] use bare ToolingLog rather than stub (#36862) 2019-05-21 21:57:59 -07:00
spalger
8d7e1237bb loosen check for security exception and log metadata so we can investigate "offline" 2019-05-21 20:24:33 -07:00
Tyler Smalley
caaf68742f
[kbn-es] Support ES security running from source/archive (#36733)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-21 10:37:29 -07:00
Spencer
508424ef07
[kbn/es] await native realm setup, error if there are failures (#36475)
* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (#36290)""

This reverts commit a102ca1f97.

* [kbn/es] retry setPassword call if it fails
2019-05-20 13:24:18 -07:00
spalger
a102ca1f97 Revert "[kbn-es] await native realm setup, error if there are failures (#36290)"
This reverts commit 6ef9fbe423.
2019-05-10 12:52:37 -07:00
Spencer
6ef9fbe423
[kbn-es] await native realm setup, error if there are failures (#36290)
* [kbn-es] await native realm setup, error if there are failures

* cleanup listener from started promise too

* fix typo

* log native realm setup errors on run and handle 400 when xpack isn't available

* avoid creating potentially unused promises

* avoid Promise constructor that's larger than necessary

* group similar operations, add some light comments

* set passwords in parallel so that logging is grouped

* update native realm tests

* update es_bin fixture to handle _xpack requests

* log started after server is listening

* remove unused mockEsBin args

* use more realistic http address log
2019-05-10 07:42:26 -07:00
Tyler Smalley
860977694b
[kbn-es] Security updates (#36288)
* In kbn-test, provide password to cluster.start()
* Security enabled by default, overwrite by passing `-E "xpack.security.enabled=false"`

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-08 12:25:26 -07:00
Tyler Smalley
8f782a8dbd
[kbn-es] Set password for native realm accounts (#35586)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-05-01 09:34:14 -07:00
Aaron Zhao
f0b47f1a68 [kbn-es] Add network archive downloading to installArchive (#31535) 2019-04-23 21:48:14 -07:00
Spencer
cce64060d0
[npm] upgrade getopts (#34603)
getopts added the `string` option in a recent version, which I didn't realize when I made updates to src/dev/run/run.ts, which rely on this option. To make sure it is available this updates getopts in all projects from `2.x` to `^2.2.4`
2019-04-15 14:39:34 -07:00
Spencer
c982856d31
[kbn/es] print headers when requests fail (#34237) 2019-03-31 18:17:12 -07:00
Spencer
eefa2a54af
[kbn/es] print first 500 bytes of invalid content (#34133)
We added checksum verification to `@kbn/es` in https://github.com/elastic/kibana/pull/33893, which is working, and in turn proves that using the Artifact API to download snapshots wasn't enough to fix the issues we've been seeing. In https://kibana-ci.elastic.co/job/elastic+kibana+7.0/412/JOB=kibana-ciGroup8,node=immutable/console we see:

```
00:14:20.762  │ info Installing from snapshot
00:14:20.763      │ info version: 7.0.0
00:14:20.763      │ info install path: /var/lib/jenkins/workspace/elastic+kibana+7.0/JOB/kibana-ciGroup8/node/immutable/kibana/.es/test-97zzlri1nyb
00:14:20.764      │ info license: oss
00:14:20.764      │ info downloading artifact info from https://artifacts-api.elastic.co/v1/versions/7.0.0-SNAPSHOT/builds/latest/projects/elasticsearch
00:14:20.852      │ info downloading artifact from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz
00:14:22.765      │ info downloading artifact checksum from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz
00:14:22.818 
00:14:22.818 artifact downloaded from https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz does not match expected checksum
00:14:22.819   expected: 1e5188c1410e0299777dd1cb9638aa12de4067ea80a77ff6723986746bbc7b1a35d580023f605e6e1ae84887e675ba10184032da80240a42289ee6c5d5361c66
00:14:22.820   received: 7630553ab83828196931917aa15f17bf1f31d541fd11e0f1c82d89f294fbe47dca7be99d96da5baa0a9545712e3c9340f9b3d989e5056a3f9471f43bb582becc
```

The new logging shows that we used the explicit URL for the artifact `https://snapshots.elastic.co/7.0.0-ea741e68/downloads/elasticsearch/elasticsearch-oss-7.0.0-SNAPSHOT-linux-x86_64.tar.gz` and still got invalid data, which confirms our suspicion that something funky is happening with the CDN. In order to understand more about what's going on here this PR adds a buffer that caches the first 500 bytes from the artifact response and will log it in utf8 format if the checksum doesn't match. I'm hoping we're going to see some HTML or something that will help explain what's happening.
2019-03-28 19:41:07 -07:00
Spencer
bedf74faf2
[kbn/es] use artifact api to determine snapshot url (#33893)
* [kbn/es] use artifact api to determine snapshot url

* actually use AbortController

* [kbn/es] handle optional os and jdk indicators in artifact name
2019-03-28 14:29:39 -07:00
Jonathan Budzenski
61f999304b [kbn-es] Platform specific snapshots (#29673) 2019-02-01 20:59:59 -08:00
Tyler Smalley
c70aeec6fd
[kbn-es] Support platform specific distributions (#29678)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2019-01-31 08:41:29 -08:00
Josh Dover
45a4d3bce0
[kbn-es] Add support to kbn-es and kbn-test for data archives (#28723)
* Add support to kbn-es and kbn-test for data archives

* Fix log indent

* Decompress directly to data directory

Co-Authored-By: joshdover <me@joshdover.com>

* Improve logs
2019-01-23 16:45:19 -06:00
Josh Dover
0ba1db2d5b
Add ENV_VAR for overriding ES snapshot version (#28768) 2019-01-15 11:27:54 -06:00