6c62c686cf
* chore(NA): upgrade oss to lodash4 chore(NA): migrate cli, cli_plugin, cli_keystore, dev, test_utils and apm src script to lodash4 chore(NA): missing file for cli plugin chore(NA): add src core chore(NA): es archiver and fixtures chore(NA): try to fix functional test failure chore(NA): migrate src/legacy entirely to lodash4 except src/legacy/core_plugins chore(NA): move legacy core plugins to lodash4 chore(NA): upgrade optimize to lodash4 chore(NA): upgrade to lodash4 on advanced_settings, charts, console and dashboard chore(NA): migrate to lodash4 on dev_tools, discover, embeddable, es_ui)shared, expressions, home plugins chore(NA): upgrade data plugin to lodash4 chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4 chore(NA): missing data upgrades to lodash4 Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4" This reverts commit 137055c5fed2fc52bb26547e0bc1ad2e3d4fe309. Revert "Revert "chore(NA): upgrade usage_collection, ui_actions, tile_map, telemtry, share, saved_objects, saved_objects_management, region_map and navigation to lodash4"" This reverts commit f7e73688782998513d9fb6d7e8f0765e9beb28d1. Revert "chore(NA): missing data upgrades to lodash4" This reverts commit 92b85bf947a89bfc70cc4052738a6b2128ffb076. Revert "chore(NA): upgrade data plugin to lodash4" This reverts commit 88fdb075ee1e26c4ac979b6681d8a2b002df74c6. chore(NA): upgrade idx_pattern_mgt, input_control_vis, inspector, kbn_legacy, kbn_react, kbn_usage_collections, kbn_utils, management and maps_legacy to lodash4 chore(NA): map src plugin data to lodash3 chore(NA): missing lodash.clonedeep dep chore(NA): change packages kbn-config-schema deps chore(NA): update renovate config chore(NA): upgrade vis_type plugins to lodash4 chore(NA): move vis_type_vislib to lodash3 chore(NA): update visualizations and visualize to lodash4 chore(NA): remove lodash 3 types from src and move test to lodash4 chore(NA): move home, usage_collection and management to lodash 3 Revert "chore(NA): move home, usage_collection and management to lodash 3" This reverts commit f86e8585f02d21550746569af54215b076a79a3d. chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3 chore(NA): update x-pack test to mock lodash4 Revert "chore(NA): move kibana_legacy, saved_objects saved_objects_management into lodash3" This reverts commit 2d10fe450533e1b36db21d99cfae3ce996a244e0. * chore(NA): move x-pack and packages to lodash 4 * chore(NA): remove mention to lodash from main package.json * chore(NA): remove helper alias for lodash4 and make it the default lodash * chore(NA): fix last failing types in the repo * chore(NA): fix public api * chore(NA): fix types for agg_row.tsx * chore(NA): fix increment of optimizer modules in the rollup plugin * chore(NA): migrate `src/core/public/http/fetch.ts` (#5) * omit undefined query props * just remove merge usage * fix types * chore(NA): fixes for feedback from apm team * chore(NA): recover old behaviour on apm LoadingIndeicatorContext.tsx * chore(NA): fixes for feedback from watson * Platform lodash4 tweaks (#6) * chore(NA): fix types and behaviour on src/core/server/elasticsearch/errors.ts * Canvas fixes for lodash upgrade * [APM] Adds unit test for APM service maps transform (#7) * Adds a snapshot unit test for getConnections and rearranges some code to make testing easier * reverts `ArrayList` back to `String[]` in the painless script within `fetch_service_paths_from_trace_ids.ts` * chore(NA): update yarn.lock * chore(NA): remove any and use a real type for alerts task runner Co-authored-by: Gidi Meir Morris <github@gidi.io> * chore(NA): used named import for triggers_actions_ui file * chore(NA): fix eslint * chore(NA): fix types * Delete most uptime lodash references. * Simplify. Clean up types. * [Uptime] Delete most uptime lodash references (#8) * Delete most uptime lodash references. * Simplify. Clean up types. * chore(NA): add eslint rule to avoid using lodash3 * chore(NA): apply changes on feedback from es-ui team * fix some types (#9) * Clean up some expressions types. * chore(NA): missing ts-expect-error statements * Upgrade lodash 4 vislib (#11) * replace lodash 3 with lodash 4 on vislib plugin * Further changes * further replacement of lodash3 to 4 * further work on upgrading to lodash 4 * final changes to update lodash * chore(NA): upgrade data plugin to lodash4 chore(NA): upgrade data plugin public to lodash4 chore(NA): fix typecheck task chore(NA): fix agg_config with hasIn chore(NA): assign to assignIn and has to hasIn chore(NA): upgrade data plugin server to lodash4 chore(NA): new signature for core api fix(NA): match behaviour between lodash3 and lodash4 for set in search_source * chore(NA): remove lodash3 completely from the repo * chore(NA): fix x-pack/test/api_integration/apis/metrics_ui/snapshot.ts missing content * chore(NA): fix lodash usage on apm * chore(NA): fix typecheck for maps * Patch lodash template (#12) * Applying changes from https://github.com/elastic/kibana/pull/64985 * Using isIterateeCall, because it seems less brittle * Also patching `lodash/template` and `lodash/fp/template` * Reorganizing some files... * Revising comment * Ends up `_` is a function also... I hate JavaScript Co-authored-by: Pierre Gayvallet <pierre.gayvallet@gmail.com> Co-authored-by: Josh Dover <me@joshdover.com> Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co> Co-authored-by: Oliver Gupte <ogupte@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Gidi Meir Morris <github@gidi.io> Co-authored-by: Justin Kambic <justin.kambic@elastic.co> Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com> Co-authored-by: Luke Elmers <luke.elmers@elastic.co> Co-authored-by: Brandon Kobel <brandon.kobel@gmail.com> Co-authored-by: kobelb <brandon.kobel@elastic.co> |
||
---|---|---|
.. | ||
src | ||
types | ||
package.json | ||
README.md | ||
tsconfig.json |
@kbn/config-schema
— The Kibana config validation library
@kbn/config-schema
is a TypeScript library inspired by Joi and designed to allow run-time validation of the
Kibana configuration entries providing developers with a fully typed model of the validated data.
Table of Contents
- Why
@kbn/config-schema
? - Schema building blocks
- Custom validation
- Default values
Why @kbn/config-schema
?
Validation of externally supplied data is very important for Kibana. Especially if this data is used to configure how it operates.
There are a number of reasons why we decided to roll our own solution for the configuration validation:
- Limited API surface - having a future rich library is awesome, but it's a really hard task to audit such library and make sure everything is sane and secure enough. As everyone knows complexity is the enemy of security and hence we'd like to have a full control over what exactly we expose and commit to maintain.
- Custom error messages - detailed validation error messages are a great help to developers, but at the same time they can contain information that's way too sensitive to expose to everyone. We'd like to control these messages and make them only as detailed as really needed. For example, we don't want validation error messages to contain the passwords for internal users to show-up in the logs. These logs are commonly ingested into Elasticsearch, and accessible to a large number of users which shouldn't have access to the internal user's password.
- Type information - having run-time guarantees is great, but additionally having compile-time guarantees is even better. We'd like to provide developers with a fully typed model of the validated data so that it's harder to misuse it after validation.
- Upgradability - no matter how well a validation library is implemented, it will have bugs and may need to be improved at some point anyway. Some external libraries are very well supported, some aren't or won't be in the future. It's always a risk to depend on an external party with their own release cadence when you need to quickly fix a security vulnerability in a patch version. We'd like to have a better control over lifecycle of such an important piece of our codebase.
Schema building blocks
The schema is composed of one or more primitives depending on the shape of the data you'd like to validate.
const simpleStringSchema = schema.string();
const moreComplexObjectSchema = schema.object({ name: schema.string() });
Every schema instance has a validate
method that is used to perform a validation of the data according to the schema. This method accepts three arguments:
data: any
- required, data to be validated with the schemacontext: Record<string, any>
- optional, object whose properties can be referenced by the context referencesnamespace: string
- optional, arbitrary string that is used to prefix every error message thrown during validation
const valueSchema = schema.object({
isEnabled: schema.boolean(),
env: schema.string({ defaultValue: schema.contextRef('envName') }),
});
expect(valueSchema.validate({ isEnabled: true, env: 'prod' })).toEqual({
isEnabled: true,
env: 'prod',
});
// Use default value for `env` from context via reference
expect(valueSchema.validate({ isEnabled: true }, { envName: 'staging' })).toEqual({
isEnabled: true,
env: 'staging',
});
// Fail because of type mismatch
expect(() =>
valueSchema.validate({ isEnabled: 'non-bool' }, { envName: 'staging' })
).toThrowError(
'[isEnabled]: expected value of type [boolean] but got [string]'
);
// Fail because of type mismatch and prefix error with a custom namespace
expect(() =>
valueSchema.validate({ isEnabled: 'non-bool' }, { envName: 'staging' }, 'configuration')
).toThrowError(
'[configuration.isEnabled]: expected value of type [boolean] but got [string]'
);
Notes:
validate
method throws as soon as the first schema violation is encountered, no further validation is performed.- when you retrieve configuration within a Kibana plugin
validate
function is called by the Core automatically providing appropriate namespace and context variables (environment name, package info etc.).
Basic types
schema.string()
Validates input data as a string.
Output type: string
Options:
defaultValue: string | Reference<string> | (() => string)
- defines a default value, see Default values section for more details.validate: (value: string) => string | void
- defines a custom validator function, see Custom validation section for more details.minLength: number
- defines a minimum length the string should have.maxLength: number
- defines a maximum length the string should have.hostname: boolean
- indicates whether the string should be validated as a valid hostname (per RFC 1123).
Usage:
const valueSchema = schema.string({ maxLength: 10 });
Notes:
- By default
schema.string()
allows empty strings, to prevent that use non-zero value forminLength
option. - To validate a string using a regular expression use a custom validator function, see Custom validation section for more details.
schema.number()
Validates input data as a number.
Output type: number
Options:
defaultValue: number | Reference<number> | (() => number)
- defines a default value, see Default values section for more details.validate: (value: number) => string | void
- defines a custom validator function, see Custom validation section for more details.min: number
- defines a minimum value the number should have.max: number
- defines a maximum value the number should have.
Usage:
const valueSchema = schema.number({ max: 10 });
Notes:
- The
schema.number()
also supports a string as input if it can be safely coerced into number.
schema.boolean()
Validates input data as a boolean.
Output type: boolean
Options:
defaultValue: boolean | Reference<boolean> | (() => boolean)
- defines a default value, see Default values section for more details.validate: (value: boolean) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.boolean({ defaultValue: false });
Notes:
- The
schema.boolean()
also supports a string as input if it equals'true'
or'false'
(case-insensitive).
schema.literal()
Validates input data as a string, numeric or boolean literal.
Output type: string
, number
or boolean
literals
Options:
defaultValue: TLiteral | Reference<TLiteral> | (() => TLiteral)
- defines a default value, see Default values section for more details.validate: (value: TLiteral) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = [
schema.literal('stringLiteral'),
schema.literal(100500),
schema.literal(false),
];
schema.buffer()
Validates input data as a NodeJS Buffer
.
Output type: Buffer
Options:
defaultValue: TBuffer | Reference<TBuffer> | (() => TBuffer)
- defines a default value, see Default values section for more details.validate: (value: TBuffer) => Buffer | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.buffer({ defaultValue: Buffer.from('Hi, there!') });
schema.stream()
Validates input data as a NodeJS stream
.
Output type: Stream
, Readable
or Writtable
Options:
defaultValue: TStream | Reference<TStream> | (() => TStream)
- defines a default value, see Default values section for more details.validate: (value: TStream) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.stream({ defaultValue: new Stream() });
Composite types
schema.arrayOf()
Validates input data as a homogeneous array with the values being validated against predefined schema.
Output type: TValue[]
Options:
defaultValue: TValue[] | Reference<TValue[]> | (() => TValue[])
- defines a default value, see Default values section for more details.validate: (value: TValue[]) => string | void
- defines a custom validator function, see Custom validation section for more details.minSize: number
- defines a minimum size the array should have.maxSize: number
- defines a maximum size the array should have.
Usage:
const valueSchema = schema.arrayOf(schema.number());
Notes:
- The
schema.arrayOf()
also supports a json string as input if it can be safely parsed usingJSON.parse
and if the resulting value is an array.
schema.object()
Validates input data as an object with a predefined set of properties.
Output type: { [K in keyof TProps]: TypeOf<TProps[K]> } as TObject
Options:
defaultValue: TObject | Reference<TObject> | (() => TObject)
- defines a default value, see Default values section for more details.validate: (value: TObject) => string | void
- defines a custom validator function, see Custom validation section for more details.unknowns: 'allow' | 'ignore' | 'forbid'
- indicates whether unknown object properties should be allowed, ignored, or forbidden. It'sforbid
by default.
Usage:
const valueSchema = schema.object({
isEnabled: schema.boolean({ defaultValue: false }),
name: schema.string({ minLength: 10 }),
});
Notes:
- Using
unknowns: 'allow'
is discouraged and should only be used in exceptional circumstances. Consider usingschema.recordOf()
instead. - Currently
schema.object()
always has a default value of{}
, but this may change in the near future. Try to not rely on this behaviour and specify default value explicitly or useschema.maybe()
if the value is optional. schema.object()
also supports a json string as input if it can be safely parsed usingJSON.parse
and if the resulting value is a plain object.
schema.recordOf()
Validates input data as an object with the keys and values being validated against predefined schema.
Output type: Record<TKey, TValue>
Options:
defaultValue: Record<TKey, TValue> | Reference<Record<TKey, TValue>> | (() => Record<TKey, TValue>)
- defines a default value, see Default values section for more details.validate: (value: Record<TKey, TValue>) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.recordOf(schema.string(), schema.number());
Notes:
- You can use a union of literal types as a record's key schema to restrict record to a specific set of keys, e.g.
schema.oneOf([schema.literal('isEnabled'), schema.literal('name')])
. schema.recordOf()
also supports a json string as input if it can be safely parsed usingJSON.parse
and if the resulting value is a plain object.
schema.mapOf()
Validates input data as a map with the keys and values being validated against the predefined schema.
Output type: Map<TKey, TValue>
Options:
defaultValue: Map<TKey, TValue> | Reference<Map<TKey, TValue>> | (() => Map<TKey, TValue>)
- defines a default value, see Default values section for more details.validate: (value: Map<TKey, TValue>) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.mapOf(schema.string(), schema.number());
Notes:
- You can use a union of literal types as a record's key schema to restrict record to a specific set of keys, e.g.
schema.oneOf([schema.literal('isEnabled'), schema.literal('name')])
. schema.mapOf()
also supports a json string as input if it can be safely parsed usingJSON.parse
and if the resulting value is a plain object.
Advanced types
schema.oneOf()
Allows a list of alternative schemas to validate input data against.
Output type: TValue1 | TValue2 | TValue3 | ..... as TUnion
Options:
defaultValue: TUnion | Reference<TUnion> | (() => TUnion)
- defines a default value, see Default values section for more details.validate: (value: TUnion) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.oneOf([schema.literal('∞'), schema.number()]);
Notes:
- Since the result data type is a type union you should use various TypeScript type guards to get the exact type.
schema.any()
Indicates that input data shouldn't be validated and returned as is.
Output type: any
Options:
defaultValue: any | Reference<any> | (() => any)
- defines a default value, see Default values section for more details.validate: (value: any) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.any();
Notes:
schema.any()
is essentially an escape hatch for the case when your data can really have any type and should be avoided at all costs.
schema.maybe()
Indicates that input data is optional and may not be present.
Output type: T | undefined
Usage:
const valueSchema = schema.maybe(schema.string());
Notes:
- Don't use
schema.maybe()
if a nested type defines a default value.
schema.nullable()
Indicates that input data is optional and defaults to null
if it's not present.
Output type: T | null
Usage:
const valueSchema = schema.nullable(schema.string());
Notes:
schema.nullable()
also treats explicitly specifiednull
as a valid input.
schema.never()
Indicates that input data is forbidden.
Output type: never
Usage:
const valueSchema = schema.never();
Notes:
schema.never()
has a very limited application and usually used within conditional schemas to fully or partially forbid input data.
schema.uri()
Validates input data as a proper URI string (per RFC 3986).
Output type: string
Options:
defaultValue: string | Reference<string> | (() => string)
- defines a default value, see Default values section for more details.validate: (value: string) => string | void
- defines a custom validator function, see Custom validation section for more details.scheme: string | string[]
- limits allowed URI schemes to the one(s) defined here.
Usage:
const valueSchema = schema.uri({ scheme: 'https' });
Notes:
- Prefer using
schema.uri()
for all URI validations even though it may be possible to replicate it with a custom validator forschema.string()
.
schema.byteSize()
Validates input data as a proper digital data size.
Output type: ByteSizeValue
Options:
defaultValue: ByteSizeValue | string | number | Reference<ByteSizeValue | string | number> | (() => ByteSizeValue | string | number)
- defines a default value, see Default values section for more details.validate: (value: ByteSizeValue | string | number) => string | void
- defines a custom validator function, see Custom validation section for more details.min: ByteSizeValue | string | number
- defines a minimum value the size should have.max: ByteSizeValue | string | number
- defines a maximum value the size should have.
Usage:
const valueSchema = schema.byteSize({ min: '3kb' });
Notes:
- The string value for
schema.byteSize()
and its options supports the following optional suffixes:b
,kb
,mb
,gb
andtb
. The default suffix isb
. - The number value is treated as a number of bytes and hence should be a positive integer, e.g.
100
is equal to'100b'
. - Currently you cannot specify zero bytes with a string format and should use number
0
instead.
schema.duration()
Validates input data as a proper duration.
Output type: moment.Duration
Options:
defaultValue: moment.Duration | string | number | Reference<moment.Duration | string | number> | (() => moment.Duration | string | number)
- defines a default value, see Default values section for more details.validate: (value: moment.Duration | string | number) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.duration({ defaultValue: '70ms' });
Notes:
- The string value for
schema.duration()
supports the following optional suffixes:ms
,s
,m
,h
,d
,w
,M
andY
. The default suffix isms
. - The number value is treated as a number of milliseconds and hence should be a positive integer, e.g.
100
is equal to'100ms'
.
schema.conditional()
Allows a specified condition that is evaluated at the validation time and results in either one or another input validation schema.
The first argument is always a reference while the second one can be:
- another reference, in this cases both references are "dereferenced" and compared
- schema, in this case the schema is used to validate "dereferenced" value of the first reference
- value, in this case "dereferenced" value of the first reference is compared to that value
The third argument is a schema that should be used if the result of the aforementioned comparison evaluates to true
, otherwise schema.conditional()
should fallback
to the schema provided as the fourth argument.
Output type: TTrueResult | TFalseResult
Options:
defaultValue: TTrueResult | TFalseResult | Reference<TTrueResult | TFalseResult> | (() => TTrueResult | TFalseResult
- defines a default value, see Default values section for more details.validate: (value: TTrueResult | TFalseResult) => string | void
- defines a custom validator function, see Custom validation section for more details.
Usage:
const valueSchema = schema.object({
key: schema.oneOf([schema.literal('number'), schema.literal('string')]),
value: schema.conditional(schema.siblingRef('key'), 'number', schema.number(), schema.string()),
});
Notes:
- Conditional schemas may be hard to read and understand and hence should be used only sparingly.
References
schema.contextRef()
Defines a reference to the value specified through the validation context. Context reference is only used as part of a conditional schema or as a default value for any other schema.
Output type: TReferenceValue
Usage:
const valueSchema = schema.object({
env: schema.string({ defaultValue: schema.contextRef('envName') }),
});
valueSchema.validate({}, { envName: 'dev' });
Notes:
- The
@kbn/config-schema
neither validates nor coerces the "dereferenced" value and the developer is responsible for making sure that it has the appropriate type. - The root context that Kibana provides during config validation includes lots of useful properties like
environment name
that can be used to provide a strict schema for production and more relaxed one for development.
schema.siblingRef()
Defines a reference to the value of the sibling key. Sibling references are only used a part of conditional schema or as a default value for any other schema.
Output type: TReferenceValue
Usage:
const valueSchema = schema.object({
node: schema.object({ tag: schema.string() }),
env: schema.string({ defaultValue: schema.siblingRef('node.tag') }),
});
Notes:
- The
@kbn/config-schema
neither validates nor coerces the "dereferenced" value and the developer is responsible for making sure that it has the appropriate type.
Custom validation
Using built-in schema primitives may not be enough in some scenarios or sometimes the attempt to model complex schemas with built-in primitives only may result in unreadable code.
For these cases @kbn/config-schema
provides a way to specify a custom validation function for almost any schema building block through the validate
option.
For example @kbn/config-schema
doesn't have a dedicated primitive for the RegExp
based validation currently, but you can easily do that with a custom validate
function:
const valueSchema = schema.string({
minLength: 3,
validate(value) {
if (!/^[a-z0-9_-]+$/.test(value)) {
return `must be lower case, a-z, 0-9, '_', and '-' are allowed`;
}
},
});
// ...or if you use that construct a lot...
const regexSchema = (regex: RegExp) => schema.string({
validate: value => regex.test(value) ? undefined : `must match "${regex.toString()}"`,
});
const valueSchema = regexSchema(/^[a-z0-9_-]+$/);
Custom validation function is run only after all built-in validations passed. It should either return a string
as an error message
to denote the failed validation or not return anything at all (void
) otherwise. Please also note that validate
function is synchronous.
Another use case for custom validation functions is when the schema depends on some run-time data:
const gesSchema = randomRunTimeSeed => schema.string({
validate: value => value !== randomRunTimeSeed ? 'value is not allowed' : undefined
});
const schema = gesSchema('some-random-run-time-data');
Default values
If you have an optional config field that you can have a default value for you may want to consider using dedicated defaultValue
option to not
deal with "defined or undefined"-like checks all over the place in your code. You have three options to provide a default value for almost any schema primitive:
- plain value that's known at the compile time
- reference to a value that will be "dereferenced" at the validation time
- function that is invoked at the validation time and returns a plain value
const valueSchemaWithPlainValueDefault = schema.string({ defaultValue: 'n/a' });
const valueSchemaWithReferencedValueDefault = schema.string({ defaultValue: schema.contextRef('env') });
const valueSchemaWithFunctionEvaluatedDefault = schema.string({ defaultValue: () => Math.random().toString() });
Notes:
@kbn/config-schema
neither validates nor coerces default value and developer is responsible for making sure that it has the appropriate type.