diff --git a/dev_docs/assets/data_view_diagram.png b/dev_docs/assets/data_view_diagram.png new file mode 100644 index 000000000000..7a0a47abc195 Binary files /dev/null and b/dev_docs/assets/data_view_diagram.png differ diff --git a/dev_docs/key_concepts/data_views.mdx b/dev_docs/key_concepts/data_views.mdx new file mode 100644 index 000000000000..e2b64c8705c4 --- /dev/null +++ b/dev_docs/key_concepts/data_views.mdx @@ -0,0 +1,32 @@ +--- +id: kibDataViewsKeyConcepts +slug: /kibana-dev-docs/data-view-intro +title: Data Views +summary: Data views are the central method of defining queryable data sets in Kibana +date: 2021-08-11 +tags: ['kibana','dev', 'contributor', 'api docs'] +--- + +*Note: Kibana index patterns are currently being renamed to data views. There will be some naming inconsistencies until the transition is complete.* + +Data views (formerly Kibana index patterns or KIPs) are the central method of describing sets of indices for queries. Usage is strongly recommended +as a number of high level rely on them. Further, they provide a consistent view of data across +a variety Kibana apps. + +Data views are defined by a wildcard string (an index pattern) which matches indices, data streams, and index aliases, optionally specify a +timestamp field for time series data, and are stored as a . They have a field list which comprises all the fields in matching indices plus fields defined specifically +on the data view via runtime fields. Schema-on-read functionality is provided by data view defined runtime fields. + +![image](../assets/data_view_diagram.png) + + + +The data view API is made available via the data plugin (`data.indexPatterns`, soon to be renamed) and most commonly used with +(`data.search.search.SearchSource`) to perform queries. SearchSource will apply existing filters and queries from the search bar UI. + +Users can create data views via [Data view management](https://www.elastic.co/guide/en/kibana/current/index-patterns.html). +Additionally, they can be created through the data view API. + +Data views also allow formatters and custom labels to be defined for fields. + diff --git a/dev_docs/tutorials/data_views.mdx b/dev_docs/tutorials/data_views.mdx new file mode 100644 index 000000000000..453146d81c4d --- /dev/null +++ b/dev_docs/tutorials/data_views.mdx @@ -0,0 +1,86 @@ +--- +id: kibDevTutorialDataViews +slug: /kibana-dev-docs/tutorials/data-views +title: Data views API +summary: Data views API +date: 2021-08-11 +tags: ['kibana', 'onboarding', 'dev', 'architecture'] +--- + +*Note: Kibana index patterns are currently being renamed to data views. There will be some naming inconsistencies until the transition is complete.* + +### Data views API + +- Get list of data views +- Get default data view and examine fields +- Get data view by id +- Find data view by title +- Create data view +- Create data view and save it +- Modify data view and save it +- Delete data view + +#### Get list of data view titles and ids + +``` +const idsAndTitles = await data.indexPatterns.getIdsWithTitle(); +idsAndTitles.forEach(({id, title}) => console.log(`Data view id: ${id} title: ${title}`)); +``` + +#### Get default data view and examine fields + +``` +const defaultDataView = await data.indexPatterns.getDefault(); +defaultDataView.fields.forEach(({name}) => { console.log(name); }) +``` + +#### Get data view by id + +``` +const id = 'xxxxxx-xxx-xxxxxx'; +const dataView = await data.indexPatterns.get(id); +``` + +#### Find data view by title + +``` +const title = 'kibana-*'; +const [dataView] = await data.indexPatterns.find(title); +``` + +#### Create data view + +``` +const dataView = await data.indexPatterns.create({ title: 'kibana-*' }); +``` + +#### Create data view and save it immediately + +``` +const dataView = await data.indexPatterns.createAndSave({ title: 'kibana-*' }); +``` + +#### Create data view, modify, and save + +``` +const dataView = await data.indexPatterns.create({ title: 'kibana-*' }); +dataView.setFieldCustomLabel('customer_name', 'Customer Name'); +data.indexPatterns.createSavedObject(dataView); +``` + +#### Modify data view and save it + +``` +dataView.setFieldCustomLabel('customer_name', 'Customer Name'); +await data.indexPatterns.updateSavedObject(dataView); +``` + +#### Delete index pattern + +``` +await data.indexPatterns.delete(dataViewId); +``` + +### Data view HTTP API + +Rest-like HTTP CRUD+ API - [docs](https://www.elastic.co/guide/en/kibana/master/index-patterns-api.html) diff --git a/src/plugins/data/README.mdx b/src/plugins/data/README.mdx index c2da9a686cac..40e82d3034ee 100644 --- a/src/plugins/data/README.mdx +++ b/src/plugins/data/README.mdx @@ -49,471 +49,14 @@ This is helpful when you want to provide a user with options, for example when c ``` -## Index Patterns +## Data Views -The Index Patterns API provides a consistent method of structuring and formatting documents +The data views API provides a consistent method of structuring and formatting documents and field lists across the various Kibana apps. Its typically used in conjunction with -SearchSource for composing queries. + for composing queries. -### Index Patterns API +*Note: Kibana index patterns are currently being renamed to data views. There will be some naming inconsistencies until the transition is complete.* -- Get list of index patterns -- Get default index pattern and examine fields -- Get index pattern by id -- Find index pattern by title -- Create index pattern -- Create index pattern and save it -- Modify index pattern and save it -- Delete index pattern - -#### Get list of index pattern titles and ids - -``` -const idsAndTitles = await data.indexPatterns.getIdsWithTitle(); -idsAndTitles.forEach(({id, title}) => console.log(`Index pattern id: ${id} title: ${title}`)); -``` - -#### Get default index pattern and examine fields - -``` -const defaultIndexPattern = await data.indexPatterns.getDefault(); -defaultIndexPattern.fields.forEach(({name}) => { console.log(name); }) -``` - -#### Get index pattern by id - -``` -const id = 'xxxxxx-xxx-xxxxxx'; -const indexPattern = await data.indexPatterns.get(id); -``` - -#### Find index pattern by title - -``` -const title = 'kibana-*'; -const [indexPattern] = await data.indexPatterns.find(title); -``` - -#### Create index pattern - -``` -const indexPattern = await data.indexPatterns.create({ title: 'kibana-*' }); -``` - -#### Create index pattern and save it immediately - -``` -const indexPattern = await data.indexPatterns.createAndSave({ title: 'kibana-*' }); -``` - -#### Create index pattern, modify, and save - -``` -const indexPattern = await data.indexPatterns.create({ title: 'kibana-*' }); -indexPattern.setFieldCustomLabel('customer_name', 'Customer Name'); -data.indexPatterns.createSavedObject(indexPattern); -``` - -#### Modify index pattern and save it - -``` -indexPattern.setFieldCustomLabel('customer_name', 'Customer Name'); -await data.indexPatterns.updateSavedObject(indexPattern); -``` - -#### Delete index pattern - -``` -await data.indexPatterns.delete(indexPatternId); -``` - -### Index Patterns HTTP API - -Index patterns provide Rest-like HTTP CRUD+ API with the following endpoints: - -- Index Patterns API - - Create an index pattern — `POST /api/index_patterns/index_pattern` - - Fetch an index pattern by `{id}` — `GET /api/index_patterns/index_pattern/{id}` - - Delete an index pattern by `{id}` — `DELETE /api/index_patterns/index_pattern/{id}` - - Partially update an index pattern by `{id}` — `POST /api/index_patterns/index_pattern/{id}` -- Fields API - - Update field — `POST /api/index_patterns/index_pattern/{id}/fields` -- Scripted Fields API - - Create a scripted field — `POST /api/index_patterns/index_pattern/{id}/scripted_field` - - Upsert a scripted field — `PUT /api/index_patterns/index_pattern/{id}/scripted_field` - - Fetch a scripted field — `GET /api/index_patterns/index_pattern/{id}/scripted_field/{name}` - - Remove a scripted field — `DELETE /api/index_patterns/index_pattern/{id}/scripted_field/{name}` - - Update a scripted field — `POST /api/index_patterns/index_pattern/{id}/scripted_field/{name}` - -### Index Patterns API - -Index Patterns REST API allows you to create, retrieve and delete index patterns. I also -exposes an update endpoint which allows you to update specific fields without changing -the rest of the index pattern object. - -#### Create an index pattern - -Create an index pattern with a custom title. - -``` -POST /api/index_patterns/index_pattern -{ - "index_pattern": { - "title": "hello" - } -} -``` - -Customize creation behavior with: - -- `override` — if set to `true`, replaces an existing index pattern if an - index pattern with the provided title already exists. Defaults to `false`. -- `refresh_fields` — if set to `true` reloads index pattern fields after - the index pattern is stored. Defaults to `false`. - -``` -POST /api/index_patterns/index_pattern -{ - "override": false, - "refresh_fields": true, - "index_pattern": { - "title": "hello" - } -} -``` - -At creation all index pattern fields are option and you can provide them. - -``` -POST /api/index_patterns/index_pattern -{ - "index_pattern": { - "id": "...", - "version": "...", - "title": "...", - "type": "...", - "intervalName": "...", - "timeFieldName": "...", - "sourceFilters": [], - "fields": {}, - "typeMeta": {}, - "fieldFormats": {}, - "fieldAttrs": {} - } -} -``` - -The endpoint returns the created index pattern object. - -```json -{ - "index_pattern": {} -} -``` - -#### Fetch an index pattern by ID - -Retrieve an index pattern by its ID. - -``` -GET /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -``` - -Returns an index pattern object. - -```json -{ - "index_pattern": { - "id": "...", - "version": "...", - "title": "...", - "type": "...", - "intervalName": "...", - "timeFieldName": "...", - "sourceFilters": [], - "fields": {}, - "typeMeta": {}, - "fieldFormats": {}, - "fieldAttrs": {} - } -} -``` - -#### Delete an index pattern by ID - -Delete and index pattern by its ID. - -``` -DELETE /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -``` - -Returns an '200 OK` response with empty body on success. - -#### Partially update an index pattern by ID - -Update part of an index pattern. Only provided fields will be updated on the -index pattern, missing fields will stay as they are persisted. - -These fields can be update partially: - -- `title` -- `timeFieldName` -- `intervalName` -- `fields` (optionally refresh fields) -- `sourceFilters` -- `fieldFormatMap` -- `type` -- `typeMeta` - -Update a title of an index pattern. - -``` -POST /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -{ - "index_pattern": { - "title": "new_title" - } -} -``` - -All update fields are optional, you can specify the following fields. - -``` -POST /api/index_patterns/index_pattern -{ - "index_pattern": { - "title": "...", - "timeFieldName": "...", - "intervalName": "...", - "sourceFilters": [], - "fieldFormats": {}, - "type": "...", - "typeMeta": {}, - "fields": {} - } -} -``` - -- `refresh_fields` — if set to `true` reloads index pattern fields after - the index pattern is stored. Defaults to `false`. - -``` -POST /api/index_patterns/index_pattern -{ - "refresh_fields": true, - "index_pattern": { - "fields": {} - } -} -``` - -This endpoint returns the updated index pattern object. - -```json -{ - "index_pattern": {} -} -``` - -### Fields API - -Fields API allows to change field metadata, such as `count`, `customLabel`, and `format`. - -#### Update fields - -Update endpoint allows you to update fields presentation metadata, such as `count`, -`customLabel`, and `format`. You can update multiple fields in one request. Updates -are merges with persisted metadata. To remove existing metadata specify `null` as value. - -Set popularity `count` for field `foo`: - -``` -POST /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/fields -{ - "fields": { - "foo": { - "count": 123 - } - } -} -``` - -Update multiple metadata values and fields in one request: - -``` -POST /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/fields -{ - "fields": { - "foo": { - "count": 123, - "customLabel": "Foo" - }, - "bar": { - "customLabel": "Bar" - } - } -} -``` - -Use `null` value to delete metadata: - -``` -POST /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/fields -{ - "fields": { - "foo": { - "customLabel": null - } - } -} -``` - -This endpoint returns the updated index pattern object. - -```json -{ - "index_pattern": {} -} -``` - -### Scripted Fields API - -Scripted Fields API provides CRUD API for scripted fields of an index pattern. - -#### Create a scripted field - -Create a field by simply specifying its name, will default to `string` type. Returns -an error if a field with the provided name already exists. - -``` -POST /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/scripted_field -{ - "field": { - "name": "my_field" - } -} -``` - -Create a field by specifying all field properties. - -``` -POST /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/scripted_field -{ - "field": { - "name": "", - "type": "", - "searchable": false, - "aggregatable": false, - "count": 0, - "script": "", - "scripted": false, - "lang": "", - "conflictDescriptions": {}, - "format": {}, - "esTypes": [], - "readFromDocValues": false, - "subType": {}, - "indexed": false, - "customLabel": "", - "shortDotsEnable": false - } -} -``` - -#### Upsert a scripted field - -Creates a new field or updates an existing one, if one already exists with the same name. - -Create a field by simply specifying its name. - -``` -PUT /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/scripted_field -{ - "field": { - "name": "my_field" - } -} -``` - -Create a field by specifying all field properties. - -``` -PUT /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/scripted_field -{ - "field": { - "name": "", - "type": "", - "searchable": false, - "aggregatable": false, - "count": 0, - "script": "", - "scripted": false, - "lang": "", - "conflictDescriptions": {}, - "format": {}, - "esTypes": [], - "readFromDocValues": false, - "subType": {}, - "indexed": false, - "customLabel": "", - "shortDotsEnable": false - } -} -``` - -#### Fetch a scripted field - -Fetch an existing index pattern field by field name. - -``` -GET /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/scripted_field/ -``` - -Returns the field object. - -```json -{ - "field": {} -} -``` - -#### Delete a scripted field - -Delete a field of an index pattern. - -``` -DELETE /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/scripted_field/ -``` - -#### Update a an existing scripted field - -Updates an exiting field by mergin provided properties with the existing ones. If -there is no existing field with the specified name, returns a `404 Not Found` error. - -You can specify any field properties, except `name` which is specified in the URL path. - -``` -POST /api/index_patterns/index_pattern/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/scripted_field/ -{ - "field": { - "type": "", - "searchable": false, - "aggregatable": false, - "count": 0, - "script": "", - "scripted": false, - "lang": "", - "conflictDescriptions": {}, - "format": {}, - "esTypes": [], - "readFromDocValues": false, - "subType": {}, - "indexed": false, - "customLabel": "", - "shortDotsEnable": false - } -} -``` ## Query