persistable state docs (#105202)

dev_docs/key_concepts/persistable_state.mdx

@@ -0,0 +1,83 @@
+---
+id: kibDevDocsPersistableStateIntro
+slug: /kibana-dev-docs/persistable-state-intro
+title: Persistable State
+summary: Persitable state is a key concept to understand when building a Kibana plugin.
+date: 2021-02-02
+tags: ['kibana','dev', 'contributor', 'api docs']
+---
+
+ “Persistable state” is developer-defined state that supports being persisted by a plugin other than the one defining it. Persistable State needs to be serializable and the owner can/should provide utilities to migrate it, extract and inject any <DocLink id="kibDevDocsSavedObjectsIntro" section="references" text="references to Saved Objects"/> it may contain, as well as telemetry collection utilities.
+ 
+## Exposing state that can be persisted
+
+Any plugin that exposes state that another plugin might persist should implement <DocLink id="kibKibanaUtilsPluginApi " section="def-common.PersistableStateService" text="`PersistableStateService`"/> interface on their `setup` contract. This will allow plugins persisting the state to easily access migrations and other utilities.
+
+Example: Data plugin allows you to generate filters. Those filters can be persisted by applications in their saved 
+objects or in the URL. In order to allow apps to migrate the filters in case the structure changes in the future, the Data plugin implements `PersistableStateService` on <DocLink id="kibDataQueryPluginApi " section="def-public.FilterManager" text="`data.query.filterManager`"/>.
+
+note: There is currently no obvious way for a plugin to know which state is safe to persist. The developer must manually look for a matching `PersistableStateService`, or ad-hoc provided migration utilities (as is the case with Rule Type Parameters).
+In the future, we hope to make it easier for producers of state to understand when they need to write a migration with changes, and also make it easier for consumers of such state, to understand whether it is safe to persist.
+
+## Exposing state that can be persisted but is not owned by plugin exposing it (registry)
+
+Any plugin that owns collection of items (registry) whose state/configuration can be persisted should implement `PersistableStateService`
+interface on their `setup` contract and each item in the collection should implement <DocLink id="kibKibanaUtilsPluginApi" section="def-common.PersistableStateDefinition" text="`PersistableStateDefinition`"/> interface.
+
+Example: Embeddable plugin owns the registry of embeddable factories to which other plugins can register new embeddable factories. Dashboard plugin
+stores a bunch of embeddable panels input in its saved object and URL. Embeddable plugin setup contract implements `PersistableStateService`
+interface and each `EmbeddableFactory`  needs to implement `PersistableStateDefinition` interface.
+
+Embeddable plugin exposes this interfaces:
+```
+// EmbeddableInput implements Serializable
+
+export interface EmbeddableRegistryDefinition extends PersistableStateDefinition<EmbeddableInput> {
+  id: string;
+  ...
+}
+
+export interface EmbeddableSetup extends PersistableStateService<EmbeddableInput>;
+```
+
+Note: if your plugin doesn't expose the state (it is the only one storing state), the plugin doesn't need to implement the `PersistableStateService` interface.
+If the state your plugin is storing can be provided by other plugins (your plugin owns a registry) items in that registry still need to implement `PersistableStateDefinition` interface.
+
+## Storing persistable state as part of saved object
+
+Any plugin that stores any persistable state as part of their saved object should make sure that its saved object migration 
+and reference extraction and injection methods correctly use the matching `PersistableStateService` implementation for the state they are storing.
+
+Take a look at [example saved object](https://github.com/elastic/kibana/blob/master/examples/embeddable_examples/server/searchable_list_saved_object.ts#L32) which stores an embeddable state. Note how the `migrations`, `extractReferences` and `injectReferences` are defined.
+
+## Storing persistable state as part of URL
+
+When storing persistable state as part of URL you must make sure your URL is versioned. When loading the state `migrateToLatest` method
+of `PersistableStateService` should be called, which will migrate the state from its original version to latest.
+
+note: Currently there is no recommended way on how to store version in url and its up to every application to decide on how to implement that.
+
+## Available state operations
+ 
+### Extraction/Injection of References
+
+In order to support import and export, and space-sharing capabilities, Saved Objects need to explicitly list any references they contain to other Saved Objects.
+To support persisting your state in saved objects owned by another plugin, the <DocLink id="kibKibanaUtilsPluginApi" section="def-common.PersistableState.extract" text="`extract`"/> and <DocLink id="kibKibanaUtilsPluginApi" section="def-common.PersistableState.inject" text="`inject`"/> methods of Persistable State interface should be implemented.
+
+<DocLink id="kibDevTutorialSavedObject" section="references" text="Learn how to define Saved Object references"/>
+
+[See example embeddable providing extract/inject functions](https://github.com/elastic/kibana/blob/master/examples/embeddable_examples/public/migrations/migrations_embeddable_factory.ts)
+
+### Migrations and Backward compatibility
+
+As your plugin evolves, you may need to change your state in a breaking way. If that happens, you should write a migration to upgrade the state that existed prior to the change.
+
+<DocLink id="kibDevTutorialSavedObject" section="migrations" text="How to write a migration"/>.
+
+[See an example saved object storing embeddable state implementing saved object migration function](https://github.com/elastic/kibana/blob/master/examples/embeddable_examples/server/searchable_list_saved_object.ts)
+
+[See example embeddable providing migration functions](https://github.com/elastic/kibana/blob/master/examples/embeddable_examples/public/migrations/migrations_embeddable_factory.ts)
+
+## Telemetry
+
+You might want to collect statistics about how your state is used. If that is the case you should implement the telemetry method of Persistable State interface. 

examples/embeddable_examples/kibana.json

@@ -9,7 +9,7 @@
     "githubTeam": "kibana-app-services"
   },
   "description": "Example app that shows how to register custom embeddables",
-  "requiredPlugins": ["embeddable", "uiActions", "savedObjects", "dashboard"],
+  "requiredPlugins": ["embeddable", "uiActions", "savedObjects", "dashboard", "kibanaUtils"],
   "optionalPlugins": [],
   "extraPublicDirs": ["public/todo", "public/hello_world", "public/todo/todo_ref_embeddable"],
   "requiredBundles": ["kibanaReact"]

examples/embeddable_examples/public/migrations/migrations_embeddable_factory.ts

@@ -7,6 +7,7 @@
  */
 
 import { i18n } from '@kbn/i18n';
+import { EmbeddableStateWithType } from '../../../../src/plugins/embeddable/common';
 import {
   IContainer,
   EmbeddableInput,
@@ -35,6 +36,16 @@ export class SimpleEmbeddableFactoryDefinition
     '7.3.0': migration730,
   };
 
+  public extract(state: EmbeddableStateWithType) {
+    // this embeddable does not store references to other saved objects
+    return { state, references: [] };
+  }
+
+  public inject(state: EmbeddableStateWithType) {
+    // this embeddable does not store references to other saved objects
+    return state;
+  }
+
   /**
    * In our simple example, we let everyone have permissions to edit this. Most
    * embeddables should check the UI Capabilities service to be sure of

examples/embeddable_examples/server/merge_migration_function_maps.ts

@@ -0,0 +1,25 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0 and the Server Side Public License, v 1; you may not use this file except
+ * in compliance with, at your election, the Elastic License 2.0 or the Server
+ * Side Public License, v 1.
+ */
+
+import { mergeWith } from 'lodash';
+import type { SerializableRecord } from '@kbn/utility-types';
+import { MigrateFunctionsObject, MigrateFunction } from '../../../src/plugins/kibana_utils/common';
+
+export const mergeMigrationFunctionMaps = (
+  obj1: MigrateFunctionsObject,
+  obj2: MigrateFunctionsObject
+) => {
+  const customizer = (objValue: MigrateFunction, srcValue: MigrateFunction) => {
+    if (!srcValue || !objValue) {
+      return srcValue || objValue;
+    }
+    return (state: SerializableRecord) => objValue(srcValue(state));
+  };
+
+  return mergeWith({ ...obj1 }, obj2, customizer);
+};

examples/embeddable_examples/server/searchable_list_saved_object.ts

@@ -9,9 +9,12 @@
 import { mapValues } from 'lodash';
 import { SavedObjectsType, SavedObjectUnsanitizedDoc } from 'kibana/server';
 import { EmbeddableSetup } from '../../../src/plugins/embeddable/server';
+// NOTE: this should rather be imported from 'plugins/kibana_utils/server' but examples at the moment don't
+// allow static imports from plugins so this code was duplicated
+import { mergeMigrationFunctionMaps } from './merge_migration_function_maps';
 
 export const searchableListSavedObject = (embeddable: EmbeddableSetup) => {
-  return {
+  const searchableListSO: SavedObjectsType = {
     name: 'searchableList',
     hidden: false,
     namespaceType: 'single',
@@ -30,14 +33,22 @@ export const searchableListSavedObject = (embeddable: EmbeddableSetup) => {
       },
     },
     migrations: () => {
-      // we assume all the migration will be done by embeddables service and that saved object holds no extra state besides that of searchable list embeddable input\
-      // if saved object would hold additional information we would need to merge the response from embeddables.getAllMigrations with our custom migrations.
-      return mapValues(embeddable.getAllMigrations(), (migrate) => {
+      // there are no migrations defined for the saved object at the moment, possibly they would be added in the future
+      const searchableListSavedObjectMigrations = {};
+
+      // we don't know if embeddables have any migrations defined so we need to fetch them and map the received functions so we pass
+      // them the correct input and that we correctly map the response
+      const embeddableMigrations = mapValues(embeddable.getAllMigrations(), (migrate) => {
         return (state: SavedObjectUnsanitizedDoc) => ({
           ...state,
           attributes: migrate(state.attributes),
         });
       });
+
+      // we merge our and embeddable migrations and return
+      return mergeMigrationFunctionMaps(searchableListSavedObjectMigrations, embeddableMigrations);
     },
-  } as SavedObjectsType;
+  };
+
+  return searchableListSO;
 };

src/plugins/kibana_utils/server/index.ts

@@ -15,6 +15,7 @@ export {
   Get,
   Set,
   url,
+  mergeMigrationFunctionMaps,
 } from '../common';
 
 export { KbnServerError, reportServerError, getKbnServerError } from './report_server_error';