* [Maps] use index_exists route instead of /api/index_management/indices
* fix functional test
* add retry and correct permissions to fix functional tests
* fix upload functional test
* [ML] Moving file data vizualizer to file upload plugin
* removing maps plug dependency
* fixing imports
* small refactor
* adding missing endpoints
* fixing translations
* fxing table controls
* fixing types and disabling geo point test
* actually disabling geo point test
* making endpoints internal
* moving UI code to separate plugin
* enabling maps integration
* cleaning up dependencies
* fixing translation ids
* moving analyze file endpoint out of file upload plugin
* fixing transtations issues
* refactor for lazy loading of component
* updating limits
* updating plugin asciidoc
* code clean up
* further clean up
* adding comment
* fixing really obvious CI error
* removing commented out include
* reenabling geo point test
* fixing incorrectly changed import
* removing ml from labels and identifiers
* renaming function
* moving analyse file endpoint to file upload plugin
* reverting import path changes
* adding esUiShared back in
* fixing navigation tabs alignment in basic license
* adding key to tab wrapper
* reverting test label
* further removal of ml references
* removing ml label from more identifiers
* fixing tests
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* add base config for all the TS projects
* all the project use new tsconfig.project.json
* compile test files in the high-level tsconfig.json
* fix TS error in maps plugin
* fix TS error in infra plugin
* exclude mote test and test until folders
* uptime. do not import test code within prod code
* expressions. do not import test code within prod code
* data: export mocks from high level folder
* task_manager: comply with es client typings
* infra: remove unused enzyme_helpers
* check_ts_project requires "include" key
* ts_check should handle parent configs
* all ts configs should extend base one
* exclude test folders from plugins
* update patterns to fix ts_check errors
* Apply suggestions from code review
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
* uptime: MountWithReduxProvider to test helpers
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [file_upload] add has_import_permission route
* remove ml hasImportPermissions
* fix tsconfig path
* tslint
* review feedback
* make pipeline check optional since geojson upload does not use pipeline
* ts cleanup
* make geojson permission failure message actionable
* revert privilege change in functional test
* add global_index_pattern_management_all permission to functional test
* rename hasPipeline to checkHasManagePipeline
* add api integration test
* tslint
* revert change to es_search_source
* simpilify error message when users can't create index pattern
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Use client from branch
* Get type checking working in core
* Fix types in other plugins
* Update client types + remove type errors from core
* migrate Task Manager Elasticsearch typing from legacy library to client library
* use SortOrder instead o string in alerts
* Update client types + fix core type issues
* fix maps ts errors
* Update Lens types
* Convert Search Profiler body from a string to an object to conform to SearchRequest type.
* Fix SOT types
* Fix/mute Security/Spaces plugins type errors.
* Fix bootstrap types
* Fix painless_lab
* corrected es typing in Event Log
* Use new types from client for inferred search responses
* Latest type defs
* Integrate latest type defs for APM/UX
* fix core errors
* fix telemetry errors
* fix canvas errors
* fix data_enhanced errors
* fix event_log errors
* mute lens errors
* fix or mute maps errors
* fix reporting errors
* fix security errors
* mute errors in task_manager
* fix errors in telemetry_collection_xpack
* fix errors in data plugins
* fix errors in alerts
* mute errors in index_management
* fix task_manager errors
* mute or fix lens errors
* fix upgrade_assistant errors
* fix or mute errors in index_lifecycle_management
* fix discover errors
* fix core tests
* ML changes
* fix core type errors
* mute error in kbn-es-archiver
* fix error in data plugin
* fix error in telemetry plugin
* fix error in discover
* fix discover errors
* fix errors in task_manager
* fix security errors
* fix wrong conflict resolution
* address errors with upstream code
* update deps to the last commit
* remove outdated comments
* fix core errors
* fix errors after update
* adding more expect errors to ML
* pull the lastest changes
* fix core errors
* fix errors in infra plugin
* fix errors in uptime plugin
* fix errors in ml
* fix errors in xpack telemetry
* fix or mute errors in transform
* fix errors in upgrade assistant
* fix or mute fleet errors
* start fixing apm errors
* fix errors in osquery
* fix telemetry tests
* core cleanup
* fix asMutableArray imports
* cleanup
* data_enhanced cleanup
* cleanup events_log
* cleaup
* fix error in kbn-es-archiver
* fix errors in kbn-es-archiver
* fix errors in kbn-es-archiver
* fix ES typings for Hit
* fix SO
* fix actions plugin
* fix fleet
* fix maps
* fix stack_alerts
* fix eslint problems
* fix event_log unit tests
* fix failures in data_enhanced tests
* fix test failure in kbn-es-archiver
* fix test failures in index_pattern_management
* fixing ML test
* remove outdated comment in kbn-es-archiver
* fix error type in ml
* fix eslint errors in osquery plugin
* fix runtime error in infra plugin
* revert changes to event_log cluser exist check
* fix eslint error in osquery
* fixing ML endpoint argument types
* fx types
* Update api-extractor docs
* attempt fix for ese test
* Fix lint error
* Fix types for ts refs
* Fix data_enhanced unit test
* fix lens types
* generate docs
* Fix a number of type issues in monitoring and ml
* fix triggers_actions_ui
* Fix ILM functional test
* Put search.d.ts typings back
* fix data plugin
* Update typings in typings/elasticsearch
* Update snapshots
* mute errors in task_manager
* mute fleet errors
* lens. remove unnecessary ts-expect-errors
* fix errors in stack_alerts
* mute errors in osquery
* fix errors in security_solution
* fix errors in lists
* fix errors in cases
* mute errors in search_examples
* use KibanaClient to enforce promise-based API
* fix errors in test/ folder
* update comment
* fix errors in x-pack/test folder
* fix errors in ml plugin
* fix optional fields in ml api_integartoon tests
* fix another casting problem in ml tests
* fix another ml test failure
* fix fleet problem after conflict resolution
* rollback changes in security_solution. trying to fix test
* Update type for discover rows
* uncomment runtime_mappings as its outdated
* address comments from Wylie
* remove eslint error due to any
* mute error due to incompatibility
* Apply suggestions from code review
Co-authored-by: John Schulz <github.com@jfsiii.org>
* fix type error in lens tests
* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.ts
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* Update x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_service.test.ts
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* update deps
* fix errors in core types
* fix errors for the new elastic/elasticsearch version
* remove unused type
* remove unnecessary manual type cast and put optional chaining back
* ML: mute Datafeed is missing indices_options
* Apply suggestions from code review
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
* use canary pacakge instead of git commit
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: Josh Dover <1813008+joshdover@users.noreply.github.com>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
Co-authored-by: Nathan Reese <reese.nathan@gmail.com>
Co-authored-by: Wylie Conlon <wylieconlon@gmail.com>
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
Co-authored-by: restrry <restrry@gmail.com>
Co-authored-by: James Gowdy <jgowdy@elastic.co>
Co-authored-by: John Schulz <github.com@jfsiii.org>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
* [ml] migrate file_data_visualizer/analyze_file to file_upload plugin
* tslint
* give analyze route access to ml user
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Maps] chunk geojson upload to keep import requests under 1MB
* fix geojson_importer tests
* update failure.item to reflect location in file
* remove console statement
* clean up
* return instead of break if upload is no longer active
* add unit test for createChunks
* update file_upload API
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* rename ml:fileDataVisualizerMaxFileSize to fileUppload:maxFileSize
* add saved object migration
* file preview
* importing status
* remove console statement
* import complete view
* fix geojson_importer test
* tslint
* i18n fixes
* cleanup
* update documenation for advanced setting rename
* advanced settings usage_collection
* remove ml:fileDataVisualizerMaxFileSize from schemas and types
* add copy buttons for import response and fix geojson upload functional tests
* tslint
* remove clipboard-read check
* return early if env does not support reading from clipboard
* fix reporting tests
* review feedback
* update GeoJsonFileSource to support showing results trimmed icon and tooltip
* add fileUpload to useMlKibana context and replace dependencyCache with useMlKibana
* tslint
* review feedback
* lower case file name
* default to selecting geo_shape when file contains both points and shapes
* fix wizard onError callback to not advance to next step
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* migrate file_upload plugin to maps_file_upload
* update plugins list
* migrate ml import endpoint
* migrate ml telemetry to file_upload plugin
* add fileUpload plugin to ml
* add TS project
* update ML to use file_upload endpoint
* move types to file_upload plugin
* ignore error
* clean up
* i18n clean-up
* remove schemas from ml
* remove usageCollection from ml
* node scripts/build_plugin_list_docs
* update telemety collector
* revert changes to ingestPipeline schema
* change name of TELEMETRY_DOC_ID to unique value
* remove ImportFile from ml/server/routes/apidoc.json
* fix typo in x=pack/tsconfig.json
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Add file upload x-pack plugin
* Clean up
* Remove unneeded cluster config
* Remove unneeded test
* First pass basic telemetry (not connected).
* Basic telemetry connected
* Review feedback
* Revise telemetry to use savedObjectRepository. Capture metrics on app and file types
* Lots of cleanup, consolidation of logic
* Clean up, reorg
* Update telem tests and telem functions
* Add back import data model
* Clean up and update telemetry tests
* Fix telemetry test issues and update corresponding code
* Up chunk limit to 30 MB
* Add file upload telemetry to saved objects management builder
* Missing space
* Add descriptive comments to dynamic keys in telemetry fields
* [Maps] [File upload] Geojson upload (#36410)
* Client side basics
* File added and default named correctly
* Connect transient layer removal to file import component
* Simplify transient layer removal
* Move file import ui over to new file upload plugin and make more generic
* Add post-processing option to file upload. Make component json-specific
* Add flag for source indexing
* Revise import layer workflow to be separate from add layer workflow
* Differentiate between normal sources and import sources. Add back layer add/next button
* Update indexing boolean in component after file upload & parse
* First pass rough indexing from maps working on specific use cases
* Update parsing logic to handle geojson formats
* Index pattern added following index creation
* Pass onsuccess callback to file_upload to add layer to maps app
* Handle mulltipolygon type in es geo utils
* Add functionality to make es layer permanent and switch to edit panel
* Add to index if exists or create new
* Make plugin API more intuitive. Set up to handle remove action
* Pass transient removal call through to file import
* Clean up layer viewing logic for temp and perm layers
* Remove change source from import screen
* Add option to provide mappings array and pass geo_point and geo_shape to array
* Add support for multiple mappings select and index naming
* Match style of import file button to add layer for now
* Remove duplicate case handling resulting from merge
* Move geo processing logic over to file upload plugin for reuse
* Remove old geo_shape formatting from geo_point code
* Set default index data type. Remove unneeded stringify request logic
* Check for custom processor object which contains function
* Move file picker to separate component
* Some cleaning. Add geojson clean & validate code
* Catch file parsing errors and notify user
* Disable index type if valid file not referenced
* Set error messaging on invalid index name used
* Add index pattern checking logic and error handling
* Dynamically populate geo index options
* Set index data type earlier in the workflow. Don't duplicate requests
* Pass back index ready status from plugin and connect to layer next button
* Increase max bytes to ~50MB
* Don't parse files over max size & warn user. Also, remove toasts and warn similarly to other components
* Uploaded file default label: 'fileToImport' -> 'Import File'
* Expand out feature properties for mapping
* Pass through telemetry data to plugin back-end
* Clean up indexing flow. Separate creation of index patterns. Add new index pattern create callbacks
* Pass back info for indexing failures but don't connect to UI yet
* Fix telemetry test issues and update corresponding code
# Conflicts:
# x-pack/plugins/file_upload/server/telemetry/telemetry.test.ts
* Add file upload telemetry to saved objects management builder
* Missing space
* Add descriptive comments to dynamic keys in telemetry fields
* Divide up the import layer add workflow on the maps side and tweak to avoid layer color change
* Fix bug affecting file preview replacement not updating coordinate index type
* Remove index pattern. Organize effects. General clean up
* preIndexTransform -> transformDetails
* Update proptypes
* Cleaning, organizing
* Add index name guidelines. Show conditionally
* Add file size, type guidance. Filter on file size, not chunk size. Small tweaks to file/index tips
* Zoom to layer extent on preview of imported data
* Revert "Revise import layer workflow to be separate from add layer workflow"
This reverts commit 3b35f5371d.
* Handle import file determination to accomodate add layer flow. Update card wording
* Decompose addLayer component into smaller pieces. Add placeholder import progress component
* Dynamically change footer button text for context
* Move import card to top of sources
* Get basic progress tracking in place
* Allow second layer (indexed layer) to get removed. Reconnect file remove. Small UI tweaks
* Add link to management for further index mods
* Fix i18n failures
* Add file parsing progress indicator & text
* Reset importView on source reset
* Add dynamic chunking to handle fluctuating data sizes common to geo features
* Don't duplicate index request if request in flight
* Modify json upload and import to use JS classes & react class component state
* Transition remaining file_upload components over to class/component structure with state
* Move functions into index_settings class
* Review feedback
* Add fetch with timeout
* Split out import editor into separate component and clean up logic
* Clean up add layer panel/button title logic
* More cleanup
* Tweaks to success and error handling flow
* Handle success/error handling on add/view indexed data
* Jump to indexing complete on error. Handle nothing returned to client app
* Update name/location of source select css file
* Update import source card border color and icon
* Suggest name for index based upon file name
* Add validation to auto-suggested index name/pattern
* Use constants for geo_point and geo_shape in mappings
* Update geojson upload card/description
* Catch-all review feedback
* Fix internationalization syntax errors
* Review feedback
* Get index names and patterns only when needed
* Make addLayer async again to fix zoom to extent issue, waits on just the async syncDataForLayer function (which is async)
* Remove panel description as derived state and shift to function
* Remove geojson fit to extent for now
* Remove unused class selector
* Remove unneeded i18 wrapper for what's already passed in as an i18 element
* Revise import state to be handle via redux. Some json upload args changes
* Review feedback. Some cleanup and bug fixes
* Roll back store actions changes and layer_addpanel changes related to color change
* Follow scss file naming conventions for source_select
* Review feedback
* Restore clobbered layer_control view to master state. Add source_select css
* Update import to use plugin local indexPatternService, not maps'
* Review feedback, mostly i18n. Also add index to scss path
* i18n translation updates
* Assign error message to values rather than error object
* Update getMapColors to filter out transient layer
* Wrap Feature as FeatureCollection in Maps
* Add jest tests for geo processing functions. Add fixes for single feature handling
* i18n
* Review feedback. Test cleanup/fixes
* Update layer add panel footer logic to still show when source not selected
* Fix issue of not recognizing MultiPoint type. Remove throw logic for now
* Update telemetry with newly required placeholder function
* Prevent external modification of nested geojson objects
* i18n translation updates
* Revert "Fix issue of not recognizing MultiPoint type. Remove throw logic for now"
This reverts commit d692f913f8.
* Revert "Prevent external modification of nested geojson objects"
This reverts commit 0ea9fd3336.
* yarn.lock update
* [File upload] Remove dynamic fields from mappings, code and telemetry test (#38902)
* Remove dynamic fields from mappings, code and telemetry test
* Add file-upload-telemetry to spaces and es archiver test mappings
* Don't create telemetry saved object if none exists, create on first update instead
* Back out es archiver mappings update
