## Summary
Assert unreachable was created through advice given by both the Typescript community and through the techniques that TyepScript is trying to achieve type safety with switch statements.
This fixes recent bugs by:
* Re-adding the never type
* Reduces the two different types by putting the helper within the common section so there's not duplication
* Fixes on type that looks like it was a regular string rather than a one of the enum types
The reasoning for exhaustive checks within switch statements and techniques can be seen in numerous areas such as here:
https://stackoverflow.com/questions/39419170/how-do-i-check-that-a-switch-block-is-exhaustive-in-typescript
You can do it either way with TypeScript as long as you ensure you have a explicit return type and you do early return statements you can actually avoid having to call into the assertUnreachable.
If introduced and used correctly it is there to help out like this error it is telling us that this string type is not exhaustive:
<img width="921" alt="Screen Shot 2020-08-24 at 10 39 42 AM" src="https://user-images.githubusercontent.com/1151048/91075618-9b1ad380-e5fb-11ea-9200-1c355faf5dca.png">
You can notice that for this pull request I actually remove the assertion like so if someone accidentally removes one of the switch statements:
<img width="1014" alt="Screen Shot 2020-08-24 at 10 42 08 AM" src="https://user-images.githubusercontent.com/1151048/91075662-a968ef80-e5fb-11ea-8d74-a92eedd63892.png">
And since the function has an explicit return type it is not needed. You will see that TypeScript improved its never types behind the scenes where it actually will tell you that it will never reach the `assertUnreachable` and want to remove it as an auto-refactor. That is ok as long as we have explicit return types and what I did with one line of code here.
<img width="536" alt="Screen Shot 2020-08-24 at 11 21 05 AM" src="https://user-images.githubusercontent.com/1151048/91075861-efbe4e80-e5fb-11ea-9991-dda111a04f1d.png">
Without this fix, and having the never type become an unknown it introduces less safety where any code that is utilizing the assertUnknown without explicit return types will be prone to having run time errors being thrown when something new is added to their switch enum types.
* changing transaction type filter to radio group
* fixing unit test
* changing transaction type filter to radio group
* adding onclick to the badge component
* adding onclick to the badge component
* adding i18n to aria
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
resolves https://github.com/elastic/kibana/issues/68209
Since routing key figures fairly prominently throughout PagerDuty APIs,
and ours, it seems like it make sense to include it in the single validation
message we have for it, as well as using the term we use for it in the product:
"integration key".
See the referenced issue for more background.
* Fixed alerting_api_integration/security_and_spaces tests failing if actions proxy set on for parallel process running using commands 'scripts/functional_tests_server' and 'scripts/functional_test_runner'
* -
* Fixed get port from range for Slack and webhook simulators, removed some test warnings
* Added check for listening proxy server
* changed logger to debug removed not useful error
* -
* changed proxy to dynamic target in a single place
* test retry
* -
* -
* -
* -
* test with no cleanup
* -
* -
* -
* -
* Added environment variable ALERTING_PROXY_PORT
* fixed type checks
* fixed clean up proxy server port
* Rename Whitelist to AllowList in Actions and Alerting
* revert not related change
* Fixed due to comments and tests failing
* Fixed failing tests
* Fixed due to comments
The release manager is currently expecting a Docker asset image with
the format of `kibana-8.0.0-SNAPSHOT-docker-image.tar.gz`. If this
target is not found, it will re-export the image. Making this change to
produce the expected filename will remove that duplicated effort.
Additionally, the release manager plans to remove this fallback in the
future anyways.
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [Setup] Change error connecting status code to 502
- For clearer error handling
* Set up new HttpProvider/Logic Kea store & listeners
- This allows us to:
- connect() directly to HttpLogic in other Kea logic files that need to make http calls, instead of passing in http manually via args
- Set http interceptors & remove them interceptors on unmount within Kea
- Share state derived from http (e.g. errorConnecting, readOnlyMode) between both AS & WS (but allow each app to handle that state differently if needed)
+ Refactors necessary for these changes:
- Kea types - add events key, clarify that mount returns an unmount function, fix reducer state type
- ReactDOM unmount - remove resetContext({}), was preventing logic from unmounting properly
* Update AS & WS to show error connecting component at app level
* [WS] Remove errorConnecting logic & http arg from Overview
- Since main app is now handling errorConnecting
- http can now be connected directly from HttpLogic Kea store, so no need to pass it
+ minor cleanup in logic_overview.test.ts - remove unneeded unmount(), act(), switch to HttpLogic mock
* [AS] Add top-level ErrorConnecting component & remove error logic from EngineOverview
* [AS] Clean up/move EngineOverview child components into subfolder
- delete old ErrorState component
- move LoadingState, EmptyState, and EngineOverviewHeader into subfolders in engine_overview
* PR feedback: Update test assertions 404 copy
* Simplify our kibana mocks
* Simpler mock factory that returns an object instead of a thunk
* We can use mockReturnValue instead of mockImplementation to
accomplish the same
* Allows us to replace createStartServices mock
* Uses unknown instead of any for mocks
* Clean up our manual use of kibana mocks in tests
* Since our useKibana mock returns a consistent mock, we can modify its
return value instead of re-mocking the entire thing
* Removes unnecessary uses of clearing/resetting mocks
* If your mocks are configured at the beginning of each test this is
usually unnecessary.
* I left one case of clearAllMocks in all_cases/index.test since it
defined several mock functions that were persistent across tests, and
it was easier than moving their definitions to a beforeEach
* Removes some unnecessary overrides that seemed due to storage
previously not being mocked
* Rename some old occurrences of SIEM
* Cross-reference similar hooks via JSDoc
There's a good chance that the consumer might want the OTHER hook, so
let's make that discoverable.
* Adds jest tests for our useListsConfig hook
* adds mocks for the hooks upon which it depends
* Add a mock for our useListsConfig hook
Leverages this mock factory in our manual mock for this hook.
* Remove unneeded eslint exception
* Move kibana_react mocks into their own .mock file
We're trying to consolidate mocks to this pattern so that they're easier
to find and reuse.
* Remove intermediate mock factory
This was only being consumed by our general createStartServicesMock.
* Replace security_solution's alias for a core mock
This is just noise/tech debt, we should use the core mock directly when
we can.
* Remove unnecessary wrapper around core mocks
Instead let's just reference the core mocks themselves.
* Remove outdated references from upstream
* More accurate mock
Throw an error of the same type if an unexpected key is used.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* migrate ui settings to core
* add basic test on service
* add unit tests
* adapt buildNum schema
* use any for buildNum...
* move i18n keys to core prefix
* translate added validation messages
* using number for schema for buildNum
* move state:storeInSessionStorage setting to core
* remove overrides config validation
* remove defaultRoute from config schema
## Summary
On the `security_solution` project from different customers we have been getting reports of scaling issues and excessive NodeJS event blocking times. After in-depth tracing through some of the index and field capabilities calls we identified two of the "hot paths" running through `field_capabilities` to where it is using double looped arrays rather than hashes. By switching these two hot spots out for hashes we are now able to reduce the event loop block times by an order of magnitude.
Before this PR you can see event loop block times as high as:
```ts
field_cap: 575.131ms
```
And after this PR you will see event loop block times drop by an order of magnitude to:
```ts
field_cap: 31.783ms
```
when you're calling into indexes as large as `filebeat-*`. This number can be higher if you're concatenating several large indexes together trying to get capabilities from each one all at once. We already only call `getFieldCapabilities` with one index at a time to spread out event block times.
The fix is to use a hash within two key areas within these two files:
```ts
src/plugins/data/server/index_patterns/fetcher/lib/field_capabilities/field_capabilities.ts
src/plugins/data/server/index_patterns/fetcher/lib/field_capabilities/field_caps_response.ts
```
This effect happens during the query of `SourceQuery`/`IndexFields` within `security_solution` but you should be able to trigger it with any application code who calls into those code paths with large index sizes such as `filebeat-*` anywhere in Kibana.
An explanation of how to see the block times for before and after
---
Add, `console.time('field_cap');` and `console.timeEnd('field_cap');` to where the synchronize code is for testing the optimizations of before and after.
For example around lines 45 with the original code:
```ts
const esFieldCaps: FieldCapsResponse = await callFieldCapsApi(callCluster, indices);
console.time('field_cap'); // <--- start timer
const fieldsFromFieldCapsByName = keyBy(readFieldCapsResponse(esFieldCaps), 'name');
const allFieldsUnsorted = Object.keys(fieldsFromFieldCapsByName)
.filter((name) => !name.startsWith('_'))
.concat(metaFields)
.reduce(concatIfUniq, [] as string[])
.map<FieldDescriptor>((name) =>
defaults({}, fieldsFromFieldCapsByName[name], {
name,
type: 'string',
searchable: false,
aggregatable: false,
readFromDocValues: false,
})
)
.map(mergeOverrides);
const sorted = sortBy(allFieldsUnsorted, 'name');
console.timeEnd('field_cap'); // <--- outputs the end timer
return sorted;
```
And around lines 45 with this pull request:
```ts
const esFieldCaps: FieldCapsResponse = await callFieldCapsApi(callCluster, indices);
console.time('field_cap'); // <--- start timer
const fieldsFromFieldCapsByName = keyBy(readFieldCapsResponse(esFieldCaps), 'name');
const allFieldsUnsorted = Object.keys(fieldsFromFieldCapsByName)
.filter((name) => !name.startsWith('_'))
.concat(metaFields)
.reduce<{ names: string[]; hash: Record<string, string> }>(
(agg, value) => {
// This is intentionally using a "hash" and a "push" to be highly optimized with very large indexes
if (agg.hash[value] != null) {
return agg;
} else {
agg.hash[value] = value;
agg.names.push(value);
return agg;
}
},
{ names: [], hash: {} }
)
.names.map<FieldDescriptor>((name) =>
defaults({}, fieldsFromFieldCapsByName[name], {
name,
type: 'string',
searchable: false,
aggregatable: false,
readFromDocValues: false,
})
)
.map(mergeOverrides);
const sorted = sortBy(allFieldsUnsorted, 'name');
console.timeEnd('field_cap'); // <--- outputs the end timer
return sorted;
```
And then reload the security solutions application web page or generically anything that is going to call filebeat-* index or another large index or you could concatenate several indexes together as well to test out the performance. For security solutions we can just visit any page such as this one below which has a filebeat-* index:
```
http://localhost:5601/app/security/timelines/
```
Be sure to load it _twice_ for testing as NodeJS will sometimes report better numbers the second time as it does optimizations after the first time it encounters some code paths.
You should begin to see numbers similar to this in the before:
```ts
field_cap: 575.131ms
```
This indicates that it is blocking the event loop for around half a second before this fix. If an application adds additional indexes on-top of `filebeat`, or if it tries to execute other code after this (which we do in security solutions) then the block times will climb even higher.
However, after this fix, the m^n are changed to use hashing so this only climb by some constant * n where n is your fields and for filebeat-* it will should very low around:
```ts
field_cap: 31.783ms
```
### Checklist
Unit tests already present, so this shouldn't break anything 🤞 .
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* add asciidoc support for generated plugin list
Try level offset "=+2" instead of "=+1" to stop the inlining of the includes.
remove +2 back to +1
* Remove asciidoc, switch to regex. Rearrange dev guide to avoid nesting limit.
* Add tests for regex
* add a description to not throw off the table. Remove the heading from the paragraph snippet.
* Fix more READMEs so table renders correctly
* Update plugin list
* Remove code-exploration file, moved to plugin-list
* fix typo
* Add link to developer examples
* Update plugin list
* fix typo
* Add support for decorating 429 errors in the saved objects client
* Update the docs
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* check for packages stuck installing and reinstall
* update mock endpoint package
* diferentiate between reinstall and reupdate type of install, remove isUpdate, add integration test
* create new EpmPackageInstallStatus type instead of using InstallStatus
* fix merge conflict
* change EpmPackageInstallStatus to a union type
* change time to install to 1 minute
* used saved object find
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* fixes 'sets and reads the url state for timeline by id' timeline ttest
* makes test more reliable
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
