Commit graph

1260 commits

Author SHA1 Message Date
Dirk Baeumer 1a3fd783fe
Disable locale info check. 2021-07-06 15:38:21 +02:00
Dirk Baeumer ae52915f0c
Improve localization tests 2021-07-06 13:15:43 +02:00
Benjamin Pasero 7a0714a6e5
smoke - harden data migration test 2021-07-06 12:10:08 +02:00
Sandeep Somavarapu 062e6519f8
use vscode smoke test check extension 2021-07-02 10:46:31 +02:00
Benjamin Pasero 99a831f6e8
web - allow integration tests to open a workspace 2021-07-01 15:38:30 +02:00
João Moreno ca61a42383
web smoke test: bump up port number 2021-06-30 12:01:31 +02:00
João Moreno a44611d4c6
tests: hide output 2021-06-30 11:07:35 +02:00
Dirk Baeumer a63e0a0a21
Reenable German translation test based on new API 2021-06-29 09:23:52 +02:00
Robo 92999c339e
fix: setuid helper permission (#126798)
* ci: fix setuid helper binary permission

* ci: allow privilege for CLONE_NEWPID | CLONE_NEWNET

* ci: fix chrome-sandbox permission with rpm

* ci: fix permissions on snap

* ci: download electron earlier for arm builds
2021-06-22 08:42:30 -07:00
Dirk Baeumer 9cfc4bcb5c
Improve wording for failing test case 2021-06-22 13:30:29 +02:00
Dirk Baeumer 1f2a6cc331
Improve localization smoke test 2021-06-21 13:54:16 +02:00
João Moreno 3d2dea7a9f
fix integration browser tests 2021-06-15 15:15:41 +02:00
João Moreno 9d4274bf64
skip the entire notebook test suite so Code isn't even launched 2021-06-15 14:07:35 +02:00
Benjamin Pasero 81317867a3
web - disable welcome in smoke tests (fix #126221)
//cc @JacksonKearl
2021-06-15 10:57:29 +02:00
Benjamin Pasero 2ddc1e4fc5
Produce logs during test runs (integration, smoke) and publish as artefacts (#126065)
* first cut upload log files from integration test runs

* always publish log files

* move logs into integration-tests folder

* also produce logs for smoke tests

* store remote log files

* wire in logspath arg properly

* always publish logs

* enable logs for browser based smoke tests

* log only in verbose
2021-06-15 08:33:26 +02:00
João Moreno 85a36ae074 enable smoketest on linux 2021-06-11 12:19:17 +02:00
Alexandru Dima 0c87771e7b
Merge pull request #125754 from microsoft/alex/update-playwright
Update to `playwright@1.11.1`
2021-06-10 23:00:14 +02:00
Ladislau Szomoru 291ec03bc3
Smoke tests - run each suite in its own user-data-dir (#125670)
* Initial commit for isolated smoke tests

* Do not wait for the explorer view when restarting

* Debug failed localization test

* User data path suffix length

* Kill server when application exits

* Do not run yarn install

* Refactor application start/stop

* Reverted some changes

* Missed one

* One more change to revert

* Restored one more file

* Pull request feedback
2021-06-10 06:40:57 -07:00
Alexandru Dima 911ef74bbb
Merge remote-tracking branch 'origin/main' into alex/update-playwright 2021-06-10 09:27:13 +02:00
Alexandru Dima b1bc7be9d1
Do not rely on console.log for exiting tests 2021-06-09 21:47:18 +02:00
Benjamin Pasero 33f84b36fd
tests - disable getting started in integration tests and smoke tests (#125808)
* tests - disable getting started in integration tests and smoke tests

* --skip-getting-started => --skip-welcome

Co-authored-by: Jackson Kearl <jakearl@microsoft.com>
2021-06-09 18:36:42 +00:00
Henning Dieterichs 00123b32a7
Updates sinon from 1.17 to 11.1. 2021-06-09 10:03:46 +02:00
Henning Dieterichs 294b3bdf16
Revert "Include sinon-test in renderer.html / index.html"
This reverts commit e4e0f8be89.
2021-06-08 17:18:34 +02:00
Henning Dieterichs e4e0f8be89
Include sinon-test in renderer.html / index.html 2021-06-08 17:16:54 +02:00
Ladislau Szomoru 4e0cf885d4
Don't wait for the explorer when restarting 2021-06-07 15:34:37 +02:00
Ladislau Szomoru b003560579
Use --disable-workspace-trust for the test runs 2021-06-07 09:59:26 +02:00
Ladislau Szomoru e2d0e05038
Do not run yarn install as part of the smoke tests 2021-06-06 21:14:37 +02:00
Benjamin Pasero 802648f2a1 electron - adopt some newer APIs and 💄 main code 2021-06-04 09:22:38 +02:00
Ladislau Szomoru b0509196f6
Disable workspace trust during tests 2021-06-03 10:21:19 +02:00
João Moreno 541a5d4fa7
smoke: trust dialog should appear in 5 seconds, and we can ignore if it doesn't 2021-05-28 11:38:10 +02:00
SteVen Batten e5f70e8e2a
fix smoke tests (#124814) 2021-05-28 09:30:47 +02:00
Benjamin Pasero 5eec3c86e3 smoke test - disable experiments 2021-05-27 07:30:23 +02:00
Benjamin Pasero 4cda850edf disable welcome editor in smoke tests (fix #124674) 2021-05-26 14:22:13 +02:00
Matt Bierner b1823157d5
Fall back to existing published webview commit (#119295) 2021-05-24 22:36:22 -07:00
Jackson Kearl 5ac4f5f5be
Fix didn't work 2021-05-21 11:21:27 -07:00
Jackson Kearl 852b70eafe
Try fix (#124384) 2021-05-21 17:36:20 +00:00
Benjamin Pasero 4b5db9098c
smoke test - disable failing search test
//cc @JacksonKearl
2021-05-21 09:12:01 +02:00
Jackson Kearl e66c62f38a
Generalize smoke text 2021-05-20 22:21:19 -07:00
Jackson Kearl 89c8f91988
Add smoke test for #124146 2021-05-20 22:20:48 -07:00
Ladislau Szomoru a13da59f95
Fix #123621 2021-05-12 10:04:46 +02:00
Rob Lourens 5c22e55a74 Disable broken smoke test #123621 2021-05-11 19:26:30 -07:00
dependabot[bot] 000e064e61
Bump hosted-git-info from 2.8.8 to 2.8.9 in /test/smoke (#123481)
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-11 07:57:52 -07:00
Dirk Baeumer 2fc3214ba4
Fixes #123083: Localization: starts with 'DE' locale and verifies title and viewlets text is in German 2021-05-10 13:57:43 +02:00
João Moreno 9a3bfc1ea1
add leak test scaffolding 2021-05-06 14:45:12 +02:00
Benjamin Pasero e2954beb4b
Remove --no-sandbox (#122909)
* linux - remove `--no-sandbox` flag

We should not disable sandbox on Linux.
We only added this to support running on Kernel <= 3.8 where setuid sandbox is used.

* fix ci

* add some docs
2021-05-04 11:18:19 +02:00
Robo bce4d71f64
chore: bump node@14.x in devDependencies (#122905) 2021-05-04 10:04:13 +02:00
Connor Peet 24ea8408c3
testing: propoagate timeout option for unit tests 2021-04-29 12:19:56 -07:00
Connor Peet b1569995f9
testing: allow waiting for server before running unit tests
Fixes https://github.com/microsoft/vscode/issues/121751
2021-04-26 16:31:36 -07:00
Benjamin Pasero 7095f4706c
unit tests (electron) - avoid sending circular structures over 2021-04-21 08:13:08 +02:00
Benjamin Pasero f8c7dd64a4 tests - ensure preload promises are all resolved 2021-04-09 08:28:21 +02:00
Benjamin Pasero ff0f7f3cf2
Make product configuration available to non node.js environments (fix #120243) 2021-04-06 08:55:31 +02:00
Benjamin Pasero 1db94dbb10
sandbox - add additional arguments also to test runner 2021-04-01 08:10:40 +02:00
Ladislau Szomoru 335bc0196c
Remove unused dependency 2021-03-31 10:11:35 +02:00
Ladislau Szomoru 45f061b9c8
Swap cpx for cpx2 2021-03-31 09:24:44 +02:00
Logan Ramos d5a670eaeb
Switch to deepStrictEqual part of #118667 2021-03-30 12:05:06 -04:00
Ladislau Szomoru 7c498ec3a8
Update tests to use consistent version of mkdirp 2021-03-30 14:00:37 +02:00
Connor Peet 48e8463a89
testing: tear down app if renderer crashes
Fixes https://github.com/microsoft/vscode/issues/117068
2021-03-29 16:41:03 -07:00
Logan Ramos 31fd94b660
First round of getting rid of deprecated asserts (#120125) 2021-03-29 15:48:46 -04:00
Dirk Baeumer 08c4cd87e3 Reenable localization tests. 2021-03-25 10:42:35 +01:00
Megan Rogge 2c232fed6a
Update README.md 2021-03-24 12:40:21 -07:00
rebornix 2b66dfe021
skip flaky tests. 2021-03-23 21:49:48 -07:00
Rob Lourens cca8a48b17 Restore notebook smoketests, Fix #113882 2021-03-23 08:49:57 -07:00
Erich Gamma 73b9e6679d
Update README.md 2021-03-23 14:54:30 +01:00
Connor Peet 808266d4de
Allow tests to be loaded asynchronously (#119537)
* initial wip in the extension host

* wip

* wip

* wip

* continued progress

* update api from discussion

* update for new proposal

* wip

* update to lastest testing api, almost everything working

Refs https://github.com/microsoft/vscode/issues/115089
Design https://gist.github.com/connor4312/73f1883d720654834b7fd40550d3b6e0

* re-wire retirement state

* update actions to new async test structure

* minor cleanup

* remove unused es2018 that failed build
2021-03-22 18:08:13 -07:00
Alexandru Dima 242ddc4bdf
Merge pull request #118400 from thebinarysearchtree/main
Fix transformToSnakecase
2021-03-18 22:02:49 +01:00
Johannes Rieken b353f06339
comment out failing notebook smoke tests, https://github.com/microsoft/vscode/issues/119164 2021-03-17 09:22:44 +01:00
Jackson Kearl ac0008a25b
Remove waiting for welcome page while we are experimenting with what the welcome page will be
Fixes #118748
2021-03-11 09:23:38 -08:00
Jackson Kearl 556e6d8310
Fixup search smoke tests 2021-03-09 12:50:36 -08:00
Jackson Kearl 9eceb50498
Move label about for #115244 after more tests are failing 2021-03-09 11:25:45 -08:00
Jackson Kearl e5a10ca55d
Skip search tests while diagnosing flakeyness 2021-03-09 11:16:09 -08:00
Sandeep Somavarapu edfbf27e3a #118443 - change name 2021-03-09 09:34:19 +01:00
Sandeep Somavarapu 9d3a1d9f59 Fix #118443 2021-03-09 09:30:56 +01:00
Jackson Kearl ac229ced07
Revert "Reenable smoke test to see if still flakey" #115244
This reverts commit 5d80c30e5b.
2021-03-08 19:31:58 -08:00
Benjamin Pasero 9c78fa40ca
skip failing test (#118443) 2021-03-08 15:22:24 +01:00
Andrew Jones 2a0f9aa8dc Reminder to compile between tests 2021-03-08 19:52:39 +10:00
Andrew Jones 3f46e7f7b5 add ui flag to mocha tests 2021-03-08 15:04:26 +10:00
Jackson Kearl 5d80c30e5b
Reenable smoke test to see if still flakey
Ref #115244
2021-03-05 22:05:32 -08:00
Sandeep Somavarapu 1a9df6127c fix smoke tests 2021-03-05 15:27:03 +01:00
Sandeep Somavarapu c6202bb3cc Adopt to use new table widget 2021-03-04 19:51:03 +01:00
Benjamin Pasero e916637528 tests - enable color output for integration tests 2021-03-01 15:10:15 +01:00
Benjamin Pasero 5fabc0d9dc smoke test - remove code only needed for stable january release 2021-02-25 09:31:55 +01:00
Benjamin Pasero 041e4d9b1f fix smoke tests with stable build (fix #117420) 2021-02-23 19:04:51 +01:00
Connor Peet a0e24147e0
testing: remove duplicate extension and internal IDs
Fixes #117084
2021-02-19 16:57:29 -08:00
Raymond Zhao ee820dffbf
Update smoke test README (#115582) 2021-02-19 09:05:11 -08:00
João Moreno 2ca2792a65
fix smoketest 2021-02-19 16:03:01 +01:00
Jackson Kearl 096e5555b6 Allow welcome page to not be active on account of focus potentially going to a getting started item. 2021-02-18 18:57:24 -08:00
Benjamin Pasero 9de38fc9a8
Disable keytar in our integration tests (#116852)
* tests - extract and use ALL_PLATFORMS_API_TESTS_EXTRA_ARGS

* tests - throw when accessing keytar from API tests
2021-02-17 15:09:40 +01:00
João Moreno 38ae92f4b1
remove unused ui tests 2021-02-16 11:59:30 +01:00
João Moreno 980becda8d main rename
fixes #116341
2021-02-14 20:10:24 +01:00
Rob Lourens a699ffaee6 Reenable notebook smoke test
Fix #116535
2021-02-13 15:48:37 -08:00
Johannes Rieken fc31a6a558 skip failing test, https://github.com/microsoft/vscode/issues/116535 2021-02-12 11:15:56 +01:00
Jackson Kearl d10f2e6032 Fix #116412: Use getting started page as default startupEditor. 2021-02-11 12:34:14 -08:00
Jackson Kearl 8a3a5d9b74 Fix search smoke test 2021-02-09 15:44:07 -08:00
Sandeep Somavarapu cd97629c44 #116152 enable smoke tests 2021-02-09 08:58:39 +01:00
Rob Lourens be5126d37c Also skip 'before', #116156 2021-02-08 22:08:17 -08:00
Rob Lourens 0829608afb Change disabled tests to .skip 2021-02-08 21:37:02 -08:00
Megan Rogge 29844a0baf disable smoke tests that time out 2021-02-08 18:21:39 -08:00
Jackson Kearl 4623a50995 Fix #116154 2021-02-08 17:46:12 -08:00
Sandeep Somavarapu 63310d91d3 remove install extension 2021-02-04 20:48:25 +01:00
Sandeep Somavarapu a9ccd48a1f enable extension tests in web 2021-02-04 08:44:06 +01:00
Sandeep Somavarapu 4e6d9ce2b7 add server cli test 2021-02-03 21:42:12 +01:00
Sandeep Somavarapu 066c584405 enable remote smoke tests 2021-02-03 19:20:44 +01:00
Benjamin Pasero 7a6a45fb2c browser tests - fix regression from updating playwright 2021-02-02 16:53:30 +01:00
Sandeep Somavarapu abb05eb058 Fix #115586 2021-02-02 10:25:50 +01:00
Benjamin Pasero 04d4310a8a fix smoketest (#115586) 2021-02-02 08:20:07 +01:00
Rob Lourens 662a698ef9 Skip failing test #115244 2021-01-28 12:00:17 -08:00
Alexandru Dima 0a943766a5
Fix monaco checks 2021-01-28 14:48:41 +01:00
Martin Aeschlimann 4bd2d367e7 Remote smoketest timeout (fixes #115159) 2021-01-27 15:52:25 +01:00
Sandeep Somavarapu 4b9b2ab9a6 Fix #114707 2021-01-21 17:46:40 +01:00
João Moreno f745a912ae
fixes #114420 2021-01-15 15:19:19 +01:00
Alexandru Dima 00f8540d79
Fixes #114332
`yarn` no longer compiles the smoke test as a postinstall script. This breaks the flow of running `yarn && yarn smoketest` to execute the smoketest.

So change `yarn smoketest` to also do the compiling of the smoke test. But that generates too much extra work on the build machine, which needs to invoke `yarn smoketest` twice, once for Electron and once for web.

To workaround that, `yarn smoketest-no-compile` is added, which can be used by the build machine to avoid needlessly compiling.

While here, the nodejs version is already checked in the `yarn` preinstall script, so remove the check from the smoke test.
2021-01-14 10:35:42 +01:00
Connor Peet 2b5ae783bf
testing: add full json reporter to show more complete output 2021-01-13 15:30:11 -08:00
Johannes Rieken b90166177b reuse stats collector for EchoRunner, fix missing titlePath-property, fixes https://github.com/microsoft/vscode/issues/114190 2021-01-12 11:21:32 +01:00
Alexandru Dima 533d094020
Allow logging FS access with stacks 2021-01-11 15:32:39 +01:00
Alexandru Dima 2fd00ba9fe
Add a way to troubleshoot fs calls 2021-01-11 09:11:33 +01:00
Benjamin Pasero eb40962288 tests (unit, electron) - set forbidOnly when running on build machine 2021-01-08 09:09:23 +01:00
Benjamin Pasero 96e2981c91 skip failing test (#113882) 2021-01-08 08:16:28 +01:00
Alexandru Dima 8ad08b04f5
Do not compile /test/ and /build/ scripts during postinstall, the scripts get compiled explicitly during CI 2021-01-07 16:47:11 +01:00
Rob Lourens d29388d255 Reenable test, fix #113768 2021-01-05 12:21:50 -08:00
Alexandru Dima 689fbbd960
Increase timeouts 2020-12-30 14:15:14 +01:00
Alexandru Dima 4b6280aa8c
Skip webkit 2020-12-28 21:53:35 +01:00
Alexandru Dima 9945754a62
Add editor smoke test (ported from monaco-editor) 2020-12-28 21:44:42 +01:00
Alexandru Dima f74ad9692f
Align @types/* dev dependencies (#112718) 2020-12-18 13:28:39 +01:00
Alexandru Dima aa2864d53f
Remove most of the yarn license warnings 2020-12-18 10:26:14 +01:00
Alexandru Dima fee6dbf4e0
Merge remote-tracking branch 'origin/master' into alex/node-modules-vscode 2020-12-17 22:02:47 +01:00
Alexandru Dima 7ca71e763d
Move away from deprecated 'vscode' node module (#112718) 2020-12-17 21:40:14 +01:00
Connor Peet 88f32df1d7
Merge branch 'mocha-update' 2020-12-17 12:08:24 -08:00
Connor Peet 04d7411785
eng: update mocha 2 -> 8 2020-12-17 11:24:15 -08:00
Alexandru Dima ed19f6082f
Share mocha related node modules (#112718) 2020-12-17 11:03:49 +01:00
Alexandru Dima bb157721fb
/build/ and /test/ should only have devDependencies 2020-12-17 10:27:39 +01:00
João Moreno 1100f27681
fix build 2020-12-16 11:18:28 +01:00
João Moreno 7743d971b1
concurrently -> run-run-all 2020-12-15 15:21:24 +01:00
Sandeep Somavarapu 0ac5e95251 Fix #109644 2020-12-02 17:50:49 +01:00
Martin Aeschlimann 3472159cb1
Merge pull request #111408 from microsoft/aeschli/notebookIcons
externalize notebook items
2020-12-01 09:29:13 +01:00
Sandeep Somavarapu 47ce2a8d6d #111845 fix localization tests 2020-11-30 20:26:00 +01:00
Sandeep Somavarapu 2890fd321f Fix #111485 2020-11-30 11:58:43 +01:00
Martin Aeschlimann 6c8f62432f Merge branch 'master' into aeschli/notebookIcons 2020-11-27 12:19:23 +01:00
Martin Aeschlimann 8012c255e5 fix keybinding smoke test after icon name changes 2020-11-27 12:19:03 +01:00
Martin Aeschlimann 4e45d27a72 fix stopIcon name 2020-11-27 12:12:54 +01:00
Sandeep Somavarapu 6a2535e0ab Fix #107965 2020-10-28 19:49:42 +01:00
Sandeep Somavarapu 758240249a Fix #109210 2020-10-23 09:20:36 +02:00
Benjamin Pasero 781e0e3192 windows - remove not needed and confusing disable-restore-windows CLI arg 2020-10-01 11:36:45 +02:00
Christof Marti f48da22bdf
Update README.md 2020-09-30 09:16:06 +02:00
Rob Lourens 20b9010c39
Update README.md 2020-09-29 16:20:51 -07:00
Benjamin Pasero 7f035b4be7 debt - introduce and adopt FileAccess
deprecate amd.ts and require.toUrl methods
2020-09-25 14:11:05 +02:00
Rob Lourens b47a60c67f Fix notebook smoketest
Fix #107219
2020-09-22 17:14:15 -05:00
Benjamin Pasero 5497e60ed2 sandbox - consolidate fileUriFromPath into one 2020-09-22 08:14:45 +02:00
ChaseKnowlden e8760a5d6c Fix capitalization of GitHub org 2020-09-17 11:43:03 +02:00
Rob Lourens 1893b55c25 Skip smoketest #105330 2020-09-16 17:06:40 -05:00
Benjamin Pasero 87847325a6 electron - set spellcheck: false again for windows 2020-09-11 09:49:18 +02:00