Commit graph

1199 commits

Author SHA1 Message Date
João Moreno 3d2dea7a9f
fix integration browser tests 2021-06-15 15:15:41 +02:00
João Moreno 9d4274bf64
skip the entire notebook test suite so Code isn't even launched 2021-06-15 14:07:35 +02:00
Benjamin Pasero 81317867a3
web - disable welcome in smoke tests (fix #126221)
//cc @JacksonKearl
2021-06-15 10:57:29 +02:00
Benjamin Pasero 2ddc1e4fc5
Produce logs during test runs (integration, smoke) and publish as artefacts (#126065)
* first cut upload log files from integration test runs

* always publish log files

* move logs into integration-tests folder

* also produce logs for smoke tests

* store remote log files

* wire in logspath arg properly

* always publish logs

* enable logs for browser based smoke tests

* log only in verbose
2021-06-15 08:33:26 +02:00
João Moreno 85a36ae074 enable smoketest on linux 2021-06-11 12:19:17 +02:00
Alexandru Dima 0c87771e7b
Merge pull request #125754 from microsoft/alex/update-playwright
Update to `playwright@1.11.1`
2021-06-10 23:00:14 +02:00
Ladislau Szomoru 291ec03bc3
Smoke tests - run each suite in its own user-data-dir (#125670)
* Initial commit for isolated smoke tests

* Do not wait for the explorer view when restarting

* Debug failed localization test

* User data path suffix length

* Kill server when application exits

* Do not run yarn install

* Refactor application start/stop

* Reverted some changes

* Missed one

* One more change to revert

* Restored one more file

* Pull request feedback
2021-06-10 06:40:57 -07:00
Alexandru Dima 911ef74bbb
Merge remote-tracking branch 'origin/main' into alex/update-playwright 2021-06-10 09:27:13 +02:00
Alexandru Dima b1bc7be9d1
Do not rely on console.log for exiting tests 2021-06-09 21:47:18 +02:00
Benjamin Pasero 33f84b36fd
tests - disable getting started in integration tests and smoke tests (#125808)
* tests - disable getting started in integration tests and smoke tests

* --skip-getting-started => --skip-welcome

Co-authored-by: Jackson Kearl <jakearl@microsoft.com>
2021-06-09 18:36:42 +00:00
Henning Dieterichs 00123b32a7
Updates sinon from 1.17 to 11.1. 2021-06-09 10:03:46 +02:00
Henning Dieterichs 294b3bdf16
Revert "Include sinon-test in renderer.html / index.html"
This reverts commit e4e0f8be89.
2021-06-08 17:18:34 +02:00
Henning Dieterichs e4e0f8be89
Include sinon-test in renderer.html / index.html 2021-06-08 17:16:54 +02:00
Ladislau Szomoru 4e0cf885d4
Don't wait for the explorer when restarting 2021-06-07 15:34:37 +02:00
Ladislau Szomoru b003560579
Use --disable-workspace-trust for the test runs 2021-06-07 09:59:26 +02:00
Ladislau Szomoru e2d0e05038
Do not run yarn install as part of the smoke tests 2021-06-06 21:14:37 +02:00
Benjamin Pasero 802648f2a1 electron - adopt some newer APIs and 💄 main code 2021-06-04 09:22:38 +02:00
Ladislau Szomoru b0509196f6
Disable workspace trust during tests 2021-06-03 10:21:19 +02:00
João Moreno 541a5d4fa7
smoke: trust dialog should appear in 5 seconds, and we can ignore if it doesn't 2021-05-28 11:38:10 +02:00
SteVen Batten e5f70e8e2a
fix smoke tests (#124814) 2021-05-28 09:30:47 +02:00
Benjamin Pasero 5eec3c86e3 smoke test - disable experiments 2021-05-27 07:30:23 +02:00
Benjamin Pasero 4cda850edf disable welcome editor in smoke tests (fix #124674) 2021-05-26 14:22:13 +02:00
Matt Bierner b1823157d5
Fall back to existing published webview commit (#119295) 2021-05-24 22:36:22 -07:00
Jackson Kearl 5ac4f5f5be
Fix didn't work 2021-05-21 11:21:27 -07:00
Jackson Kearl 852b70eafe
Try fix (#124384) 2021-05-21 17:36:20 +00:00
Benjamin Pasero 4b5db9098c
smoke test - disable failing search test
//cc @JacksonKearl
2021-05-21 09:12:01 +02:00
Jackson Kearl e66c62f38a
Generalize smoke text 2021-05-20 22:21:19 -07:00
Jackson Kearl 89c8f91988
Add smoke test for #124146 2021-05-20 22:20:48 -07:00
Ladislau Szomoru a13da59f95
Fix #123621 2021-05-12 10:04:46 +02:00
Rob Lourens 5c22e55a74 Disable broken smoke test #123621 2021-05-11 19:26:30 -07:00
dependabot[bot] 000e064e61
Bump hosted-git-info from 2.8.8 to 2.8.9 in /test/smoke (#123481)
Bumps [hosted-git-info](https://github.com/npm/hosted-git-info) from 2.8.8 to 2.8.9.
- [Release notes](https://github.com/npm/hosted-git-info/releases)
- [Changelog](https://github.com/npm/hosted-git-info/blob/v2.8.9/CHANGELOG.md)
- [Commits](https://github.com/npm/hosted-git-info/compare/v2.8.8...v2.8.9)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-11 07:57:52 -07:00
Dirk Baeumer 2fc3214ba4
Fixes #123083: Localization: starts with 'DE' locale and verifies title and viewlets text is in German 2021-05-10 13:57:43 +02:00
João Moreno 9a3bfc1ea1
add leak test scaffolding 2021-05-06 14:45:12 +02:00
Benjamin Pasero e2954beb4b
Remove --no-sandbox (#122909)
* linux - remove `--no-sandbox` flag

We should not disable sandbox on Linux.
We only added this to support running on Kernel <= 3.8 where setuid sandbox is used.

* fix ci

* add some docs
2021-05-04 11:18:19 +02:00
Robo bce4d71f64
chore: bump node@14.x in devDependencies (#122905) 2021-05-04 10:04:13 +02:00
Connor Peet 24ea8408c3
testing: propoagate timeout option for unit tests 2021-04-29 12:19:56 -07:00
Connor Peet b1569995f9
testing: allow waiting for server before running unit tests
Fixes https://github.com/microsoft/vscode/issues/121751
2021-04-26 16:31:36 -07:00
Benjamin Pasero 7095f4706c
unit tests (electron) - avoid sending circular structures over 2021-04-21 08:13:08 +02:00
Benjamin Pasero f8c7dd64a4 tests - ensure preload promises are all resolved 2021-04-09 08:28:21 +02:00
Benjamin Pasero ff0f7f3cf2
Make product configuration available to non node.js environments (fix #120243) 2021-04-06 08:55:31 +02:00
Benjamin Pasero 1db94dbb10
sandbox - add additional arguments also to test runner 2021-04-01 08:10:40 +02:00
Ladislau Szomoru 335bc0196c
Remove unused dependency 2021-03-31 10:11:35 +02:00
Ladislau Szomoru 45f061b9c8
Swap cpx for cpx2 2021-03-31 09:24:44 +02:00
Logan Ramos d5a670eaeb
Switch to deepStrictEqual part of #118667 2021-03-30 12:05:06 -04:00
Ladislau Szomoru 7c498ec3a8
Update tests to use consistent version of mkdirp 2021-03-30 14:00:37 +02:00
Connor Peet 48e8463a89
testing: tear down app if renderer crashes
Fixes https://github.com/microsoft/vscode/issues/117068
2021-03-29 16:41:03 -07:00
Logan Ramos 31fd94b660
First round of getting rid of deprecated asserts (#120125) 2021-03-29 15:48:46 -04:00
Dirk Baeumer 08c4cd87e3 Reenable localization tests. 2021-03-25 10:42:35 +01:00
Megan Rogge 2c232fed6a
Update README.md 2021-03-24 12:40:21 -07:00
rebornix 2b66dfe021
skip flaky tests. 2021-03-23 21:49:48 -07:00
Rob Lourens cca8a48b17 Restore notebook smoketests, Fix #113882 2021-03-23 08:49:57 -07:00
Erich Gamma 73b9e6679d
Update README.md 2021-03-23 14:54:30 +01:00
Connor Peet 808266d4de
Allow tests to be loaded asynchronously (#119537)
* initial wip in the extension host

* wip

* wip

* wip

* continued progress

* update api from discussion

* update for new proposal

* wip

* update to lastest testing api, almost everything working

Refs https://github.com/microsoft/vscode/issues/115089
Design https://gist.github.com/connor4312/73f1883d720654834b7fd40550d3b6e0

* re-wire retirement state

* update actions to new async test structure

* minor cleanup

* remove unused es2018 that failed build
2021-03-22 18:08:13 -07:00
Alexandru Dima 242ddc4bdf
Merge pull request #118400 from thebinarysearchtree/main
Fix transformToSnakecase
2021-03-18 22:02:49 +01:00
Johannes Rieken b353f06339
comment out failing notebook smoke tests, https://github.com/microsoft/vscode/issues/119164 2021-03-17 09:22:44 +01:00
Jackson Kearl ac0008a25b
Remove waiting for welcome page while we are experimenting with what the welcome page will be
Fixes #118748
2021-03-11 09:23:38 -08:00
Jackson Kearl 556e6d8310
Fixup search smoke tests 2021-03-09 12:50:36 -08:00
Jackson Kearl 9eceb50498
Move label about for #115244 after more tests are failing 2021-03-09 11:25:45 -08:00
Jackson Kearl e5a10ca55d
Skip search tests while diagnosing flakeyness 2021-03-09 11:16:09 -08:00
Sandeep Somavarapu edfbf27e3a #118443 - change name 2021-03-09 09:34:19 +01:00
Sandeep Somavarapu 9d3a1d9f59 Fix #118443 2021-03-09 09:30:56 +01:00
Jackson Kearl ac229ced07
Revert "Reenable smoke test to see if still flakey" #115244
This reverts commit 5d80c30e5b.
2021-03-08 19:31:58 -08:00
Benjamin Pasero 9c78fa40ca
skip failing test (#118443) 2021-03-08 15:22:24 +01:00
Andrew Jones 2a0f9aa8dc Reminder to compile between tests 2021-03-08 19:52:39 +10:00
Andrew Jones 3f46e7f7b5 add ui flag to mocha tests 2021-03-08 15:04:26 +10:00
Jackson Kearl 5d80c30e5b
Reenable smoke test to see if still flakey
Ref #115244
2021-03-05 22:05:32 -08:00
Sandeep Somavarapu 1a9df6127c fix smoke tests 2021-03-05 15:27:03 +01:00
Sandeep Somavarapu c6202bb3cc Adopt to use new table widget 2021-03-04 19:51:03 +01:00
Benjamin Pasero e916637528 tests - enable color output for integration tests 2021-03-01 15:10:15 +01:00
Benjamin Pasero 5fabc0d9dc smoke test - remove code only needed for stable january release 2021-02-25 09:31:55 +01:00
Benjamin Pasero 041e4d9b1f fix smoke tests with stable build (fix #117420) 2021-02-23 19:04:51 +01:00
Connor Peet a0e24147e0
testing: remove duplicate extension and internal IDs
Fixes #117084
2021-02-19 16:57:29 -08:00
Raymond Zhao ee820dffbf
Update smoke test README (#115582) 2021-02-19 09:05:11 -08:00
João Moreno 2ca2792a65
fix smoketest 2021-02-19 16:03:01 +01:00
Jackson Kearl 096e5555b6 Allow welcome page to not be active on account of focus potentially going to a getting started item. 2021-02-18 18:57:24 -08:00
Benjamin Pasero 9de38fc9a8
Disable keytar in our integration tests (#116852)
* tests - extract and use ALL_PLATFORMS_API_TESTS_EXTRA_ARGS

* tests - throw when accessing keytar from API tests
2021-02-17 15:09:40 +01:00
João Moreno 38ae92f4b1
remove unused ui tests 2021-02-16 11:59:30 +01:00
João Moreno 980becda8d main rename
fixes #116341
2021-02-14 20:10:24 +01:00
Rob Lourens a699ffaee6 Reenable notebook smoke test
Fix #116535
2021-02-13 15:48:37 -08:00
Johannes Rieken fc31a6a558 skip failing test, https://github.com/microsoft/vscode/issues/116535 2021-02-12 11:15:56 +01:00
Jackson Kearl d10f2e6032 Fix #116412: Use getting started page as default startupEditor. 2021-02-11 12:34:14 -08:00
Jackson Kearl 8a3a5d9b74 Fix search smoke test 2021-02-09 15:44:07 -08:00
Sandeep Somavarapu cd97629c44 #116152 enable smoke tests 2021-02-09 08:58:39 +01:00
Rob Lourens be5126d37c Also skip 'before', #116156 2021-02-08 22:08:17 -08:00
Rob Lourens 0829608afb Change disabled tests to .skip 2021-02-08 21:37:02 -08:00
Megan Rogge 29844a0baf disable smoke tests that time out 2021-02-08 18:21:39 -08:00
Jackson Kearl 4623a50995 Fix #116154 2021-02-08 17:46:12 -08:00
Sandeep Somavarapu 63310d91d3 remove install extension 2021-02-04 20:48:25 +01:00
Sandeep Somavarapu a9ccd48a1f enable extension tests in web 2021-02-04 08:44:06 +01:00
Sandeep Somavarapu 4e6d9ce2b7 add server cli test 2021-02-03 21:42:12 +01:00
Sandeep Somavarapu 066c584405 enable remote smoke tests 2021-02-03 19:20:44 +01:00
Benjamin Pasero 7a6a45fb2c browser tests - fix regression from updating playwright 2021-02-02 16:53:30 +01:00
Sandeep Somavarapu abb05eb058 Fix #115586 2021-02-02 10:25:50 +01:00
Benjamin Pasero 04d4310a8a fix smoketest (#115586) 2021-02-02 08:20:07 +01:00
Rob Lourens 662a698ef9 Skip failing test #115244 2021-01-28 12:00:17 -08:00
Alexandru Dima 0a943766a5
Fix monaco checks 2021-01-28 14:48:41 +01:00
Martin Aeschlimann 4bd2d367e7 Remote smoketest timeout (fixes #115159) 2021-01-27 15:52:25 +01:00
Sandeep Somavarapu 4b9b2ab9a6 Fix #114707 2021-01-21 17:46:40 +01:00
João Moreno f745a912ae
fixes #114420 2021-01-15 15:19:19 +01:00
Alexandru Dima 00f8540d79
Fixes #114332
`yarn` no longer compiles the smoke test as a postinstall script. This breaks the flow of running `yarn && yarn smoketest` to execute the smoketest.

So change `yarn smoketest` to also do the compiling of the smoke test. But that generates too much extra work on the build machine, which needs to invoke `yarn smoketest` twice, once for Electron and once for web.

To workaround that, `yarn smoketest-no-compile` is added, which can be used by the build machine to avoid needlessly compiling.

While here, the nodejs version is already checked in the `yarn` preinstall script, so remove the check from the smoke test.
2021-01-14 10:35:42 +01:00