Commit graph

410 commits

Author SHA1 Message Date
rnhmjoj
bc2188b083
nixos: fix qemu_test being used in normal VMs
This is an attempt to fixup PR #49403.
2020-10-21 16:38:04 +02:00
Andreas Rammhold
20893b3a70
nixos/tests: expose both the interactive and non-interactive driver
For a lot of the work the non-interactive drivers are enough and it is
probably a good idea to keep it accessible for debugging without
touching the Nix expression.
2020-10-19 17:39:48 +02:00
Andreas Rammhold
c096880d46
nixos/tests: make the driver attribute use a rich qemu
Since we previously stripped down the features of `qemu_test` some of
the features users are used to while running tests through the (impure)
driver didn't work anymore. Most notably we lost support for graphical
output and audio. With this change the `driver` attribute uses are more
feature complete version of QEmu compared to the one used in the pure
Nix builds.

This gives us the best of both worlds. Users are able to see the
graphical windows of VMs while CI and regular nix builds do not have to
download all the (unnecessary) dependencies.
2020-10-19 17:39:48 +02:00
zowoq
008de9ca3c nixos/{containers,cri-o,podman}: move copyFile to nixos/lib/utils 2020-09-24 10:01:47 +10:00
Janne Heß
9f33ab62d9
nixos/testing: Add support for specialArgs
Since using flakes disallows the usage of <unstable> (which I use in
some tests), this adds an alternative. By setting specialArgs, all VMs
can get the `unstable` flake input as an arg. This is not possible with
extraConfigurations, as that would lead to infinite recursions.
2020-09-14 00:10:21 +02:00
Félix Baylac-Jacqué
e571b315e2
test-driver.py: defaulting keepVmState in Machine init
ecb73fd555 introduced a new keepVmState
CLI flag for test-driver.py. This CLI flags gets forwarded to the
Machine class through create_machine.

It created a regression for the boot tests where __main__ end up not
being evaluated. See
https://github.com/NixOS/nixpkgs/pull/97346#issuecomment-690951837 for
bug report.

Defaulting keepVmState to false when __main__ ends up not being
evaluated.
2020-09-11 10:47:03 +02:00
Félix Baylac-Jacqué
ecb73fd555
test-driver.py: fix VM state directory deletion
The previous version of the code would only kick in if the state
directory path pointed at a *file*, which never occurs. Making that
codepath actually work reveals an ordering bug, which this patch fixes
as well.

It also replaces the confusing, imperative case log message "delete VM
state directory" with "deleting VM state directory".

Finally, we hint the user about how to prevent this deletion. IE. by
passing the --keep-vm-state flag.

Bug report:
https://github.com/NixOS/nixpkgs/pull/91046#issuecomment-685568750

Credit goes to Edef for the rebase on top of a recent nixpkgs commit
and for writing most of this commit message.

Co-authored-by: edef <edef@edef.eu>
2020-09-07 12:26:40 +02:00
Florian Klink
98d6b55fdc nixos/testing: remove remaining coverage-data logic
This isn't used anymore as per
https://github.com/NixOS/nixpkgs/pull/72354#discussion_r451031449.
2020-09-05 16:07:59 +02:00
WORLDofPEACE
18348c7829
Merge pull request #96042 from rnhmjoj/loaOf
treewide: completely remove types.loaOf
2020-09-02 08:45:37 -04:00
rnhmjoj
3f8a3246f4
nixos/lib/make-options-doc: remove loaOf subs
Remove the substitution for the <name?> placeholder used by loaOf,
now that the type has been deprecated.
2020-09-02 00:42:51 +02:00
Anders Kaseorg
59b6664f15 Revert "Merge pull request #96254 from Mic92/logging"
This reverts commit 4fc708567f, reversing
changes made to 0e54f3a6d8.

Fixes #96699.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-08-30 15:46:39 -07:00
Anders Kaseorg
a0a421bf5e Revert "Merge pull request #96152 from JJJollyjim/colour-test-machines-staging"
This reverts commit 1bff6fe17c, reversing
changes made to 2995fa48cb.

There’s presumably nothing wrong with this PR, except that it
conflicts with reverting #96254 which broke several tests (#96699).

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2020-08-30 15:45:44 -07:00
aszlig
b0ac24ae41
nixos/test-driver: Use guest time when using sleep
With the Perl driver, machine.sleep(N) was doing a sleep on the guest
machine instead of the host machine. The new Python test driver however
uses time.sleep(), which instead sleeps on the host.

While this shouldn't make a difference most of the time, it *does*
however make a huge difference if the test machine is loaded and you're
sleeping for a minimum duration of eg. an animation.

I stumbled on this while porting most of all my tests to the new Python
test driver and particularily my video game tests failed on a fairly
loaded machine, whereas they don't with the Perl test driver.

Switching the sleep() method to sleep on the guest instead of the host
fixes this.

Signed-off-by: aszlig <aszlig@nix.build>
2020-08-29 00:55:01 +02:00
Florian Klink
0620184f3f nixos/lib/test*: remove perl test driver
This has been deprecated in 20.03, and all tests have been migrated to
the python framework, effectively making this dead code.
2020-08-27 19:45:38 +02:00
Jamie McClymont
d7875caf76 nixos/test: colour machine names 2020-08-27 23:29:55 +12:00
Jörg Thalheim
4fc708567f
Merge pull request #96254 from Mic92/logging 2020-08-26 19:45:24 +01:00
Matthew Bauer
ca3fa9c32a
Merge pull request #95956 from matthewbauer/qemu-cpu-max
runInLinuxVM, test-driver: use -cpu max instead of -cpu host
2020-08-26 12:59:57 -05:00
Jörg Thalheim
87214dbd10
nixos/test-driver: re-introduce log()
Appearantly this is used in tests
2020-08-25 14:50:47 +01:00
Jörg Thalheim
f3c0a09c76
nixos/testdriver: sort imports 2020-08-25 10:15:24 +01:00
Jörg Thalheim
392415c285
nixos/test-driver: switch to pythons' logging lib
- Less code
- more thread-safe according to @flokli
2020-08-25 10:13:27 +01:00
Jörg Thalheim
c1667f85bb
nixos/test-driver: introduce main method
This way we not accidentally use introduce/use global variables.
Also it explictly mark the code for the mypy type checker.
2020-08-25 09:36:37 +01:00
Silvan Mosberger
b77d8ead28
Merge pull request #51850 from roberth/nixos-pure
nixos/lib/eval-config.nix: Add extraModules parameter for opt-in purity
2020-08-24 04:29:37 +02:00
Matthew Bauer
47b56e7c19 runInLinuxVM, test-driver: use -cpu max instead of -cpu host
This appears to avoid requiring KVM when it’s not available. This is
what I originally though -cpu host did. Unfortunately not much
documentation available from the QEMU side on this, but this appears
to square with help:

$ qemu-system-x86 -cpu help
...
x86 host                  KVM processor with all supported host features
x86 max                   Enables all features supported by the accelerator in the current host
...

Whether we actually want to support this not clear, since this only
happens when your CPU doesn’t have full KVM support. Some Nix builders
are lying about kvm support though. Things aren’t too slow without it
though.

Fixes https://github.com/NixOS/nixpkgs/issues/85394

Alternative to https://github.com/NixOS/nixpkgs/pull/83920
2020-08-21 23:42:07 -05:00
Janne Heß
ff03800d3b
nixos/testing: Fix fail() function
The docs say this behaves as succeed(), but it does not return stdout as
succeed() does. This fixes that behaviour
2020-08-21 21:28:24 +02:00
Jacek Galowicz
2a288cb1da
Merge pull request #93824 from blitz/fix-rpi4-installer
Fix Raspberry Pi 4B SD-Card Install Image
2020-08-08 13:45:02 +02:00
Maximilian Bosch
2ae62ef72f
Merge pull request #94611 from Ma27/rel-pkgs-error-msg
nixos/manual: improve error message for invalid values in `relatedPackages'
2020-08-03 20:17:46 +02:00
Maximilian Bosch
6d19c04416
nixos/manual: improve error message for invalid values in `relatedPackages'
As reported in NixOS discourse[1], tracking down invalid values in
`relatedPackages'[2] (i.e. list-items that don't exist in `pkgs`) is
fairly hard as the message "Invalid package attribute path `foobar'"
is quite unhelpful and the trace doesn't point to the source of the
problem either.

This patch improves the error message by mentioning that the issue is an
invalid `relatedPackages`-declaration in $optionName.

[1] https://discourse.nixos.org/t/invalid-package-attribute-path-nextcloud19/8403/9
[2] https://nixos.org/nixpkgs/manual/#sec-functions-library-options
2020-08-03 16:37:59 +02:00
zowoq
25d7880f17 nixos/lib/*: editorconfig fixes 2020-08-04 00:23:54 +10:00
Emery Hemingway
3f922834b8 nixos/lib/make-disk-image.nix: abritary format input
Pass unrecognized format types as the output file extension to
qemu-img. The motivation is support for "vdi" output.
2020-07-31 15:33:08 +02:00
Keshav Kini
5e86bba082 nixos/boot: some documentation improvements
- Give a more accurate description of how fileSystems.<name/>.neededForBoot
  works

- Give a more detailed description of how fileSystems.<name/>.encrypted.keyFile
  works
2020-07-29 14:39:21 -07:00
Lassulus
77cf1a6581
Merge pull request #89331 from Lassulus/make-disk-image
make-disk-image: add hybrid and dynamic sized images
2020-07-28 20:34:20 +02:00
lassulus
883a6079fd make-disk-image: add hybrid and dynamic sized images 2020-07-27 19:45:55 +02:00
Julian Stecklina
fd2047c642 nixos/lib/make-ext4-fs: fix after mkfs.ext4 refactoring
In 9ac1ab10c9 this library function was
refactored to use mkfs.ext4 instead of cptofs. There are two problems:

If populateImageCommands would create no files (which is fine), a cp
invocation would fail due to missing source arguments.

Another problem is that mkfs.ext4 relies on fakeroot to have sane
uid/gids in the generated filesystem image. This currently doesn't
work for cross compiling.
2020-07-26 22:49:55 +02:00
Timo Kaufmann
830a8d6ee1
Merge pull request #74174 from raboof/fix-74047-stable-gpt-disk-guid
make-iso9660-image: produce stable GPT disk GUID
2020-07-21 00:30:57 +02:00
Bas van Dijk
6e7822b8f3 lib: toHex -> toHexString & toBase -> toBaseDigits
This makes the type of these functions more apparent from the name.
2020-07-20 13:14:19 +02:00
Bas van Dijk
e15815e885 nixos/tests/networking.nix: test the services.dhcpd4.machines option
This modifies the `router` to not give out a range of IP addresses but
only give out a fixed address based on the MAC address using the
`services.dhcpd4.machines` option.

To get access to the MAC address the `qemuNicMac` function is defined
and exported from `qemu-flags.nix`.
2020-07-20 13:09:27 +02:00
Bas van Dijk
0410f5dff9 nixos/tests: support up to 255 nodes in NixOS tests 2020-07-20 13:09:27 +02:00
Arnout Engelen
be006eab1f
make-iso9660-image: produce stable GPT disk GUID
By generating a version-5 GUID based on $out (which contains
the derivation hash) and preventing isohybrid from overwriting
the GPT table (which already is populated correctly by xorriso).

Tested by:
* booting from USB disk on a UEFI system
* booting from USB disk on a non-UEFI system
* booting from CD on a UEFI system
* booting from CD on a non-UEFI system
* booting from CD on an OSX system

Also tested that "nix-build ./nixos/release-combined.nix -A
nixos.iso_minimal.x86_64-linux -I nixpkgs=~/nixpkgs-r13y --check"
now succeeds.

Fixes #74047
2020-07-20 11:16:59 +02:00
lassulus
8d05772134
nixos make-disk-image: fix permissions of /build
This was broken in 460c0d6 (PR #90431); now the nixos-unstable channel
should get unblocked.
vcunat modified this commit to use env-var instead of hardcoding /build
2020-07-10 10:31:26 +02:00
Justinas Stankevičius
f815cb6adf
nixos/test-driver: print a traceback when testScript fails (#92369)
* print a traceback: assertion message can be empty

* change print back to eprint
2020-07-06 23:29:40 +02:00
Samuel Dionne-Riel
736c7ca712
Merge pull request #82718 from misuzu/armv7l-ext4-fs-fix
nixos/lib/make-ext4-fs: use mkfs.ext4 instead of cptofs
2020-07-01 21:38:07 -04:00
misuzu
9ac1ab10c9 nixos/lib/make-ext4-fs: use mkfs.ext4 instead of cptofs
This fixes image creation on armv7l when image is bigger than 2G.
Also fix some reproducibility issues and other cptofs issues.
2020-07-01 11:32:28 +03:00
Michele Guerini Rocco
dab676b2d7
Merge pull request #65231 from buckley310/grub-password
grub: add support for passwords
2020-07-01 09:04:30 +02:00
Alexandre Esteves
e10e7d6a8b
testing-python: fix typo 2020-06-30 22:31:32 -05:00
Florian Klink
9e248c9ec9
Merge pull request #91046 from NinjaTrappeur/nin-delete-vm-state
test-driver.py: delete VM state directory after test run
2020-06-28 18:41:38 +02:00
rnhmjoj
b520055df6
nixos/lib/test-driver: add wait_for_console_text
This method is similar to wait_for_text but is based on matching
serial console lines instead of the VGA output.
2020-06-24 10:22:53 +02:00
Félix Baylac-Jacqué
7e7aa529d9
test-driver.py: delete VM state directory after test run
Keeping the VM state test across several run sometimes lead to subtle
and hard to spot errors in practice. We delete the VM state which
contains (among other things) the qcow volume.

We also introduce a -K (--keep-vm-state) flag making VM state to
persist after the test run. This flag makes test-driver.py to match
its previous behaviour.
2020-06-21 10:27:51 +02:00
Florian Klink
079de14fd3 nixos/make-ext4-fs: increase fudge factor from 1.03 to 1.10
Turns out, on smaller images (~800MiB uncompressed sdcard image size),
the current fudge factor is way too small to even get the system to the
phase where it can resize itself.

I first tried with 1.05, but it wasn't enough.
2020-06-21 09:47:02 +02:00
Peter Hoeg
eb62c7edc8
Merge pull request #90261 from prusnak/installer-zstd-rm
images: remove original files when using zstd for compression
2020-06-14 10:39:10 +08:00
Pavol Rusnak
490fc040be
images: remove original files when using zstd for compression 2020-06-14 00:19:11 +02:00