Commit graph

1256 commits

Author SHA1 Message Date
fanquake 8334ee31f8
scripts: add MACHO LAZY_BINDINGS test to test-security-check.py
I didn't add the relevant test in #18295.
2020-04-21 11:32:06 +08:00
fanquake 7b99c7454c
scripts: add MACHO Canary check to security-check.py 2020-04-21 11:32:01 +08:00
MarcoFalke 54f812d9d2
Merge #18673: scripted-diff: Sort test includes
fa4632c417 test: Move boost/stdlib includes last (MarcoFalke)
fa488f131f scripted-diff: Bump copyright headers (MarcoFalke)
fac5c37300 scripted-diff: Sort test includes (MarcoFalke)

Pull request description:

  When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.

  This pull preempts both issues by just sorting all includes in one commit.

  Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.

  Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.

ACKs for top commit:
  practicalswift:
    ACK fa4632c417
  jonatack:
    ACK fa4632c417, light review and sanity checks with gcc build and clang fuzz build

Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
2020-04-17 10:12:13 -04:00
Wladimir J. van der Laan 8f2497941e
Merge #18598: gitian: Add missing automake package to gitian-win-signer.yml
e44aeefaae gitian: Add missing automake package to gitian-win-signer.yml (Andrew Chow)

Pull request description:

  automake is needed to build osslsigncode otherwise autogen.sh fails with the docker virtualization method.

ACKs for top commit:
  hebasto:
    ACK e44aeefaae, for `osslsigncode-1.7.1` we did not run `autogen.sh` in the past.
  fanquake:
    ACK e44aeefaae
  jonatack:
    ACK e44aeef

Tree-SHA512: a0e615c1b099ee1c469ce41f886f2ece6746234a5a800743a4e8be671e4114fd30e1c35bc0ddcb75778409564129d0fde7ac4e3d70b0f7691f97f729f34c8e0c
2020-04-16 22:27:46 +02:00
MarcoFalke fa488f131f
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-04-16 13:33:09 -04:00
fanquake e831f18b1e
Merge #18619: gitian: add jonatack gpg key fingerprint
905e2e85ba gitian: add jonatack gpg key fingerprint (Jon Atack)

Pull request description:

  per request https://github.com/bitcoin-core/gitian.sigs/pull/1221#issuecomment-612778063

ACKs for top commit:
  laanwj:
    ACK 905e2e85ba
  fanquake:
    ACK 905e2e85ba

Tree-SHA512: bddd734f13c53859280db2fa94b47cbb2a8b3f17ed6a6fdda2bf04f7e201e310ae24930e8f4be2f8b65a949659a9d3369704ed70031da9653a66a513fe597b67
2020-04-15 15:38:02 +08:00
fanquake 5447d57bff
Merge #18624: Added my fingerprint Stephan Oeste (Emzy)
c47adf8df4 Added my fingerprint Stephan Oeste (Emzy) (Stephan Oeste)

Pull request description:

  By request from laanwj added my PGP fingerprint.
  See: https://github.com/bitcoin-core/gitian.sigs/pull/1220#issuecomment-612778442

ACKs for top commit:
  Sjors:
    ACK c47adf8. Fingerprint matches Twitter profile: https://twitter.com/emzy (haven't verified it in any other way)
  fanquake:
    ACK c47adf8df4

Tree-SHA512: 3e39ae88f507a12f11fb2d5c779eba79ee2daeddecd0dc3f1fddfa29ce963d0e9af3fa5a10357157812597c10205a6beae31cc70af9471a782da23d8753b7cbd
2020-04-15 15:31:40 +08:00
Stephan Oeste c47adf8df4
Added my fingerprint Stephan Oeste (Emzy)
By request from added my PGP fingerprint.
See: https://github.com/bitcoin-core/gitian.sigs/pull/1220#issuecomment-612778442
2020-04-13 16:39:42 +02:00
Jon Atack 905e2e85ba
gitian: add jonatack gpg key fingerprint 2020-04-13 14:23:06 +02:00
fanquake f2b5b0a3b4
build: add linker optimization flags to guix
Any -O argument will enable optimizations in GNU ld. We can use -O2
here, as this matches our compile flags. Note that this would also
enable additional optimizations if using the lld or gold linkers,
when compared to -O0.
2020-04-12 18:38:00 +08:00
fanquake b8b050a8d6
build: add linker optimization flags to gitian descriptors
Any -O argument will enable optimizations in GNU ld. We can use -O2
here, as this matches our compile flags. Note that this would also
enable additional optimizations if using the lld or gold linkers,
when compared to -O0.
2020-04-12 18:36:56 +08:00
fanquake ed3b8eada8
Merge #17595: guix: Enable building for x86_64-w64-mingw32 target
a35e323589 guix: Appease travis. (Carl Dong)
0b66d22da5 guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong)
ba0b99bdd6 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong)
93439a71ed guix: Bump to upstream commit with mingw-w64 changes (Carl Dong)
35a96792dd guix: Check mingw symbols, improve SSP fix docs (Carl Dong)
449d8fe25b guix: Expand on INT trap message (Carl Dong)
3f1f03c67a guix: Spelling fixes (Carl Dong)
ff821dd2a1 guix: Reinstate make-ssp-fixed-gcc (Carl Dong)
360a9e0ad5 guix: Bump time-machine for mingw-w64 patches (Carl Dong)
93e41b7e3b guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong)
ef4f7e4c45 guix: Set the well-known timezone env var (Carl Dong)
acf4b3b3b5 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong)
c4cce00eac guix: Remove dead links from README. (Carl Dong)
df953a4c9a guix: Appease shellcheck. (Carl Dong)
91897c95e1 guix: Improve guix-build.sh documentation (Carl Dong)
570d769c6c guix: Build support for Windows (Carl Dong)

Pull request description:

  ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~
  Based on: #17933 (Time Machines are... shall we say... superior 😁)

  This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon)

ACKs for top commit:
  fanquake:
    ACK a35e323589 2/3

Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 18:34:18 +08:00
Andrew Chow e44aeefaae gitian: Add missing automake package to gitian-win-signer.yml
automake is needed to build osslsigncode otherwise autogen.sh fails.
2020-04-11 14:15:05 -04:00
Wladimir J. van der Laan dabe2bb11a build: Bump gitian descriptors to 0.21
Per the release process.
2020-04-10 19:46:39 +02:00
fanquake d486991aa5
Merge #18295: scripts: add MACHO lazy bindings check to security-check.py
5ca90f8b59 scripts: add MACHO lazy bindings check to security-check.py (fanquake)

Pull request description:

  This is a slightly belated follow up to #17686 and some discussion with Cory. It's not entirely clear if we should make this change due to the way the macOS dynamic loader appears to work. However I'm opening this for some discussion. Also related to #17768.

  #### Issue:
  [`LD64`](https://opensource.apple.com/source/ld64/) doesn't set the [MH_BINDATLOAD](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) bit in the header of MACHO executables, when building with `-bind_at_load`. This is in contradiction to the [documentation](https://opensource.apple.com/source/ld64/ld64-450.3/doc/man/man1/ld.1.auto.html):
  ```bash
  -bind_at_load
       Sets a bit in the mach header of the resulting binary which tells dyld to
       bind all symbols when the binary is loaded, rather than lazily.
  ```

  The [`ld` in Apples cctools](https://opensource.apple.com/source/cctools/cctools-927.0.2/ld/layout.c.auto.html) does set the bit, however the [cctools-port](https://github.com/tpoechtrager/cctools-port/) that we use for release builds, bundles `LD64`.

  However; even if the linker hasn't set that bit, the dynamic loader ([`dyld`](https://opensource.apple.com/source/dyld/)) doesn't seem to ever check for it, and from what I understand, it looks at a different part of the header when determining whether to lazily load symbols.

  Note that our release binaries are currently working as expected, and no lazy loading occurs.

  #### Example:

  Using a small program, we can observe the behaviour of the dynamic loader.

  Conducted using:
  ```bash
  clang++ --version
  Apple clang version 11.0.0 (clang-1100.0.33.17)
  Target: x86_64-apple-darwin18.7.0

  ld -v
  @(#)PROGRAM:ld  PROJECT:ld64-530
  BUILD 18:57:17 Dec 13 2019
  LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23)
  TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)
  ```

  ```cpp
  #include <iostream>
  int main() {
  	std::cout << "Hello World!\n";
  	return 0;
  }
  ```

  Compile and check the MACHO header:
  ```bash
  clang++ test.cpp -o test
  otool -vh test
  ...
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE

  # Run and dump dynamic loader bindings:
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=no_bind.txt ./test
  Hello World!
  ```

  Recompile with `-bind_at_load`. Note still no `BINDATLOAD` flag:
  ```bash
  clang++ test.cpp -o test -Wl,-bind_at_load
  otool -vh test
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
  ...
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=bind.txt ./test
  Hello World!
  ```

  If we diff the outputs, you can see that `dyld` doesn't perform any lazy bindings when the binary is compiled with `-bind_at_load`, even if the `BINDATLOAD` flag is not set:
  ```diff
  @@ -1,11 +1,27 @@
  +dyld: bind: test:0x103EDF030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x103EDF030 = 0x7FFF70C9FA58
  +dyld: bind: test:0x103EDF038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x103EDF038 = 0x7FFF70CA12C2
  +dyld: bind: test:0x103EDF068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x103EDF068 = 0x7FFF70CA12B6
  +dyld: bind: test:0x103EDF070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x103EDF070 = 0x7FFF70CA1528
  +dyld: bind: test:0x103EDF080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x103EDF080 = 0x7FFF70C9FAE6
  <trim>
  -dyld: lazy bind: test:0x10D4AC0C8 = libsystem_platform.dylib:_strlen, *0x10D4AC0C8 = 0x7FFF73C5C6E0
  -dyld: lazy bind: test:0x10D4AC068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x10D4AC068 = 0x7FFF70CA12B6
  -dyld: lazy bind: test:0x10D4AC038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x10D4AC038 = 0x7FFF70CA12C2
  -dyld: lazy bind: test:0x10D4AC030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x10D4AC030 = 0x7FFF70C9FA58
  -dyld: lazy bind: test:0x10D4AC080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x10D4AC080 = 0x7FFF70C9FAE6
  -dyld: lazy bind: test:0x10D4AC070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x10D4AC070 = 0x7FFF70CA1528
  ```

  Note: `dyld` also has a `DYLD_BIND_AT_LAUNCH=1` environment variable, that when set, will force any lazy bindings to be non-lazy:
  ```bash
  dyld: forced lazy bind: test:0x10BEC8068 = libc++.1.dylib:__ZNSt3__113basic_ostream
  ```

  #### Thoughts:
  After looking at the dyld source, I can't find any checks for `MH_BINDATLOAD`. You can see the flags it does check for, such as MH_PIE or MH_BIND_TO_WEAK [here](https://opensource.apple.com/source/dyld/dyld-732.8/src/ImageLoaderMachO.cpp.auto.html).

  It seems that the lazy binding of any symbols depends on whether or not [lazy_bind_size](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) from the `LC_DYLD_INFO_ONLY` load command is > 0. Which was mentioned in [#17686](https://github.com/bitcoin/bitcoin/pull/17686#issue-350216254).

  #### Changes:
  This PR is one of [Corys commits](7b6ba26178), that I've rebased and modified to make build. I've also included an addition to the `security-check.py` script to check for the flag.

  However, given the above, I'm not entirely sure this patch is the correct approach. If the linker no-longer inserts it, and the dynamic loader doesn't look for it, there might be little benefit to setting it. Or, maybe this is an oversight from Apple and needs some upstream discussion. Looking for some thoughts / Concept ACK/NACK.

  One alternate approach we could take is to drop the patch and modify security-check.py to look for `lazy_bind_size` == 0 in the `LC_DYLD_INFO_ONLY` load command, using `otool -l`.

ACKs for top commit:
  theuni:
    ACK 5ca90f8b59

Tree-SHA512: 444022ea9d19ed74dd06dc2ab3857a9c23fbc2f6475364e8552d761b712d684b3a7114d144f20de42328d1a99403b48667ba96885121392affb2e05b834b6e1c
2020-04-10 07:42:20 +08:00
Carl Dong a35e323589
guix: Appease travis. 2020-04-07 19:27:31 -04:00
Carl Dong 0b66d22da5
guix: Use gcc-9 for mingw-w64 instead of 8
The libtool unsorted 'find' determinism issue seemed to have been solved
in gcc-9's git: d41cd173e23ebea7c758644d6ad6e0fde1c2e3a6 or SVN: r262451

Furthermore, it seems that Ubuntu Focal 20.04 LTS is going to ship with
gcc 9 and mingw-w64 7, which will match what we have now.

-----

A note on this:

Careful observers will see that previously I stated that all released
versions of gcc were bootstrapped with a libtool 2.2.7a, meaning that
they all had the unsorted 'find' determinism issue first resolved in
libtool 2.2.7b.

However, I was mistaken, gcc's ltmain.sh CLAIMS it was generated by
libtool 2.2.7a, but it was in fact edited manually. It seems that gcc
maintains their own versions of ltmain.sh and libtool.m4, and only
sometimes backports patches from upstream.

Quite confusing.
2020-04-07 19:01:26 -04:00
Carl Dong ba0b99bdd6
guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends
This is no longer needed after 3bef7c22 in the mingw-w64 git repository,
which is first included in mingw-w64 v7.0.0.

As of the previous bump to our Guix time machine, we now use mingw-w64
v7.0.0.
2020-04-07 19:00:50 -04:00
Carl Dong 93439a71ed
guix: Bump to upstream commit with mingw-w64 changes
Most of the mingw-w64 toolchain changes have now been upstreamed, we can
point to a commit that exists upstream.

NOTE: I'm not changing the URL yet until we see that Guix upstream will
accept all my patches for macOS.

-----

The Guix tree that's referred to by this commit contains the following
changes relevant to our mingw-w64 build:

b066c25026

  Adds a PACKAGES-WITH-*PATCHES procedure which we can use in the future
  to apply patches to packages if those patches are not considered
  appropriate to upstream Guix

4719b71572

  Adds mingw-w64 (the libc itself) reproducibility patches, taken from
  debian.

79825bee07 + 401d28e433 + c1c50cb5b0

  Add mingw-w64 specific binutils patches, taken from debian.
  Specifically, the "Make DLL import libraries reproducible" patch made
  libbitcoinconsensus.dll.a build reproducibly. The followup commits
  were hotfixes for my mistakes.

0f864175dc

  Bumps mingw-w64 to v7.0.0. This is the first release that enables
  secure APIs by default (which we need), and gains _FORTIFY_SOURCE
  support. This will also be what Ubuntu Focal 20.04 LTS releases with.

cdf00cf75d

  Bumps NSIS to v3.05. This is the first release that includes a fix for
  a reproducibility bug found by some of the electrum developers. See
  details here: https://sourceforge.net/p/nsis/bugs/1230/
2020-04-07 19:00:49 -04:00
Wladimir J. van der Laan adac12ae73
Merge #18506: net: Hardcoded seeds update for 0.20
0eeb0468e7 net: Hardcoded seeds update for 0.20 (Wladimir J. van der Laan)

Pull request description:

  Update hardcoded seeds from http://bitcoin.sipa.be/seeds.txt.gz,
  according to release process.

  Output from makeseeds.py:
  ```
    IPv4   IPv6  Onion Pass
  1364173 244127   2454 Initial
  1364173 244127   2454 Skip entries with invalid address
  1129552 213117   2345 After removing duplicates
  1129548 213117   2345 Skip entries from suspicious hosts
  338216 191944   2249 Enforce minimal number of blocks
  336851 188993   2189 Require service bit 1
    6998   1520    150 Require minimum uptime
    5682   1290     89 Require a known and recent user agent
    5622   1279     89 Filter out hosts with multiple bitcoin ports
     512    146     89 Look up ASNs and limit results per ASN and per net
  ```

Top commit has no ACKs.

Tree-SHA512: ce1c2cda18dd5bd22586a5283a0877f3bd890437cc29dc1d85452ba4a4d28032f591c8b37f3329e8e649556cf83750b6949a068fad76d1773853d93014609da0
2020-04-06 13:38:32 +02:00
fanquake 5ca90f8b59
scripts: add MACHO lazy bindings check to security-check.py 2020-04-04 09:54:25 +08:00
Wladimir J. van der Laan 0eeb0468e7 net: Hardcoded seeds update for 0.20
Update hardcoded seeds from seeds_emzy.txt seeds_lukejr.txt
seeds_sipa.txt seeds_sjors.txt, according to release process.

Output from makeseeds.py:
```
  IPv4   IPv6  Onion Pass
1364173 244127   2454 Initial
1364173 244127   2454 Skip entries with invalid address
1129552 213117   2345 After removing duplicates
1129548 213117   2345 Skip entries from suspicious hosts
338216 191944   2249 Enforce minimal number of blocks
336851 188993   2189 Require service bit 1
  6998   1520    150 Require minimum uptime
  5682   1290     89 Require a known and recent user agent
  5622   1279     89 Filter out hosts with multiple bitcoin ports
   512    146     89 Look up ASNs and limit results per ASN and per net
```
2020-04-03 16:29:26 +02:00
fanquake 6ec42df32b
Merge #18426: scripts: previous_release: improve behaviour on failed download
332f373a9d [scripts] previous_release: improve failed download error message (Sebastian Falbesoner)

Pull request description:

  Currently, if the earlier release build/fetch script `previous_release.sh` is invoked with the option `-b` (intending to fetch a binary package from `https://bitcoin.org`) and the download fails, the user sees the following confusing output:
  ```
  $ contrib/devtools/previous_release.sh -r -b v0.9.5
  [...]
  gzip: stdin: not in gzip format
  tar: Child returned status 1
  tar: Error is not recoverable: exiting now
  ```
  This implies that the download worked, but the archive is corrupted, when in reality the HTML document containing the delivery fail reason (most likely 404 Not Found) is saved and tried to get unpacked. In contrast to wget, curl is a bit stubborn and needs explicit instructions to react to server errors via the flag `-f` (outputs error message and returns error code, ideal for scripts): https://curl.haxx.se/docs/manpage.html#-f

  On the PR branch, the output on failed download looks now the following:
  ```
  $ contrib/devtools/previous_release.sh -r -b v0.9.5
  [...]
  curl: (22) The requested URL returned error: 404 Not Found
  Download failed.
  ```

ACKs for top commit:
  fanquake:
    ACK 332f373a9d

Tree-SHA512: 046c931ad9e78aeb2d13faa4866d46122ed325aa142483547c2b04032d03223ed2411783b00106fcab0cd91b2f78691531ac526ed7bb3ed7547b6e2adbfb2e93
2020-04-03 18:10:28 +08:00
Carl Dong 35a96792dd
guix: Check mingw symbols, improve SSP fix docs 2020-04-02 17:20:05 -04:00
Carl Dong 449d8fe25b
guix: Expand on INT trap message 2020-04-02 17:20:04 -04:00
Carl Dong 3f1f03c67a
guix: Spelling fixes 2020-04-02 17:20:03 -04:00
Carl Dong ff821dd2a1
guix: Reinstate make-ssp-fixed-gcc
Unfortunately, gcc is still not smart enough to detect whether or not
mingw-w64 provides ssp, so let's put it back just for mingw-w64.
2020-04-02 17:20:02 -04:00
Carl Dong 360a9e0ad5
guix: Bump time-machine for mingw-w64 patches
This bump will includes a couple of commits which improve the
reproducibility of the mingw-w64 toolchain. Most of which came from
debian. They will be upstreamed as upstream Guix release timeline
allows.
2020-04-02 17:20:01 -04:00
Carl Dong 93e41b7e3b
guix: Use gcc-8 for mingw-w64 instead of 7
We're using mingw-w64 6.0.0, which is paired with gcc-8 in most distros.
2020-04-02 17:20:00 -04:00
Carl Dong ef4f7e4c45
guix: Set the well-known timezone env var 2020-04-02 17:19:59 -04:00
Carl Dong acf4b3b3b5
guix: Make x86_64-w64-mingw32 builds reproducible
- Add "--no-insert-timestamp" LDFLAG for x86_64-w64-mingw32 builds

"The option --no-insert-timestamp can be used to insert a zero value for
the timestamp, this ensuring that binaries produced from identical
sources will compare identically." - ld(1)

- Set "SetDateSave off" in NSIS script

From https://nsis.sourceforge.io/Docs/Chapter4.html#flags

"This command sets the file date/time saving flag which is used by the
File command to determine whether or not to save the last write date and
time of the file, so that it can be restored on installation. Valid
flags are 'on' and 'off'. 'on' is the default."

- Add commented out NSIS options for reproducibility debugging in NSIS
  script

- Make ZIPs deterministic by reseting file modification times to
  SOURCE_DATE_EPOCH using touch(1) (Reference:
  https://reproducible-builds.org/docs/archives/)
2020-04-02 17:19:57 -04:00
Carl Dong c4cce00eac
guix: Remove dead links from README. 2020-04-02 17:19:56 -04:00
Carl Dong df953a4c9a
guix: Appease shellcheck. 2020-04-02 17:19:55 -04:00
Carl Dong 91897c95e1
guix: Improve guix-build.sh documentation 2020-04-02 17:19:54 -04:00
Carl Dong 570d769c6c
guix: Build support for Windows 2020-04-02 17:19:53 -04:00
fanquake 7142d50ac3
scripts: rename test_64bit_PE to test_PE 2020-03-26 11:39:34 +08:00
fanquake edaca2dd12
scripts: add MACHO NX check to security-check.py 2020-03-26 11:39:34 +08:00
fanquake 1a4e9f32ef
scripts: add MACHO tests to test-security-check.py 2020-03-26 11:39:34 +08:00
Wladimir J. van der Laan 60a39a96fc
Merge #18425: releases: Update with new Windows code signing certificate
3e0df92bf2 Update with new Windows code signing certificate (Andrew Chow)

Pull request description:

  The current Windows code signing certificate is about expire (on March 26th 2020). As I have volunteered to take over the Windows code signing duties, I've purchased a new Windows code signing certificate with the same CA and under the same organization (Bitcoin Core Code Signing Association).

  A signature by the old certificate over the new certificate has been provided to me. This signature can be verified using
  ```
  openssl cms -verify -inform pem -purpose any -content path/to/new/win-codesign.cert -CAfile path/to/old/win-codesign.cert -certfile path/to/old/win-codesign.cert
  ```
  The verification should succeed and the new certificate will be printed out. This can be compared to the contents of `win-codesign.cert`.

  ```
  -----BEGIN PKCS7-----
  MIIC3AYJKoZIhvcNAQcCoIICzTCCAskCAQExDzANBglghkgBZQMEAgEFADALBgkq
  hkiG9w0BBwExggKkMIICoAIBATCBkTB8MQswCQYDVQQGEwJHQjEbMBkGA1UECBMS
  R3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9T
  ZWN0aWdvIExpbWl0ZWQxJDAiBgNVBAMTG1NlY3RpZ28gUlNBIENvZGUgU2lnbmlu
  ZyBDQQIRALWcUnSOxv9FQW3xdaMDO6swDQYJYIZIAWUDBAIBBQCggeQwGAYJKoZI
  hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAwMzI0MjA0ODM3
  WjAvBgkqhkiG9w0BCQQxIgQgtLkmnuSQyczDlJSnJeqbi61p3iJ/rpFABrY8JWBO
  o74weQYJKoZIhvcNAQkPMWwwajALBglghkgBZQMEASowCwYJYIZIAWUDBAEWMAsG
  CWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcN
  AwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZIhvcNAQEBBQAEggEA
  XaCl3Q8HwI9VpLCb9OY9eQh0QOPyl1KWEc3TP3UvwZwR4/gXkfPOKKf19UnS8eRB
  48SgUKRMYWoDYfSVUJRMda9BLkbJbQlHG3LFXhSY2alajpPXEHcMto/XPhVAmqzL
  w6aSNY0Gaorow696JHpetpKqAAlL1r2GjeaPYi2aZyIAifuhay/qwA+ig0SqzGOw
  UdgFZWMyS5yanq8/WlLCCql6kKOzT4tEqUaleD7R1q8BTcG2+fmhWR8WwJLpIV6y
  7GAqt0Cocu8sYpTNBNk8iKHxzZ2hMZKJpH9lHZuiJ/9vSercrvDy2R4/MG+KnBWb
  OyiFAt2mC51+63RhLOMJfg==
  -----END PKCS7-----
  ```

ACKs for top commit:
  laanwj:
    ACK 3e0df92bf2
  theuni:
    ACK 3e0df92bf2.

Tree-SHA512: 4210f4db1e805ab11231fbae49ea197257c6f7e44f1f6219685b63831704984d824ac2f9e0a3b1bd2655953af72636a474f077cb859fb35852551f5a9f8fbde3
2020-03-25 17:04:53 +01:00
Wladimir J. van der Laan 3e50fdbe4e
Merge #18395: scripts: add PE dylib checking to symbol-check.py
1a0993ae35 scripts: add PE dylib checking to symbol-check.py (fanquake)

Pull request description:

  Uses `objdump -x` and looks for `DLL Name:` lines. i.e:
  ```bash
  objdump -x src/qt/bitcoin-qt.exe | grep "DLL Name:"
  	DLL Name: ADVAPI32.dll
  	DLL Name: dwmapi.dll
  	DLL Name: GDI32.dll
  	DLL Name: IMM32.dll
  	DLL Name: IPHLPAPI.DLL
  	DLL Name: KERNEL32.dll
  	DLL Name: msvcrt.dll
  	DLL Name: ole32.dll
  	DLL Name: OLEAUT32.dll
  	DLL Name: SHELL32.dll
  	DLL Name: SHLWAPI.dll
  	DLL Name: USER32.dll
  	DLL Name: UxTheme.dll
  	DLL Name: VERSION.dll
  	DLL Name: WINMM.dll
  	DLL Name: WS2_32.dll
  ```

ACKs for top commit:
  dongcarl:
    Concept ACK 1a0993ae35
  hebasto:
    ACK 1a0993ae35, tested on Linux Mint 19.3:

Tree-SHA512: 0099a50e2c616d5239a15cafa9a7c483e9c40244af41549e4738be0f5360f27a2afb956eb50b47cf446b242f4cfc6dc9d111306a056fb83789eefbd71eddabd2
2020-03-25 15:35:36 +01:00
Wladimir J. van der Laan baa72cd9a2
Merge #18331: build: Use git archive as source tarball
e4d366788b build: Drop needless EXTRA_DIST content (Hennadii Stepanov)
6c4da59f5b build: Drop SOURCEDIST reordering (Hennadii Stepanov)
5e6b8b3912 build: Use git archive as source tarball (Hennadii Stepanov)

Pull request description:

  This PR:
  - is an alternative to #17104
  - closes #16734
  - closes #6753

  The idea is clear described by some developers:
  - [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540691850):
  > This whole concept of explicitly listing each and every file manually (or with a fragile wildcard) is an obvious sisyphean task. I'd say all we need to do is run git archive and be done with it forever, see #16734, #6753, #11530 ...

  - [laanwj](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540706025):
  > I agree, I've never been a fan of it. I don't think we have any files in the git repository we don't want to ship in the source tarball.

  ---

  The suggested changes have a downside which is pointed by [**luke-jr**](https://github.com/bitcoin/bitcoin/pull/17104#issuecomment-540828045):
  > ... but the distfile needs to include autogen-generated files.

  This means that a user is not able to run `./configure && make` right away. One must run `./autogen.sh` at first.

  Here are opinions about mandatory use of `./autogen.sh`:
  - [ryanofsky](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-534139356):
  > It's probably ok to require autogen. I think historically configure scripts were supposed to work on obscure unix systems that would just have a generic shell + make tool + c compiler, and not necessarily need gnu packages like m4 which are needed for autogen.

  - [laanwj](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-540729483):
  > I also think it's fine to require autogen. What is one dependency more, if you're building from source.

  ---

  ~Also this PR provides Windows users with ZIP archives of the sources. Additionally the commit ID is stored in these ZIP files as a file comment:~

  ---

  Note for reviewers: please verify is `git archive` output deterministic?

ACKs for top commit:
  MarcoFalke:
    re-ACK e4d366788b, only change is adding two dots in a the path 🛳
  laanwj:
    ACK e4d366788b

Tree-SHA512: d1153d3ca4a580696019b92be3555ab004d197d9a2146aacff9d3150eb7093b7d40eebd6eea12d861d93ff62d62b68706e04e64dbe5ea796ff6757486e462193
2020-03-25 14:48:44 +01:00
Sebastian Falbesoner 332f373a9d [scripts] previous_release: improve failed download error message
before:
------------------------------------------------------------
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
------------------------------------------------------------

now:
------------------------------------------------------------
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
curl: (22) The requested URL returned error: 404 Not Found
Download failed.
------------------------------------------------------------
2020-03-25 09:06:48 +01:00
Andrew Chow 3e0df92bf2 Update with new Windows code signing certificate 2020-03-24 12:22:46 -04:00
fanquake 1a0993ae35
scripts: add PE dylib checking to symbol-check.py 2020-03-22 10:47:38 +08:00
Hennadii Stepanov e4d366788b
build: Drop needless EXTRA_DIST content
Some EXTRA_DIST content is needless since a git archive is used as the
source tarball.
2020-03-15 18:34:27 +02:00
Hennadii Stepanov 6c4da59f5b
build: Drop SOURCEDIST reordering
Making SOURCEDIST deterministic is needless since a git archive is used
as the source tarball.
2020-03-12 11:43:16 +02:00
Hennadii Stepanov 5e6b8b3912
build: Use git archive as source tarball 2020-03-12 11:34:43 +02:00
Carl Dong 0ae42a16c7
guix: Remove now-unnecessary gcc make flag
Previously, Guix would produce a gcc which did not know to use the SSP
function from glibc, and required a gcc make flag for it to do so, in my
attempt to fix it upstream I realized that this is no longer the case.

This can be verified by performing a Guix build and doing

  readelf -s ... | grep __stack_chk

to check that symbols are coming from glibc, and doing

  readelf -d ... | grep NEEDED | grep ssp

to see that libssp.so is not being depended on
2020-03-11 16:16:31 -04:00
fanquake 530d02addb
build: pass -fno-ident in Windows gitian descriptor
This prevents compilers from emitting compiler name and
version number info that can needlessly bloat binaries.

Accepted by Clang and GCC. See:

https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-qn

https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fno-ident
2020-02-13 18:20:43 +08:00
Sjors Provoost c7ca630896
[scripts] support release candidates of earlier releases 2020-02-11 21:46:22 +01:00