Commit graph

91 commits

Author SHA1 Message Date
Carl Dong
395c1137f6
gitian: Limit sourced script to just assignments
Previously, the sourced script would create the source tarball. Now, it
only assigns variables and the source-ing script has more flexibility in
determining what to do with these variables.

See later commit showing how this flexibility is useful in our Guix
builds.
2020-04-28 10:36:35 -04:00
fanquake
65fb3dfc8d
Merge #18556: build: Drop make dist in gitian builds
2aa48edec0 refactor: Drop unused ${WRAP_DIR}/${HOST} directory (Hennadii Stepanov)
1362be0447 build: Drop make dist in gitian builds (Hennadii Stepanov)

Pull request description:

  After the merge of #18331, the packaged source tarball is created by `git archive`, but the binaries are built from another one which is made by `make dist`.

  With this PR the only source tarball, created by `git archive`, is used both for binaries building and for packaging to users.

  Close #16588.
  Close #18547.

  As a good side-effect, #18349 becomes redundant.

  **Change in behavior**

  The following variables 1b151e3ffc/configure.ac (L2-L6)

  are no longer used for naming of directories and tarballs.

  Instead of them the gitian descriptors use a git tag (if available) or a commit hash.

  ---

  Also a small refactor commit picked from #18404.

ACKs for top commit:
  dongcarl:
    ACK 2aa48edec0
  MarcoFalke:
    ACK 2aa48edec0
  fanquake:
    ACK 2aa48edec0 - I've had a quick look over this, and don't want to block merging if this actually gets as closer to finally having this all sorted out. Obviously we've still got #18741, and after speaking to Carl this morning, there will likely be even more changes after that (not Guix specific).

Tree-SHA512: d3b16f87e48d1790a3264940c28acd5d881bfd10f3ce94fb0c8a6af76d8039289d01e0cd4972adac49ae24362857251f6c1e5e09e3e9fbf636c10708b4015a7c
2020-04-28 16:44:17 +08:00
Hennadii Stepanov
1362be0447
build: Drop make dist in gitian builds 2020-04-17 16:09:04 +03:00
fanquake
b8b050a8d6
build: add linker optimization flags to gitian descriptors
Any -O argument will enable optimizations in GNU ld. We can use -O2
here, as this matches our compile flags. Note that this would also
enable additional optimizations if using the lld or gold linkers,
when compared to -O0.
2020-04-12 18:36:56 +08:00
Wladimir J. van der Laan
dabe2bb11a build: Bump gitian descriptors to 0.21
Per the release process.
2020-04-10 19:46:39 +02:00
Hennadii Stepanov
e4d366788b
build: Drop needless EXTRA_DIST content
Some EXTRA_DIST content is needless since a git archive is used as the
source tarball.
2020-03-15 18:34:27 +02:00
Hennadii Stepanov
6c4da59f5b
build: Drop SOURCEDIST reordering
Making SOURCEDIST deterministic is needless since a git archive is used
as the source tarball.
2020-03-12 11:43:16 +02:00
Hennadii Stepanov
5e6b8b3912
build: Use git archive as source tarball 2020-03-12 11:34:43 +02:00
MarcoFalke
fae9084ac5
build: Skip i686 build by default in guix and gitian 2020-02-09 13:22:53 -08:00
MarcoFalke
0a8b68cdf7
Merge #17483: build: Set gitian arch back to amd64
fae75306ba scripted-diff: Set gitian arch back to amd64 (MarcoFalke)

Pull request description:

  This was required to allow gitian builds on non-amd64 architecture, however, it seems to break the current builds (with lxc), see https://github.com/bitcoin/bitcoin/pull/17409#issuecomment-554099626

  Also, the gititan builds wouldn't be deterministic across arches anyway, see #17468

  So instead of wasting more time on this, revert the change and hope that guix allows to compile on non-amd64 architectures.

Top commit has no ACKs.

Tree-SHA512: 801e9a30ae1b0882ef45d5eb3a3cf80f3ace3b99db046069dbd95b6162119e977e3cf3134287d1ac5d09483906206acc71e1ac34d6b74dbc533d46aaf73f5cc2
2020-01-22 09:50:26 -05:00
Wladimir J. van der Laan
1f59885d27
Merge #17361: script: Lint Gitian descriptors with ShellCheck
17f81e9648 script: Enable SC2001 rule for Gitian scripts (Hennadii Stepanov)
61bb21b418 script: Enable SC2155 rule for Gitian scripts (Hennadii Stepanov)
577682d9e8 script: Enable SC2006 rule for Gitian scripts (Hennadii Stepanov)
14aded46df script: Lint Gitian descriptors with ShellCheck (Hennadii Stepanov)

Pull request description:

  This PR extracts shell scripts from Gitian descriptors (`contrib/gitian-descriptors/`) and checks for ShellCheck warnings as any other one.

  Some non-controversial warnings are fixed.

ACKs for top commit:
  practicalswift:
    ACK 17f81e9648 -- diff looks correct

Tree-SHA512: bdfa3d35bbb65ff634c90835d75c3df63e958b558599771d21366724f5cf64da83a68957d926e926a99c3704b9529e96a17697dc8d9ff3adf7154d9cb1999a8d
2019-11-28 10:18:22 +01:00
Hennadii Stepanov
17f81e9648
script: Enable SC2001 rule for Gitian scripts 2019-11-27 19:27:56 +02:00
Hennadii Stepanov
61bb21b418
script: Enable SC2155 rule for Gitian scripts
Also pwd command is replaced with $PWD variable everywhere for
consistency.
2019-11-27 19:25:29 +02:00
Wladimir J. van der Laan
eafd259367 build: Add NX workaround for RV64
Work around https://bugs.launchpad.net/ubuntu/+source/gcc-8-cross-ports/+bug/1853740.
2019-11-24 13:40:11 +01:00
MarcoFalke
fae75306ba
scripted-diff: Set gitian arch back to amd64
-BEGIN VERIFY SCRIPT-
sed -i -e 's|"linux64"|"amd64"|g' $(git grep -l '"linux64"')
-END VERIFY SCRIPT-
2019-11-14 17:10:18 -05:00
MarcoFalke
333362991c
doc: Explain $LIB in LD_PRELOAD in gitian descriptors 2019-11-12 12:06:34 -05:00
MarcoFalke
fab9850ef4
scripted-diff: Avoid hardcoded libfaketime dir in gitian
-BEGIN VERIFY SCRIPT-
sed -i -e "s|'export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1'|\"export LD_PRELOAD='/usr/\\\\\$LIB/faketime/libfaketime.so.1'\"|g" $(git grep -l 'x86_64-linux-gnu/faketime')
sed -i -e 's|"amd64"|"linux64"|g' $(git grep -l '"amd64"')
-END VERIFY SCRIPT-
2019-11-07 22:35:51 -05:00
Hennadii Stepanov
577682d9e8
script: Enable SC2006 rule for Gitian scripts 2019-11-06 15:11:54 +02:00
Wladimir J. van der Laan
9b4dfec831
Merge #17029: gitian: Various improvements for Windows descriptor
9d1f971c75 gitian: Put things in the right place to begin with (Carl Dong)
71949a97a7 gitian: Eliminate rename dependency (Carl Dong)
999a9a5f5b gitian: Smaller diff with gitian-linux.yml (Carl Dong)
c4a3c25ba1 gitian: Fix README inclusion in archives (Carl Dong)
93cb974980 gitian: Use split-debug.sh for Win builds (Carl Dong)

Pull request description:

  It would seem that our `gitian-win.yml` has not been keeping up with `gitian-linux.yml`, this PR:

  1. Minimizes the diff size between `gitian-{win,linux}.yml`
  2. Eliminates the `rename` dependency

ACKs for top commit:
  laanwj:
    ACK 9d1f971c75

Tree-SHA512: 84ed47c685e12d0064c02811907ae3d0fd3c47db8773d497dcc38f0defbfb3040fd82899fb026cf355f229b906d05a1c8038a95642bb90d044afbc2e0b239af2
2019-10-09 09:38:00 +02:00
Carl Dong
c4a3c25ba1
gitian: Fix README inclusion in archives
Linux:
The README was originally added in 8550f1fb2, but included the README
under the docs directory, which has a bunch of internal links that won't
make sense in a release tarball. In this patch, we include the root
level README instead, which makes more sense.

Windows:
.md files are inconvenient to open on windows and the line endings
differ, so we use README_windows.txt instead.
2019-10-08 13:36:50 -04:00
MarcoFalke
fa1ad8f06e
build: Bump gitian descriptor versions 2019-10-04 09:17:57 -04:00
Carl Dong
993aa414d3
scripted-diff: gitian: Use REFERENCE_DATETIME directly.
Fixes regression introduced by #16141

-BEGIN VERIFY SCRIPT-
sed -i 's#\$REFERENCE_DATE\\\\\\ \$REFERENCE_TIME#\$REFERENCE_DATETIME#g' contrib/gitian-descriptors/*
-END VERIFY SCRIPT-
2019-06-10 15:14:05 -04:00
fanquake
bc8863b819
depends: remove usage of TAR_OPTIONS 2019-06-05 15:39:17 +02:00
fanquake
3ff1f2a319
build: remove export GZIP from gitian descriptors 2019-06-03 15:51:59 -04:00
Hennadii Stepanov
b8705a0915
Set 'distro' explicitly 2019-03-16 07:40:50 +02:00
Wladimir J. van der Laan
12408d33c6
Merge #15549: gitian: Improve error handling
32da92bdf6 gitian: Improve error handling (Wladimir J. van der Laan)

Pull request description:

  Improve error handling in gitian builds:

  - Set fail-on-error and pipefail flag, this causes a command to fail when either of the pipe stages fails, not only when the last of the stages fails, so this improves error detection.
  - Also use `xargs` instead of `find -exec`, because `find` will not propagate errors in the executed command, but `xargs` will.

  This will avoid some issues like #15541 where non-determinism is silently introduced due to errors caused by environment conditions (such as lack of disk space in that case).

Tree-SHA512: d5d3f22ce2d04a75e5c25e935744327c3adc704c2d303133f2918113573a564dff3d3243d5569a2b93ee7eb0e97f8e1b1ba81767e966af9015ea711a14091035
2019-03-09 06:45:19 +01:00
MarcoFalke
923d87497c
Merge #15528: contrib: Bump gitian descriptors for 0.19
fa58a2e335 contrib: Bump gitian descriptors for 0.19 (MarcoFalke)

Pull request description:

  Bump the cache directory (as in 263b3777e7)

Tree-SHA512: 2f9a4f8c14c3c6cef30bcdfed638486c6b957068b1e2380cbde1719c9bf8b53bcaff35833272a879af5b031e3ea0a32cc16f78cc7e4d7399037de89132311c5b
2019-03-08 09:12:03 -05:00
Wladimir J. van der Laan
32da92bdf6 gitian: Improve error handling 2019-03-07 17:21:46 +01:00
MarcoFalke
fa58a2e335
contrib: Bump gitian descriptors for 0.19 2019-03-04 14:44:23 -05:00
MarcoFalke
fab5a1e0f4
build: Require python 3.5 2019-03-02 10:40:23 -05:00
Wladimir J. van der Laan
0c9bea5a00
Merge #14037: Add README.md to linux release tarballs
8550f1fb28 Add README.md to linux release tarballs (Hennadii Stepanov)

Pull request description:

  fix #8160

  Gitian building report for 0.17.0rc2:
  ```
  7d89d7dc3488915ec2380253a69fb3b8f8065592e24c5b2a99a91da30f2142cc  bitcoin-0.17.0-aarch64-linux-gnu-debug.tar.gz
  fcb292fd2c4fca88e5cc5a97ee7fa3390d3c7221aada166fe7822d64a2ee9dfa  bitcoin-0.17.0-aarch64-linux-gnu.tar.gz
  0ec6f979a823a6b6084d2e80605dffd3ccdda359e8459cebec25092c1087348f  bitcoin-0.17.0-arm-linux-gnueabihf-debug.tar.gz
  45af8757a2315125afe2f4d4f276d9b9cf616b8ab814284ce2f82b9a345971d8  bitcoin-0.17.0-arm-linux-gnueabihf.tar.gz
  b37b6d9bda864af968dfab6eebb245e75ecc56eb18b139b946270933381ea288  bitcoin-0.17.0-i686-pc-linux-gnu-debug.tar.gz
  20c96a5509eeb3e8ec505f18914ef9231beef1fec5e9e1c4b33ec6c6b613d146  bitcoin-0.17.0-i686-pc-linux-gnu.tar.gz
  d505888594a04dab2b34ccd6863b8f25eb97d9cb76650e39d93f4d6c09d4c55a  bitcoin-0.17.0-x86_64-linux-gnu-debug.tar.gz
  f55b16716c3295e309c816e170911380a5a26e9be3a336b213f2f412f0b159b3  bitcoin-0.17.0-x86_64-linux-gnu.tar.gz
  01c6b5ce15b9f3fcdcce96baae14eb04ab2605f2294d333e96b66e004594eea6  src/bitcoin-0.17.0.tar.gz
  ```
  Release tarball content:
  ```
  $ tar -tf bitcoin-binaries/0.17.0rc2/bitcoin-0.17.0-x86_64-linux-gnu.tar.gz
  bitcoin-0.17.0/
  bitcoin-0.17.0/bin/
  bitcoin-0.17.0/bin/bitcoin-cli
  bitcoin-0.17.0/bin/bitcoind
  bitcoin-0.17.0/bin/bitcoin-qt
  bitcoin-0.17.0/bin/bitcoin-tx
  bitcoin-0.17.0/bin/test_bitcoin
  bitcoin-0.17.0/include/
  bitcoin-0.17.0/include/bitcoinconsensus.h
  bitcoin-0.17.0/lib/
  bitcoin-0.17.0/lib/libbitcoinconsensus.so
  bitcoin-0.17.0/lib/libbitcoinconsensus.so.0
  bitcoin-0.17.0/lib/libbitcoinconsensus.so.0.0.0
  bitcoin-0.17.0/README.md
  bitcoin-0.17.0/share/
  bitcoin-0.17.0/share/man/
  bitcoin-0.17.0/share/man/man1/
  bitcoin-0.17.0/share/man/man1/bitcoin-cli.1
  bitcoin-0.17.0/share/man/man1/bitcoind.1
  bitcoin-0.17.0/share/man/man1/bitcoin-qt.1
  bitcoin-0.17.0/share/man/man1/bitcoin-tx.1
  ```

Tree-SHA512: 2a0c069d6533502a95a83eaba57b9828bddd03ab4a4fc47027b0068c9f04837f107abc448d82c929aa1f45441d2459cf6f2ad74b97a4d953f66dc81031bd521a
2018-08-31 13:05:15 +02:00
Hennadii Stepanov
8550f1fb28
Add README.md to linux release tarballs
fix #8160
2018-08-28 00:33:37 +03:00
Chun Kuan Lee
c516c3a770 [contrib] Support ARM and RISC-V symbol check 2018-08-25 21:08:16 +08:00
Wladimir J. van der Laan
3c8d1ae153
Merge #13665: [build] Add risc-v support to gitian
c4aecd1d80 Add risc-v 64-bit to gitian (Chun Kuan Lee)
96dda8b058 [depends] Add riscv qt depends support for cross compiling bitcoin-qt (Chun Kuan Lee)

Pull request description:

  Based on ~#13660~ #13710 ,  add gitian tarball for RISC-V

Tree-SHA512: 8db73545a2ea7fe03fa156598479335ea3c79aa3fb9c5cc44b8563094b1deb7c94d29c1dab47fac129dbfa2e3e774301b526474beeeb59c9b0087d3ea087dbd6
2018-08-16 20:33:50 +02:00
fanquake
263b3777e7
gitian: bump descriptors for (0.)18 2018-08-13 21:44:22 +08:00
Chun Kuan Lee
c4aecd1d80 Add risc-v 64-bit to gitian 2018-08-09 03:35:51 +08:00
Fuzzbawls
6b5506a286
Fix Qt's rcc determinism for depends/gitian
Backport of https://bugreports.qt.io/browse/QTBUG-62511 to resolve
locale determinism during the build process.
2018-07-25 13:53:53 -07:00
Chun Kuan Lee
c1afe3232f Change gitian-descriptors to use bionic instead 2018-07-16 10:42:32 +00:00
DesWurstes
000000035b Obsolete #!/bin/bash shebang 2018-06-20 11:12:41 +03:00
Cory Fields
d7f438a026 gitian: bump descriptors for (0.)17 2018-02-15 17:29:39 -05:00
Cory Fields
4452829b10 gitian: quick hack to fix version string in releases
Release version strings were broken in Gitian by 7522. This is a minimal fix
suitable for 0.15.

After this, we should fix up version handling for good so that gitian packages
the correct string in the release tarball, so that git is not required to get
the tag name.
2017-08-20 00:31:05 -04:00
Wladimir J. van der Laan
140de14a12
gitian: bump descriptors for master
Tree-SHA512: dc56ab285ea3fd293794341d7e2f8452730d3efb59f793112d4e1b036a051f9d221a7e577a460b426ecfb1578558203fa6a432efc62e6cabc534059719a2b437
2017-08-14 17:38:10 +02:00
Wladimir J. van der Laan
d25449f858
Merge #7522: Bugfix: Only use git for build info if the repository is actually the right one
ed1fcdc Bugfix: Detect genbuild.sh in repo correctly (Luke Dashjr)
e98e3dd Bugfix: Only use git for build info if the repository is actually the right one (Luke Dashjr)

Tree-SHA512: 510d7ec8cfeff4e8e0c7ac53631eb32c7acaada7017e7f8cc2e6f60d86afe1cd131870582e01022f961c85a783a130bcb8fef971f8b110070c9c02afda020726
2017-05-17 11:07:01 +02:00
Cory Fields
8e5cca027f gitian: bump descriptors for master
This was skipped for the 0.14 cycle.
2017-02-17 16:11:43 -05:00
Luke Dashjr
df634908ba Merge tag 'branch-0.13' into bugfix_gitdir 2016-08-09 05:45:50 +00:00
Cory Fields
099d4b0b65 gitian: use a wrapped gcc/g++ to avoid the need for a system change
C_INCLUDE_PATH and CPLUS_INCLUDE_PATH work globally as though -isystem was used
for each invocation.

Since that changes the build results, force a rebuild of x86 depends by adding
the value to $HOST_ID_SALT.
2016-07-07 20:13:08 -04:00
Wladimir J. van der Laan
cca1c8cff0
Merge #8194: [gitian] set correct PATH for wrappers
fa61756 [gitian] set correct PATH for wrappers (MarcoFalke)
2016-06-14 09:13:44 +02:00
MarcoFalke
fa61756842 [gitian] set correct PATH for wrappers 2016-06-12 14:22:07 +02:00
Cory Fields
9d25362087 build: add armhf/aarch64 gitian builds
- create a script to handle split debug. This will also eventually need to check
  targets, and use dsymutil for osx.
- update config.guess/config.sub for bdb for aarch64.
- temporarily disable symbol checks for arm/aarch64
- quit renaming to linux32/linux64 and use the host directly

This also adds a hack to work around an Ubuntu bug in the gcc-multilib package:
https://bugs.launchpad.net/ubuntu/+source/gcc-defaults-armhf-cross/+bug/1347820

The problem is that gcc-multilib conflicts with the aarch toolchain.
gcc-multilib installs a symlink that points
/usr/include/asm -> /usr/include/x86_64-linux-gnu/asm.

Without this link, gcc -m32 can't find asm/errno.h (and others), since
/usr/include/x86_64-linux-gnu isn't in its default include path. But
/usr/include/i386-linux-gnu is (though it doesn't exist on disk).

So work around the problem by linking
/usr/include/i386-linux-gnu/asm -> /usr/include/x86_64-linux-gnu/asm.

The symlink fix is actually quite reasonable, but echoing the password into
sudo is nasty, and should probably be addressed in gitian itself. It makes more
sense to enable passwordless sudo for the build user by default.
2016-06-10 05:34:50 -04:00
Wladimir J. van der Laan
fd9881ae67
Merge #7283: [gitian] Default reference_datetime to commit author date
fa42a67 [gitian] hardcode datetime for depends (MarcoFalke)
fa58c76 [gitian] Default reference_datetime to commit author date (MarcoFalke)
2016-06-09 11:14:20 +02:00