* Update build_chromium README * more edits * Update init.py # Conflicts: # x-pack/build_chromium/README.md
This commit is contained in:
parent
80aae02a90
commit
600aff96a1
|
@ -6,49 +6,65 @@ to accept a commit hash from the Chromium repository, and initialize the build
|
|||
environments and run the build on Mac, Windows, and Linux.
|
||||
|
||||
## Before you begin
|
||||
|
||||
If you wish to use a remote VM to build, you'll need access to our GCP account,
|
||||
which is where we have two machines provisioned for the Linux and Windows
|
||||
builds. Mac builds can be achieved locally, and are a great place to start to
|
||||
gain familiarity.
|
||||
|
||||
**NOTE:** Linux builds should be done in Ubuntu on x86 architecture. ARM builds
|
||||
are created in x86. CentOS is not supported for building Chromium.
|
||||
|
||||
1. Login to our GCP instance [here using your okta credentials](https://console.cloud.google.com/).
|
||||
2. Click the "Compute Engine" tab.
|
||||
3. Ensure that `chromium-build-linux` and `chromium-build-windows-12-beefy` are there.
|
||||
4. If #3 fails, you'll have to spin up new instances. Generally, these need `n1-standard-8` types or 8 vCPUs/30 GB memory.
|
||||
5. Ensure that there's enough room left on the disk. `ncdu` is a good linux util to verify what's claming space.
|
||||
3. Find `chromium-build-linux` or `chromium-build-windows-12-beefy` and start the instance.
|
||||
4. Install [Google Cloud SDK](https://cloud.google.com/sdk) locally to ssh into the GCP instance
|
||||
5. System dependencies:
|
||||
- 8 CPU
|
||||
- 30GB memory
|
||||
- 80GB free space on disk (Try `ncdu /home` to see where space is used.)
|
||||
- git
|
||||
- python2 (`python` must link to `python2`)
|
||||
- lsb_release
|
||||
- tmux is recommended in case your ssh session is interrupted
|
||||
6. Copy the entire `build_chromium` directory into a GCP storage bucket, so you can copy the scripts into the instance and run them.
|
||||
|
||||
## Usage
|
||||
## Build Script Usage
|
||||
|
||||
```
|
||||
# Allow our scripts to use depot_tools commands
|
||||
export PATH=$HOME/chromium/depot_tools:$PATH
|
||||
|
||||
# Create a dedicated working directory for this directory of Python scripts.
|
||||
mkdir ~/chromium && cd ~/chromium
|
||||
|
||||
# Copy the scripts from the Kibana repo to use them conveniently in the working directory
|
||||
cp -r ~/path/to/kibana/x-pack/build_chromium .
|
||||
# Install the OS packages, configure the environment, download the chromium source
|
||||
gsutil cp -r gs://my-bucket/build_chromium .
|
||||
|
||||
# Install the OS packages, configure the environment, download the chromium source (25GB)
|
||||
python ./build_chromium/init.sh [arch_name]
|
||||
|
||||
# Run the build script with the path to the chromium src directory, the git commit id
|
||||
python ./build_chromium/build.py <commit_id>
|
||||
python ./build_chromium/build.py <commit_id> x86
|
||||
|
||||
# You can add an architecture flag for ARM
|
||||
# OR You can build for ARM
|
||||
python ./build_chromium/build.py <commit_id> arm64
|
||||
```
|
||||
|
||||
**NOTE:** The `init.py` script updates git config to make it more possible for
|
||||
the Chromium repo to be cloned successfully. If checking out the Chromium fails
|
||||
with "early EOF" errors, the instance could be low on memory or disk space.
|
||||
|
||||
## Getting the Commit ID
|
||||
Getting `<commit_id>` can be tricky. The best technique seems to be:
|
||||
The `build.py` script requires a commit ID of the Chromium repo. Getting `<commit_id>` can be tricky. The best technique seems to be:
|
||||
1. Create a temporary working directory and intialize yarn
|
||||
2. `yarn add puppeteer # install latest puppeter`
|
||||
3. Look through puppeteer's node module files to find the "chromium revision" (a custom versioning convention for Chromium).
|
||||
3. Look through Puppeteer documentation and Changelogs to find information
|
||||
about where the "chromium revision" is located in the Puppeteer code. The code
|
||||
containing it might not be distributed in the node module.
|
||||
- Example: https://github.com/puppeteer/puppeteer/blob/b549256/src/revisions.ts
|
||||
4. Use `https://crrev.com` and look up the revision and find the git commit info.
|
||||
|
||||
The official Chromium build process is poorly documented, and seems to have
|
||||
breaking changes fairly regularly. The build pre-requisites, and the build
|
||||
flags change over time, so it is likely that the scripts in this directory will
|
||||
be out of date by the time we have to do another Chromium build.
|
||||
|
||||
This document is an attempt to note all of the gotchas we've come across while
|
||||
building, so that the next time we have to tinker here, we'll have a good
|
||||
starting point.
|
||||
- Example: http://crrev.com/818858 leads to the git commit e62cb7e3fc7c40548cef66cdf19d270535d9350b
|
||||
|
||||
## Build args
|
||||
|
||||
|
@ -114,8 +130,8 @@ The more cores the better, as the build makes effective use of each. For Linux,
|
|||
|
||||
- Linux:
|
||||
- SSH in using [gcloud](https://cloud.google.com/sdk/)
|
||||
- Get the ssh command in the [GCP console](https://console.cloud.google.com/) -> VM instances -> your-vm-name -> SSH -> gcloud
|
||||
- Their in-browser UI is kinda sluggish, so use the commandline tool
|
||||
- Get the ssh command in the [GCP console](https://console.cloud.google.com/) -> VM instances -> your-vm-name -> SSH -> "View gcloud command"
|
||||
- Their in-browser UI is kinda sluggish, so use the commandline tool (Google Cloud SDK is required)
|
||||
|
||||
- Windows:
|
||||
- Install Microsoft's Remote Desktop tools
|
||||
|
|
|
@ -33,10 +33,10 @@ source_version = sys.argv[1]
|
|||
base_version = source_version[:7].strip('.')
|
||||
|
||||
# Set to "arm" to build for ARM on Linux
|
||||
arch_name = sys.argv[2] if len(sys.argv) >= 3 else 'x64'
|
||||
arch_name = sys.argv[2] if len(sys.argv) >= 3 else 'unknown'
|
||||
|
||||
if arch_name != 'x64' and arch_name != 'arm64':
|
||||
raise Exception('Unexpected architecture: ' + arch_name)
|
||||
raise Exception('Unexpected architecture: ' + arch_name + '. `x64` and `arm64` are supported.')
|
||||
|
||||
print('Building Chromium ' + source_version + ' for ' + arch_name + ' from ' + src_path)
|
||||
print('src path: ' + src_path)
|
||||
|
|
|
@ -8,18 +8,19 @@ from build_util import runcmd, mkdir, md5_file, configure_environment
|
|||
# call this once the platform-specific initialization has completed.
|
||||
|
||||
# Set to "arm" to build for ARM on Linux
|
||||
arch_name = sys.argv[1] if len(sys.argv) >= 2 else 'x64'
|
||||
arch_name = sys.argv[1] if len(sys.argv) >= 2 else 'undefined'
|
||||
build_path = path.abspath(os.curdir)
|
||||
src_path = path.abspath(path.join(build_path, 'chromium', 'src'))
|
||||
|
||||
if arch_name != 'x64' and arch_name != 'arm64':
|
||||
raise Exception('Unexpected architecture: ' + arch_name)
|
||||
raise Exception('Unexpected architecture: ' + arch_name + '. `x64` and `arm64` are supported.')
|
||||
|
||||
# Configure git
|
||||
print('Configuring git globals...')
|
||||
runcmd('git config --global core.autocrlf false')
|
||||
runcmd('git config --global core.filemode false')
|
||||
runcmd('git config --global branch.autosetuprebase always')
|
||||
runcmd('git config --global core.compression 0')
|
||||
|
||||
# Grab Chromium's custom build tools, if they aren't already installed
|
||||
# (On Windows, they are installed before this Python script is run)
|
||||
|
@ -35,13 +36,14 @@ else:
|
|||
runcmd('git pull origin master')
|
||||
os.chdir(original_dir)
|
||||
|
||||
configure_environment(arch_name, build_path, src_path)
|
||||
|
||||
# Fetch the Chromium source code
|
||||
chromium_dir = path.join(build_path, 'chromium')
|
||||
if not path.isdir(chromium_dir):
|
||||
mkdir(chromium_dir)
|
||||
os.chdir(chromium_dir)
|
||||
runcmd('fetch chromium')
|
||||
runcmd('fetch chromium --nohooks=1 --no-history=1')
|
||||
else:
|
||||
print('Directory exists: ' + chromium_dir + '. Skipping chromium fetch.')
|
||||
|
||||
# This depends on having the chromium/src directory with the complete checkout
|
||||
configure_environment(arch_name, build_path, src_path)
|
||||
|
|
Loading…
Reference in a new issue