mirror of
https://github.com/torvalds/linux.git
synced 2026-03-08 01:04:41 +01:00
Documentation: dev-tools: add container.rst page
Add a dev-tools/container.rst documentation page for the scripts/container tool. This covers the basic usage with additional information about environment variables and user IDs. It also includes a number of practical examples with a reference to the experimental kernel.org toolchain images. Update MAINTAINERS accordingly with a reference to the added file. Signed-off-by: Guillaume Tucker <gtucker@gtucker.io> Reviewed-by: Onur Özkan <work@onurozkan.dev> Tested-by: Nicolas Schier <nsc@kernel.org> Acked-by: Nicolas Schier <nsc@kernel.org> Link: https://patch.msgid.link/af886533cc5cbdd6ef1d909793b79a1ad42c74ca.1769090419.git.gtucker@gtucker.io Signed-off-by: Nathan Chancellor <nathan@kernel.org>
This commit is contained in:
parent
8f989b3b6f
commit
6eac13c876
3 changed files with 229 additions and 0 deletions
227
Documentation/dev-tools/container.rst
Normal file
227
Documentation/dev-tools/container.rst
Normal file
|
|
@ -0,0 +1,227 @@
|
|||
.. SPDX-License-Identifier: GPL-2.0-only
|
||||
.. Copyright (C) 2025 Guillaume Tucker
|
||||
|
||||
====================
|
||||
Containerized Builds
|
||||
====================
|
||||
|
||||
The ``container`` tool can be used to run any command in the kernel source tree
|
||||
from within a container. Doing so facilitates reproducing builds across
|
||||
various platforms, for example when a test bot has reported an issue which
|
||||
requires a specific version of a compiler or an external test suite. While
|
||||
this can already be done by users who are familiar with containers, having a
|
||||
dedicated tool in the kernel tree lowers the barrier to entry by solving common
|
||||
problems once and for all (e.g. user id management). It also makes it easier
|
||||
to share an exact command line leading to a particular result. The main use
|
||||
case is likely to be kernel builds but virtually anything can be run: KUnit,
|
||||
checkpatch etc. provided a suitable image is available.
|
||||
|
||||
|
||||
Options
|
||||
=======
|
||||
|
||||
Command line syntax::
|
||||
|
||||
scripts/container -i IMAGE [OPTION]... CMD...
|
||||
|
||||
Available options:
|
||||
|
||||
``-e, --env-file ENV_FILE``
|
||||
|
||||
Path to an environment file to load in the container.
|
||||
|
||||
``-g, --gid GID``
|
||||
|
||||
Group id to use inside the container.
|
||||
|
||||
``-i, --image IMAGE``
|
||||
|
||||
Container image name (required).
|
||||
|
||||
``-r, --runtime RUNTIME``
|
||||
|
||||
Container runtime name. Supported runtimes: ``docker``, ``podman``.
|
||||
|
||||
If not specified, the first one found on the system will be used
|
||||
i.e. Podman if present, otherwise Docker.
|
||||
|
||||
``-s, --shell``
|
||||
|
||||
Run the container in an interactive shell.
|
||||
|
||||
``-u, --uid UID``
|
||||
|
||||
User id to use inside the container.
|
||||
|
||||
If the ``-g`` option is not specified, the user id will also be used for
|
||||
the group id.
|
||||
|
||||
``-v, --verbose``
|
||||
|
||||
Enable verbose output.
|
||||
|
||||
``-h, --help``
|
||||
|
||||
Show the help message and exit.
|
||||
|
||||
|
||||
Usage
|
||||
=====
|
||||
|
||||
It's entirely up to the user to choose which image to use and the ``CMD``
|
||||
arguments are passed directly as an arbitrary command line to run in the
|
||||
container. The tool will take care of mounting the source tree as the current
|
||||
working directory and adjust the user and group id as needed.
|
||||
|
||||
The container image which would typically include a compiler toolchain is
|
||||
provided by the user and selected via the ``-i`` option. The container runtime
|
||||
can be selected with the ``-r`` option, which can be either ``docker`` or
|
||||
``podman``. If none is specified, the first one found on the system will be
|
||||
used while giving priority to Podman. Support for other runtimes may be added
|
||||
later depending on their popularity among users.
|
||||
|
||||
By default, commands are run non-interactively. The user can abort a running
|
||||
container with SIGINT (Ctrl-C). To run commands interactively with a TTY, the
|
||||
``--shell`` or ``-s`` option can be used. Signals will then be received by the
|
||||
shell directly rather than the parent ``container`` process. To exit an
|
||||
interactive shell, use Ctrl-D or ``exit``.
|
||||
|
||||
.. note::
|
||||
|
||||
The only host requirement aside from a container runtime is Python 3.10 or
|
||||
later.
|
||||
|
||||
.. note::
|
||||
|
||||
Out-of-tree builds are not fully supported yet. The ``O=`` option can
|
||||
however already be used with a relative path inside the source tree to keep
|
||||
separate build outputs. A workaround to build outside the tree is to use
|
||||
``mount --bind``, see the examples section further down.
|
||||
|
||||
|
||||
Environment Variables
|
||||
=====================
|
||||
|
||||
Environment variables are not propagated to the container so they have to be
|
||||
either defined in the image itself or via the ``-e`` option using an
|
||||
environment file. In some cases it makes more sense to have them defined in
|
||||
the Containerfile used to create the image. For example, a Clang-only compiler
|
||||
toolchain image may have ``LLVM=1`` defined.
|
||||
|
||||
The local environment file is more useful for user-specific variables added
|
||||
during development. It is passed as-is to the container runtime so its format
|
||||
may vary. Typically, it will look like the output of ``env``. For example::
|
||||
|
||||
INSTALL_MOD_STRIP=1
|
||||
SOME_RANDOM_TEXT=One upon a time
|
||||
|
||||
Please also note that ``make`` options can still be passed on the command line,
|
||||
so while this can't be done since the first argument needs to be the
|
||||
executable::
|
||||
|
||||
scripts/container -i docker.io/tuxmake/korg-clang LLVM=1 make # won't work
|
||||
|
||||
this will work::
|
||||
|
||||
scripts/container -i docker.io/tuxmake/korg-clang make LLVM=1
|
||||
|
||||
|
||||
User IDs
|
||||
========
|
||||
|
||||
This is an area where the behaviour will vary slightly depending on the
|
||||
container runtime. The goal is to run commands as the user invoking the tool.
|
||||
With Podman, a namespace is created to map the current user id to a different
|
||||
one in the container (1000 by default). With Docker, while this is also
|
||||
possible with recent versions it requires a special feature to be enabled in
|
||||
the daemon so it's not used here for simplicity. Instead, the container is run
|
||||
with the current user id directly. In both cases, this will provide the same
|
||||
file permissions for the kernel source tree mounted as a volume. The only
|
||||
difference is that when using Docker without a namespace, the user id may not
|
||||
be the same as the default one set in the image.
|
||||
|
||||
Say, we're using an image which sets up a default user with id 1000 and the
|
||||
current user calling the ``container`` tool has id 1234. The kernel source
|
||||
tree was checked out by this same user so the files belong to user 1234. With
|
||||
Podman, the container will be running as user id 1000 with a mapping to id 1234
|
||||
so that the files from the mounted volume appear to belong to id 1000 inside
|
||||
the container. With Docker and no namespace, the container will be running
|
||||
with user id 1234 which can access the files in the volume but not in the user
|
||||
1000 home directory. This shouldn't be an issue when running commands only in
|
||||
the kernel tree but it is worth highlighting here as it might matter for
|
||||
special corner cases.
|
||||
|
||||
.. note::
|
||||
|
||||
Podman's `Docker compatibility
|
||||
<https://podman-desktop.io/docs/migrating-from-docker/managing-docker-compatibility>`__
|
||||
mode to run ``docker`` commands on top of a Podman backend is more complex
|
||||
and not fully supported yet. As such, Podman will take priority if both
|
||||
runtimes are available on the system.
|
||||
|
||||
|
||||
Examples
|
||||
========
|
||||
|
||||
The TuxMake project provides a variety of prebuilt container images available
|
||||
on `Docker Hub <https://hub.docker.com/u/tuxmake>`__. Here's the shortest
|
||||
example to build a kernel using a TuxMake Clang image::
|
||||
|
||||
scripts/container -i docker.io/tuxmake/korg-clang -- make LLVM=1 defconfig
|
||||
scripts/container -i docker.io/tuxmake/korg-clang -- make LLVM=1 -j$(nproc)
|
||||
|
||||
.. note::
|
||||
|
||||
When running a command with options within the container, it should be
|
||||
separated with a double dash ``--`` to not confuse them with the
|
||||
``container`` tool options. Plain commands with no options don't strictly
|
||||
require the double dashes e.g.::
|
||||
|
||||
scripts/container -i docker.io/tuxmake/korg-clang make mrproper
|
||||
|
||||
To run ``checkpatch.pl`` in a ``patches`` directory with a generic Perl image::
|
||||
|
||||
scripts/container -i perl:slim-trixie scripts/checkpatch.pl patches/*
|
||||
|
||||
As an alternative to the TuxMake images, the examples below refer to
|
||||
``kernel.org`` images which are based on the `kernel.org compiler toolchains
|
||||
<https://mirrors.edge.kernel.org/pub/tools/>`__. These aren't (yet) officially
|
||||
available in any public registry but users can build their own locally instead
|
||||
using this `experimental repository
|
||||
<https://gitlab.com/gtucker/korg-containers>`__ by running ``make
|
||||
PREFIX=kernel.org/``.
|
||||
|
||||
To build just ``bzImage`` using Clang::
|
||||
|
||||
scripts/container -i kernel.org/clang -- make bzImage -j$(nproc)
|
||||
|
||||
Same with GCC 15 as a particular version tag::
|
||||
|
||||
scripts/container -i kernel.org/gcc:15 -- make bzImage -j$(nproc)
|
||||
|
||||
For an out-of-tree build, a trick is to bind-mount the destination directory to
|
||||
a relative path inside the source tree::
|
||||
|
||||
mkdir -p $HOME/tmp/my-kernel-build
|
||||
mkdir -p build
|
||||
sudo mount --bind $HOME/tmp/my-kernel-build build
|
||||
scripts/container -i kernel.org/gcc -- make mrproper
|
||||
scripts/container -i kernel.org/gcc -- make O=build defconfig
|
||||
scripts/container -i kernel.org/gcc -- make O=build -j$(nproc)
|
||||
|
||||
To run KUnit in an interactive shell and get the full output::
|
||||
|
||||
scripts/container -s -i kernel.org/gcc:kunit -- \
|
||||
tools/testing/kunit/kunit.py \
|
||||
run \
|
||||
--arch=x86_64 \
|
||||
--cross_compile=x86_64-linux-
|
||||
|
||||
To just start an interactive shell::
|
||||
|
||||
scripts/container -si kernel.org/gcc bash
|
||||
|
||||
To build the HTML documentation, which requires the ``kdocs`` image built with
|
||||
``make PREFIX=kernel.org/ extra`` as it's not a compiler toolchain::
|
||||
|
||||
scripts/container -i kernel.org/kdocs make htmldocs
|
||||
|
|
@ -38,6 +38,7 @@ Documentation/process/debugging/index.rst
|
|||
gpio-sloppy-logic-analyzer
|
||||
autofdo
|
||||
propeller
|
||||
container
|
||||
|
||||
|
||||
.. only:: subproject and html
|
||||
|
|
|
|||
|
|
@ -6383,6 +6383,7 @@ F: include/linux/console*
|
|||
CONTAINER BUILD SCRIPT
|
||||
M: Guillaume Tucker <gtucker@gtucker.io>
|
||||
S: Maintained
|
||||
F: Documentation/dev-tools/container.rst
|
||||
F: scripts/container
|
||||
|
||||
CONTEXT TRACKING
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue