Packaging Guide¶
Building packages with dh-virtualenv is relatively easy to start with, but it also supports lot of customization to match your specific needs.
By default, dh-virtualenv installs your packages under
/opt/venvs/«packagename»
. The package name is provided by
the debian/control
file.
To use an alternative install prefix, add a line like the following
to the top of your debian/rules
file.
export DH_VIRTUALENV_INSTALL_ROOT=«/your/custom/install/dir»
dh_virtualenv will now use the value of
DH_VIRTUALENV_INSTALL_ROOT
instead of /opt/venvs
when it constructs the install path.
To use an install suffix other than the package name, call
dh_virtualenv
using the --install-suffix
command
line option. See Advanced usage for further information on passing
options.
Simple usecase¶
To signal debhelper to use dh-virtualenv for building your
package, you need to pass --with python-virtualenv
to the debhelper
sequencer.
In a nutshell, the simplest debian/rules
file to build using
dh-virtualenv looks like this:
#!/usr/bin/make -f
%:
dh $@ --with python-virtualenv
However, the tool makes a few assumptions of your project’s structure:
- For installing requirements, you need to have a file called
requirements.txt
in the root directory of your project. The requirements file is not mandatory.- The project must have a
setup.py
file in the root of the project.dh_virtualenv
will runsetup.py install
to add your project to the virtualenv.
After these preparations, you can just build the package with your favorite tool!
Environment variables¶
Certain environment variables can be used to customise the behaviour of the debhelper sequencer in addition to the standard debhelper variables.
-
DH_VIRTUALENV_INSTALL_ROOT
¶ Define a custom root location to install your package(s). The resulting location for a specific package will be
$DH_VIRTUALENV_INSTALL_ROOT/«<packagename»
, unless--install-suffix
is also used to change«<packagename»
.
Command line options¶
To change its default behavior, the dh_virtualenv
command accepts a
few command line options:
-
-p
<package>
,
--package
<package>
¶ Act on the package named <package>.
-
-N
<package>
,
--no-package
<package>
¶ Do not act on the specified package.
-
-v
,
--verbose
¶
Turn on verbose mode. This has a few effects: it sets the root logger level to
DEBUG
, and passes the verbose flag topip
when installing packages. This can also be provided using the standardDH_VERBOSE
environment variable.
-
--install-suffix
<suffix>
¶ Override virtualenv installation suffix. The suffix is appended to
/opt/venvs
, or theDH_VIRTUALENV_INSTALL_ROOT
environment variable if specified, to construct the installation path.
-
--extra-index-url
<url>
¶ Use extra index url <url> when running
pip
to install packages. This can be provided multiple times to pass multiple URLs topip
. A common use-case is enabling a private Python package repository.
-
--preinstall
<package>
¶ Package to install before processing the requirements. This flag can be used to provide a package that is installed by
pip
before processing the requirements file. It is handy if you need to install a custom setup script or other packages needed to parsesetup.py
, and can be provided multiple times to pass multiple packages for pre-install.
-
--extras
<name>
¶ New in version 1.1.
Name of extras defined in the main package (specifically its
setup.py
, inextras_require
). You can pass this multiple times to add different extra requirements.
-
--pip-tool
<exename>
¶ Executable that will be used to install requirements after the preinstall stage. Usually you’ll install this program by using the
--preinstall
argument. The replacement is expected to be found in the virtualenv’sbin/
directory.
-
--upgrade-pip
¶
New in version 1.0.
Force upgrading to the latest available release of
pip
. This is the first thing done in the pre-install stage, and uses a separatepip
call.Options provided via
--extra-pip-arg
are ignored here, because the defaultpip
of your system might not support them (since version 1.1).Note: This can produce non-repeatable builds.
-
--index-url
<URL>
¶ Base URL of the PyPI server. This flag can be used to pass in a custom URL to a PyPI mirror. It’s useful if you have an internal PyPI mirror, or you run a special instance that only exposes selected packages of PyPI. If this is not provided, the default will be whatever
pip
uses as default (usually the API ofhttps://pypi.org/
).
-
--extra-pip-arg
<PIP ARG>
¶ Extra arguments to pass to the pip executable. This is useful if you need to change the behaviour of pip during the packaging process. You can use this flag multiple times to pass in different pip flags.
As an example, adding
--extra-pip-arg --no-compile
in the call of aoverride_dh_virtualenv
rule in thedebian/rules
file will disable the generation of*.pyc
files.
-
--extra-virtualenv-arg
<VIRTUALENV ARG>
¶ Extra parameters to pass to the virtualenv executable. This is useful if you need to change the behaviour of virtualenv during the packaging process. You can use this flag multiple times to pass in different virtualenv flags.
-
--requirements
<REQUIREMENTS FILE>
¶ Use a different requirements file when installing. Some packages such as pbr expect the
requirements.txt
file to be a simple list of requirements that can be copied verbatim into theinstall_requires
list. This command option allows specifying a differentrequirements.txt
file that may include pip specific flags such as-i
,-r-
and-e
.
-
--setuptools
¶
Use setuptools instead of distribute in the virtualenv.
-
--setuptools-test
¶
New in version 1.0.
Run
python setup.py test
when building the package. This was the old default behaviour before version 1.0. This option is incompatible with the deprecated--no-test
.
-
--python
<path>
¶ Use a specific Python interpreter found in
path
as the interpreter for the virtualenv. Default is to use the system default, usually/usr/bin/python
.
-
--builtin-venv
¶
Enable the use of the build-in
venv
module, i.e. usepython -m venv
to create the virtualenv. It will only work with Python 3.4 or later, e.g. by using the option--python
/usr/bin/python3.4
.
-
-S
,
--use-system-packages
¶
Enable the use of system site-packages in the created virtualenv by passing the
--system-site-packages
flag tovirtualenv
.
-
--skip-install
¶
Skip running
pip install .
after dependencies have been installed. This will result in anything specified insetup.py
being ignored. If this package is intended to install a virtualenv and a program that uses the supplied virtualenv, it is up to the user to ensure that ifsetup.py
exists, any installation logic or dependencies contained therein are handled.This option is useful for web application deployments, where the package’s virtual environment merely supports an application installed via other means. Typically, the
debian/«packagename».install
file is used to place the application at a location outside of the virtual environment.
-
--pypi-url
<URL>
¶ Deprecated since version 1.0: Use
--index-url
instead.
-
--no-test
¶
Deprecated since version 1.0: This option has no effect. See
--setuptools-test
.
Advanced usage¶
To provide command line options to the dh_virtualenv
step,
use debhelper’s override mechanism.
The following debian/rules
will provide http://example.com as
an additional source of Python packages:
#!/usr/bin/make -f
%:
dh $@ --with python-virtualenv
override_dh_virtualenv:
dh_virtualenv --extra-index-url http://example.com
pbuilder and dh-virtualenv¶
Building your Debian package in a pbuilder environment can help to ensure proper dependencies and repeatable builds. However, precisely because pbuilder creates its own build environment, build failues can be much more difficult to understand and troubleshoot. This is especially true when there is a pip error inside the pbuilder environment. For that reason, make sure that you can build your Debian package successfully outside of a pbuilder environment before trying to build it inside.
With those caveats, here are some tips for making pip and dh_virtual work inside pbuilder.
If you want pip to retrieve packages from the network, you need to
add USENETWORK=yes
to your /etc/pbuilderrc or ~/.pbuilderrc file.
pip has several options that can be used to make it more compatible with pbuilder.
Use --no-cache-dir
to stop creating wheels in your home directory,
which will fail when running in a pbuilder environment, because
pbuilder sets the HOME environment variable to “/nonexistent”.
Use --no-deps
to make pip builds more repeatable.
Use --ignore-installed
to ensure that pip installs every package in
requirements.txt
in the virtualenv. This option is especially important if
you are using the –system-site-packages option in your virtualenv.
Here’s an example of how to use these arguments in your rules
file.
override_dh_virtualenv:
dh_virtualenv \
--extra-pip-arg "--ignore-installed" \
--extra-pip-arg "--no-deps" \
--extra-pip-arg "--no-cache-dir"
Experimental buildsystem support¶
Important
This section describes a completely experimental functionality of dh-virtualenv.
Starting with version 0.9 of dh-virtualenv, there is a buildsystem alternative.
The main difference in use is that instead of the --with python-virtualenv
option, --buildsystem=dh_virtualenv
is passed to debhelper. The debian rules
file should look like this:
#!/usr/bin/make -f
%:
dh $@ --buildsystem=dh_virtualenv
Using the buildsystem instead of the part of the sequence (in other
words, instead of the --with python-virtualenv
) one can get more
flexibility into the build process.
Flexibility comes from the fact that buildsystem will have individual
steps for configure, build, test and install and those can be
overridden by adding override_dh_auto_<STEP>
target into the
debian/rules
file. For example:
#!/usr/bin/make -f
%:
dh $@ --buildsystem=dh_virtualenv
override_dh_auto_test:
py.test test/
In addition the separation of build and install steps makes it
possible to use debian/install
files to include built files into
the Debian package. This is not possible with the sequencer addition.
The build system honors the DH_VIRTUALENV_INSTALL_ROOT
environment variable. Following other environment variables can be
used to customise the functionality:
-
DH_VIRTUALENV_ARGUMENTS
¶ Pass given extra arguments to the
virtualenv
commandFor example:
export DH_VIRTUALENV_ARGUMENTS="--no-site-packages --always-copy"
The default is to create the virtual environment with
--no-site-packages
.
-
DH_VIRTUALENV_INSTALL_SUFFIX
¶ Override the default virtualenv name, instead of source package name.
For example:
-
DH_REQUIREMENTS_FILE
¶ New in version 1.0.
Override the location of requirements file. See
--requirements
.
-
DH_UPGRADE_PIP
¶ New in version 1.0.
Force upgrade of the
pip
tool by settingDH_UPGRADE_PIP
to empty (latest version) or specific version. For example:
-
DH_UPGRADE_SETUPTOOLS
¶ New in version 1.0.
Force upgrade of setuptools by setting
DH_UPGRADE_SETUPTOOLS
to empty (latest version) or specific version.
-
DH_UPGRADE_WHEEL
¶ New in version 1.0.
Force upgrade of wheel by setting
DH_UPGRADE_WHEEL
to empty (latest version) or specific version.
-
DH_PIP_EXTRA_ARGS
¶ New in version 1.0.
Pass additional parameters to the
pip
command. For example:export DH_PIP_EXTRA_ARGS="--no-index --find-links=./requirements/wheels"