Chapter 2. Debian package management

Here are some key points for package configuration on the Debian system.

Despite my warnings above, I know many readers of this document wish to run the testing or unstable suites of Debian as their main system for self-administered Desktop environments. This is because they work very well, are updated frequently, and offer the latest features.

	Caution
	For your production server, the stable suite with the security updates is recommended. The same can be said for desktop PCs on which you can spend limited administration efforts, e.g. for your mother's PC.

It takes no more than simply setting the distribution string in the "/etc/apt/sources.list" to the suite name: "testing" or "unstable"; or the codename: "buster" or "sid". This makes you live the life of eternal upgrades.

The use of testing or unstable is a lot of fun but comes with some risks. Even though the unstable suite of the Debian system looks very stable for most of the times, there have been some package problems on the testing and unstable suite of the Debian system and a few of them were not so trivial to resolve. It may be quite painful for you. Sometimes, you may have a broken package or missing functionality for a few weeks.

Here are some ideas to ensure quick and easy recovery from bugs in Debian packages.

Debian is 100% free software because of the followings:

Some people wonder if the following 2 facts contradict or not.

The Debian system offers a consistent set of binary packages through its versioned binary dependency declaration mechanism in the control file fields. Here is a bit over simplified definition for them.

	Note
	Please note that defining "Provides", "Conflicts" and "Replaces" simultaneously to an virtual package is the sane configuration. This ensures that only one real package providing this virtual package can be installed at any one time.

Here is a summary of the simplified event flow of the package management by APT.

Although aptitude is a very nice interactive tool which the author mainly uses, you should know some cautionary facts:

Here are basic package management operations with the commandline using aptitude(8) and apt-get(8) /apt-cache(8).

	Note
	Although the aptitude command comes with rich features such as its enhanced package resolver, this complexity has caused (or may still causes) some regressions such as Bug #411123, Bug #514930, and Bug #570377. In case of doubt, please use the apt-get and apt-cache commands over the aptitude command.

	Note
	Since apt-get and aptitude share auto-installed package status (see Section 2.5.5, “The package state for APT”) after lenny, you can mix these tools without major troubles (see Bug #594490).

The "aptitude why <regex>" can list more information by "aptitude -v why <regex>". Similar information can be obtained by "apt-cache rdepends <package>".

When aptitude command is started in the commandline mode and faces some issues such as package conflicts, you can switch to the full screen interactive mode by pressing "e"-key later at the prompt.

You may provide command options right after "aptitude".

See aptitude(8) and "aptitude user's manual" at "/usr/share/doc/aptitude/README" for more.

	Tip
	The dselect package is still available and was the preferred full screen interactive package management tool in previous releases.

For the interactive package management, you start aptitude in interactive mode from the console shell prompt as follows.

$ sudo aptitude -u
Password:

This updates the local copy of the archive information and display the package list in the full screen with menu. Aptitude places its configuration at "~/.aptitude/config".

	Tip
	If you want to use root's configuration instead of user's one, use "sudo -H aptitude …" instead of "sudo aptitude …" in the above expression.

	Tip
	Aptitude automatically sets pending actions as it is started interactively. If you do not like it, you can reset it from menu: "Action" → "Cancel pending actions".

Notable key strokes to browse status of packages and to set "planned action" on them in this full screen mode are the following.

The file name specification of the command line and the menu prompt after pressing "l" and "//" take the aptitude regex as described below. Aptitude regex can explicitly match a package name using a string started by "~n and followed by the package name.

	Tip
	You need to press "U" to get all the installed packages upgraded to the candidate version in the visual interface. Otherwise only the selected packages and certain packages with versioned dependency to them are upgraded to the candidate version.

In the interactive full screen mode of aptitude(8), packages in the package list are displayed as the next example.

idA   libsmbclient                             -2220kB 3.0.25a-1  3.0.25a-2

Here, this line means from the left as the following.

	Tip
	The full list of flags are given at the bottom of Help screen shown by pressing "?".

	Note
	Please help us improving tagging packages with debtags!

The standard "Package View" categorizes packages somewhat like dselect with few extra features.

Aptitude offers several options for you to search packages using its regex formula.

	Tip
	The string for <package_name> is treated as the exact string match to the package name unless it is started explicitly with "~" to be the regex formula.

	Tip
	When <regex_pattern> is a null string, place "~T" immediately after the command.

Here are some short cuts.

Users familiar with mutt pick up quickly, as mutt was the inspiration for the expression syntax. See "SEARCHING, LIMITING, AND EXPRESSIONS" in the "User's Manual" "/usr/share/doc/aptitude/README".

	Note
	With the lenny version of aptitude(8), the new long form syntax such as "?broken" may be used for regex matching in place for its old short form equivalent "~b". Now space character " " is considered as one of the regex terminating character in addition to tilde character "~". See "User's Manual" for the new long form syntax.

The selection of a package in aptitude not only pulls in packages which are defined in its "Depends:" list but also defined in the "Recommends:" list if the menu "F10 → Options → Preferences → Dependency handling" is set accordingly. These auto installed packages are removed automatically if they are no longer needed under aptitude.

The flag controlling the "auto install" behavior of the aptitude command can also be manipulated using the apt-mark(8) command from the apt package.

You can check package activity history in the log files.

The following command lists packages with regex matching on package names.

$ aptitude search '~n(pam|nss).*ldap'
p libnss-ldap - NSS module for using LDAP as a naming service
p libpam-ldap - Pluggable Authentication Module allowing LDAP interfaces

This is quite handy for you to find the exact name of a package.

The regex "~dipv6" in the "New Flat Package List" view with "l" prompt, limits view to packages with the matching description and let you browse their information interactively.

You can purge all remaining configuration files of removed packages.

Check results of the following command.

# aptitude search '~c'

If you think listed packages are OK to be purged, execute the following command.

# aptitude purge '~c'

You may want to do the similar in the interactive mode for fine grained control.

You provide the regex "~c" in the "New Package View" view with "l" prompt. This limits the package view only to regex matched packages, i.e., "removed but not purged". All these regex matched packages can be shown by pressing "[" at top level headings.

Then you press "_" at top level headings such as "Not Installed Packages". Only regex matched packages under the heading are marked to be purged by this. You can exclude some packages to be purged by pressing "=" interactively for each of them.

This technique is quite handy and works for many other command keys.

Here is how I tidy auto/manual install status for packages (after using non-aptitude package installer etc.).

The "m" action over "Tasks" is an optional one to prevent mass package removal situation in future.

Note

When moving to a new release etc, you should consider to perform a clean installation of new system even though Debian is upgradable as described below. This provides you a chance to remove garbages collected and exposes you to the best combination of latest packages. Of course, you should make a full backup of system to a safe place (see Section 10.2, “Backup and recovery”) before doing this. I recommend to make a dual boot configuration using different partition to have the smoothest transition.

You can perform system wide upgrade to a newer release by changing contents of the "/etc/apt/sources.list" file pointing to a new release and running the "apt-get update; apt-get dist-upgrade" command.

Here are list of other package management operations for which aptitude is too high-level or lacks required functionalities.

	Note
	For a package with the multi-arch feature, you may need to specify the architecture name for some commands. For example, use "dpkg -L libglib2.0-0:amd64" to list contents of the libglib2.0-0 package for the amd64 architecture.

	Caution
	Lower level package tools such as "dpkg -i …" and "debi …" should be carefully used by the system administrator. It does not automatically take care required package dependencies. Dpkg's commandline options "--force-all" and similar (see dpkg(1)) are intended to be used by experts only. Using them without fully understanding their effects may break your whole system.

Please note the following.

Many users prefer to follow the unstable release of the Debian system for its new features and packages. This makes the system more prone to be hit by the critical package bugs.

The installation of the apt-listbugs package safeguards your system against critical bugs by checking Debian BTS automatically for critical bugs when upgrading with APT system.

The installation of the apt-listchanges package provides important news in "NEWS.Debian" when upgrading with APT system.

Meta data files for each distribution are stored under "dist/<codename>" on each Debian mirror sites, e.g., "http://deb.debian.org/debian/". Its archive structure can be browsed by the web browser. There are 6 types of key meta data.

In the recent archive, these meta data are stored as the compressed and differential files to reduce network traffic.

	Tip
	The top level "Release" file is used for signing the archive under the secure APT system.

Each suite of the Debian archive has a top level "Release" file, e.g., "http://deb.debian.org/debian/dists/unstable/Release", as follows.

Origin: Debian
Label: Debian
Suite: unstable
Codename: sid
Date: Sat, 14 May 2011 08:20:50 UTC
Valid-Until: Sat, 21 May 2011 08:20:50 UTC
Architectures: alpha amd64 armel hppa hurd-i386 i386 ia64 kfreebsd-amd64 kfreebsd-i386 mips mipsel powerpc s390 sparc
Components: main contrib non-free
Description: Debian x.y Unstable - Not Released
MD5Sum:
 bdc8fa4b3f5e4a715dd0d56d176fc789 18876880 Contents-alpha.gz
 9469a03c94b85e010d116aeeab9614c0 19441880 Contents-amd64.gz
 3d68e206d7faa3aded660dc0996054fe 19203165 Contents-armel.gz
...

	Note
	Here, you can find my rationale to use the "suite", and "codename" in Section 2.1.4, “Debian archive basics”. The "distribution" is used when referring to both "suite" and "codename". All archive "area" names offered by the archive are listed under "Components".

	Tip
	The archive level "Release" files are used for the rule of apt_preferences(5).

There are archive level "Release" files for all archive locations specified by "deb" line in "/etc/apt/sources.list", such as "http://deb.debian.org/debian/dists/unstable/main/binary-amd64/Release" or "http://deb.debian.org/debian/dists/sid/main/binary-amd64/Release" as follows.

Archive: unstable
Origin: Debian
Label: Debian
Component: main
Architecture: amd64

	Caution
	For "Archive:" stanza, suite names ("stable", "testing", "unstable", …) are used in the Debian archive while codenames ("dapper", "feisty", "gutsy", "hardy", "intrepid", …) are used in the Ubuntu archive.

For some archives, such as experimental, and stretch-backports, which contain packages which should not be installed automatically, there is an extra line, e.g., "http://deb.debian.org/debian/dists/experimental/main/binary-amd64/Release" as follows.

Archive: experimental
Origin: Debian
Label: Debian
NotAutomatic: yes
Component: main
Architecture: amd64

In addition to the remotely fetched meta data, the APT tool after lenny stores its locally generated installation state information in the "/var/lib/apt/extended_states" which is used by all APT tools to track all auto installed packages.

In addition to the remotely fetched meta data, the aptitude command stores its locally generated installation state information in the "/var/lib/aptitude/pkgstates" which is used only by it.

All the remotely fetched packages via APT mechanism are stored in the "/var/cache/apt/archives" until they are cleaned.

This cache file cleaning policy for aptitude can be set under "Options" → "Preferences" and it may be forced by its menu "Clean package cache" or "Clean obsolete files" under "Actions".

Debian package files have particular name structures.

dpkg(1) is the lowest level tool for the Debian package management. This is very powerful and needs to be used with care.

While installing package called "<package_name>", dpkg process it in the following order.

The "status" file is also used by the tools such as dpkg(1), "dselect update" and "apt-get -u dselect-upgrade".

The specialized search command grep-dctrl(1) can search the local copies of "status" and "available" meta data.

	Tip
	In the debian-installer environment, the udpkg command is used to open udeb packages. The udpkg command is a stripped down version of the dpkg command.

The Debian system has mechanism to install somewhat overlapping programs peacefully using update-alternatives(8). For example, you can make the vi command select to run vim while installing both vim and nvi packages.

$ ls -l $(type -p vi)
lrwxrwxrwx 1 root root 20 2007-03-24 19:05 /usr/bin/vi -> /etc/alternatives/vi
$ sudo update-alternatives --display vi
...
$ sudo update-alternatives --config vi
  Selection    Command
 ----------------------------------------------
      1        /usr/bin/vim
*+    2        /usr/bin/nvi

Enter to keep the default[*], or type selection number: 1

The Debian alternatives system keeps its selection as symlinks in "/etc/alternatives/". The selection process uses corresponding file in "/var/lib/dpkg/alternatives/".

Stat overrides provided by the dpkg-statoverride(8) command are a way to tell dpkg(1) to use a different owner or mode for a file when a package is installed. If "--update" is specified and file exists, it is immediately set to the new owner and mode.

	Caution
	The direct alteration of owner or mode for a file owned by the package using chmod or chown commands by the system administrator is reset by the next upgrade of the package.

	Note
	I use the word file here, but in reality this can be any filesystem object that dpkg handles, including directories, devices, etc.

File diversions provided by the dpkg-divert(8) command are a way of forcing dpkg(1) not to install a file into its default location, but to a diverted location. The use of dpkg-divert is meant for the package maintenance scripts. Its casual use by the system administrator is deprecated.

If a desktop GUI program experienced instability after significant upstream version upgrade, you should suspect interferences with old local configuration files created by it. If it is stable under a newly created user account, this hypothesis is confirmed. (This is a bug of packaging and usually avoided by the packager.)

To recover stability, you should move corresponding local configuration files and restart the GUI program. You may need to read old configuration file contents to recover configuration information later. (Do not erase them too quickly.)

When a command in the package script returns error for some reason and the script exits with error, the package management system aborts their action and ends up with partially installed packages. When a package contains bugs in its removal scripts, the package may become impossible to remove and quite nasty.

For the package script problem of "<package_name>", you should look into following package scripts.

Edit the offending package script from the root using following techniques.

Configure all partially installed packages with the following command.

# dpkg --configure -a

Since dpkg is very low level package tool, it can function under the very bad situation such as unbootable system without network connection. Let's assume foo package was broken and needs to be replaced.

If "/var/lib/dpkg/status" becomes corrupt for any reason, the Debian system loses package selection data and suffers severely. Look for the old "/var/lib/dpkg/status" file at "/var/lib/dpkg/status-old" or "/var/backups/dpkg.status.*".

Keeping "/var/backups/" in a separate partition may be a good idea since this directory contains lots of important system data.

For serious breakage, I recommend to make fresh re-install after making backup of the system. Even if everything in "/var/" is gone, you can still recover some information from directories in "/usr/share/doc/" to guide your new installation.

Reinstall minimal (desktop) system.

# mkdir -p /path/to/old/system

Mount old system at "/path/to/old/system/".

# cd /path/to/old/system/usr/share/doc
# ls -1 >~/ls1.txt
# cd /usr/share/doc
# ls -1 >>~/ls1.txt
# cd
# sort ls1.txt | uniq | less

Then you are presented with package names to install. (There may be some non-package names such as "texmf".)

You can seek packages which satisfy your needs with aptitude from the package description or from the list under "Tasks".

When you encounter more than 2 similar packages and wonder which one to install without "trial and error" efforts, you should use some common sense. I consider following points are good indications of preferred packages.

Debian being a volunteer project with distributed development model, its archive contains many packages with different focus and quality. You must make your own decision what to do with them.

	Caution
	Installing packages from mixed source of archives is not supported by the official Debian distribution except for officially supported particular combinations of archives such as stable with security updates and stretch-updates.

Here is an example of operations to include specific newer upstream version packages found in unstable while tracking testing for single occasion.

You do not create the "/etc/apt/preferences" file nor need to worry about apt-pinning with this manual approach. But this is very cumbersome.

	Caution
	When using mixed source of archives, you must ensure compatibility of packages by yourself since the Debian does not guarantee it. If package incompatibility exists, you may break system. You must be able to judge these technical requirements. The use of mixed source of random archives is completely optional operation and its use is not something I encourage you to use.

General rules for installing packages from different archives are the following.

Without the "/etc/apt/preferences" file, APT system choses the latest available version as the candidate version using the version string. This is the normal state and most recommended usage of APT system. All officially supported combinations of archives do not require the "/etc/apt/preferences" file since some archives which should not be used as the automatic source of upgrades are marked as NotAutomatic and dealt properly.

	Tip
	The version string comparison rule can be verified with, e.g., "dpkg --compare-versions ver1.1 gt ver1.1~1; echo $?" (see dpkg(1)).

The target release archive can be set by several methods.

If you wish not to pull in particular packages automatically by "Recommends", you must create the "/etc/apt/preferences" file and explicitly list all those packages at the top of it as the following.

Package: <package-1>
Pin: version *
Pin-Priority: -1

Package: <package-2>
Pin: version *
Pin-Priority: -1

Here is an example of apt-pinning technique to include specific newer upstream version packages found in unstable regularly upgraded while tracking testing. You list all required archives in the "/etc/apt/sources.list" file as the following.

deb http://deb.debian.org/debian/ testing main contrib non-free
deb http://deb.debian.org/debian/ unstable main contrib non-free
deb http://security.debian.org/ testing/updates main contrib

Set the "/etc/apt/preferences" file as as the following.

Package: *
Pin: release a=unstable
Pin-Priority: 100

When you wish to install a package named "<package-name>" with its dependencies from unstable archive under this configuration, you issue the following command which switches target release with "-t" option (Pin-Priority of unstable becomes 990).

$ sudo apt-get install -t unstable <package-name>

With this configuration, usual execution of "apt-get upgrade" and "apt-get dist-upgrade" (or "aptitude safe-upgrade" and "aptitude full-upgrade") upgrades packages which were installed from testing archive using current testing archive and packages which were installed from unstable archive using current unstable archive.

	Caution
	Be careful not to remove "testing" entry from the "/etc/apt/sources.list" file. Without "testing" entry in it, APT system upgrades packages using newer unstable archive.

	Tip
	I usually edit the "/etc/apt/sources.list" file to comment out "unstable" archive entry right after above operation. This avoids slow update process of having too many entries in the "/etc/apt/sources.list" file although this prevents upgrading packages which were installed from unstable archive using current unstable archive.

	Tip
	If "Pin-Priority: 1" is used instead of "Pin-Priority: 100" in the "/etc/apt/preferences" file, already installed packages having Pin-Priority value of 100 are not upgraded by unstable archive even if "testing" entry in the "/etc/apt/sources.list" file is removed.

If you wish to track particular packages in unstable automatically without initial "-t unstable" installation, you must create the "/etc/apt/preferences" file and explicitly list all those packages at the top of it as the following.

Package: <package-1>
Pin: release a=unstable
Pin-Priority: 700

Package: <package-2>
Pin: release a=unstable
Pin-Priority: 700

These set Pin-Priority value for each specific package. For example, in order to track the latest unstable version of this "Debian Reference" in English, you should have following entries in the "/etc/apt/preferences" file.

Package: debian-reference-en
Pin: release a=unstable
Pin-Priority: 700

Package: debian-reference-common
Pin: release a=unstable
Pin-Priority: 700

	Tip
	This apt-pinning technique is valid even when you are tracking stable archive. Documentation packages have been always safe to install from unstable archive in my experience, so far.

Here is another example of apt-pinning technique to include specific newer upstream version packages found in experimental while tracking unstable. You list all required archives in the "/etc/apt/sources.list" file as the following.

deb http://deb.debian.org/debian/ unstable main contrib non-free
deb http://deb.debian.org/debian/ experimental main contrib non-free
deb http://security.debian.org/ testing/updates main contrib

The apt package comes with its own cron script "/etc/cron.daily/apt" to support the automatic download of packages. This script can be enhanced to perform the automatic upgrade of packages by installing the unattended-upgrades package. These can be customized by parameters in "/etc/apt/apt.conf.d/02backup" and "/etc/apt/apt.conf.d/50unattended-upgrades" as described in "/usr/share/doc/unattended-upgrades/README".

The unattended-upgrades package is mainly intended for the security upgrade for the stable system. If the risk of breaking an existing stable system by the automatic upgrade is smaller than that of the system broken by the intruder using its security hole which has been closed by the security update, you should consider using this automatic upgrade with configuration parameters as the following.

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "1";

If you are running an unstable system, you do not want to use the automatic upgrade since it certainly breaks system some day. Even for such unstable case, you may still want to download packages in advance to save time for the interactive upgrade with configuration parameters as the following.

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::Unattended-Upgrade "0";

If you want to limit the download bandwidth for APT to e.g. 800Kib/sec (=100kiB/sec), you should configure APT with its configuration parameter as the following.

APT::Acquire::http::Dl-Limit "800";

Although the maintainer name listed in "/var/lib/dpkg/available" and "/usr/share/doc/package_name/changelog" provide some information on "who is behind the packaging activity", the actual uploader of the package is somewhat obscure. who-uploads(1) in the devscripts package identifies the actual uploader of Debian source packages.

If you are to compile a program from source to replace the Debian package, it is best to make it into a real local debianized package (*.deb) and use private archive.

If you chose to compile a program from source and to install them under "/usr/local" instead, you may need to use equivs as a last resort to satisfy the missing package dependency.

Package: equivs
Priority: extra
Section: admin
Description: Circumventing Debian package dependencies
 This is a dummy package which can be used to create Debian
 packages, which only contain dependency information.

For partial upgrades of the stable system, rebuilding a package within its environment using the source package is desirable. This avoids massive package upgrades due to their dependencies.

Add the following entries to the "/etc/apt/sources.list" of a stable system.

deb-src http://deb.debian.org/debian unstable  main contrib non-free

Install required packages for the compilation and download the source package as the following.

# apt-get update
# apt-get dist-upgrade
# apt-get install fakeroot devscripts build-essential
$ apt-get build-dep foo
$ apt-get source foo
$ cd foo*

Update some tool chain packages such as dpkg, and debhelper from the backport packages if they are required for the backporting.

Execute the following.

$ dch -i

Bump package version, e.g. one appended with "+bp1" in "debian/changelog"

Build packages and install them to the system as the following.

$ debuild
$ cd ..
# debi foo*.changes

You can make a local copy of the package and debconf selection states by the following.

# dpkg --get-selections '*' > selection.dpkg
# debconf-get-selections    > selection.debconf

Here, "*" makes "selection.dpkg" to include package entries for "purge" too.

You can transfer these 2 files to another computer, and install there with the following.

# dselect update
# debconf-set-selections < myselection.debconf
# dpkg --set-selections  < myselection.dpkg
# apt-get -u dselect-upgrade    # or dselect install

If you are thinking about managing many servers in a cluster with practically the same configuration, you should consider to use specialized package such as fai to manage the whole system.

alien(1) enables the conversion of binary packages provided in Red Hat rpm, Stampede slp, Slackware tgz, and Solaris pkg file formats into a Debian deb package. If you want to use a package from another Linux distribution than the one you have installed on your system, you can use alien to convert it from your preferred package format and install it. alien also supports LSB packages.

	Warning
	alien(1) should not be used to replace essential system packages, such as sysvinit, libc6, libpam-modules, etc. Practically, alien(1) should only be used for non-free binary-only packages which are LSB compliant or statically linked. For free softwares, you should use their source packages to make real Debian packages.

You can learn more on the package management from following documentations.