(LinuxWorld) — It is time for a radically new approach to software installation for Linux. Starting with this article, I will examine the problems with existing approaches to software installation, look at the technical issues, look at some of the alternatives, and propose a framework for a solution. Let me be clear that this is not an attempt to evaluate existing package-managers. I most often recommend Red Hat Linux, I like Mandrake and Knoppix, I use Debian, and I'm about to try Gentoo, so I'm not here to push one package-format or distribution-type over another. I may like one more than another, but none of them make it sufficiently easy and error-free to access, install and upgrade software.

The first thing we need to do is examine the issues from a realistic perspective.

Here are some of the factors that I believe should determine how one approaches the problem of installing software on Linux:

• Administration and troubleshooting are very expensive
• Any given application may share files with other applications
• Systems don't necessarily store shared files in the same places
• People often install the latest versions of software
• Unofficial versions of software are often required
• RAM is cheap
• Disk storage is cheap
• CPU power is cheap
• Broadband access, where available, is (relatively) cheap
• CDs are cheaper still

The money pit

I learned a lesson early in my career as a programmer. I was assigned the task of designing the graphics portion of a real-time digital signal processing system shortly after I landed my first job. I stopped one of the managers in the hallway to discuss the possible hardware options. At some point during the conversation, I asked if it would be presumptuous of me to order a $150 graphics board to see if it was appropriate for the box we were building. I was afraid to spend the company's money, even if I had the right to do so, because someone might view it as a waste. He pointed out that by discussing the issue, we had already spent that much in terms of our salaries.

Being an entry-level programmer with a pitiful salary, I have no doubt his portion of the money comprised most of the amount we spent discussing graphics. But I got the point. Time is not only money, it is often more money than hardware or software. What I don't understand is why, in view of the decreasing costs of hardware and software, others don't get this point with respect to Linux distributions and software installation.

It is sometimes very expensive to install software — most expensive when the installation suffers from dependency problems or causes dependency conflicts. These arise out of the phenomenon described in my second bullet point: Any given application may share files with other applications. Two applications may use the same shared library, but not necessarily the same version. The version of the library may exist on your system, but the software-installation process doesn't know it exists because the installation program is looking in the wrong directories or because the library is not registered in the database of installed software.

Here's how missing dependency and dependency conflicts usually surface. I will use purely hypothetical examples in order to avoid singling out a distribution or package system. This list does not represent every possible problem, nor does it list them in any order of probability.

• You attempt to install a word processor and the installation fails, because you haven't installed the shared library that imports Microsoft Word documents.
• You install a word processor successfully, but the word processor does not run.

Among the possible causes:

• You have the wrong version of a library or configuration file.
• The installation failed to detect that a needed library or configuration file is missing.
• You have the right version of a library, but for any of several reasons, the application can't find it.

Assuming you discover that the problem is a missing library, your attempt to resolve the problem can lead to more problems:

• You attempt to install the missing shared library, and it complains that you don't have the latest XML library installed.
• You install the XML library successfully, but it breaks compatibility with your spreadsheet application.
• You install the correct library, but the installation program puts it in a location where many applications that need it won't find it. Thus, the installation fails to update the system-configuration to look there.
• You allow the application to update or resolve a configuration file conflict, but it mangles the configuration into something useless or modifies it in such a way that it breaks other applications.

The packaging approach

The traditional solution to the above problems is to organize and categorize software in packages, then manage the packages according to how they depend upon one another. Package-management systems such as RPM or Debian apt-get/dpkg rarely include all the files the application needs, but the packages and packaging system keep track of the interdependencies and the software you have installed. This way, either a clever program or clever operator can get the application installed and working.

Here are some of the packaging advantages and disadvantages. I'll list the advantages without comment. The intent of this series is to arrive at a better solution. We will want to sustain as many advantages as possible, but our focus will be primarily on the disadvantages of the existing methods we'll have to address with any proposed new method of installation.

Advantages

• When it works, it's reliable
• Categorization helps users select software
• Official packages usually interoperate flawlessly
• Even command-line installations and upgrades are easy

Disadvantages

SUBHEAD2: Interdependencies are package-centric instead of dependency-centric

In most cases, the interdependencies are defined by the package, not by the specific files that an application may need. To use the aforementioned example of the word processor, the word-processor package might include the names of all of the other packages that must be installed before the word processor will work. The word processor may need something as specific as the libsomething.so.5.0 library, but the dependency is recorded by the name of the package that supplies the libsomething.so.5.0 library. That package may also supply many other files the word processor does not use or need.

In practice, this approach is reliable, but only as long as you get your packages from official or approved sources. For example, if you use a stable version of Debian, Red Hat, Mandrake, SuSE or any other distribution, and you only install packages intended for these distributions — preferably packages created and managed by the distributors themselves — you should rarely, if ever, run into an unresolved dependencies or conflicts.

SUBHEAD2: Official packages aren't always available

Unfortunately, there are two items on our initial bulleted list of factors that make this a difficult proposition. First, people often install the latest versions of software whether or not you tell them to do otherwise. Fortunately, a good administrator with the power to enforce policy can prevent this problem by restricting the rights of users on their own machines. This doesn't help if you're not in a corporate environment, but it is an option for some.

The next problem is a lot more difficult to solve. Sometimes you have no choice but to install a specific application or a new version of an application that the distribution does not yet officially support. When this is the case, you have to find that software from some other source.

SUBHEAD2: Custom-compiled software can break software and confuse the packaging system

If it is open source, you can download the source for that package and compile it. This can be the best solution in some cases, but it has several potential traps. It can be a lengthy and difficult process to download the application and every bit of related software it needs. There is also no guarantee that the application will build properly with the compilers and toolkits you happen to have on your system.

Unexperienced programmers run the risk of compiling and installing shared libraries over the official versions of the same libraries that come with your distribution. That could improve other applications or it could break other applications, but you may find out the worst after it's too late.

Finally, anything you compile and install on your own will not be recorded in the package database. It won't matter that you already have custom versions of all the necessary dependencies for the next application you want to install. It won't matter that these libraries actually work better with your applications from official packages. When you attempt to install an official package, the package manager will complain that these shared files don't exist because they are not listed in the package database. You need to know one of the ways you can bypass the dependency requirements or you'll end up overwriting your custom libraries with the official ones. Again, sometimes these can be good things, sometimes bad, but the unpredictability of the outcome is a serious concern. More important, the potentially heavy requirement for troubleshooting violates our primary concern: to minimize the time invested.

SUBHEAD2: Anything other than official packages can introduce problems

Another alternative is to find an unofficial, pre-compiled binary package of the application you need. This can also introduce difficult-to-solve dependency problems and conflicts. In many cases, the unofficial package depends on other unofficial packages. Sometimes those other packages come from another unofficial source. Good luck to you if the person who created the application you need didn't bother to document where to find those other unofficial packages.

Sometimes, clever package-managers simply add to the difficulty by trying to be smart about resolving conflicts. Take the above hypothetical situation, which is a real problem: an XML library works with a word processor but breaks a spreadsheet. Some package-managers or installers are smart enough to prevent you from creating the conflict, so they automatically uninstall the spreadsheet when you install the correct XML library for the word processor. This is obviously not an acceptable solution for anyone who needs both the word processor and the spreadsheet.

SUBHEAD2: Dependency requirements are sometimes perceived but not real

It's a shame that a variety of the conflicts you may encounter may only be perceived problems. Perceived problems occur when the package-dependency information or installation process is arbitrarily or even accidentally restrictive. For example, the person who created the word-processor package may assume that it needs version 1.0.5 or later of the XML library. It is possible that the application will work just fine with version 1.0.4, which you may have installed on your system, but there's no way for you to know that in advance.

Worse, you may have the compatible version 1.0.5a of the XML library installed, but the package manager incorrectly identifies this as an earlier, rather than later, version of the package. In both cases, neither you nor the package-manager have any way to know that the existing libraries are actually compatible with the application. If you knew the problems were only perceived problems, you might use an option that forces the package-manager to ignore any conflicts. If we could all see into the future, this might be useful. In most situations, it is unwise to force an installation to proceed when it suspects there will be problems.

SUBHEAD2: Not well-suited to incremental upgrades, must install an entire new package instead of a subset of upgraded files

This is probably the least significant problem for package-management, but it is worth mentioning. It may be possible to do so, but I have never seen an upgrade package that includes only the files you need to go from one version to the next. If you want to upgrade your word processor from version 1.0.1 to version 1.0.1a, you generally have to download the entire package for 1.0.1a and install everything... even if only a few of the included files have changed. This is a waste, but it is usually easy to live with given that CD-ROMs are cheap and downloading is usually cheap. Those with dial-up connections are likely to complain the most, and rightly so.

In Part 2, we'll look at the specific issues, such as the problem of shared libraries, file location and configuration files, in more detail.