Comments on: Evils of Source

By: .:: CentOS VN ::. - Linux Howtos and Tutorials | Jim Perrin: Evils of Source | Centosvn.com

Sat, 21 Feb 2009 03:43:50 +0000

[...] It’s a near daily occurrence that I see an administrator building something from source because either they fail to understand how backporting works, or they want to install software which isn’t packaged in the default repositories. Source installs used to be common, but the problems inherent in them drove the creation of package management software. Since you have package management software, USE IT! I cannot stress this enough. Source installs suck for the following reasons: They conflict with packaged installs of software. They do not inform the package management system that they exist, which leads back to #1. They generally leave no easy way to uninstall them later on. There’s no record of what an installation does to your system, or what files are placed where. Searching them becomes a pain (what files did that install of openssl drop on my system?) They are not easily duplicated across multiple systems over time (think remote support for users) There is no distribution update mechanism. If you updated via source for security reasons then congratulations, the onus is now on you to maintain that install, and track product security for its lifetime on your server. The benefits of using package management: Reliable reproduction of installs over time, and across multiple systems Easily search for installs, files, and modifications of packages. Easily get package meta-data such as who built it, when, build options used, and scripts required to set up the package for use. Audit trails for allow you to know what files have changed since install, what permissions were modified, etc. Simplified install, update and removal of packages. No hunting for remnants of previous installs. Simplified dependency management and version tracking. Distribution supplied updates. Rather than 1 admin tracking a dozen packages for updates, you have dozens of builders and hundreds of users testing and tracking the software. Just about every distribution out there has some form of support structure, either via irc, email, forums, or carrier pidgeon. In many cases this community will be able to point you to a location where you can find packages for the software you’re wanting to install. In the case of CentOS, there are a number of very good 3rd party repositories providing quite literally thousands of packages which are not included in the base distribution. By using your distribution’s support mechanisms, you can provide a voice for the users and potentially guide the development efforts for future software releases. In short, if you need a particular piece of software, please use your distribution’s package management system and if you cannot find a pre-existing package, ASK. If one doesn’t exist, and won’t be created, build a package yourself (there are tutorials aplenty online) and submit it for inclusion. Source installs should only be done when all other options have been exhausted. Update : Well, since this one seems to have generated some interesting responses, I thought I’d post a couple links that I’d neglected to mention. Thanks to toracat for pointing these out to me, since they fit in perfectly with what we’ve discussed here. http://wiki.centos.org/PackageManagement/SourceInstalls http://www.centos.org/modules/newbb/viewtopic.php?topic_id=14408&forum=47 Continued here: Jim Perrin: Evils of Source [...]

By: Dimitar

Dimitar — Mon, 12 Jan 2009 08:17:21 +0000

I built emacs 22.3 myself but I used ./configure –with-x-toolkit=motif –prefix=/opt/emacs/
because:
a.) the gtk version has some very weird bugs and probably conflicts with metacity
b.) when its in /opt/emacs/ all it takes to remove it is “rm -r /opt/emacs/”

Is using a safe “–prefix=/dir/” OK?
Also is there a way I can install more conflicting software using the package management system in other directories?

Thanks.

By: Gerard Braad

Gerard Braad — Sun, 11 Jan 2009 18:59:50 +0000

For production systems I totally agree with point of view. Else I would have also been unable to perform a franken-upgrade: http://blog.gbraad.nl/2009/01/wizard-of-yum-upgrade-from-fc3-to.html. Building from source also means you need to plan… if you would build with the standard prefix, your files might start to conflict with packages or other files. You can not easily uninstall the binaries, etc. Preferably, if you really need to build from source, you would need to create a package… which you can then also distribute for other server. This is at least how we do it for certain unavailable packages. Try to keep your server as much as possible to the mainline… or as I would call it ‘the yellow brick road’.

By: Matthew Berg

Matthew Berg — Thu, 08 Jan 2009 19:28:49 +0000

Andrew – more often than not GUI tools are split out into seperate packages; e.g. vim-minimal vs vim-X11, wireshark vs wireshark-gnome. Likewise optional modules are often packaged seperately, as with php.

Yes, there are plenty of badly built packages out there in third party repositories. On the other hand it is almost always easier to grab the SRPM and %exlcude a binary you don’t want or add a new flag to the %configure. And when a new version comes out it’s often as easy as changing a single tag and running a rebuild, as opposed to digging through your notes or shell history trying to figure out what compile options you used the last time.

Seriously, running with –force or –nodeps is almost universally a mistake. In almost a decade of administration of (at peak) several hundred peaks I can count the number of times I’ve used those flags on my fingers and could tell you exactly why I did it and how I was avoiding inconsistency in my RPM database.

By: Jim Perrin

Jim Perrin — Sat, 03 Jan 2009 18:17:44 +0000

You’re exactly the type of user who gets caught on the sidelines during the debate of packaging workload vs package use. Essentially you’re relegated to the one of the last lines I included in the article. In your instance, a source build is a last resort, and in your case you’d be right to do so. Ideally, you’d have enough free time to create a repository of photo software packages to share with others. In the real world you may not have the time or the resources to do so, and for you a source build is likely the only way.

Assuming you plan them out and know what you’re doing with a source build they shouldn’t interfere since no one is packaging that software anyway. It’s when you start overwriting the distribution packages with your own brew that things get interesting.

As to you gentoo comments, I actually consider what gentoo does ‘package management’ because they have a command structure similar to the bsd ‘ports’ system. It’s not quite as robust as RPM, but it also doesn’t restrict you in the manner rpm does also. In the end, it’s all about finding the right tool for the job, and learning to use that tool appropriately. I simply wish that more people would use the tools provided in the distribution, rather than ignoring them.

By: Ted Miller

Ted Miller — Sat, 03 Jan 2009 03:23:18 +0000

But what about when I need a whole set of packages, none of which are available on any of the repos, I don’t know how to build an .rpm file from scratch, and don’t have a year of spare time to learn how?

The lack of serious photo packages like cinepaint, hugin, krita, ufraw and kphotoalbum drove me run virtual machines in VMWare to get around what I could not persuade Centos to do. I tried Fedora, but it only had some of what I wanted, so I ended up with a virtual install of Gentoo (which seems to be the ultimate example of the oxymoron of a well-managed, built-from-source distro). So far so good for my need, but I have not had it running long enough to declare it a success.

By: Ned

Ned — Fri, 02 Jan 2009 19:23:13 +0000

I think Bill touched on one of the key issues above – planning. A package management system allows one to plan and manage deployments effectively whereas source installs have little if no planning/management structure.

It’s simply more cost/time effective to spend the time upfront building a spec/package than to spend the time later dealing with snafu’s caused by source installs. Even if you keep impeccable records, what happens when you move on and the next System Admin has to try to pick up the pieces and maintain that system?

As Jim says – the package management system works, so use it!

By: Jim Perrin

Jim Perrin — Fri, 02 Jan 2009 17:46:37 +0000

Okay, let’s talk compile-time options. Why build from source when you can simply fetch the src.rpm, and modify the spec to build as you see fit? I know several folks that do this, and infact there’s a comment already about doing just that from Bill. He describes what I consider to be just about exactly the proper method for making the modifications you’re talking about. Doing so, you can still use the system package management for security audits, file verification, install, update and removal. Keep in mind that this is about package management in general. Nearly every major distribution on the planet has gone the way of package management for a reason. Using it appropriately rather than fighting with it has saved me (and probably countless other admins) loads of time and frustration. If you choose to do things differently and that method works for you, then that’s fine. It’s good to get a dissenting voice every now and then.

By: Bill

Bill — Fri, 02 Jan 2009 17:28:40 +0000

I think the idea of compiling from source should be a last resort. Usually I compile from source by hand – but only when I’m trying to build/fix a spec file. EG, to see what is wrong and fix it!

For things like apache – that is what dynamic module support is for.

For perl, well – perl is messed up on redhat anyway :-> ( see here as an example, http://linux.slashdot.org/linux/08/08/29/1423201.shtml ). The solution here should not be to roll your own perl dist but rather to take the default .spec file and adjust it to your needing. Then submit a patch to redhat/other explaining what you did and why. Try to get the upstream to fix it.

Once you have that updated spec file you can rpmbuild to your hearts content on each arch and deploy it. Install perl into /usr/local or /opt/perl or whatever. Make your code use that local install until the upstream fixes the problem.

Done…

This does take a bit of planning and such, but it makes for a more repeatable environment. What happens if that box suddenly pukes? Backups are gone, assume all worst cases. Now you have to remember just how you got around that problem 6 months ago on top of all the other crap associated with that box!

By: Andrew Crawford

Andrew Crawford — Fri, 02 Jan 2009 17:21:23 +0000

I note that you ignored the issue about compile-time options, which was really my main point.

However, to respond to your point about dependencies due to features that some users might need, I think you’ve actually highlighted one of the weaknesses of package management. Maybe (for example) X11 really IS a dependency for some GUI tool that comes with a package. But I know that I will never run that tool on this machine. And I’m sure as heck not going to install X11 just to satisfy the dependency. Building from source is the least-worst option.