People Hate Perl – OK, not a blog about php, but 😉
Perl is a scripting language originally written by Larry Wall and still maintained by him, though at this point it has thousands of developers all around the world, just like virtually any successful FLOSS project.
It is a critical component of virtually and GNU operating system, the autoconf tool that is at the heart of virtually every portable code project relies upon Perl.
I hate it. It isn’t so much the language itself that I hate, it’s the damn CPAN bloated module infrastructure that I hate. Virtually every little function is it’s own module that depends upon other modules that depend upon other modules and so on.
The model that the Perl developers have is that you use their cpan tool to install what you need and it grabs everything and manages for you. Unfortunately that model does not work well with a modern FLOSS operating system, one of the advantages of a FLOSS operating system is you can have universal package management that takes care of things like automated dependency resolution. Installing perl modules via the cpan binary, the package manager does not know about them. For cpan modules with binary dependencies, cpan requires that you have the binary dependency including its header files already installed or it can’t build you the module.
This blog is about the extent of the dependency bloat the Perl has. It’s really bad.
Right now I am working on building my own Linux distribution. I like CentOS but a lot of the software in CentOS is rather dated. That’s fine for servers, but not for the desktop. Fedora on the other hand does a lot of things in a way I really don’t like, especially GNOME 3. I could blog forever about my gripes with GNOME3 – the list is very long.
The source is open though, so use it, damnit. I’m currently building my own distribution for the purpose of doing things the way I think they ought to be done. Perl is an essential part of any Linux system, hate it or not you gotta have it. While the vast majority of my RPMs I am writing from scratch, there are just too many perl modules. I certainly do not need them all, but Perl likes to have a dozen plus different modules for the same functionallity that is often within a single library binary library package in C/C++ land.
So I am doing perl in a way that allows me to just take Fedora perl module source RPMs and rebuild them in YellowJacket GNU/Linux. My layout is a bit different, I but binary modules in /usr/lib{32,64}/perl5 and noarch modules in /usr/lib/perl5. Fedora just uses /usr/lib for 32-bit libraries, they don’t have a /usr/lib32, so it puts binary modules in /usr/lib{,64}/perl5 and noarch modules in /usr/share/perl5 – but the different layout is handled automagically by macros in the RPM spec file at build time.
My spec file for perl results in a total of 65 packages (originally 66, soon 66 again), it has to be split up into that many packages because some of the modules have necessary updates that in CPAN and need to upgrade the module as it was distributed in the perl tarball.
Most source tarballs when compiled and packaged in an RPM result in 2 or 3 packages. one for the executables, one for development files (headers and such), and if it has shared libraries, one for the shared libraries.
My system was just about finished being RPM bootstrapped, meaning it was a minimal system where every piece of software installed from source had been replaced by packaged RPM versions of the software. OpenSSL was all that was left. At that time, there were 65 perl RPMs installed from the single source perl RPM.
OpenSSL installed a script that required the perl(WWW::Curl) module, a module not provided by the core perl system.
Building that module requires the perl(Test::Base) module to test the build of the module. It is very important to run the test suite, if there is a problem in the package, finding it by running the test suite can save you a lot of headaches.
Building perl(Test::Base) required a several other modules, which in turn required several other modules, which in turn required several other modules, and so on.
Right now, looking in my SRPM directory for perm modules I have built in pursuit of getting perl(Test::Base) so that I can build perl(WWW::Curl) there are currently 114 different source RPMs. One hundred and fucking fourteen.
Some of that may be Fedora’s fault. For example, one perl module wanted the Valgrind perl module to build for its test suite. My system is not yet ready for Valgrind, Valgrind has special instructions on how certain libraries are to be stripped, and to be honest, I’m not really that worried about every having Valgrind installed. I know in that particular case, Valgrind really wasn’t necessary for the test suite, so I modified the spec file to not require it. But most of the dependencies are in fact genuine.
In fairness, a users system won’t need all those modules installed to use perl(WWW::Curl), quite a few of those modules (I’m guessing more than half) are only needed to benefit the automated testing of the module after it is built. But it still is fucking way too much.
In the rest of the software world, something like Dejagnu is used for the test suite. Dejagnu requires Expect which requires Tcl. That’s two dependencies the test suite has. Perl on the other hand doesn’t seem to have any kind of standardized test suite, the Perl developers need to write one.
Perl needs a standardized test suite in a single tarball and perl modules that need something else for the test better have a damn good reason to do so.
And enough of this 20 different but closely related modules. That’s just bull shit.
Perl gurus pride themselves on their perl one-liners, how compactly they can do something with Perl. They need to apply that same philosophy to CPAN because right now it is a big fucking bloated mess. The result, I really hate perl.
I guess on the bright side, having built and installed that many modules going through the test process for each one successfully, I guess it’s a good indication my Perl install is working properly…
yellowjacketgnu 
Seriously? Your complaint is that Perl has too many useful, maintained, decoupled modules, provided free-of-charge for your use? Well at least every other language hasn’t tried to emulate CPAN, that would really mess you up. And what is this you say? After all this unnecessary work you’re doing, the system still functions correctly. Boy yeah, you’re life is tough. C’mon man, this post is really ridiculous.
Not that it has too many, but that there is no real oversight to the process. To properly build and test, A requires B which requires C which requires A again.
And what should be one single tarball of closely related functions with a single maintainer is often 20 different tarballs with 12 different maintainers – and that’s what results in Perl’s dependency bloat, because those 12 different maintainers have different ideas about how something should be done so they end up requiring a lot of different modules all to do similar related things.
Before trying to install perl(WWW::Curl) I had 65 different perl RPM packages installed, all from the core perl tarball. Would have had 66 but CPANPLUS had a couple dependencies on modules not in core perl (e.g. DBD::SQLite). Now there are 181 different Perl RPM packages installed, all just so I could properly build and test perl(WWW::Curl).
And that number is actually low because to deal with the circular dependencies, I had to skip some dependencies meaning that the test suites did not fully run. I have to rebuild them all now without that bootstrap skip so I can investigate any failings. And I know that will pull in some additional module dependencies.
An example is aspell and hunspell. Yes, I had to build and install friggin spell checkers to satisfy dependencies needed to build and test the necessary dependencies of Test::Base
Some of the spell stuff I had to install wanted aspell and some wanted hunspell. That’s the kind of redundant dependency bloat you get from a poorly managed project.
It is shortcomming of your disto, not shortcomming of perl/CPAN. Debian has dh-make-perl that packages CPAN modules as debs that could be tracked and managed by dpkg/apt. There was CPAN before all linux package managers existed, so yes, it has no idea about distro package managers. Looking the other popular scripting languages I see no difference, I do not see how pyhton’s pip/easy_setup ot ruby gems or nodejs npm are integrated with distro package managers.
On other side, using system’s perl is just one option. I prefere to install separate perl and use it for development and production code. You could look for perlbrew in order to manage multiple perl versions/environments on the same host. If you do not like the cpan program there are cpanm and cpanp that are great.
My distro is Linux From Scratch being RPM bootstrapped.
Yes I know there is cpan2rpm but try it, try it on a LFS system and see what happens.
For that matter, build an LFS system, install dpkg, bootstrap your system in dpkg, and watch what happens when you try you cpan-dpkg solution. See what happens. I know what will happen.
I do have a question, how do those automated cpan2{rpm,dpkg} packages work in a proper clean chroot build environment without network access?
It sounds to me like your main problem is the mismatch between CPAN and RPM package management. And, while I agree that having one place to manage packages is great, you’ve creating nonstandard RPM’s that I suspect won’t ever be updated, so I’m not convinced you’re actually going to come out ahead.
The rest of your gripes come from the lack of a central authority in the free software/open source world. But that’s hardly Perl specific. Python and Java have the same proliferation of useful modules ( http://martijndashorst.com/blog/2009/05/19/downloading-the-internet-with-maven-in-a-new-light/ ). Why doesn’t somebody tell the automake guys and the CMake guys to merge their projects? Why doesn’t Linus Torvalds tell Red Hat, Novell, Canonical and Debian to get over their petty differences? How long can the world live with OpenBSD, FreeBSD and DragonflyBSD? Why does PHP have several popular templating engines, especially since the language started out as a templating engine?
I, for one, am glad that there is no central person with the delusion of having control over open source/free software developers. I, for one, would never listen to him.
Actually updating is very important to me. I was very careful in my initial perl spec file to make sure my perl platform would be compatible with perl packaging by Fedora.
That allows me to monitor Fedora updates, grab updates, and rebuild them. Automatically in most cases.
Fedora uses a pm suffix on man pages, that I don’t use, so I have to run a simple sed on the spec file to fix broken spec files that assume a pm suffix
sed -i -e 's/\(\.[1-8]\)pm\*/\1*/g' ${SPEC}
(I only run that starting with first lines number %files is on and ending with first line number %changelog appears on)
There are two ways of dealing with filtering requires and provides. One that was used in RPM 4.8 and earlier, and one after.
Fedora’s RPM infrastructure defines both and fedora spec files specify both ways, instead of checking for the RPM version and using the method that matches.
sed -i -e 's/^\([[:space:]]*\)%filter_\(from_requires\)/#\1%%filter_\2/g' ${SPEC}
sed -i -e 's/^\([[:space:]]*\)%filter_\(from_provides\)/#\1%%filter_\2/g' ${SPEC}
sed -i -e 's/^\([[:space:]]*\)%filter_\(setup\)/#\1%%filter_\2/g' ${SPEC}
fixes that (I only run that from line number 1 through the first line %prep appears on)
That disables the old filtering method leaving only the new RPM 4.9+ method alone.
So assuming I have the proper BuildRequires – most fedora perl spec files rebuild just fine with those asjustments.
Sometimes other asjustments are needed, e.g. sometimes they do a man page iconv conversion to UTF-8 in the %install section instead of in %prep where it belongs, resulting in a pm suffix issue on the generated man page they are converting, but those cases are rare and I send a fix to the Fedora maintainer with the iconv moved into %prep where it belongs. We’ll see if they ever apply the fixes.
I’m impressed, then. Given the amount of work involved in creating the packages, I wasn’t expecting you to go through the trouble to create updated packages later when bug fixes are available. If you actually pull it off, my hat’s off to you.
Don’t blame Perl.
Just don’t use bad perl scripts, bad CPAN modules or bad perl scripts that use bad CPAN modules.
Use only Perl software that only use low-dependency CPAN modules, and modules available as RPM packages (example WWW::Curl is available as perl-WWW-Curl on CentOS 5)
If you’re building your own Linux Distro – you can build own packages from CPAN modules (I think you can re-use scripts from existing distro)
If you cannot create package for a module – either you don’t know Perl and the module code good enough or this module is crap or you should not build your own Distro.
Yes, I’ve written a shell script that “fixes” most Fedora/RHEL perl spec files so they build in my RPM environment. (the two most common issues are packaging mistakes by fedora packagers that break build in some none fedora environments, fortunately usually easy to fix via sed)
The real problem is the tests. Most of the dependency bloat in building the modules is dependencies needed to properly run make check to make sure the module is working properly.
It would be nice if there was a unified test platform (like dejagnu) that could be used for make test instead of dozens of individual perl modules who’s only purpose is tests, that then each bring in their own dependencies (sometimes with circular dependencies meaning you have to build without tests to install the modules needed to then build again and run the tests)
Hi. I am both a Perl developer and an OBS user and I am knowledgable about the Perl toolchain and RPM packaging. I can’t even reproduce your openssl tsget → WWW::Curl dep problem. Contrary to what you say, it does *not* depend on Test::Base. Proof: http://search.cpan.org/dist/WWW-Curl/META.yml and there isn’t any mention of Test::Base in all of the source of the WWW-Curl distribution, either. Further correction: Larry does not maintain Perl5 for many years anymore, perl5-porters do.
You wrote a rant completely devoid of any useful information – it’s just your interpretation of what you’re seeing, without naming cold hard facts that are verifiable by someone else. That makes me wonder, are you just interested in venting and badmouthing Perl or in solving problems? I’m offering you help: irc://irc.perl.org/toolchain or mailto:daxim~~AT~~cpan.org
Test::Base is in the BuildRequires for WWW::Curl in Fedora and when attempting to build the rpm without it, it failed tests resulting in a failed build. I suppose I could have skipped the %check section and probably been fine, since it passed the tests when Test::Base was installed (and Test::Base certainly isn’t required by the module itself) but Test::Base was required to successfully run the check target.
Perhaps the META.yml is not up to date?
OK – just did a test where I removed perl-Test-Base and it does build without it, so that appears to be a bogus BuildRequires that a fedora packager put in the spec file in order to pull in the real requirements. Gaw.