Friday, July 3, 2009

How we should address the DarkPan problem

I just finished reading nperez's blog concerning DarkPan

No, DarkPan is not Larry Wall telling you he's your father. It's a reference to the immense amount of code out there in businesses that are written using old perl practices that most modern developers stopped using 10 years ago. As a result, when we discuss putting new features into perl5, the attitude is to make no changes to the language that would break previous code.

I lost a good 10 hours of sleep at YAPC discussing DarkPan issues. Reading NPEREZ's the blog, I See the same sub issues we discussed at YAPC: CPAN, vendor dependence, fear of upgrade.

CPAN: This is it's own discussion, but I think CPAN really suffers from not being perl version aware. This is why I think it comes up as a DarkPan topic every time. Right now I can say version 1.2 of module XX supports 5.8.8 but 1.3 requires 5.10. The problem is if I use 5.8.8 perl and run 'report updates' in CPAN, it's not smart enough to say you have the most up to date copy of module XX. Instead it reports that version 1.3 is available for download and install.

Dependence on vendor perl: Most UNIX operating systems depend on perl. For this reason, it's messy to tweak the perl that comes with the OS. FINE! Compile your own.

I propose we establish a best practices policy that custom perl be compiled to /usr/local/perl/5.10 and /usr/local/perl/5.8.8, etc. I'm sorry, but it's just not that hard to compile your own and you only have to do it once. As for dependence on binaries like openssl, etc: Compile them too with prefix=/usr/local/perl/utils. Then when compiling your perl modules, make sure LD_RUN_PATH points to /usr/local/perl/utils/lib. Perl is now completely self contained.

Vendor problem solved. You've now got a handy dandy tarball you can throw around on all your hosts that doesn't get in the way of anything vendor related. This goes to the comment Schwern made at YAPC about what PHP does right: You pull it out of the box, install it, and it just works.

Fear of upgrade: How do you eat a 2 ton elephant? 1 Bite at a time. If we have standards for custom perl locations for various versions, then the upgrade can be phased in. When you've decided your script is now 5.10 capable, simply change your first line to be #!/usr/local/perl/510/bin/perl and you're now declaring the script to be 5.10 compliant.

This said, I have a hard time feeling sympathy for DarkPan (my own company included) whining that they have to fix their code if they want their 5.8.3 code to work on 5.10. How is this different from any other language or platform? Off the top of my head, I believe this is a problem in .net, java, IBM WebSphere, etc. Most of these platforms would tell you if you complained: "You want the new features in the new version of the language? You're going to have to upgrade your code"

Modern Perl:
Perl5 does a bad job encouraging good programming practices. While at YAPC, I saw this topic come up in many forms: Modern Perl, perl5i, DarkPan, etc. There's alotta frustration with perl 5 because as we release new versions of perl5, we continue to allow all of the old sloppy programming practices.

My take:
1. Enable strict/warnings by default.
2. add warnings when &subname, is called and tell the coder that their perl4 code needs to be upgraded to use subname();
3. turn on frickin say/switch, etc by default
4. I'm sure there's more.

If the DarkPan code breaks, tell em to go put 1 line of code in their scripts: use sloppyperl;, which would effectively revert the language without requiring a re-code. If all these DarkPan users are so eager to upgrade, make em sweat a little by forcing them to add that one line of code to their scripts.


  1. Hmm, &subname is still valid (it is roughly equivalent to subname(@_), I would say warn when you see &subname(1, 2, 3), but even that has its uses (it avoids prototype checking).

  2. The people I know who are just dabbling or copying from someone else's example invoke subs with no params "&subname", completely unaware they're calling subname(@_). This leads to unexpected behavior sometimes and the author often has no clue what they did wrong.

  3. One thing that I think is often overlooked when the back-compat decisions are criticized is that different changes have different impacts. Deprecating and dieing while introducing a newer, better version that has died before is completely different from simply changing behaviour, or introducing something new which might already be there.

  4. Honestly, I guess I don't see the point of providing an excess of backward compatibility. If you don't like the new features in 5.10, why can't you stay on 5.8? See my comments about custom perl installs

  5. If there were compile time options to make new and incompatible features present or absent by default (e.g. use strict), then those building perl could build it for compatibility or for new features according to the needs of their users, the conflict would be resolved and there would be no need to impose one decision on everyone.