Python import improvements

[originally posted to comp.lang.python — but worth repeating]

Ruby has been getting pummeled for the last year or more on the performance subject. They’ve been working hard at improving it. From my arm chair perspective Python is sitting on it’s laurels and not taking this as seriously as it probably should. In general it’s possible to make many comments that swirl around religion and approach, one of the things that I’ve noticed is that wile Python has a much more disciplined developer community they’re not using this for the greater good.

Specifically, in looking at a benchmark that was posted, python was an order of magnitude (20 secs vs. 1 sec) slower than ruby. In investigating the performance bottleneck is the random module. This got me to think, why does python have “pickle” and “cPickle”? Comes down to lowest common denominator, or the inability for somebody to write an optimized package that can mimic a python package.

To that end why would somebody write big try catch blocks to see if modules exist and if they exist alias their names. Wouldn’t it be better if there was a way that if I have an “interface compatible” native (aka C) module that has better performance that there could be a way that python would give it preference.

e.g.

1import random(version=1.2, lang=c)

or

1# use the python version by default
2import random(version=1.2, lang=py)   

or

1#  use the latest version in the "fastest" code (C given preference)
2import random     

where there could be a nice set of “standard” key value pairs that could provide addtional hints as to what language and version of a library was to be used.

**

Notes from the comp.lang.python discussion**

People fell into a few camps:

• There’s many “optimization” projects, like PyPy or ShedSkin (compile Python to C/C++ code).  In essence JIT the code, though not as passive as a true JIT.
• “most bottlenecks are from IO (disk, network) or interaction” so worrying about library performance is not worthwhile.
• Why worry about it, what we’ve got is good enough.

What amazes me is that time and time again, people don’t realize who their customers’ are.  Languages like PHP realize their customers are hack VB programmers looking to build web pages.  So, having solid libraries and other things that can be totally abused (used) is a good use of “higher order design”.

In general, systems should support progressive improvement.  Somehow I fundamentally belive the python community doesn’t grasp that, but belive they’re building for themselves, not tools that enable a different class of engineer.