Python's feature set is basically what's easy to do in a naive interpreter. Everything is a dictionary. Anything can be changed from anywhere. With "getattr", you can patch one thread from another. It's elegant, and very difficult to speed up. Google tried, with von Rossum on board. Their "Unladen Swallow" JIT compiler project crashed and burned.
The PyPy group has made a fast Python compiler/interpreter/JIT system. It's really hard. The initial funding from the European Union got them started, but wasn't enough. They really try to handle all the hard cases. This requires two interpreters and a JIT. They have to handle "oh no, someone patched object A from thread R, invalidating code that's running in thread S". There's a "backup interpreter" that kicks in for hard cases, and once control is out of the area in trouble, the JIT can recompile it. (This is an old and oversimplified description.)
This transactional memory thing is very clever. It has to separate things at run time that probably should have been separated at compile time, of course. It's impressive that they can get it to work. It's a lot like how a superscalar CPU works, including transaction commit and backup at the retirement unit.
Python gets into this mess because, like C and C++, the language doesn't really know about concurrency. (Threads came late to UNIX, and C predates threads. So C has an excuse for backing into concurrency.) Python has the C model of concurrency; at the user level, it's treated as a library issue. Internally, though, it needs a lot of locking, because there's so much mutable state in the interpreter.
It would be a lot easier if the language were restricted a little. But then It Wouldn't Be Python(tm). The price of this is huge complexity layered on a simple model, and probably years of obscure bugs in PyPy.
> Python has the C model of concurrency; at the user level, it's treated as a library issue.
Other than Erlang, which modern language doesn't?
Java has e.g. "synchronized" keyword, but I regard it as merely syntactic sugar over a standard library implementing thread - semantically, it still basically has the C model.
Go has channels and stuff - but it's still library level (in fact, it's basically syntactic sugar over the Plan9/Inferno/Aleph/DontRememberName standard library primitives)
There were other OS with threads and co-routines being explored as design while UNIX was being developed at AT&T.
As for the rest I agree with you.
Personally I don't have any use for Python besides the occasional shell script, but as a user of applications written in Python I would like them to perform fast.
PyPy does not really try to do that. CPython is great for a lot of things, small, easy to install on a myriad of platforms and is, after all, Python. PyPy instead tries to push what's possible to do with Python as a language and with dynamic language interpreters. You can do real time image processing using only Python with PyPy, now STM, fast numerics in the future etc. The goal is to expand Python ecosystem, not to "replace" CPython.
I don't think this is applicable to python. Only so many developers can work on cpython without stepping on toes. This led to the decision not to have cpython to be an implementation standard, not an experimental language, while pypy led the effort for expanding with a different team of developers. This is a good decision, because there are still more than enough developers to maintain a stable, full-featured cpython that pypy can conform to (to varying degrees). There aren't any competing standard libraries, for instance, and it's made very clear which libraries work on which implementations.
Maybe if there were competing implementations to ruby, it would still be popular outside the rails community. But it seems as if the vast majority of ruby core developers work on the mainline implementation, or forks thereof.
Generally speaking, for a language like Python, having only one implementation is considered a bad sign. Anything serious has multiple implementations.
CPython is hurting the ecosystem by being slow. I'll let you in on a little secret. People used to using C, C++, D even Java and C# laugh at the slowness of languages like Python and Ruby. Blah, blah, blah productivity gains. We laugh. We think, "I could write this in Java and go to production with three servers or I could write it in Python and use eleven."
People don't like to say it to your face, but Python is often derided and considered to be pretty silly. Oh, and its syntax is like Lego. Neat at first until you want something like a multiline lambda.
In most cases I would rather pay for 7 more servers than for the 10,000 extra lines of code your C++ or Java implementation is going to cost me. Hardware is cheap. Engineers are expensive.
C, C++, D, Java and C# are hurting the ecosystem by being slow. I'll let you in on a little secret. People used to coding assembly laugh at the slowness of languages like C and C++. Blah, blah, blah productivity gains. We laugh. We think, "I could write this in assembly and go to production on an Arduino or I could write it in C and have to use expensive servers."
People don't like to say it to your face, but C is often derided and considered to be pretty silly. Oh, and its syntax is like Lego. Neat at first until you want to understand what something is a pointer to.
The difference is that you can achieve similar performance to assembly by writing in those other languages, while you cannot achieve anything close to similar performance with Python.
It used to be the assumption that there was a direct tradeoff between performance and convenience/productivity of a programming language. I think newer languages are showing that this doesn't have to be true, at least not to nearly the same degree.
It was warranted. CPython needs to be derided and PyPy needs to keep up the good work until Python isn't unreasonably slow. Python should be humiliated into being fast or just go away.
