At one point, I started writing a 3D game engine in Python, with the understanding that I'd profile and if necessary re-write the necessary parts.
I've also used Panda3D, which has a lot of Python-wrapped C++ code.
Now, I don't have specific benchmarks to point to, but it's important not "prematurely pessimize", just as it's important not to optimize prematurely. In high-performance 3D games, you'd switch compilers for say a 10% gain in optimized code performance.
Ever since I've become aware of how deep the GIL problems go, I can't really justify starting to write an engine in Python, if I'm just going to have to re-write critical and error-prone parts in C++ anyway.
There's a more general principle here that applies to languages and runtimes, as well as library design. You want to be able to get started with a new component or language / runtime easily, trading ease of use for performance, BUT as the project progresses you want a way to be sure that you could beat-down or recover from any problems in a way that doesn't essentially require a re-write.
These GIL issues remove that confidence in Python, for me. It's not a win if I can only write the easy parts of my code in Python. I specifically want to use Python to write the hard parts of my code.
I agree that there doesn't seem to be an easy, or even moderately difficult solution to this problem. That makes it even worse from the point of view of advocating Python use on a project.
To summarize, it's reasonable to say "don't optimize until you know there's a problem", but you can come at it the other way from a risk-management perspective. "What is the risk that there's some dead-end due to this language / runtime that we just won't be able to overcome (because the smartest dudes who know the language inside-out, can't fix it either)."
It's sad, because I'm a huge fan of higher-level languages, but even dealing with other people's broken or retarded bindings to C or C++ libraries is a huge time sink.
At a certain point it just becomes more productive to write in C++, never mind the FUD about programmer productivity.
That said, for my current project I'm using Unity, and writing script code in Javascript and C#, but this is more about having a toolset that Just Works out of the box. It's not clear that I would choose that approach if I had to write it from scratch.
I've also used Panda3D, which has a lot of Python-wrapped C++ code.
Now, I don't have specific benchmarks to point to, but it's important not "prematurely pessimize", just as it's important not to optimize prematurely. In high-performance 3D games, you'd switch compilers for say a 10% gain in optimized code performance.
Ever since I've become aware of how deep the GIL problems go, I can't really justify starting to write an engine in Python, if I'm just going to have to re-write critical and error-prone parts in C++ anyway.
There's a more general principle here that applies to languages and runtimes, as well as library design. You want to be able to get started with a new component or language / runtime easily, trading ease of use for performance, BUT as the project progresses you want a way to be sure that you could beat-down or recover from any problems in a way that doesn't essentially require a re-write.
These GIL issues remove that confidence in Python, for me. It's not a win if I can only write the easy parts of my code in Python. I specifically want to use Python to write the hard parts of my code.
I agree that there doesn't seem to be an easy, or even moderately difficult solution to this problem. That makes it even worse from the point of view of advocating Python use on a project.
To summarize, it's reasonable to say "don't optimize until you know there's a problem", but you can come at it the other way from a risk-management perspective. "What is the risk that there's some dead-end due to this language / runtime that we just won't be able to overcome (because the smartest dudes who know the language inside-out, can't fix it either)."