(originally posted on G+)
Yesterday I was just reading Armin Rigo’s post related to having Software Transactional Memory in PyPy. If you haven’t heard of Armin before, I’ll just set the context by telling you that he’s one of the most amazing developers I’ve ever met. I also appreciate the context of the post. Armin is trying to address the long standing problem of the Global Interpreter Lock in Python.
This is indeed unbelievably exciting. For many years now I see PyPy as being the real future of Python (even more when the BDFL is too busy even to format Python code properly), and this year feels so far like the PyPy year, with breakthroughs in terms of speed and much more.
What strikes me as curious in the STM post, though, is that rather than addressing the problem at its root and opening up concurrency, it sounds like the desire is to go through great lengths to preserve the GIL user-level semantics of Python.
I definitely share the feeling that whatever the code does, the interpreter itself should not just break down due to an internal inconsistency. That said, there’s a significant delta between not breaking down and attempting to preserve the user-level atomic semantics of a dictionary insert and pop, for instance.
That approach feels a little bit like wearing rubber boots every day because there’s a chance it might rain. Of course, it might actually rain, but even when it does rain the rubber boots will solve a very small part of the overall problem, and for solving that small part you’re paying the price every other day with the discomfort of wearing rubber boots. Sure, some people do use the rubber boots for good reason, but some people requiring rubber boots in specific situations does not make it a compelling reason for everybody to wear them all the time.
(also see comments on G+)