Rob Pike just wrote an article/talk that is the best background on the origins of Go yet.
It surprises me how much his considerations match my world view pre-Go, and in a sense give me a fulfilling explanation about why I got hooked into the language. I still recall sitting in a hotel years ago with Jamu Kakar while we went through the upcoming C++0x standard (now C++11) and got perplexed about how someone could think that having details such as rvalue references and move constructors into the language specification was something reasonable.
Rob also expressed again the initial surprise that developers using languages such as Python and Ruby were more often the ones willing to migrate towards Go, rather than ones using C++, with some reasonable explanations about why that is so. While I agree with his considerations, I see Python going through the same kind of issue that caused C++ to be what it is today.
Consider this excerpt from PEP 0380 as evidence:
If yielding of values is the only concern, this can be performed without much difficulty using a loop such as
for v in g: yield vHowever, if the subgenerator is to interact properly with the caller in the case of calls to send(), throw() and close(), things become considerably more difficult. As will be seen later, the necessary code is very complicated, and it is tricky to handle all the corner cases correctly.
A new syntax will be proposed to address this issue. In the simplest use cases, it will be equivalent to the above for-loop, but it will also handle the full range of generator behaviour, and allow generator code to be refactored in a simple and straightforward way.
This description has the same DNA that creates the C++ problem Rob talks about. Don’t get me wrong, I’m sure yield from will make a lot of people very happy, and that’s exactly the tricky part. It’s easy and satisfying to please a selection of users, but often that leads to isolated solutions that create new cognitive load and new corner cases that in turn lead to new requirements.
The history of generators in Python is specially telling:
- PEP 0234 [30-Jan-2001] – Iterators – Accepted
- PEP 0255 [18-May-2001] – Simple Generators – Accepted
- PEP 0288 [21-Mar-2002] – Generators Attributes and Exceptions – Withdrawn
- PEP 0289 [30-Jan-2002] – Generator Expressions – Accepted
- PEP 0325 [25-Aug-2003] – Resource-Release Support for Generators – Rejected
- PEP 0342 [10-May-2005] – Coroutines via Enhanced Generators – Accepted
- PEP 0380 [13-Feb-2009] – Syntax for Delegating to a Subgenerator – Accepted
You see the rabbit hole getting deeper? I’ll clarify it further by rephrasing the previous quote from PEP 0380:
If [feature from PEP 0255] is the only concern, this can be performed without much difficulty using a loop […] However, if the subgenerator is to interact properly with [changes from PEP 0342] things become considerably more difficult. [So we need feature from PEP 0380.]
Yet, while the language grows handling self-inflicted micro-problems, the real issue is still not solved. All of these features are simplistic forms of concurrency and communication, that don’t satisfy the developers, causing community fragmentation.
This happened to C++, to Python, and to many other languages. Go seems slightly special in that regard in the sense that its core development team has an outstanding respect for simplicity, yet dares to solve the difficult problems at their root, while keeping these solutions orthogonal so that they support each other. Less is more, and is not always straightforward.
Heh, I remember that conversation. It’s funny you mention this because even examples that are less complex than the generator PEP you mention cause problems. Just today I was explaining how the ‘is’ keyword is about identity but that it’s confusing because primitives like strings and ints are interned in Python and so it looks like equality from the outside. It’s a simple thing but even simple things can cause confusion, especially when simple exploration doesn’t reveal the subtle issues that can crop up.
Every feature is justifiable in isolation. Nobody plans to make things worse, it’s just a side effect of the lack of perspective of the decision makers. Everything can be justified in some particular context, to the loss of clarity in the greater systemic view.
When someone argues for a feature, I don’t want to hear that it will make programming faster, or require less code to type, or that it makes things more efficient. I want to hear how that feature helps the reader ten years from now understand the code. I want to hear how can we make sure that feature does not hinder any other feature we might consider doing in the future. I want to hear what impact does that feature have on collaboration between members in a team. These things are important and there are a lot of historical precedents, many from programming, many more from other fields that can help answer those questions.
Gustavo, this is a bit like praising a newly launched ship for having no barnacles.
If you looked at early Python, it certainly avoided this particular problem (no generators), and was a smaller language in general. C circa 1980, had none of the complexity of C++11. Also Unix, and the WWW. As languages or systems grow from their original use and face new demands, the real test is whether their leadership, and the core concepts, are strong enough not to accrete complexity.
There is _some_ amount of nonlinearity in Go now: make vs new comes to mind. Time will tell whether and at what rate it multiplies.
I like Go now, and I respect the team, but I’m going to be much more impressed if ten years from now it is both widely used and not much more complex.
Another point of view is that you do just need to throw out your tools every N years and make new ones that abandon compatibility and include only what you now think you need.
On Python vs C adoption:
For myself, as a Python programmer, the performance is apparently not all that much better, but the static typing is highly attractive.
As a C programmer: C and C++ are useful because they can fit in with so many existing code bases; you can’t Go in also those places yet. For example, you can’t write Python extensions; you can’t patch existing codebases; until recently it would be hard to distribute code everyone could build. I don’t know how many people are starting brand new C++ projects where Go could possibly fit; if you don’t _have_ to use C, many people would tend to use something else.
Hi Martin,
I’m rather praising the mindset of well known shipwrights for being brave enough for tackling sailing complexity at its heart, while developing working ships of simple maneuvering. I’m also providing evidence comparing both mindsets.
This is all unrelated to the current state of either Python or Go. It would be too cheap to bait the feature creep of Python.