class MyClass:
def mymethod(self, …):
some_func_defined_in_module(self, …)

and to someone having only an instance he has no easy way to call that function which has the same abilities as a method defined in the class body, of course. To access that function one from the caller side one needs to perform an import of the module and fish the function. This is probably something that you don’t consider sufficient. But still wanted to mention it – for my purposes it works well because there really is no convenient way to import the module, in fact

So let’s presume we have a private keyword and i would not be able to get at the attribute i want. I’d file an issue with the developer (if i find out how) and then what?

As I said earlier, if the public API of the library does not satisfy you, either copy the library and adapt to your needs, or work with the maintainer to improve it as wanted. In both cases, both the maintainer of the library and your users will be happier, since your software won’t break when the next stable version of the library comes out.

I need to get my job done and being confronted with private/public distinctions doesn’t make my job easier.

I never said it would make that job easier. In fact, I’ve explicitly stated that the problem is exactly that the “job” of accessing private attributes is too easy right now. What I suggest in this post is that stricter member access control makes the job easier when you have to maintain nicely working software for thousands/millions of users and hundreds/thousands of developers in an evolving environment.

you wrote in your original post:

“It’s way easier to simply read the code and get to the
private guts of a library than to try to get agreement
over the right way to do something, or sending a patch
with a suggested change which was carefully architected.”

This is a valid observation and there is what i’d call “cost of communication” here. It can be high even in controlled team environments even, but often is very high between very loosely or not-at-all coupled developers. And we all know, developers are lazy.

You say in your comment:

“Having tons of software in the wild which use the private API vs. having tons of software in the wild which use the private API and two more lines of comment doesn’t really change the picture presented in the post and detailed in the comments above.”.

So let’s presume we have a private keyword and i would not be able to get at the attribute i want. I’d file an issue with the developer (if i find out how) and then what? Would he be more likely to fix the issue because there is a “private” keyword? Would he rush to remove my blocking issue? Here is what i think is likely: while waiting i’d hack around in *even weirder ways* to get at what i need, maybe duplicating code, copy-pasting his class or whatever. I need to get my job done and being confronted with private/public distinctions doesn’t make my job easier.

Concluding, I see more value in easing code-centered communication between developers then shifting Python programming paradigms to use private/protected methods.

cheers,
holger (wonders how formatting works on the blog here)

Rather would like to see better tools for communicating around code, see http://tetamap.wordpress.com/2009/05/18/code-centered-issue-tracking/ for some more discussion on this.

The problem isn’t means of communication. There are plenty of ways people can already communicate with maintainers. We’re talking about mass behavior here, not coding under a controlled team environment. If people quite frequently do not respect the private member convention, adding an entirely new convention on top of it which goes beyond any kind of known coding practices and expecting people to follow it in the large would be futile.

Also, even if we assume that we’d be able to cause such a major shift in behavior across the board, sending a couple of lines from a comment to the code maintainer doesn’t fix the issue by itself. Having tons of software in the wild which use the private API vs. having tons of software in the wild which use the private API and two more lines of comment doesn’t really change the picture presented in the post and detailed in the comments above.

I prefer not to have private/protected members and methods in Python, because those things assume that libraries are perfect and library users are jerks that like to kludge.

Quite simply and without trying to judge people’s personal characteristics, access control assumes that there’s value in being able to separate the public interface from the implementation. It’s a contract between the library maintainer and users of the library. In open source, you can very easily break these kinds of contracts by forking the library and doing whatever you please with it. As I mentioned in the post, this will provide you with what you want, and will reduce problems for the real maintainer since you are the maintainer of the fork.

The opposite is often the true in practice; libraries are buggy and
library users like me have been forced to patch them to meet deadlines and
deliverables. (And don’ tell me that “you should report the bug, fix the
library and/or wait for next version” thing.)

Not having protected/private members in Python is one problem less when you need to handle buggy libraries. If such things are added to Python, libraries will still be buggy and still will have to be patched in user code — it will just be more complicated to do, like C++.

You’re helping to prove my point. Convention doesn’t work because there’s a ton of people like you out there, which hop over it for pretty much anything, and feel that interacting with the maintainer is too much work.

The fact that you should report the bug, fix the library and/or wait for the next version is certainly a fantastic advice for someone that plans to work nicely in an open source environment, but being a good player or not is really up to you.

The fact that you’re spreading software which depends on private APIs of libraries in the wild is not just your problem, though, as I tried to explain in the post.

Just for a moment, put yourself into the position of a maintainer which has to maintain a library for tens of thousands of users. If everyone jumps over what is described as the public interface of the library, suddenly the maintainer of the library can’t offer any kind of stability guarantee anymore to users of software which depend on such a library. As a follow up consequence, distributors that bundle the library and dependent software (e.g. Linux distributions) will have trouble to assert that upgrades which feel safe to the maintainer are not actually breaking dependent software (security upgrades be damd). Of course, the fact that a lot of software has poor or no test coverage at all, and that Python is a dynamically typed language, only helps to make matters worse in this scenario.

The opposite is often the true in practice; libraries are buggy and library users like me have been forced to patch them to meet deadlines and deliverables. (And don’ tell me that “you should report the bug, fix the library and/or wait for next version” thing.)

Not having protected/private members in Python is one problem less when you need to handle buggy libraries. If such things are added to Python, libraries will still be buggy and still will have to be patched in user code — it will just be more complicated to do, like C++.

I wonder when the rule is important enough to change the people though. For instance of you development process is oriented to TDD and people don’t write the tests or do the job poorly will you change them then ?

Tim, that’s an interesting question. I’ve posted a new entry to better explain what I meant, and used your questioning as an example.

It’s possible in python to enforce the rule via an external tool (like pylint) that you can launch on you code in you quality assurance checks.

Guillaume, I totally agree with your point. Note, though, that this case falls into what I describe in the post: This methodology may work quite well in situations where the code scope is within a very controlled environment, with one or more teams which follow strictly a single development guideline, and have the power to refactor the affected code base somewhat easily when the original decisions are too limiting.

Your English is pretty good, by the way.

You said that “I’m also a believer that when the rules don’t work for people, the rules should be changed, not the people.”

I wonder when the rule is important enough to change the people though. For instance of you development process is oriented to TDD and people don’t write the tests or do the job poorly will you change them then ?

Rgds

Tim

Yes, in general it is allowed to improve a programming language over time. This argument makes even less sense if we take into consideration how much Python 3 changed in the name of improving things up.