Python’s os.environ

As Chris Armstrong pointed out yesterday, os.environ.pop() is broken in Python versions at least up to 2.5. The method will simply remove the entry from the in-memory dictionary which holds a copy of the environment:

>>> import os
>>> os.system("echo $ASD")
 
0
>>> os.environ["ASD"] = "asd"
>>> os.system("echo $ASD")
asd
0
>>> os.environ.pop("ASD")
'asd'
>>> os.system("echo $ASD")
asd
0

I can understand that the interface of dictionaries has evolved since os.environ was originally planned, and the os.environ.pop method was overlooked for a while. What surprises me a bit, though, is why it was originally designed the way it is. First, the interface will completely ignore new methods added to the dictionary interface, and they will apparently work. Then, why use a copy of the environment in the first place? This will mean that any changes to the real environment are not seen.

This sounds like something somewhat simple to do right. Here is a working hack using ctypes to show an example of the behavior I’d expect out of os.environ (Python 2.5 on Ubuntu Linux):

from ctypes import cdll, c_char_p, POINTER
from UserDict import DictMixin
import os


c_char_pp = POINTER(c_char_p)


class Environ(DictMixin):

    def __init__(self):
        self._process = cdll.LoadLibrary(None)
        self._getenv = self._process.getenv
        self._getenv.restype = c_char_p
        self._getenv.argtypes = [c_char_p]

    def keys(self):
        result = []
        environ = c_char_pp.in_dll(self._process, "environ")
        i = 0
        while environ[i]:
            result.append(environ[i].split("=", 1)[0])
            i += 1
        return result

    def __getitem__(self, key):
        value = self._getenv(key)
        if value is None:
            raise KeyError(key)
        return value

    def __setitem__(self, key, value):
        os.putenv(key, value)

    def __delitem__(self, key):
        os.unsetenv(key)

I may be missing some implementation detail which would explain the original design. If not, I suggest we just change the implementation to something equivalent (without ctypes).

2 thoughts on “Python’s os.environ

  1. Graham Dumpleton

    One of the problems is that unsetenv() system call is not available on all platforms from what I remember. For example, not available on Solaris that I can find. Thus it isn’t actually possibly to implement del/pop properly if working direct from C environment variables table. This is possibly why it was done the way it was.

  2. Gustavo Niemeyer Post author

    Might be. This isn’t a reason for this behavior in systems that do have the needed infrastructure for a good implementation, though. It should be fine to have operating-system dependent behavior on the os module for cases like that.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>