Gustavo Niemeyer Archive at Labix Blog

Author Archive for Gustavo Niemeyer

MagLev and distributed VMs

Published

on 2008-06-01

. 0 Comments

Avi Bryant is working on MagLev, a Ruby interpreter, based on Gemstone’s Smalltalk VM, with some very amazing features, like transactioned objects distributed across several VMs:

The integrated VMs, cache, and storage conspire to create an illusion that global state is shared across all instances: no matter how many VMs you add, over however many machines, they all see and work with the same set of Ruby objects.

My geek side finds this highly exciting, and eager to see it released to see how people will deal with it in practice.

At the same time, my let’s-build-stable-and-maintainable-software side is a bit skeptic. As Joe Armstrong puts so enthusiastically, shared state is hard to manage correctly, global transactions reduce scalability, and transparent RPC is seductive, but dangerous.

I’m also curious about the speed gains pointed out. It’s well known that the Ruby VM isn’t very fast, which means that there must be opportunities for speedups. Even then, 100x faster is impressive, and history shows that sometimes the significant improvements are harder when the semantics are precisely the same. Let’s hope Avi can manage to run the Ruby tests successfully.

Improving reading habits

Published

Gustavo Niemeyer

on 2008-06-01

in Other

. 0 Comments

Today, Sunday, on the mailman day, I decided to change my reading habits.

You’d certainly laugh if I told you how many mailing lists, blogs, and IRC channels I try to follow (won’t include IM networks here as I don’t really read them asynchronously). What I look for is pretty obvious: I want to exchange volume for quality.

The first thing I’m doing is unsubscribing from all high-traffic lists I’m part of. The reason is clear: one hundred messages a day can’t possibly be all interesting. I’m not saying there are no interesting posts among these, of course. But with such a vibrant community of followers, a few smart readers usually bring up the most interesting discussions in more selective formats. I’ll track these instead.

For the same reason, I’m unsubscribing from most feed aggregators. Planet and similars are a great way to subscribe to many feeds quickly, but let’s face it.. how many posts in an aggregator with lots of feeds are interesting to a single individual? While getting off from them, I’m selectively peaking the feeds that interest me and subscribing to each.

Then, for the not-so-high volume sources, I’m checking the last 5 posts or so (or days, for IRC channels). Anything that hasn’t had information worth tracking will be phased out too. Interesting topics eventually will find their way to the sources I’ll still follow.

I want to read less, to read more. I want to go faster through the queue of pending books, and also follow a wider variety of topics with less pain.

Google using Geohash

Published

Gustavo Niemeyer

on 2008-05-20

in Project and Python

. 6 Comments

According to Dave Troy, Google seems to be using the Geohash algorithm:

Google is employing the GeoHash algorithm I’ve been pushing to do spatial searching using BigTable. Since database schemes like BigTable don’t support traditional GIS extensions/spatial indexes, GeoHash allows for a simple bounding box search using truncated GeoHash substrings. I will post separately about this shortly, as I am working on some GeoHash tools to expand this functionality. This is of particular interest to AppEngine developers.

Nice!

dateutil 1.4 is out

Published

Gustavo Niemeyer

on 2008-03-03

in Project and Python

. 2 Comments

Friday I’ve released version 1.4 of dateutil. There are some interesting fixes there, so please upgrade if you have the chance.

Enhancements on geohash.org

Published

Gustavo Niemeyer

on 2008-03-01

in GPS, Project and Python

. 0 Comments

Some improvements to geohash.org were made. Some of them were
motivated by a conversation with Rodrigo Stulzer.

Support for geocoding addresses (city names, whatever). E.g. http://geohash.org/?q=21 Millbank, London
Support for moving the Geohash marker in the embedded map, so that modifying the position visually is easier.
Support for providing a “name” to Geohashes, by appending a colon and the name, in a nice format. E.g. http://geohash.org/c216ne:Mt_Hood
Provided a bookmark to get a Geohash while in Google Maps.
Provided a Google Maps Mapplet. When enabled, it adds a Geohash marker identifying the Geohash position in Google Maps, and it may be moved around. Here is a screenshot:

Check out the Tips & Tricks page for details on these features.

geohash.org is public!

Published

Gustavo Niemeyer

on 2008-02-26

in Article, GPS, Project and Python

. 7 Comments

After about one year writing this service in my spare time, it’s finally out.

geohash.org offers short URLs which encode a latitude/longitude pair, so that referencing them in emails, forums, and websites is more convenient.

Geohashes offer properties like arbitrary precision, similar prefixes for nearby positions, and the possibility of gradually removing characters from the end of the code to reduce its size (and gradually lose precision). I’ve put the algorithm created in the public domain. Some details may be seen in the Wikipedia article about it (hopefully that’ll help establishing prior art, and prevent Microsoft from patenting it).

To obtain the Geohash, the user provides latitude and longitude coordinates in a single input box (most commonly used formats for latitude and longitude pairs are accepted), and performs the request.

Besides showing the latitude and longitude corresponding to the given Geohash, users who navigate to a Geohash at geohash.org are also presented with an embedded map, and may download a GPX file, or transfer the waypoint directly to certain GPS receivers. Links are also provided to external sites that may provide further details around the specified location.

Mocker 0.10 and trivial patch-mocking of existing objects

Published

Gustavo Niemeyer

on 2007-12-09

in Project, Python, Snippet and Test

. 2 Comments

Mocker 0.10 is out, with a number of improvements!

While we’re talking about Mocker, here is another interesting use case, exploring a pretty unique feature it offers.

Suppose we want to test that a method hello() on an object will call self.show(”Hello world!”) at some point. Let’s say that the code we want to test is this:

 class Greeting(object):

     def show(self, sentence):
         print sentence

     def hello(self):
         self.show("Hello world!")

This is the entire test method:

def test_hello(self):
    # Define expectation.
    mock = self.mocker.patch(Greeting)
    mock.show("Hello world!")
    self.mocker.replay()

    # Rock on!
    Greeting().hello()

This has helped me in practice a few times already, when testing some involved situations.

Note that you can also passthrough the call. In other words, the call may actually be made on the real method, and mocker will just assert that the call was really made, whatever the effect is.

One more important point: mocker ensures that the real method exists in the real object, and has a specification compatible with the call made. If it doesn’t, and assertion error is raised in the test with a nice error message.

UPDATE: The method for doing this is actually mocker.patch() rather than mocker.mock(), as documented. Apologies.

Partial stubbing of os.path.isfile() with Mocker

Published

Gustavo Niemeyer

on 2007-11-22

in Project, Python, Snippet and Test

. 0 Comments

One neat feature which Mocker offers is the ability to very easily implement custom behavior on specific functions or methods.

Take for instance the case where you want to pretend to some code that a given file exists, but you don’t want to get on the way of everything else which needs the same function:

>>> from mocker import *
>>> mocker = Mocker()
>>> isfile = mocker.replace("os.path.isfile", count=False)
>>> _ = expect(isfile("/non/existent")).result(True)
>>> _ = expect(isfile(ANY)).passthrough()

>>> mocker.replay()

>>> import os
>>> os.path.isfile("/non/existent")
True
>>> os.path.isfile("/etc/passwd")
True
>>> os.path.isfile("/other")
False

>>> mocker.restore()

>>> os.path.isfile("/non/existent")
False

Notice that the count=False parameter is available in version 0.9.2. Without it Mocker will act in a more mocking-strict way and enforce that the given expressions should be executed precisely the given number of times (which defaults to one, and may be modified with the count() method).