Archive for the 'Project' Category

Mocker for Python released!

Published
by
Gustavo Niemeyer
on 2007-11-11
in Project, Python and Test
. 0 Comments

After being bored for a long time for the lack of a better infrastructure for creating test doubles in Python, I decided to give it a go.

I’m actually quite happy with what came out.. it took me about four weekends (was developed as a personal project), and I’ll dare to say that it’s the best mocking system for Python at the present time. Not only that, but it has features that I’ve not seen in any other mocking/stubing infrastructure, independent of language.

Here’s a feature list to catch your attention:

  • Graceful platform for test doubles in Python (mocks, stubs, fakes, and dummies).
  • Inspiration from real needs, and also from pmock, jmock, pymock, easymock, etc.
  • Expectation of expressions defined by actually using mock objects.
  • Expressions may be replayed in any order by default,
  • Trivial specification of ordering between expressions when wanted.
  • Nice parameter matching for defining expectations on method calls.
  • Good error messages when expectations are broken.
  • Mocking of many kinds of expressions (getting/setting/deleting attributes, calling, iteration, containment, etc)
  • Graceful handling of nested expressions (e.g. ”person.details.get_phone().get_prefix()”)
  • Mock ”proxies”, which allow passing through to the real object on specified expressions (e.g. useful with ”os.path.isfile()”).
  • Mocking via temporary ”patching” of existent classes and instances.
  • Trivial mocking of any external module (e.g. ”time.time()”) via ”proxy replacement”.
  • Mock objects may have method calls checked for conformance with real class/instance to prevent API divergence.
  • Type simulation for using mocks while still performing certain type-checking operations.
  • Nice (optional) integration with ”unittest.TestCase”, including additional assertions (e.g. ”assertIs”, ”assertIn”, etc).
  • More …

Worked? Check it out!

Landscape and Storm go public

Published
by
Gustavo Niemeyer
on 2007-08-15
in Conference, Project and Python
. 3 Comments

Finally, a couple of projects I’ve been working on in the last year and a half have been made public, which means that I have more freedom to talk about them openly.

Landscape

Landscape is a system we’ve created to allow administrators to comfortably manage and observe a large number of computers remotely through a centralized web interface.

This description certainly won’t strike anyone as a brand new idea. There are indeed a large number of systems for remote management. Even then, Landscape does bring new ideas into that known field, such as a very flexible package management offering. Landscape, supporting only Ubuntu at the present moment, also has the advantage of being built inside the company which supports the operating system distribution itself.

There are currently 5 core developers, with many other people contributing in various areas. My role is being a Technical Lead, even though that says very little about the kind of relationship that we have within the project. The guys I work with are very smart and goal oriented, so decisions are taken through friendly discussions and consensus, and initiative is seen coming from all directions.

Storm

Storm is a ORM we have developed for Python, to be used in Landscape, Launchpad, and other projects. The project was originally started because our attempts to perform client side partitioning (sharding) of data with existent ORMs for Python failed.

It was announced as an open source project in a talk I presented last month at EuroPython, and last week the second public release (0.10) was already made.

If you are around the Boston area in the US, my coworker and friend Christopher Armstrong will be giving a Storm talk at the Cambridge Python Meetup today. I’ll also be presenting it again at PyCon Brasil at the end of the month, in Joinville, Brazil.

python-dateutil 1.2 released

Published
by
Gustavo Niemeyer
on 2007-06-26
in Project and Python
. 2 Comments

python-dateutil version 1.2 has just been released.

It includes the following changes:

  • Now tzfile will round timezones to full-minutes if necessary, since Python’s datetime doesn’t support sub-minute offsets (reported by Ilpo Nyyssönen).
  • Removed bare string exceptions (reported and fixed by Wilfredo Sánchez Vega)
  • Fixed bug in leap count parsing (reported and fixed by Eugene Oden).

Smart 0.51 available

Published
by
Gustavo Niemeyer
on 2007-05-20
in Project and Python
. 0 Comments

Smart 0.51 has been released today. It includes a few bug fixes and some minor updates.

Shortly after the release, I’ve added a couple of new hooks on Smart’s trunk as well: cache-loaded, and cache-loaded-pre-link. These should enable people to write plugins that hack the cache for specific purposes. Axel Thimm has requested these for a while to introduce kernel-related upgrades. Hopefully these will fulfill his needs.

This release took a while.. probably because I’ve been quite immersed in our current project at Canonical, traveling very frequently, and without much time to blog or to do some of the usual open source activities I used to. The good thing is that we’re getting very close to the public announcement, and some of the work we’ve been doing will be released as open source, so we’re all likely to get more community-oriented interactions again.

Quickies

Published
by
Gustavo Niemeyer
on 2007-03-10
in Conference, GPS, Other, Project, Python and RCS
. 0 Comments

brother…

My brother Diogo is in town! Good to see him after so much time.

pycon…

PyCon 2007 was fantastic. It was great to meet everyone there, and we had two awesome sprinting weeks around it.

confluence…

I’ve recently visited a confluence with a good friend of mine. Kayaks, paddling, walking, driving, swimming, aslphalt, sand, water, grass.. it was awesome.

svn2bzr…

It looks like Bazaar tags are now really coming, so I’m doing some work on svn2bzr again. Hopefully this time I’ll really migrate some projects over.

editmoin…

Version 1.9 of editmoin was released.

smart…

Some work in Smart is coming in the upcoming weeks.

projects…

Hopefully I’ll be able to speak more openly about (some of the) interesting things I’ve been working on in the near future.

editmoin now talks to moin 1.6

Published
by
Gustavo Niemeyer
on 2006-11-20
in Project, Python and RCS
. 6 Comments

editmoin 1.8 was just released, including support for moin 1.6, submitted by both David I. Lehn and Daniele Favara, and URL aliases, as suggested by Diogo Matsubara.

One interesting thing I should mention is that Daniele has sent me a reference to a Bazaar branch containing the change. It’s the first time I receive a bzr (as it’s also known) branch when the upstream project is using a different RCS (Subversion).

I actually intend to port this and other projects to Bazaar, but I’m waiting for the tags support. Hopefully it’s not too far away now.

Smart upgrading algorithm

Published
by
Gustavo Niemeyer
on 2006-11-08
in Project and Python
. 5 Comments

One of the known issues in I’ve been trying to address in Smart for a while is the freezing effect that happens when a very complex upgrading situation (such as a full distribution upgrade) results in a combinatory explosion due to the number of choices to be analysed. Unfortunately, I never had time to really put in practice a reasonable solution for the problem. At this point, the beauty of open source software starts to shine.

A few weeks ago, Eran Tromer got close to the project and started researching and discussing about the issue. Not only that, but he produced actual patches that change the algorithm to prune the search space and find resonable solutions in acceptable timings. These patches were applied into the development version, and included in release 0.50rc1.

The preliminary results are quite impressive. David Farning has tested Smart 0.50rc1 with Fedora in several situations, and reported:

fc4->fc5, fc4->fc6, fc4->devel, fc5->fc6, all calculate updates and upgrades in a few minutes on a vm with 512M, using standard repos + Freshrpms and Dries. Much quicker than with 0.42.

This is really awesome. Thanks Eran!

Released editmoin 1.5 and patcher 0.6

Published
by
Gustavo Niemeyer
on 2006-08-07
in Project and Python
. 1 Comment

New versions of editmoin and patcher were just released. They fix a couple of issues found by Jan Anlauff and Olivier Thauvin, respectively.

Comparing package versions in PostgreSQL

Published
by
Gustavo Niemeyer
on 2006-07-30
in C/C++, PostgreSQL, Project and Python
. 1 Comment

This weekend I’ve played a bit with PostgreSQL extensions written in C.

A while ago I wrote a Python C extension for Smart to compare Debian package versions. Now I was trying to do something similar inside PostgreSQL, and thus ported the original Python C extension code to a PostgreSQL C extension. It enables queries like the following:

# SELECT 'Matched' WHERE deb_version_match('1.2', '<=', '2.0');
  ?column?
----------
 Matched

The implementation of the PostgreSQL C extension was quite straightforward, but I’m a bit disappointed by the performance of PL/PGSQL.

I’ve made tests using two environments. One of them is a PL/Python function executed as a trigger, which calls the original Python C extension and executes SQL back in PostgreSQL using the plpy module. The other is a PL/PGSQL function which uses the implemented PostgreSQL C extension directly.

Considering that the function logic consisted of one loop over a SELECT statement, a few tests, and an INSERT statement, I was expecting that the overhead introduced by going back and forth between the PostgreSQL state and the Python interpreter state would be a lot more noticeable when compared with PL/PGSQL executing an internal PostgreSQL function. Tests have shown about 10% of improvement roughly, when doing a similar logic over about 5000 items.

I’m not yet sure if the speed improvement pays off the limited debugging feedback provided by the PL/PGSQL interpreter on errors.

Support for GPS TrackMaker file format on GPSBabel

Published
by
Gustavo Niemeyer
on 2006-02-04
in GPS, Patch and Project
. 0 Comments

Continuing my put-those-bits-out-of-your-hard-drive campaign, I’ve released a patched version of GPSBabel with support for input and output of waypoints, tracks and routes in the binary file format of GPS TrackMaker.

The patch was written six months ago (sorry :-) ). I just had to port it over to a recent version of GPSBabel.

The original reason for the patch is that there is quite a good amount of information under that format for Brazil, and GPSBabel is able to deliver information directly to some brands of GPS devices. Hopefully this will get applied upstream soon.