Lunatic Python

Lunatic Python is a two-way bridge between Python and Lua, allowing these languages to intercommunicate. Being two-way means that it allows Lua inside Python, Python inside Lua, Lua inside Python inside Lua, Python inside Lua inside Python, and so on.

Here are two examples giving an idea about what it does.

Lua inside Python

 
>>> table = lua.eval("table")
>>> def show(key, value):
...   print "key is %s and value is %s" % (`key`, `value`)
... 
>>> t = lua.eval("{a=1, b=2, c=3}")
>>> table.foreach(t, show)
key is 'a' and value is 1
key is 'c' and value is 3
key is 'b' and value is 2
>>>

Python inside Lua

> function notthree(num)
>>   return (num ~= 3)
>> end
> l = python.eval("[1, 2, 3, 4, 5]")
> filter = python.eval("filter")
> =filter(notthree, l)
[1, 2, 4, 5]
Posted in C/C++, Lua, Project, Python | Leave a comment

APT-RPM Article

I’ve written an article about APT-RPM to LWN, exposing features recently introduced in the software. Check it out!

Posted in Article, Project | Leave a comment

RepositorySystem now on Subversion 0.32.1

Today we’ve finally finished the migration process of our RepositorySystem to Subversion 0.32.1. The dump and load piping process, necessary to upgrade the Subversion repository due to some changes in the format which occurred in version 0.28, took about 12 days, running over 18GB. The migration process itself, including replacing the machine it runs on for one with more disk space, took a single day. Additionally, we have now a ViewCVS running on the same machine, giving colorful access to the repository. ;-)

Posted in Other | Leave a comment

Jython links

I have collected a few cool links about Jython, and I’d like to keep them somewhere. So, here they are:

Posted in Java, Python | Leave a comment

Using pyperl to build a python-like xchat interface for perl

My weekend was taken by boring academic stuff. More specifically, I’ve built a validator in C, including a tokenizer, a parser, and a scope system, to check for a small subset of the C language syntax. Anyway.. I was still able to do something interesting after all. :-)

Taking further my research about pyperl, I’ve wondered if it would be possible to wrap the XchatPython plugin interface for perl usage. Actually, it turned out that besides being possible, it was easy. It was just a matter of putting the following wrapper in the ~/.xchat2 directory:

import xchat
import perl
perl.eval("$xchat = Python::eval('xchat')")
perl.eval("do '" + __file__[:-3] + ".pyl'")
__module_name__ = perl.eval("$__module_name__")
__module_version__ = perl.eval("$__module_version__")
__module_description__ = perl.eval("$__module_description__")

This wrapper will load a file with the same name as itself, but with a .pyl extension, which should contain perl code defining the plugin (I haven’t used the .pl extension because this would conflict with the internal perl plugin of xchat). That code may access the $xchat object to interface with xchat, using the same API as defined in XchatPython.

Here is a small sample module in perl, defining the /test command:

#!/usr/bin/perl
$__module_name__ = "perlwrap";
$__module_version__ = "1.0";
$__module_description__ = "Testing perlwrap.";

sub test {
    $xchat->prnt("Test command!");
    return $xchat->EAT_ALL;
}

$xchat->hook_command("test", &test);

$xchat->prnt("Plugin perlwrap loaded!n");

Notice that the interface is exactly the same, including the meta-information.

Unfortunately, there seems to exist some bug in the MULTI_PERL support of pyperl which makes it lose track of function references once the python support commutes the thread states. To make this trick work, remove the MULTI_PERL support by deleting the MULTI_PERL file in the root of the pyperl distribution.

Posted in Perl, Python, Snippet | 1 Comment

Webmin and Perl inside Python

Today I’ve been talking to Cavassin about Webmin, and wondering if it would be possible to write Python modules for it. I was sure that I wasn’t being original in that idea, and Cavassin also mentioned that he had read something about it somewhere, so I’ve looked in google for possible references.

Of course, we were right. I have found the python-webmin project. OTOH, I’ve looked in the code, and that’s not really what I had mind. This module reimplements the Webmin API in Python. This means that one will have to maintain it forever, and check for updates in every Webmin release, and even then, it won’t be able to do more advanced stuff which requires deeper interaction with the Webmin system.

The correct solution for that would be to somehow wrap the Webmin API in a way that Python calls the real methods, thus requiring little or no maintenance. It’d be even better if this system was generic and not bound to Webmin in any way. While looking for the solution, I found Inline, which did exactly what I had in mind, but reversed (Python inside Perl), and PyInline, which unfortunately doesn’t seem to support Perl yet. Then, I finally found what I was looking for: pyperl.

The pyperl project embeds a Perl interpreter inside a Python module, allowing arbitrary Perl functionality to be used from Python. It’s really amazing to see it working. Have a look at these simple examples:

>>> import perl
>>> perl.eval('sub func { printf("Hello $_[0]!n"); }')
>>> ret = perl.call("func", "world")
Hello world!

>>> func = perl.get_ref("func")
>>> ret = func("world")
Hello world!

>>> perl.require("Digest::MD5")
1
>>> md5 = perl.callm("new", "Digest::MD5")
>>> md5.add("something").hexdigest()
'437b930db84b8079c2dd804a71936b5f'

Isn’t it fantastic? :-)

Another amazing thing is that pyperl also includes support for doing Python inside Perl. Of course, you can do Python inside Perl inside Python:

>>> n = 5
>>> perl.eval('Python::eval("n")*2')
10

Now, it’s just a matter of finding some time to create a basic wrapper on top of the Webmin Perl API, turning it into something more pythonic and also protecting Webmin modules developed in Python from changes in the pyperl API.

References

Posted in Perl, Python, Snippet | Leave a comment

UTF8 to ISO-8859-1 xchat converter

Today I’m a little short in time, so I decided to implement one of the bit-sized tasks in my backlog. These UTF8 raw strings appearing in my xchat when someone else is using that encoding is bothering for a long time, and the fix is now straightforward, with the latest update I’ve done in the xchat python plugin (the emit_print() API function was implemented). It’s just a matter of creating a decode() function which tries to decode the string from UTF8, and reencode it as ISO-8859-1. If it succeeds, it blocks xchat, and sends the message by itself. Otherwise it just lets xchat do its work. After that, I’ve hooked this method in the many print events, including information about which of the parameters are the strings to decode. Here is an excerpt (the full version is in the examples section of the xchat-python page):

...

def decode(word, word_eol, userdata):
    event, pos = userdata
    if type(pos) is int:
        pos = (pos,)
    changed = False
    for i in pos:
        try:
            reencoded = word[i].decode('utf8') 
                               .encode('iso-8859-1')
        except UnicodeError:
            continue
        if reencoded != word[i]:
            word[i] = reencoded
            changed = True
    if changed:
        xchat.emit_print(event, *word)
        return xchat.EAT_XCHAT
    else:
        return xchat.EAT_NONE

EVENTS = [
  ("Channel Action", 1),
  ("Channel Action Hilight", 1),
  ...
]

for event in EVENTS:
    xchat.hook_print(event[0], decode, event)

...

Notice that you need the latest version of the xchat-python plugin, which will be available in the next version of xchat (coming after 2.0.5). You may also download xchat 2.0.5, and update the plugin with the file available in the Download section of the plugin page.

Posted in Project, Python, Snippet | Leave a comment

RandomFolder for Zope 3

I’ve restarted my research about Zope 3 in the last few days. This time, I went a little further. I’ve written a small product so that I was forced to debug some of the internals of Zope 3 and learn something from it.

The product

While going through the Zope 3 API, I’ve wondered how I would implement something close to the RandomQuote Moin macro, including my hack to render quotes, like images. After thinking a little bit about it, I decided to implement a RandomFolder; something which would render one of the contained items randomly.

How it was implemented

After some study, the RandomFolder was implemented like this:

  1. Defined a dummy interface IRandomFolder, inheriting from IFolder.
  2. Defined a dummy type RandomFolder, inheriting from Folder and implementing IRandomFolder.
  3. Defined RandomFolderTraverser, inheriting from ContainerTraverser, and defining a custom browserDefault() method. This method does the real trick: instead of returning in the view name something like “@@index.html”, as usual, it will return a subpage name, randomly.
  4. Created a configure.zcml file, configuring the new type. The tricky part was to include a view adapter from IRandomFolder to IBrowserPresentation with the RandomFolderTraverser.

Result

Using the given scheme, the RandomFolder may be managed exactly as a normal Folder when including items and viewing the management interface. Contained items may also be accessed individually without any side effects. Even index.html may be used to list the folder content. OTOH, when the folder is accessed without any subpath following it, it will select one of its children randomly, and render it.

Random thoughts about Zope 3

I’m really amazed by the Component Architecture used in Zope 3. While the learning curve seems to be the same due to the many details of the framework, these details look more consistent now, reusing similar algorithms all over the place. Also, I’m happy to see a product being developed the way Zope 3 is being. It looks like everything is done the right way, no matter how much time it takes. Of course, this will probably hurt the schedule; but given the inherent complexity of these systems, I’d rather wait a little bit than getting into an evolutive cycle faster than necessary.

Posted in Python | Leave a comment

Python dateutil module released

I’m a little late here, since python-dateutil was released last week. Anyway, this is a Python package providing powerful extensions to the standard python datetime module, available with Python 2.3. I wrote it because I really missed some generic date/time manipulation features to empower the nice datetime module. I confess it ended up being a little more than what I expected initially, mainly because I had a lot of fun building these algorithms.

Here is a list of features, from the homepage:

  • Computing of relative deltas (next month, next year, next monday, last week of month, etc);
  • Computing of relative deltas between two given date and/or datetime objects;
  • Computing of dates based on very flexible recurrence rules, using a superset of the iCalendar specification. Parsing of RFC strings is supported as well.
  • Generic parsing of dates in almost any string format;
  • Timezone (tzinfo) implementations for tzfile(5) format files (/etc/localtime, /usr/share/zoneinfo, etc), TZ environment string (in all known formats), iCalendar format files, given ranges (with help from relative deltas), local machine timezone, fixed offset timezone, and UTC timezone.
  • Computing of Easter Sunday dates for any given year, using Western, Orthodox or Julian algorithms;
  • More than 400 test cases.

And to give a small example, in addition to the many examples available in the web page, suppose I wanted to know the date of the three following company happy-hours, knowing that we have it once every month, in the last friday:

>>> from dateutil.rrule import *
>>> rrule(FREQ_MONTHLY, count=3, byweekday=FR(-1))[:]
[datetime.datetime(2003, 10, 31, 10, 37, 59),
 datetime.datetime(2003, 11, 28, 10, 37, 59),
 datetime.datetime(2003, 12, 26, 10, 37, 59)]

What about the next Friday 13th?

>>> from dateutil.rrule import *
>>> rrule(FREQ_MONTHLY, bymonthday=13, byweekday=FR)[0]
datetime.datetime(2004, 2, 13, 10, 42, 27)

Cool, isn’t it? :-)

Posted in Project, Python, Snippet | Leave a comment

Updated XChat’s python plugin

After a long time of inactivity, I’ve updated the python plugin of xchat. There were some pending bugs, and new xchat features to catch. Here is the list of changes:

  • Fixed the reentrance problem which made xchat freeze when some python code made xchat trigger a python callback somewhere.
  • Implemented xchat_emit_print() and xchat_get_prefs() support.
  • Fixed softspace support for python 2.3. This problem was making print statements yield an extra space at the next line start.
  • Now in the python console, every input line is printed after >>> in the query window, making it more convenient and familiar.
  • Other minor changes.

The documentation in http://labix.org/xchat-python has been updated as well. For current users of xchat 2.0.5, the updated file is in the Download section.

Posted in C/C++, Project, Python | Leave a comment