Archive for the 'Patch' Category

Support for GPS TrackMaker file format on GPSBabel

Continuing my put-those-bits-out-of-your-hard-drive campaign, I’ve released a patched version of GPSBabel with support for input and output of waypoints, tracks and routes in the binary file format of GPS TrackMaker.

The patch was written six months ago (sorry :-) ). I just had to port it over to a recent version of GPSBabel.

The original reason for the patch is that there is quite a good amount of information under that format for Brazil, and GPSBabel is able to deliver information directly to some brands of GPS devices. Hopefully this will get applied upstream soon.

Embedding Lua interpreter into RPM

I’ve recently committed into the RPM CVS HEAD the internal support for Lua scripts.

Please, notice that this is experimental stuff.

Why embedding Lua in RPM?

  • Many scripts execute simple operations which in an internal interpreter require no forking at all
  • Internal scripts reduce or eliminate external dependencies related to script slots
  • Internal scripts operate even under unfriendly situations like stripped chroots (anyone said installers?)
  • Internal scripts in Lua are really fast
  • Syntax errors in internal scripts are detected at package building time

How it works?

Just use -p <lua> in any script slot (%pre, %post, etc).

For example:

%pre -p <lua>
print("Wow! It really works!")

What is accessible from Lua?

The standard Lua library, the posix module (basic system access, by Luiz Henrique de Figueiredo and Claudio Terra), and the rex module (regular expressions, by Reuben Thomas).

Macro support

Support for Lua macros was also introduced. It means that one can create custom content using Lua macros anywhere.

For example:

%{lua: print("Requires: hello-world > 1.0") }

New RPM transaction locking

Last week I’ve worked with Jeff Johnson to fix a small RPM issue which was bothering us for quite some time. Until this change, RPM was accepting two transactions to be run at the same time but, unfortunately, this might put the database in an inconsistent state, and some headers might get lost while restoring it with –rebuilddb.

Our fix was rather simple, based on fcntl() locking while transactions are being committed to the system. This should be enough, protecting databases while the issue is not fixed The Right Way (the time consuming one ;-) ).

Developers of applications linked with RPM should not worry about additional errors, since the current behavior is to wait for the lock to be released while entering in the transaction committing function.

Compiler warnings in Python’s SRE

After a long time, and many people complaining about it, I finally took some time to fix some annoying compiler warnings in Python’s regular expression engine. Since it’s a rather uncommon case, I’ll explain it here with a quick example.

Have a look at the following code:

#include <stdio.h>

int main(int argc, char *argv[])
{
   unsigned char c = 1;
   if (c < 256)
      printf("Hello world!\n");
   return 0;
}

If you try to compile that code, you’ll probably get a warning like this:

test.c: In function `main':
test.c:7: warning: comparison is always true due to
limited range of data type

Yes, the compiler is right. Our char type will never reach the given limit. On the other hand, suppose that this code is preprocessed, and that the c variable has sometimes a multibyte character type, like wchar_t, for example. In this case our test is legitimate, and the dozens of warnings being caused by a common macro are really annoying.

There are many different ways to remove these warnings. Unfortunately, the most obvious one, which is casting the variable to a larger type, doesn’t work as expected.

My adopted solution was to reimplement the same test in another way, surpassing the compiler warnings for this specific case. Instead of writing (c < 255), I’ve written (!(c & ~255)). This is 100% safe for any unsigned type, which is my case. Ok, perhaps it’s a little bit sick, but a large comment should leave everyone aware about the rationale, and away from the warnings.

Removed SRE recursion in Python

SRE is the current regular expression engine used in Python, and the scheme currently in use is highly recursive in some cases. For example, check the result of the following expression, with the current interpreters:

>>> import re
>>> re.match('(x)*','x'*10000)
Traceback (most recent call last):
  File "", line 1, in ?
  File "/usr/lib/python2.3/sre.py", line 132, in match
    return _compile(pattern, flags).match(string)
RuntimeError: maximum recursion limit exceeded

Now, my patch to remove the SRE recursion in Python was finally committed to the CVS. Here is the same example with the CVS HEAD, but using a string which is a hundred times larger:

>>> import re
>>> re.match('(x)*','x'*1000000)
<_sre.SRE_Match object at 0x300bb970>
>>>

Unless some serious problem is found in the new algorithm, it will be available in Python 2.4.

This patch also includes the (?(id/name)yes-pattern|no-pattern) expression support I’ve written more than a year ago. This will try to match with yes-pattern if the group with given id or name exists, and with no-pattern if it doesn’t. |no-pattern is optional and can be omitted. For example, () is a poor email matching pattern, which will match with ” as well as ‘user@host.com’, but not with ‘

Java bits

For the first time in my life, I’ve been paid to implement something in the Java language. Conectiva has been contracted to implement X500 address book support in a known Brazilian groupware tool, for the Brazilian government. This tool runs on top of the TomCat server, and I must mention that I’ve used an interesting feature of the Java language to debug it remotely: the Java Platform Debugger Architecture (JPDA) (thanks JSwat developers).