It’s really amazing how little attention error handling receives in most software development. Even *tutorials* often ignore it.

It indeed does amaze me. It sometimes feels like we write code for theoretical perfect worlds.. “If the processor executes exactly in this order, and the weather is calm, this program will work.”. There are countless examples of bad assumptions.. someday I will come with some statistics of the form “Every N seconds someone forgets to check the result of write().”.

If you are a teacher, or a developer that enjoys writing snippets of code to teach people, please join me in the quest of building a better future. Do not tell us that you’re “avoiding result checking for terseness”, because that’s exactly what we people will do (terseness is good, right?). On the contrary, take this chance to make us feel bad about avoiding result checking. You might do this by putting a comment like “If you don’t do this, you’re a bad programmer.” right next to the logic which is handling the result, and might take this chance to teach people how proper result handling is done.

Of course, there’s another forgotten art related to result checking. It sits on the other side of the fence. If you are a library author, do think through about how you plan to make us check conditions which happen inside your library, and try to imagine how to make our lives easier. If we suck at handling results when there are obvious ways to handle it, you can imagine what happens when you structure your result logic badly.

Here is a clear example of what not to do, coming straight from Python’s standard library, in the imaplib module:

    def login(self, user, password):
        typ, dat = self._simple_command('LOGIN', user, self._quote(password))
        if typ != 'OK':
            raise self.error(dat[-1])
        self.state = 'AUTH'
        return typ, dat

You see the problem there? How do you handle errors from this library? Should we catch the exception, or should we verify the result code? “Both!” is the right answer, unfortunately, because the author decided to do us a little favor and check the error condition himself in some arbitrary cases and raise the error, while letting it go through and end up in the result code in a selection of other arbitrary cases.

I may provide some additional advice on result handling in the future, but for now I’ll conclude with the following suggestion: please check the results from your actions, and help others to check theirs. That’s a good life-encompassing recommendation, actually.

Please, notice that this is experimental stuff.

Why embedding Lua in RPM?

• Many scripts execute simple operations which in an internal interpreter require no forking at all
• Internal scripts reduce or eliminate external dependencies related to script slots
• Internal scripts operate even under unfriendly situations like stripped chroots (anyone said installers?)
• Internal scripts in Lua are really fast
• Syntax errors in internal scripts are detected at package building time

How it works?

Just use -p <lua> in any script slot (%pre, %post, etc).

For example:

%pre -p <lua>
print("Wow! It really works!")

What is accessible from Lua?

The standard Lua library, the posix module (basic system access, by Luiz Henrique de Figueiredo and Claudio Terra), and the rex module (regular expressions, by Reuben Thomas).

Macro support

Support for Lua macros was also introduced. It means that one can create custom content using Lua macros anywhere.

For example:

%{lua: print("Requires: hello-world > 1.0") }

pkgid() and verid()

Return a unique integer identifying a package or a version.

verpkg()

Returns the parent package of some given version.

verdeplist()

Returns a list of dependencies for a given package, including complete information about it.

These new functions were introduced to give support for something which is frequently asked by APT-RPM users: the ability to discover which installed packages are not required by any other installed package.

Here is a script using these functions to list these packages. This script will be called list-nodeps, and will be available in the contrib/ directory of the next APT-RPM release.

-- Collect dependencies from installed packages
deplist = {}
verlist = {}
for i, pkg in ipairs(pkglist()) do
    ver = pkgvercur(pkg)
    if ver then
        table.insert(verlist, ver)
        for i, dep in ipairs(verdeplist(ver)) do
            for i, depver in ipairs(dep.verlist) do
                deplist[verid(depver)] = true
            end
        end
    end
end

-- Now list all versions which are not dependencies
for i, ver in ipairs(verlist) do
    if not deplist[verid(ver)] then
        name = pkgname(verpkg(ver))
        if name ~= "gpg-pubkey" then
            -- Print package name and version without epoch
            -- (rpm -e friendly  .
            print(name.."-"..string.gsub(verstr(ver), "^%d+:", ""))
        end
    end
end

More information about the introduced functions is available in https://moin.conectiva.com.br/AptRpm/Scripting

> obj = {}
> mt = {__call=function() print("Called!") end}
> setmetatable(obj, mt)
> obj()
Called!
> table.foreach({a=1}, obj)
stdin:1: bad argument #2 to `foreach' (function expected, got table)
stack traceback:
        [C]: in function `foreach'
        stdin:1: in main chunk
        [C]: ?

The trick used in Lunatic Python to overwhelm this situation was to enclose the custom Python object type inside a real Lua C function closure. This trick might indeed be used anywhere this situation is found. Here are a few functions that allow this trick to be used from inside Lua:

static int lwrapcall(lua_State *L)
{
        lua_pushvalue(L, lua_upvalueindex(1));
        lua_insert(L, 1);
        lua_call(L, lua_gettop(L)-1, LUA_MULTRET);
        return lua_gettop(L);
}

static int lwrapfunc(lua_State *L)
{
        luaL_checkany(L, 1);
        lua_pushcclosure(L, lwrapcall, 1);
        return 1;
}

static int luaopen_wrapfunc(lua_State *L)
{
        lua_pushliteral(L, "wrapfunc");
        lua_pushcfunction(L, lwrapfunc);
        lua_rawset(L, LUA_GLOBALSINDEX);
}

And here is another implementation, by Alex Bilyk, in pure Lua:

function wrapfunc(callable)
    return function(...)
        return callable(unpack(arg))
    end
end

Using them one would be able to obtain the effect above as follows:

> table.foreach({a=1}, wrapfunc(obj))
Called!

Hopefully, in the future the standard Lua library will stop checking for a specific type in such cases, or implement some kind of lua_iscallable() checking.

Here are two examples giving an idea about what it does.

Lua inside Python

>>> table = lua.eval("table")
>>> def show(key, value):
...   print "key is %s and value is %s" % (`key`, `value`)
...
>>> t = lua.eval("{a=1, b=2, c=3}")
>>> table.foreach(t, show)
key is 'a' and value is 1
key is 'c' and value is 3
key is 'b' and value is 2
>>>

Python inside Lua

> function notthree(num)
>>   return (num ~= 3)
>> end
> l = python.eval("[1, 2, 3, 4, 5]")
> filter = python.eval("filter")
> =filter(notthree, l)
[1, 2, 4, 5]

APT-RPM is a port of the debian APT tool to RPM systems. Since I’ve started working in the project, I’ve been thinking about integrating a higher level language on it. Actually, it’s pretty easy to explain this intention. How many times have you seen distributions hacking a project to fix some misbehavior specific to their environment, or wanting some very specific functionality, which wouldn’t fit in the general context of the upstream project? Attaching an external language allows you to plug these features, without affecting how the project is conducted. Also, I’m a fan of the productivity and flexibility offered by high level languages.

Why Lua?

One might think I’ve used Lua just because I live in the same country as its core developers (Brazil), but that’s not the case. Indeed, I’ve done a pretty intensive research about embeddable languages before choosing Lua. I was looking for a fast, and small language. When you have a library which has about 500kb, you can’t embed a large interpreter to extend the functionality, otherwise you’d be extending the interpreter, not the library. One might think that the interpreter library would be in the system anyway, so that wouldn’t be a real problem. Unfortunately, that doesn’t apply to APT-RPM, since it is used in small systems, and in installer environments. The current Lua interpreter is still under 100kb, and is very fast if compared to other interpreters. I really think the Lua interpreter has no current competitors in that specific area.

Slot code example

To give you an idea about how comfortable it is to work that way, I’ve recently introduced the possibility of passing generic filenames to the “apt-get install” command. To do that, I’ve introduced a new slot in the APT-RPM core, which is called when the data entered is not found as an available package name. Notice that this slot is generic, and works for other kinds of parameters, besides filenames. In the following slot code, notice that the Lua interface has been wrapped into a more useful API for the APT-RPM environment.

_lua->SetDepCache(Cache);
_lua->SetDontFix();
_lua->SetGlobal("argument", Argument);
_lua->RunScripts("Scripts::Apt::Install::TranslateArg", false);
const char *name = _lua->GetGlobal("translated");
_lua->ResetGlobals();
_lua->ResetCaches();

Plugin code example

With the slot code above, developing the filename translation plugin was very straightforward. Of course, some kind of database containing the filename information was needed. I’ve used a compressed textfile, containing thousands of pairs like “filename packagename”, one per line. And here is the final plugin. Can you imagine how much code it’d take if implemented in C , the core language of APT-RPM?

-- Data sample:
--   argument = "/usr/bin/lua"
--   contents = "/var/state/apt/Contents.gz"
--   translated = "newname" 

if string.sub(argument, 1, 1) == "/" then
    contents = confget("Dir::State::contents/f")
    if string.sub(contents, -3) == ".gz" then
        file = io.popen("zcat "..contents)
    elseif string.sub(contents, -4) == ".bz2" then
        file = io.popen("bzcat "..contents)
    else
        file = io.open(contents)
    end
    len = string.len(argument)
    for line in file:lines() do
        if string.sub(line, 1, len) == argument then
            _, _, path, name = string.find(line, '(%S )%s (%S )')
            if path == argument then
                translated = name
                break
            end
        end
    end
    for line in file:lines() do
        -- nothing, just don't break the pipe
    end
    file:close()
end

More information

For more information, have a look at https://moin.conectiva.com.br/AptRpm and https://moin.conectiva.com.br/AptRpm/Scripting