Archive for the 'Lua' Category

Embedding Lua interpreter into RPM

Published
by
Gustavo Niemeyer
on 2004-03-23
in C/C++, Lua and Patch
. 0 Comments

I’ve recently committed into the RPM CVS HEAD the internal support for Lua scripts.

Please, notice that this is experimental stuff.

Why embedding Lua in RPM?

  • Many scripts execute simple operations which in an internal interpreter require no forking at all
  • Internal scripts reduce or eliminate external dependencies related to script slots
  • Internal scripts operate even under unfriendly situations like stripped chroots (anyone said installers?)
  • Internal scripts in Lua are really fast
  • Syntax errors in internal scripts are detected at package building time

How it works?

Just use -p <lua> in any script slot (%pre, %post, etc).

For example: 

%pre -p <lua>
print("Wow! It really works!")

What is accessible from Lua?

The standard Lua library, the posix module (basic system access, by Luiz Henrique de Figueiredo and Claudio Terra), and the rex module (regular expressions, by Reuben Thomas).

Macro support

Support for Lua macros was also introduced. It means that one can create custom content using Lua macros anywhere.

For example: 

%{lua: print("Requires: hello-world > 1.0") }

More additions to APT-RPM Lua interface

Published
by
Gustavo Niemeyer
on 2004-03-14
in C/C++, Lua and Project
. 0 Comments

The APT-RPM Lua interface is constantly being improved. This time, the following functions were added:

pkgid() and verid()

Return a unique integer identifying a package or a version.

verpkg()

Returns the parent package of some given version.

verdeplist()

Returns a list of dependencies for a given package, including complete information about it.

These new functions were introduced to give support for something which is frequently asked by APT-RPM users: the ability to discover which installed packages are not required by any other installed package.

Here is a script using these functions to list these packages. This script will be called list-nodeps, and will be available in the contrib/ directory of the next APT-RPM release. 

-- Collect dependencies from installed packages
deplist = {}
verlist = {}
for i, pkg in ipairs(pkglist()) do
    ver = pkgvercur(pkg)
    if ver then
        table.insert(verlist, ver)
        for i, dep in ipairs(verdeplist(ver)) do
            for i, depver in ipairs(dep.verlist) do
                deplist[verid(depver)] = true
            end
        end
    end
end

– Now list all versions which are not dependencies
for i, ver in ipairs(verlist) do
    if not deplist[verid(ver)] then
        name = pkgname(verpkg(ver))
        if name ~= “gpg-pubkey” then
            — Print package name and version without epoch
            — (rpm -e friendly ;-).
            print(name..”-”..string.gsub(verstr(ver), “^%d+:”, “”))
        end
    end
end

More information about the introduced functions is available in https://moin.conectiva.com.br/AptRpm/Scripting

Functions vs. Callable objects in Lua

Published
by
Gustavo Niemeyer
on 2003-12-20
in Lua, Project and Python
. 0 Comments

While working on Lunatic Python, I’ve understood that Lua does specific type checking with lua_isfunction() in some places where a callable type is expected. As a side effect, these places only accept a real Lua function when the callable object might be used. An example of such behavior is, at the time this is being written, the table.foreach() function:

> obj = {}
> mt = {__call=function() print("Called!") end}
> setmetatable(obj, mt)
> obj()
Called!
> table.foreach({a=1}, obj)
stdin:1: bad argument #2 to `foreach' (function expected, got table)
stack traceback:
        [C]: in function `foreach’
        stdin:1: in main chunk
        [C]: ?

The trick used in Lunatic Python to overwhelm this situation was to enclose the custom Python object type inside a real Lua C function closure. This trick might indeed be used anywhere this situation is found. Here are a few functions that allow this trick to be used from inside Lua:

static int lwrapcall(lua_State *L)
{
        lua_pushvalue(L, lua_upvalueindex(1));
        lua_insert(L, 1);
        lua_call(L, lua_gettop(L)-1, LUA_MULTRET);
        return lua_gettop(L);
}

static int lwrapfunc(lua_State *L)
{
        luaL_checkany(L, 1);
        lua_pushcclosure(L, lwrapcall, 1);
        return 1;
}

static int luaopen_wrapfunc(lua_State *L)
{
        lua_pushliteral(L, "wrapfunc");
        lua_pushcfunction(L, lwrapfunc);
        lua_rawset(L, LUA_GLOBALSINDEX);
}

And here is another implementation, by Alex Bilyk, in pure Lua: 

function wrapfunc(callable)
    return function(...)
        return callable(unpack(arg))
    end
end

Using them one would be able to obtain the effect above as follows: 

> table.foreach({a=1}, wrapfunc(obj))
Called!

Hopefully, in the future the standard Lua library will stop checking for a specific type in such cases, or implement some kind of lua_iscallable() checking.

Lunatic Python

Published
by
Gustavo Niemeyer
on 2003-12-14
in C/C++, Lua, Project and Python
. 0 Comments

Lunatic Python is a two-way bridge between Python and Lua, allowing these languages to intercommunicate. Being two-way means that it allows Lua inside Python, Python inside Lua, Lua inside Python inside Lua, Python inside Lua inside Python, and so on.

Here are two examples giving an idea about what it does.

Lua inside Python

>>> table = lua.eval("table")
>>> def show(key, value):
...   print "key is %s and value is %s" % (`key`, `value`)
...
>>> t = lua.eval("{a=1, b=2, c=3}")
>>> table.foreach(t, show)
key is 'a' and value is 1
key is 'c' and value is 3
key is 'b' and value is 2
>>>

Python inside Lua 

> function notthree(num)
>>   return (num ~= 3)
>> end
> l = python.eval("[1, 2, 3, 4, 5]“)
> filter = python.eval(”filter”)
> =filter(notthree, l)
[1, 2, 4, 5]

Why Lua was embedded into APT-RPM

Published
by
Gustavo Niemeyer
on 2003-06-13
in C/C++, Lua and Project
. 0 Comments

Why embedding at all?

APT-RPM is a port of the debian APT tool to RPM systems. Since I’ve started working in the project, I’ve been thinking about integrating a higher level language on it. Actually, it’s pretty easy to explain this intention. How many times have you seen distributions hacking a project to fix some misbehavior specific to their environment, or wanting some very specific functionality, which wouldn’t fit in the general context of the upstream project? Attaching an external language allows you to plug these features, without affecting how the project is conducted. Also, I’m a fan of the productivity and flexibility offered by high level languages.

Why Lua?

One might think I’ve used Lua just because I live in the same country as its core developers (Brazil), but that’s not the case. Indeed, I’ve done a pretty intensive research about embeddable languages before choosing Lua. I was looking for a fast, and small language. When you have a library which has about 500kb, you can’t embed a large interpreter to extend the functionality, otherwise you’d be extending the interpreter, not the library. One might think that the interpreter library would be in the system anyway, so that wouldn’t be a real problem. Unfortunately, that doesn’t apply to APT-RPM, since it is used in small systems, and in installer environments. The current Lua interpreter is still under 100kb, and is very fast if compared to other interpreters. I really think the Lua interpreter has no current competitors in that specific area.

Slot code example

To give you an idea about how comfortable it is to work that way, I’ve recently introduced the possibility of passing generic filenames to the “apt-get install” command. To do that, I’ve introduced a new slot in the APT-RPM core, which is called when the data entered is not found as an available package name. Notice that this slot is generic, and works for other kinds of parameters, besides filenames. In the following slot code, notice that the Lua interface has been wrapped into a more useful API for the APT-RPM environment. 

_lua->SetDepCache(Cache);
_lua->SetDontFix();
_lua->SetGlobal("argument", Argument);
_lua->RunScripts("Scripts::Apt::Install::TranslateArg", false);
const char *name = _lua->GetGlobal("translated");
_lua->ResetGlobals();
_lua->ResetCaches();

Plugin code example

With the slot code above, developing the filename translation plugin was very straightforward. Of course, some kind of database containing the filename information was needed. I’ve used a compressed textfile, containing thousands of pairs like “filename packagename”, one per line. And here is the final plugin. Can you imagine how much code it’d take if implemented in C , the core language of APT-RPM?

-- Data sample:
--   argument = "/usr/bin/lua"
--   contents = "/var/state/apt/Contents.gz"
--   translated = "newname" 

if string.sub(argument, 1, 1) == "/" then
    contents = confget("Dir::State::contents/f")
    if string.sub(contents, -3) == ".gz" then
        file = io.popen("zcat "..contents)
    elseif string.sub(contents, -4) == ".bz2" then
        file = io.popen("bzcat "..contents)
    else
        file = io.open(contents)
    end
    len = string.len(argument)
    for line in file:lines() do
        if string.sub(line, 1, len) == argument then
            _, _, path, name = string.find(line, '(%S )%s (%S )')
            if path == argument then
                translated = name
                break
            end
        end
    end
    for line in file:lines() do
        -- nothing, just don't break the pipe
    end
    file:close()
end

More information

For more information, have a look at https://moin.conectiva.com.br/AptRpm and https://moin.conectiva.com.br/AptRpm/Scripting