Smart Package Manager is out!

After 6 months of fun working on the project in silence, Smart Package Manager was finally released.

As the README says:

The Smart Package Manager project has the ambitious objective of creating smart and portable algorithms for solving adequately the problem of managing software upgrading and installation. This tool works in all major distributions, and will bring notable advantages over native tools currently in use (APT, APT-RPM, YUM, URPMI, etc).

Some of the interesting features, which are covered in more detail in the README file:

  • Smart transactions
  • Multiple interfaces
  • Support for several channel formats
  • Priority handling
  • Autobalancing mirror system
  • Parallel downloading mechanism
  • Flexible removable media support

Have fun! ;)

Posted in C/C++, Project, Python | Leave a comment

Progress Cell Renderer for pygtk

Here is a Progress Cell Renderer for pygtk based applications. It allows one to embed a progress bar inside a TreeView. It’s being used in Smart to present progress for downloads of packages and other items.

Posted in Python, Snippet | Leave a comment

Embedding Lua interpreter into RPM

I’ve recently committed into the RPM CVS HEAD the internal support for Lua scripts.

Please, notice that this is experimental stuff.

Why embedding Lua in RPM?

  • Many scripts execute simple operations which in an internal interpreter require no forking at all
  • Internal scripts reduce or eliminate external dependencies related to script slots
  • Internal scripts operate even under unfriendly situations like stripped chroots (anyone said installers?)
  • Internal scripts in Lua are really fast
  • Syntax errors in internal scripts are detected at package building time

How it works?

Just use -p <lua> in any script slot (%pre, %post, etc).

For example:

%pre -p <lua>
print("Wow! It really works!")

What is accessible from Lua?

The standard Lua library, the posix module (basic system access, by Luiz Henrique de Figueiredo and Claudio Terra), and the rex module (regular expressions, by Reuben Thomas).

Macro support

Support for Lua macros was also introduced. It means that one can create custom content using Lua macros anywhere.

For example:

%{lua: print("Requires: hello-world > 1.0") }
Posted in C/C++, Lua, Patch | Leave a comment

More additions to APT-RPM Lua interface

The APT-RPM Lua interface is constantly being improved. This time, the following functions were added:

pkgid() and verid()

Return a unique integer identifying a package or a version.

verpkg()

Returns the parent package of some given version.

verdeplist()

Returns a list of dependencies for a given package, including complete information about it.

These new functions were introduced to give support for something which is frequently asked by APT-RPM users: the ability to discover which installed packages are not required by any other installed package.

Here is a script using these functions to list these packages. This script will be called list-nodeps, and will be available in the contrib/ directory of the next APT-RPM release.

-- Collect dependencies from installed packages
deplist = {}
verlist = {}
for i, pkg in ipairs(pkglist()) do
    ver = pkgvercur(pkg)
    if ver then
        table.insert(verlist, ver)
        for i, dep in ipairs(verdeplist(ver)) do
            for i, depver in ipairs(dep.verlist) do
                deplist[verid(depver)] = true
            end
        end
    end
end

-- Now list all versions which are not dependencies
for i, ver in ipairs(verlist) do
    if not deplist[verid(ver)] then
        name = pkgname(verpkg(ver))
        if name ~= "gpg-pubkey" then
            -- Print package name and version without epoch
            -- (rpm -e friendly ;-).
            print(name.."-"..string.gsub(verstr(ver), "^%d+:", ""))
        end
    end
end

More information about the introduced functions is available in https://moin.conectiva.com.br/AptRpm/Scripting

Posted in C/C++, Lua, Project | Leave a comment

New KDE frontend for APT-RPM

Here is a screenshot of Kynaptic, a new experiment I’m working on. This is a KDE-based frontend for APT-RPM and APT. Kynaptic is based on Synaptic, and shares the same base code on top of the APT library.

On the right side you see a new toy: the interactive search dialog. This should definetly improve the user experience while looking for packages, and could eventually be ported to Synaptic itself.

Another interesting point is that during the development process of Kynaptic I’m also cleaning up the Synaptic code, something I had in my mind for some time already.

Posted in C/C++, Project | 1 Comment

APT-RPM article now in portuguese

I’ve translated to portuguese (pt_BR) the APT-RPM article I wrote for LWN. It’s available in printed media on Revista do Linux #50, and also on the LinuxIT site.

Posted in Article | Leave a comment

Toy interpreter for Linear Algebra

While working with Linear Algebra, I’ve decided to build a toy interpreter in C to play around. This was a quite interesting experiment for myself, since it was the first time I’ve built a complete (with tokenizer, compiler, and interpreter) and auto-suficient (no external dependencies, no additional tools) interpreter from the ground.

The interpreter is currently 1076 lines long, and is based on a tokenizer for the grammar, a compiler which creates a list of trees of expressions, an interpreter for the generated structure, and two modules for symbol maintenance and matrix operations.

Here is a quick example:

a = 2*3+4/2
print(a)

b = [1,2,3|4,5,6]
c = [1,2,3|4,5,6]
print((b+c)/2)
print(trans(b))

d = [1,0|-2,3|5,4|0,1]
e = [0,6,1|3,8,-2]
print(d*e)

f = [1,2|3,4]
print(det(f))
f = [2,3,4|-5,5,6|7,8,9]
print(det(f))
f = [1,-4,2,-2|4,7,-3,5|3,0,8,0|-5,-1,6,9]
print(det(f))

And here is the output:

8
[ 1, 2, 3 | 4, 5, 6 ]
[ 1, 4 | 2, 5 | 3, 6 ]
[ 0, 6, 1 | 9, 12, -8 | 12, 62, -3 | 3, 8, -2 ]
-2
-45
2042
Posted in C/C++, Math, Project | 1 Comment

New RPM transaction locking

Last week I’ve worked with Jeff Johnson to fix a small RPM issue which was bothering us for quite some time. Until this change, RPM was accepting two transactions to be run at the same time but, unfortunately, this might put the database in an inconsistent state, and some headers might get lost while restoring it with –rebuilddb.

Our fix was rather simple, based on fcntl() locking while transactions are being committed to the system. This should be enough, protecting databases while the issue is not fixed The Right Way (the time consuming one ;-) ).

Developers of applications linked with RPM should not worry about additional errors, since the current behavior is to wait for the lock to be released while entering in the transaction committing function.

Posted in C/C++, Patch | Leave a comment

Compiler warnings in Python’s SRE

After a long time, and many people complaining about it, I finally took some time to fix some annoying compiler warnings in Python’s regular expression engine. Since it’s a rather uncommon case, I’ll explain it here with a quick example.

Have a look at the following code:

#include <stdio.h>

int main(int argc, char *argv[])
{
   unsigned char c = 1;
   if (c < 256)
      printf("Hello world!n");
   return 0;
}

If you try to compile that code, you'll probably get a warning like this:

test.c: In function `main':
test.c:7: warning: comparison is always true due to
limited range of data type

Yes, the compiler is right. Our char type will never reach the given limit. On the other hand, suppose that this code is preprocessed, and that the c variable has sometimes a multibyte character type, like wchar_t, for example. In this case our test is legitimate, and the dozens of warnings being caused by a common macro are really annoying.

There are many different ways to remove these warnings. Unfortunately, the most obvious one, which is casting the variable to a larger type, doesn't work as expected.

My adopted solution was to reimplement the same test in another way, surpassing the compiler warnings for this specific case. Instead of writing (c < 255), I've written (!(c & ~255)). This is 100% safe for any unsigned type, which is my case. Ok, perhaps it's a little bit sick, but a large comment should leave everyone aware about the rationale, and away from the warnings.

Posted in C/C++, Patch, Python | Leave a comment

Functions vs. Callable objects in Lua

While working on Lunatic Python, I’ve understood that Lua does specific type checking with lua_isfunction() in some places where a callable type is expected. As a side effect, these places only accept a real Lua function when the callable object might be used. An example of such behavior is, at the time this is being written, the table.foreach() function:

> obj = {}
> mt = {__call=function() print("Called!") end}
> setmetatable(obj, mt)
> obj()
Called!
> table.foreach({a=1}, obj)
stdin:1: bad argument #2 to `foreach' (function expected, got table)
stack traceback:
        [C]: in function `foreach'
        stdin:1: in main chunk
        [C]: ?

The trick used in Lunatic Python to overwhelm this situation was to enclose the custom Python object type inside a real Lua C function closure. This trick might indeed be used anywhere this situation is found. Here are a few functions that allow this trick to be used from inside Lua:

static int lwrapcall(lua_State *L)
{
        lua_pushvalue(L, lua_upvalueindex(1));
        lua_insert(L, 1);
        lua_call(L, lua_gettop(L)-1, LUA_MULTRET);
        return lua_gettop(L);
}

static int lwrapfunc(lua_State *L)
{
        luaL_checkany(L, 1);
        lua_pushcclosure(L, lwrapcall, 1);
        return 1;
}

static int luaopen_wrapfunc(lua_State *L)
{
        lua_pushliteral(L, "wrapfunc");
        lua_pushcfunction(L, lwrapfunc);
        lua_rawset(L, LUA_GLOBALSINDEX);
}

And here is another implementation, by Alex Bilyk, in pure Lua:

function wrapfunc(callable)
    return function(...)
        return callable(unpack(arg))
    end
end

Using them one would be able to obtain the effect above as follows:

> table.foreach({a=1}, wrapfunc(obj))
Called!

Hopefully, in the future the standard Lua library will stop checking for a specific type in such cases, or implement some kind of lua_iscallable() checking.

Posted in Lua, Project, Python | Leave a comment