therning.org/ magnus

The upgrade to Gnome3 in ArchLinux a few days ago broke my previous setup that combined xmonad with Gnome. Gnome 3 has a fallback mode, but I found that the instructions for replacing metacity under Gnome 2 no longer worked. With some help from the xmonad mailing list (in particular Jens Petersen and his efforts of providing a working setup on Fedora) I now finally have a working setup again. Here’s how I did it.

Add a session file for use by Gnome Session (/usr/share/gnome-session/sessions/xmonad.session):

[GNOME Session]
Name=Xmonad session
RequiredComponents=gnome-panel;gnome-settings-daemon;
RequiredProviders=windowmanager;notifications;
DefaultProvider-windowmanager=xmonad
DefaultProvider-notifications=notification-daemon

And a desktop file for GDM (/usr/share/xsessions/xmonad-gnome-session.desktop):

[Desktop Entry]
Name=Xmonad GNOME
Comment=Tiling window manager
TryExec=/usr/bin/gnome-session
Exec=gnome-session --session=xmonad
Type=XSession

That’s all it takes. Of course I’ve raised a ticket against the Arch package.

Gnome 3 just hit the official ArchLinux repos a few days ago. It’s new, it’s slick, it’s shiny… but I don’t think it’s ready for general use just yet. It seems stable enough, but there’s just a few too many things missing to make it feel like it’s complete. Anyway, running Arch means that at times one has to live with not-quite-release-ready software anyway

The biggest issue I’ve come across with Gnome 3, and especially Gnome Shell and the window manager, is configuring the themes. I was pointed to a fairly good article on customising the Gnome Shell, but it suggests modifying system files which is a bad thing to do even on single-user systems. So this post should be read as an addendum to that one.

First of all install the User Theme Gnome Shell Extension. The AUR packages available pull the source from its git repo because there doesn’t seem to be any releases of the extensions just yet. When using the bleeding edge source I ran into problems with Gnome Shell crashing so I advise against using it. I’ve had success with the source tagged at 3.0.1, you can find an Arch source package for Gnome Shell User Theme that I put together based on one of the AUR packages. Build and install that, then restart Gnome Shell (Alt-F2, r, return). Then verify that the extension has been loaded by using Looking Glass.

Then create copies of the default themes using rsync:

% rsync -a /usr/share/themes/Adwaita ~/.themes
% mv ~/.themes/Adwaita ~/.themes/Adwaita2
% mkdir -p ~/.themes/Default/gnome-shell
% rsync -a /usr/share/gnome-shell/theme ~/.themes/Default/gnome-shell

Then modify the file ~/.themes/Adwaita2/index.theme so that each mention of Adwaita says Adwaita2 instead, except for the cursor theme.

Make sure gnome-tweak-tool is installed (it’s in a package with the same name). Run it and change the shell theme to Default,the windows theme to Adwaita2, and the interface gtk+ theme to Adwaita2 as well.

Now you return to the article on configuring Gnome Shell, but instead of modifying the system files modify the ones in your ~/.themes.

As a follow-up to my earlier post on cblrepo I thought I’d convert a snapshot of ArchHaskell HABS to cblrepo. It’s mostly done as an exercise and to serve as an example. You can find it at http://www.kiwilight.com/~magnus/habs/.

Of course I have used it to build all the packages, and I still have the result of that around, so if anyone asks I just might upload that as well.

Maintaining a large set of Haskell packages for a Linux distribution is quite a chore. Especially if one wants to track Hackage as far as possible. Several distributions have tools to automatically convert Cabal-based packages into distribution packages, e.g. cabal2arch for ArchLinux and cabal-rpm. They are just conversion tools though, and the most time-consuming activity in maintaining Haskell packages is resolving and verifying dependencies.

At least that was my experience when I was actively involved in ArchHaskell. I only saw two options when adding or upgrading a package, either I worked out dependencies manually, or I simply tried it out. Neither of them was very appealing, and both were very time-consuming. It seemed obvious that I needed some tool to help out.

Enter cblrepo!

It allows me to maintain a database of specific versions of packages, and when I want to upgrade a package, or add a new one, it’ll verify that all dependencies can be satisfied. In other words, it helps me maintain a buildable set of packages at all times.

The tool also has some functionality that helps in tracking Hackage as packages are updated there.

Something about how it works

At the moment I maintain a small repository of Arch packages, mostly just to try out cblrepo and convince myself that it works. The work environment contains a database and a directory of patches:

% ls
cblrepo.db  patches/
%

The database is a cleartext file containing the information on the packages. It’s basically just a dump of the related Haskell datatype, encoded in JSON. The patches directory holds patches for Cabal files and PKGBUILD files. They must be named patch.cabal.<hackage name> or patch.pkgbuild.<hackage name> in order to be picked up by cblrepo.

There’s also an application directory (~/.cblrepo) for caching info about the packages available on Hackage:

% ls ~/.cblrepo
00-index.tar.gz
%

How to use it

A session with cblrepo looks something like this. First we update the information about what packages are available on Hackage:

% cblrepo idxsync
%

After that it’s possible to see what packages are out-of-date:

% cblrepo updates
cmdargs: 0.6.8 (0.6.9)
test-framework-th: 0.1.3 (0.2.0)
xml: 1.3.7 (1.3.8)
language-haskell-extract: 0.1.2 (0.2.0)
blaze-builder: 0.2.1.4 (0.3.0.0)
%

Let’s check whether cmdargs can be updated:

% cblrepo add -n cmdargs,0.6.9 %

It generates no output, so that means it’s possible to update. When attempting to add all the packages we run into a problem:

% cblrepo add -n cmdargs,0.6.9 \
> test-framework-th,0.2.0 \
> xml,1.3.7 \
> language-haskell-extract,0.2.0 \
> blaze-builder,0.3.0.0
Adding blaze-builder 0.3.0.0 would break:
  haxr : blaze-builder ==0.2.*

We’ll leave blaze-builder at the current version for now:

% cblrepo add cmdargs,0.6.9 \
> test-framework-th,0.2.0 \
> xml,1.3.7 \
> language-haskell-extract,0.2.0
%

After these updates we also need to make sure that all packages that depend on these ones are re-built, that is we need to bump their release version:

% cblrepo bump -n cmdargs \
> test-framework-th \
> xml \
> language-haskell-extract
Would bump:
test-framework
test-framework-hunit
test-framework-quickcheck2
%

Just re-run that without the -n to actually perform the bump. Now that all this is done we need to generate the files necessary to build the Arch packages. We can easily check what packages need re-building, and get a good order for building them:

% cblrepo build cmdargs \
> test-framework-th \
> xml \
> language-haskell-extract
cmdargs
xml
test-framework
test-framework-quickcheck2
test-framework-hunit
language-haskell-extract
test-framework-th
%

And generating the required files is also easy:

% cblrepo pkgbuild $(!!)
% tree
.
|-- cblrepo.db
|-- haskell-cmdargs
|   |-- haskell-cmdargs.install
|   `-- PKGBUILD
|-- haskell-language-haskell-extract
|   |-- haskell-language-haskell-extract.install
|   `-- PKGBUILD
|-- haskell-test-framework
|   |-- haskell-test-framework.install
|   `-- PKGBUILD
|-- haskell-test-framework-hunit
|   |-- haskell-test-framework-hunit.install
|   `-- PKGBUILD
|-- haskell-test-framework-quickcheck2
|   |-- haskell-test-framework-quickcheck2.install
|   `-- PKGBUILD
|-- haskell-test-framework-th
|   |-- haskell-test-framework-th.install
|   `-- PKGBUILD
|-- haskell-xml
|   |-- haskell-xml.install
|   `-- PKGBUILD
`-- patches

8 directories, 15 files
%

Now all that’s left is running makepkg in each of the directories, in the order indicated by cblrepo build above.

Unfortunately they won’t all build—generating the Haddock docs for test-framework-th fails. That’s however fairly easy to remedy by patching the PKGBUILD to disable the generation of docs.

I’ll get back to that in a later post though.

Your comments, please

Please leave comments and suggestions. I’m planning on uploading the source to github shortly.

When trying to maintain set of binary packages of Haskell libraries for a Linux distribution there are a few issues that come up:

1. The set of packages must be compilable at all times, and
2. Updating one package requires all packages that depend on it, in one or more steps, to be re-compiled.

The first requires keeping track of all dependencies of the packages in the set and making sure that they are satisfiable at all times. For a while I was doing this by simple attempting to compile all updated packages and check for breakages. Which was both time-consuming and a painful when build-failures had to be resolved.

The second requires bumping the package release number for all packages that are reachable when following the dependencies in the reverse direction. Doing this manually is tedious and very error prone in my experience.

Of course it ought to be possible to make this a lot easier with the help of a tool. The last few days I’ve been writing such a tool. This is how I’ve been using it so far.

Building the initial database

GHC in ArchLinux ships with a few Haskell libraries and ArchLinux also has a few Haskell packages in its base repositories. Since I don’t need to maintain any of those packages I decided to treat these as a sort of base. Adding those is as simple as this:

% head base-pkgs
base,4.2.0.2
array,0.3.0.1
bytestring,0.9.1.7
Cabal,1.8.0.6
containers,0.3.0.0
directory,1.0.1.1
extensible-exceptions,0.1.1.1
filepath,1.1.0.4
haskell98,1.0.1.1
hpc,0.5.0.5
% cblrepo addbasepkg $(cat base-pkgs)
Success

Then I need to add the packages of the binary repo provided by ArchHaskell. I wrote a little script that extracts the package name and version from the ArchHaskell HABS tree (get-ah-cabals):

#! /bin/bash
 
habsdir=$1
 
for d in ${habsdir}/habs/*; do
    . ${d}/PKGBUILD
    case $_hkgname in
        (datetime|haskell-platform)
            ;;
        (*)
            echo ${_hkgname},${pkgver}
            ;;
    esac
done
 
echo http://hackage.haskell.org/platform/2010.2.0.0/haskell-platform.cabal

Since haskell-platform isn’t on Hackage it requires special handling. The reason why datetime is excluded is slightly different. It’s the only package that requires old base (version <4). GHC in Arch does whip with both old and new base so datetime can be built, but cblrepo can’t deal with two versions of the same package. This is a limitation, but I’m not sure it’s worth fixing it since base is the only library that comes in two versions, and datetime is the only package that hasn’t been updated to use new base.

Knowing this it’s easy to add all the ArchHaskell packages to the database:

% cblrepo idxupdate
% cblrepo add $(get-ah-cabals path/to/habs)
Success

Attempting an update

Now it’s possible to attempt to attempt an update:

% cblrepo add neither,0.2.0
Failed to satisfy the following dependencies for neither:
  monad-peel >=0.1 && <0.2
Adding neither 0.2.0 would break:
  yesod : neither >=0.1.0 && <0.2
  persistent : neither >=0.1 && <0.2

The way to read this is that there first of all is a missing dependency to satisfy for neither itself, and second there are two packages, yesod and persistent, that wouldn’t be buildable if neither were updated.

Now if it were possible to update neither, what packages would require a bump?

% cblrepo bump neither     
persistent
yesod

A few days ago I noticed that there were a few Haskell packages on AUR that had received updates. This was the excuse I had been waiting for to address the second part of keeping my Haskell packages up-to-date (I’ve written about the first part before, dealing with an update to GHC).

It’s easy to find the packages with available updates:

% yaourt -Su --aur

Unfortunately it isn’t as easy as just updating the listed packages. Any package that depends on an updated package really should be re-compiled and re-installed to guarantee that the entire system behaves as expected after the upgrade. Of course pacman can handle it:

% pacman -Rcn <all pkgs with updates>

This will list all packages that will be removed. After removing them all, they can all be re-installed. That is of course not quite as nice as it could be, since they all then will be explicitly installed, it would be nicer to just re-install the “top-level packages”. This is one way to achieve this.

I did a bit of refactoring and put the Arch-related functions from the previous post in their own module, Arch. Then I added a function that takes a package and recursively inspects Required By until a “top-level package” (i.e. a package that doesn’t require any other package) is reached:

getTopRequiredBy pkg = let
        tops = do
            first <- getRequiredBy pkg
            if null first
                then return [pkg]
                else liftM concat $ mapM getTopRequiredBy first
    in liftM nub tops

After that it’s straight forward to write up a little tool which offers some advice on what to do:

main = do
    pkgsToUpgrade <- getArgs
    pkgsToReinstall <- liftM (nub . concat) $ mapM Arch.getTopRequiredBy pkgsToUpgrade
    putStrLn $ "To remove     : pacman -Rnc " ++ unwords pkgsToUpgrade
    putStrLn $ "To re-install : yaourt -S " ++ unwords pkgsToReinstall

Using it on the packages I wanted to upgrade gave the following output:

% runhaskell PkgUpgrade.hs haskell-{configfile,haxml,json,missingh,safe,testpack,time}
To remove     : pacman -Rnc haskell-configfile haskell-haxml haskell-json haskell-missingh haskell-safe haskell-testpack haskell-time
To re-install : yaourt -S haskell-configfile haskell-haxml haskell-json haskell-hsh haskell-safe

Following that advice seemed to work just like I intended.

Arch is somewhat of a hybrid distribution in the sense that if you have any sort of ‘peculiar’ needs then you are likely to have to build packages from source. As expected developing in Haskell is a “peculiar need”

After every upgrade of GHC I find myself in the situation where the system (pacman) and GHC have different views of what packages are available. What is needed then is somehow finding out the difference, and this is how I found that difference after the recent 6.10.3 -> 6.10.4 upgrade of GHC. Once I know what packages are missing from GHC’s view of the world I can use pacman to first remove and then yaourt to rebuild the packages.

First I noted that the old package.conf file wasn’t deleted during the upgrade, apparently pacman noted the changes that installing packages resulted in and saved the file as package.conf.pacsave. Finding the name of all the ‘missing’ packages was then as simple as loading both /usr/lib/ghc-6.10.3/package.conf.pacsave and /usr/lib/ghc-6.10.4/package.conf, filter out the package names and take the difference:

printMissingPackages = let
        pkgNameStr = PackageName . display . packageName
    in do
        oldPackConf <- readFile "/usr/lib/ghc-6.10.3/package.conf.pacsave"
        curPackConf <- readFile "/usr/lib/ghc-6.10.4/package.conf"
        let oldPacks = (read oldPackConf) :: [InstalledPackageInfo_ String]
        let curPacks = (read curPackConf) :: [InstalledPackageInfo_ String]
        let gonePacks = (map pkgNameStr oldPacks) \\ (map pkgNameStr curPacks)
        putStrLn "Missing packages:"
        mapM_ (putStrLn . display) gonePacks

That isn’t the most useful output however, so I decided to modify it to print out the name of the Arch package that needed re-compilation. The following functions generates the name of the .hi of the first module in the Haskell package, it then uses pacman to look up the owner of the file:

ghcPkg2ArchPkg pkg = let
        hsFileLoc = head $ libraryDirs pkg
        hsFile = map (\ c -> if c == '.' then '/' else c) $ head $ exposedModules pkg
        hsFullFile = hsFileLoc </> hsFile <.> "hi"
    in do
        exists <- doesDirectoryExist hsFullFile
        if exists
            then liftM Just $ archOwnerOfFile hsFullFile
            else return Nothing
 
archOwnerOfFile fn = let
        pkgFromPacmanOutput = head . tail . reverse . words
    in do
        res <- rawSystemStdout silent "/usr/bin/pacman" ["-Qo", fn]
        return $ pkgFromPacmanOutput res

Now I can find the list of Arch packages that aren’t known to the new version of GHC by mapping ghcPkg2ArchPkg over gonePkgs. In other words that is the list of packages that need to be removed, but that can be different from the list of packages that needs to be rebuilt with yaourt (basically I only want to tell it to build and install the ‘top-level’ packages, i.e. packages that aren’t dependencies of any other packages. It’s of course possible to build that second list from the first one, with the help of pacman.

archGetRequiredBy pkg = let
        extractPkgs pkgDesc = let
                deps = (drop 3 . words . head . filter (isPrefixOf "Required By") . lines) pkgDesc
            in
                if deps == ["None"]
                    then []
                    else deps
    in do
        res <- rawSystemStdout silent "/usr/bin/pacman" ["-Qi", pkg]
        return $ extractPkgs res

Now I can modify printMissingPackages to print some more useful information. This is the full function:

printMissingPackages = let
        pkgNameStr = PackageName . display . packageName
    in do
        oldPackConf <- readFile "/usr/lib/ghc-6.10.3/package.conf.pacsave"
        curPackConf <- readFile "/usr/lib/ghc-6.10.4/package.conf"
        let oldPacks = (read oldPackConf) :: [InstalledPackageInfo_ String]
        let curPacks = (read curPackConf) :: [InstalledPackageInfo_ String]
        let gonePacks = (map pkgNameStr oldPacks) \\ (map pkgNameStr curPacks)
        putStrLn "Missing packages:"
        mapM_ (putStrLn . display) gonePackprints
        let gonePkgs = filter (\ p -> pkgNameStr p `elem` gonePacks) oldPacks
        archPkgs <- liftM catMaybes $ mapM ghcPkg2ArchPkg gonePkgs
        putStrLn "Packages to remove:"
        mapM_ putStrLn archPkgs
        archTopPkgs <- filterM (liftM ([] ==) . archGetRequiredBy) archPkgs
        putStrLn "\nPackages to install:"
        mapM_ putStrLn archTopPkgs

On my system it produced the following output:

Missing packages:
terminfo
vty
wl-pprint

Packages to remove:
haskell-terminfo
haskell-vty
haskell-wl-pprint

Packages to install:
haskell-vty
haskell-wl-pprint

And after a quick pacman -Rn ... and a not so quick yaourt -S ... I reran it and the output was

Packages to remove:

Packages to install:

Exactly as expected.

I read all my email via IMAP, and send all email via external SMTP servers, but at times it’s really useful to be able to run something like /usr/sbin/sendmail to send an email. Many tools depend on it, for me I want reports on cron jobs and I like using darcs send straight to an email address. For this a full-fledged SMTP server like exim/postfix/sendmail clearly is overkill. First I had a quick look at ssmtp but it only allows for a system-wide configuration file. Ideally I’d like “system-mails” (like cron) to be sent via my ISP while more personal emails should be sent via my Google account. After a suggestion on the Arch mailing list I checked out msmtp and it was exactly what I was looking for.

I created /etc/msmtprc with the following contents:

defaults
logfile ~/.msmtp.log
 
# MyISP
account myisp
host smtp.myisp.com
from my.email@myisp.com
 
account default : myisp

I then found out that [dcron][3] is severely limited; it doesn’t honour MAILTO in crontabs and it’s not possibly to use anything but /usr/sbin/sendmail to send emails. The lack of support for MAILTO is the deal breaker in this case.

Rather than create a symbolic link I modified fcron‘s configuration file, /etc/fcron/fcron.conf, to use /usr/bin/msmtp:

sendmail = /usr/bin/msmtp

I also made sure to stick MAILTO=my.email@myisp.com in the system crontab, using fcrontab -u systab -e.

Then I created the file ~/.msmtprc:

defaults
logfile ~/.msmtp.log
 
# MyISP
account mysisp
host smtp.myisp.com
from my.email@myisp.com
 
# google
account google
host smtp.gmail.com
from user@googlemail.com
tls on
tls_certcheck off
auth on
user user@googlemail.com
password MySecretPassword
 
account default : google

Finally I put MAILTO=user@googlemail.com in my user’s crontab.

So, I’m struggling somewhat with the move to Arch. Not that Arch is problematic in any way, it seems to be me and my “debianised thinking”.

I noticed that the Vim plugins that were packaged for Arch all install in system directories and hence are imposed on all users. Not a good thing. So, inspired by Debian’s vim-addons-manager, I hacked up a solution relying on pacman. Then I packaged two Vim plugins for Arch. The idea was to do rather than talk, so it wasn’t until after this that I asked for feedback on aur-general. Just to have the idea shot down

Anyway, after being presented with an existing solution more inline with “the Arch way” I decided to try it out. I’ve now subscribed to using GetLatestVimScripts. Brilliant.

Now I need to convince the author of haskellmode to put it on vim.org and I really ought to get my packages off AUR.

A few days ago I packaged the vim haskellmode for Arch Linux and uploaded it to AUR.