therning.org/ magnus

<rant>So my irritation with Debian on the desktop reached a critical moment over the last few days. Sid is a good desktop system for most things, but if you are interested in Haskell then you really should prepare yourself for being a little behind on things. If it isn’t GHC that is out of date then it’s some library that hasn’t been rebuilt for the latest version of GHC. If you’re lucky then new packages are uploaded to the NEW queue and it’s only a matter of waiting for the autobuilders to get to them. Of course that doesn’t mean you can get those packages, so not that much luck in the end anyway.</rant>

I decided to move to the only other distro I’ve used seriously since Linux’ move to 2.x kernels, Arch. I remember it as a slightly less polished than Debian, but more up-to-date than even Sid. Also, there is a vibrant group of Haskell hackers working on providing almost all of Hackage in Arch’s native packaging system.

Installing Arch turned out to be a bit of a blast from the past, even when compared to Debian. Luckily the Arch wiki is there to help with the things that are different compared to Debian. I would suggest having a second computer with a browser pointed to Arch’s home pages nearby during the entire install.

For my installation of Debian I had opted to use LVM and I wanted to keep my /home around so I had to follow the steps required to use LVM in Arch. Installation went smooth and the reboot was successful. Now starts the hard work—configuring the system and updating my home folder.

The very first thing I noticed was that vim wasn’t installed by default, only vi was available. Not a big problem, except that my muscle memory resulted in a lot of vim: command not found messages. Clearly I had to install vim just to keep my sanity.

Next thing to do was to install X.org and the video driver I need:

# pacman -S xorg xf86-video-intel

After adding hal to /etc/rc.conf I started it manually and I ran startx as non-root. Wow, it all worked perfect, without creating any configuration file for X first. I double checked and it used the driver I wanted, i.e. it didn’t pick a safe default, like vesa, as its first option. Brilliant.

After following the instructions for changing the keyboard layout in X.org using hal I even had my ‘@’ on the right key. (The most confusing thing was that in X.org the UK keyboard layout is called gb, while in the console setup it’s called uk.)

I’ll post a bit more about the setup and configuration of my system as I progress towards my ideal desktop.

I have written a few posts on debianising haskell packages in the past. Back then I stopped looking at it because of the age of the ghc-related packages in Debian Sid. Things have since improved and I’ve finally found some time to revisit the problem. This isn’t however a technical post, but rather a post on the strategy I think would be best for making Debian the best Haskell development environment in the universe.

In my opinion there is no point in trying to get all packages on Hackage into Debian proper. The few Debian developers (DD) who are interested in Haskell and maintain the existing packages are already terribly short on time and adding further burden would improve the chance of timely availability of GHC in the future. Sure, it would be possible to recruit more DDs with an interest in Haskell (I’ve considered in the past to try to become a DD, but the timing was off on both sides and I decided to postpone it) but it would take time. Maintaining 1000 packages is a huge task, and would require quite a large team I think. I suspect the recruiting would be such that if one were to plot the number of packages the team could maintain as it grew over time, compared to the number of packages on Hackage, then the lines would diverge rather than converge

Instead I think all packages on Hackage should first be debianised and put in a source-only repository. As some packages grow in importance and get added into Debian proper (e.g. a killer-app written in Haskell) then it and its dependencies are removed.

I believe that with this solution it’d be possible to quickly make a large part of Hackage available on Debian while still giving the DDs time to grow the number of “Haskell-friendlies” within Debian.

This of course rests on a few assumptions:

1. It would require a smaller team to maintain source packages than binary packages. At least it would require less time from DDs.
2. There is some space to store the source packages, ideally non-Debian space since that would mean even less involvement required by DDs.
3. There’s a tool available in Debian that makes it easy for end-users to download, compile, and install source packages.
4. There’s a tool that automates the creation of a Debian source package from a Cabal package.

In my optimism I’ve decided to not worry about the two first points and instead focus on the last two. Besides the optimism they also feel like the points I can actually address on my own, I’ll need help with the first two.

The tool in Debian for handling source packages is apt-src. In my opinion it will suffice, but if the developer would address one issue then it would be perfect.

The tool for automating the creation of Debian source packages could be cabal-debian. It is a very good tool, and it seems to be suited for maintaining Debian packages. I only have two objections to it. The first objection I have is that it is almost too well engineered; it does more than is necessary, e.g. there is no need to maintain proper changelogs in the source repo. The second is that it requires all dependencies of a package to be installed when it’s debianised. It is almost certainly the tool for the team charged with maintaining a large set of Cabal-based packages in Debian proper, but for the source-repo it might be too much. Because of that I’ve started hacking on a simplistic tool that I feel would be better suited for the source-repo.

First of all I decided to make this a new post rather then answer in the comment section in my previous post. I think that this way the chance is greater that both David and Jeremy will read this

So, first let’s set the stage… It’s almost midnight and I’ve just spent somewhere between 30 and 45 minutes hunting down packages on Haskell and navigating the directory listings on src.seereason.com. I’ve been sent in the wrong direction once (downloading a Debian package that seems to be from Linspire instead of grabbing the identically named package from the seereason site. I’m tired and slightly frustrated. I run cabal-debian, it works, then I run debuild just to be told that I really should have build cabal-debian itself into a Debian package and installed it that way and I also need another package that isn’t in Debian at all.

Anyway, having slept and let almost 24 pass by I’ve come to my senses somewhat

Here’s what think is a slightly more tempered whishlist:

1. Remove haskell-devscripts-cdbs and include hlibrary.mk in haskell-devscripts.
2. Modify dh_haskell* so that the two snippets for -doc are automatically generated instead (just like the script snippets for -dev already are).

I think it would be easy to stick that in seereason’s repos for public consumption. Then interested people, like me, can start thinking about how to use that as a base to make Debian the best Haskell platform in the world, or at least making it better than Arch

Completely unrelated question for seereason people

I have the following line in my source.list:

deb     http://deb.seereason.com/debian sid-seereason main

Why don’t I see the seereason-keyring package?

So, after receiving several pointers to seereason’s cabal-debian tool I thought I’d take it for a spin.ⁱ

After about 30 minutes of browsing through HackageDB and seereason’s source repos, building and installing, I had finally satisfied all dependencies and the build of cabal-debian succeeded. (Oh, BTW, seereason people, it’s really confusing that you have a package called debian among your source repos, when there already is a different debian package on HackageDB. Please consider renaming it!) I decided to take the tool for a spin on my own package, dataenc.

The result was fairly good. It seems the generated files depend on some packages that either aren’t in Debian or that the people at seereason have modified. With the following changes to the generated files I was happy with the contents of ./debian and I successfully built an (almost) autogenerated Debian package:

1. Download hlibrary.mk into ./debian.
2. Modify ./debian/rules so that hlibrary.mk is loaded from $(CURDIR)/debian/hlibrary.mk.
3. Delete ./debian/libghc6-dataenc-doc.post*.
4. Remove cabal-debian and haskell-devscripts-cdbs from the build dependencies in ./debian/control.

I’ll try to find the time to put those changes into cabal-debian itself. Then I’d have a rather nice tool for building Debian packages automatically.

1. I did pull down autobuilder as well, but didn’t feel I had enough time to explore it tonight.[back]

I’ve just spent the last hour or so ironing out the details required to automate the building of cabalised Haskell packages for Debian. At the same time I also built Debian packages for 5 Haskell packages (test-framework and its dependencies).

These are the basic steps I’ve followed:

1. Extract the tar-ball in the working directory.
2. Rename the directory to match the form haskell-X-0.1.
3. Re-tar the directory into an archive named haskell-X_0.1.orig.tar.gz.
4. Change into the package directory.
5. Run Cabal’s configure command followed by register --gen-pkg-conf=pkg.conf.

6. Modify the generated file and turn it into a .ini-style file:

   echo '[conf]' > pkg.conf.tmp; cat pkg.conf >> pkg.conf.tmp; mv pkg.conf.tmp pkg.conf
   

7. Use Python to extract the dependencies:

   python << EOF
   from ConfigParser import ConfigParser
   cfg = ConfigParser()
   cfg.read('pkg.conf')
   deps = cfg.get('conf', 'depends').split()
   for d in map(lambda s: s.rsplit('-', 1)[0].lower(), deps):
       print d
   EOF
   

8. Clean up by rm pkg.conf and running Cabal’s clean command.

9. Create the debian directory.
10. Create debian/compat: echo 5 > debian/compat

11. Create debian/changelog, it should be something like:

    haskell-X (0.1-1) unstable; urgency=low
    
      * Automatic build.
    
     -- Magnus Therning <magnus@therning.org>  Mon, 19 Jan 2009 22:35:00 +0000
    

12. Create debian/control. This is by far the most time consuming part to do manually. It’s also a bit long so I’ll leave it out of this post. Take a look in my APT repo if you are interested in the details.

13. Copy hlibrary.mk into debian/. At some point it will be included in some Debian package (bug #462482) but for now it can be found here.

14. Create debian/rules with

    #!/usr/bin/make -f
    include $(CURDIR)/debian/hlibrary.mk
    

    and make it executable, chmod +x debian/rules.

15. Done. Run debuild.

My intention is to at some point encode these steps in a tool so that I can spew out Debian packages for all interesting packages on HackageDB at the press of a button. Of course that tool will be written in Haskell, and it sure won’t rely on Python for any part of the process

Readers who are familiar with the rules for Debian packages will notice that this won’t create well-formed packages. i’m not particularly interested in getting these packages into Debian proper, I see the package generated this way living in an unofficial repo. So there is no need to make sure they are acceptable for inclusion in the distribution itself. The resulting packages might offer a good starting point for someone who feels that a package deserves to become official. I also suspect that a tool that generates fully compliant packages, and is able to deal with keeping them up-to-date, is too big a task for me. I’m also not sure it really is worth it, better to keep things as simple as possible in order to deal with the amazing rate of new releases on Hackage.

After reading about screenshots.debian.net I’ve now uploaded a screenshot for the only package I have in Debian, keysafe. It will be visible to the public as soon at it has been approved.

This is how it’s done.

I hope this will be out of date really soon, but here’s what I did to solve the problems I had with the nVidia driver after upgrading to the latest kernel in Sid (2.6.25). First download the file NVIDIA_kernel-169.12-2286310.diff.txt from this forum article. Then, as root do:

# cd /usr/src
# m-a clean nvidia-kernel
# tar -x -j -f nvidia-kernel.tar.bz2
# pushd modules/nvidia-kernel/nv
# patch -p3 < NVIDIA_kernel-169.12-2286310.diff.txt
# popd
# m-a -O build nvidia-kernel

Now there’s a package ready for installation

I just went through the exercise of setting up MediaWiki on a Debian Etch box, but instead of serving it using the common choice of Apache I wanted to use lighttpd. It seems the combination MediaWiki and Apache is well supported on Debian. There was even an install-time question whether I wanted the configuration files set up for me. Well, it’s boring to follow the common path, right?

It seems the combination MediaWiki and Apache is well supported on Debian. Luckily that doesn’t mean it’s difficult to serve it with lighttpd. Here’s how I configured things, and luckily it all seems to work just fine

Installation

First off, I installed mediawiki1.7, lighttpd, php5-cgi, and mysql-server. I made sure that aptitude only pulled in required packages and then I started un-choosing all Apache-related packages. By the time you get to installing MySQL you’ll be asked for a root password, that is if your debconf is set to ask medium-level questions.

Setting up MediaWiki

The Debian package for MediaWiki creates a site in /var/lib/mediawiki1.7 consisting mostly of symbolic links to the actual location of its files. I kept lighttpd’s default document root of /var/www and linked the two by creating a symbolic link /var/www/mw pointing to the MediaWiki site (/var/lib/mediawiki1.7).

Configuration of lighttpd

First enable FastCGI by linking /etc/lighttpd/conf-available/10-fastgi.conf into /etc/lighttpd/conf-enabled/. Then go in and change the bin-path to point to /usr/bin/php5-cgi. To prepare for the MediaWiki configuration, enable mod_rewrite in /etc/lighttpd/lighttpd.conf and finally create the file /etc/lighttpd/conf-enabled/20-mediawiki.conf with the following contents:

url.rewrite-once = (
    "^/$" => "/mw/index.php",
    "^/wiki/([^?]*)(?:\?(.*))?" => "/mw/index.php?title=$1&$2"
  )

Now it’s time to restart lighttpd.

Configuration of MediaWiki

Point a browser to http://your.server/mw/config/index.php to configure the site settings. When that’s done copy the file /var/lib/mediawiki1.7/config/LocalSettings.php one layer up.

That should be it! Enjoy!

I’m writing this mostly so that I’ll remember myself

If pbuilder --create fails when building a base image for Sid then it’s worth trying to build a base image for the stable release and then upgrade to Sid afterwards. The very manual way is pbuild --login --save-after-login.