therning.org/ magnus

A note to the future me:

Are your #ifdef sections greyed out despite switching to a profile where the macro is set?

Read this bug comment!

This is my input into the recent discussion on referential transparency (RT). I’m nowhere near as well versed in the subject as others, but how am I ever to learn anything unless I put my thoughts out there for them to be laughed at and ridiculed?

It all started with a post on stackoverflow.com, which received several very long and detailed responses, in particular from Uday Reddy (here and here). His answers were also linked to from Reddit. His second response contains a link to an excellent paper by Strachey, Fundamental concepts in programming languages. I’d go as far as saying that, despite it being lecture notes rather than a fully worked paper, it ought to be required reading for all software developers.

The rest of what I write here hinges on me actually understanding what Strachey writes in his paper. Of course I’m looking forward to comments/corrections/etc that help me correct my understanding.

What Strachey says about RT

In section 3.2.1 he introduces RT like this:

One of the most useful properties of expressions is that called by Quine referential transparency. In essence this means that if we wish to find the value of an expression which contains a sub-expression, the only thing we need to know about the sub-expression is its value. Any other features of the sub-expression, such as its internal structure, the number and nature of its components, the order in which they are evaluated or the colour of the ink in which they are written, are irrelevant to the value of the main expression.

There is however a crucial bit just before that:

Like the rest of mathematics, we shall be concerned only with R-values.

That is, he starts out with a very limited subset of what most people would consider a usable imperative programming language.

He then dives into some more details in section 3.2.2 by adding the concept of environment, which is handled through the use of a where-clause, or alternatively using let-statements (this ought to be making any Haskell developer feel right at home). After a few interesting sections on stuff like applicative structure, evaluation, and conditional expressions he finally tackles the issue of variables in section 3.3.1. There are two pieces to the trick, the first is to take advantage of his earlier insight that lead to a split of values into L-values and R-values:

If we consider L-values as well as R-values, however, we can preserve referential transparency as far as L-values are concerned. This is because L-values, being generalised addresses, are not altered by assignment commands. Thus the command x := x+1 leaves the address of the cell representing x (L-value of x) unchanged although it does alter the contents of this cell (R-value of x). So if we agree that the values concerned are all L-values, we can continue to use where-clauses and lambda-expressions for describing parts of a program which include assignments.

The cost of this is that the entire theory constructed earlier for operations taking R-values now has to be revised to incorporate L-values. The outline for this is in the rest of section 3.3 and it basically comes down to include an abstract store in the environment. However, before doing that he mentions that:

I think these problems are inevitable and although much of the work remains to be done, I feel hopeful that when completed it will not seem so formidable as it does at present, and that it will bring clarification to many areas of programming language study which are very obscure today. In particular the problems of side effects will, I hope, become more amenable.

He does reach his goal, but it’s a bit unfortunate that he stops short of considering the wider of problem of side effects. My assumption is that this would have to be dealt with in a similar way to assignment, but that would mean that rather than just adding an store to the environment the world, or a subset of it, would need to be added.

An open question (to me) is if anyone has built on Strachey’s work in this area and thought of the details of RT and general side effects?

RT in Haskell

The original question described RT as

it means you can replace equals with equals

which I actually think is a rather good, and very short, description of it. It’s not the full story, there are further important details, but it’s a good first intuition. Also, it’s a description usable in Haskell. Well, to be slightly more nuanced, it good for Haskell without IO (Haskell-IO). However, this is where the strict type system of Haskell really starts to shine because (here I’m probably a bit imprecise) we only have R-values in Haskell-IO. If we want to use assignment we add the use of a state monad, and we do that explicitly.

A former colleague of mine said that in Haskell we need to build up our own computational models ourselves. For instance, if we need assigment we use State, if we need to record progress we use Writer, etc. In other languages the language designer has already made all those choices for us, we don’t get to make them ourselves. For RT it means that Haskell is more explicit in what the environment of a function is.

Moving on to general side effects those are also more explicit in Haskell since they have to happen inside the IO monad. That alone is a great boon for RT in Haskell since it becomes explicit where RT as worked out by Strachey applies directly, and where there are (hopefully amenable) problems of side effects left. Even further, in Haskell it’s possible to make subsets of IO (by wrapping IO, see e.g. my own posts on wrapping IO, part 1 and wrapping IO, part 2). I’m sure that if including the world in the environment is the way to achieve RT with general side effects, then it’s highly beneficial to be able to create subsets of the world.

RT in Haskell vs. RT in (common) imperative languages

Uday writes in his first answer that:

But, today, functional programmers claim that imperative programming languages are not referentially transparent. Strachey would be turning in his grave.

This may well be true, but I think that when a Haskell programmer says it, he’s only twitching slightly. The reason? Strachey writes:

Any departure of R-value referential transparency in a R-value context should either be eliminated by decomposing the expression into several commands and simpler expressions, or, if this turns out to be difficult, the subject of a comment.

Which is something that Haskell programmers do naturally by use of IO. That is, in Haskell you either have an R-value, and you clearly see that you do, or you put in a comment, which is encoded in the type of the function.

This rather lengthy post basically arrives at the following, which is what I suspect the user [pacala is saying about RT on Reddit][reddit-pacala]:

Imperative languages my well be RT, but when trying to understand a code base the environment of each function is so large that understanding is an intractable problem. I don’t have this problem in Haskell.

Just in case you see the utter logic in developing for Windows on Linux

In the root of the unpacked Boost:

1. Run ./bootstrap.sh --with-python=$(which python2) --prefix=${HOME}/opt/boost-win --without-icu
2. Modify project-config.jam like this:

        # file.
        if ! gcc in [ feature.values  ]
        {
       -    using gcc ;
       +    using gcc : : i486-mingw32-g++ ;
        }
       
        project : default-build gcc ;

3. Compile and install by running ./bjam --layout=system variant=release threading=multi link=shared runtime-link=shared toolset=gcc target-os=windows threadapi=win32 install

Limit what is built by adding e.g. --with-program_options to that last command.

In the interest of full disclosure: this post is related to what I do for a living, development of and for embedded systems. I work for Semcon, but they don’t make me wear a suit and tie so these are my words, and mine alone.

A bit of background info

In a recent project we had a system where the turning of the wheels were controlled by a simple dial. It emitted pulses as it was turned and the pulse train was shifted slightly depending on the direction of the turn. In software this was mapped onto two signals, one for each direction, with one signal emitted for each pulse in the train. All very straight forward so far.

To avoid accidental change of direction we decided that

1. only start turning the wheels after having received four initial signals, and
2. if a full second without receiving any signal meant that the turning had stopped.

The solution

The application was to be implemented using Qt, so using the Qt state machine framework was an obvious choice. The full state machine wouldn’t have to be large, only 8 states. The initial state (sResting) would indicate that the system was in a steady state (no turning), from there any received signal would advance into a successive state (sOne, sTwo, sThree, sFour) to indicate the number of received signals. From the fourth state the machine would advance directly to a state (sTurning) where a received signal would initiate an actual turn of the wheels. The turning would happen upon the entry into two separate states (sTurnRight and sTurnLeft), each of these states would instantly return to sTurning. All of this is simple and straight forward, what wasn’t so clear was to implement the automatic return to the initial state after 1s of inactivity.

The implementation

As I like to do, I first experimented a little to find a suitable solution to the problem. What follows is the resulting code of that experiment. The final code used in the project ended up being very similar. It’s all based around the method postDelayedEvent() found in QStateMachine.

First off a new type of event is nedded, a ReturnEvent:

1
2
3
4
5

class ReturnEvent : public QEvent
{
public:
    ReturnEvent() : QEvent(QEvent::Type(QEvent::User + 1)) {}
};

There is also a need for a new type of transition, ReturnTransition:

1
2
3
4
5
6
7
8
9
10
11
12

class ReturnTransition : public QAbstractTransition
{
public:
    ReturnTransition(QState *target=0) { if(target) setTargetState(target); }
 
protected:
    virtual bool eventTest(QEvent *e) {
        return(e->type() == QEvent::Type(QEvent::User + 1));
    }
 
    virtual void onTransition(QEvent *) {}
};

For the experiment I decided to use a simple widget containing two buttons, it would also hold the state machine:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

class MButtons : public QWidget
{
    Q_OBJECT;
 
public:
    MButtons(QStateMachine &m)
        : _right("Right"), _left("Left"),
        _m(m), _delayed(0) {
        QBoxLayout *lo = new QBoxLayout(QBoxLayout::TopToBottom);
        lo->addWidget(&_right);
        lo->addWidget(&_left);
 
        setLayout(lo);
    }
    virtual ~MButtons() {}
 
    QPushButton _right,
                _left;
    QStateMachine &_m;

The widget also holds the slots for all the state entry functions:

20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

public slots:
    void sRestingEntered() {
        qDebug() << __PRETTY_FUNCTION__;
        if(_delayed) { _m.cancelDelayedEvent(_delayed); _delayed = 0; }
    }
 
    void sOneEntered() {
        qDebug() << __PRETTY_FUNCTION__;
        if(_delayed) { _m.cancelDelayedEvent(_delayed); _delayed = 0; }
        _delayed = _m.postDelayedEvent(new ReturnEvent, 1000);
    }
 
    void sTwoEntered() {
        qDebug() << __PRETTY_FUNCTION__;
        if(_delayed) { _m.cancelDelayedEvent(_delayed); _delayed = 0; }
        _delayed = _m.postDelayedEvent(new ReturnEvent, 1000);
    }
    void sThreeEntered() {
        qDebug() << __PRETTY_FUNCTION__;
        if(_delayed) { _m.cancelDelayedEvent(_delayed); _delayed = 0; }
        _delayed = _m.postDelayedEvent(new ReturnEvent, 1000);
    }
    void sFourEntered() {
        qDebug() << __PRETTY_FUNCTION__;
        if(_delayed) { _m.cancelDelayedEvent(_delayed); _delayed = 0; }
        _delayed = _m.postDelayedEvent(new ReturnEvent, 1000);
    }
    void sTurningEntered() {
        qDebug() << __PRETTY_FUNCTION__;
        if(_delayed) { _m.cancelDelayedEvent(_delayed); _delayed = 0; }
        _delayed = _m.postDelayedEvent(new ReturnEvent, 1000);
    }
    void sTurnRightEntered() {
        qDebug() << __PRETTY_FUNCTION__;
    }
    void sTurnLeftEntered() {
        qDebug() << __PRETTY_FUNCTION__;
    }

Sure, several of the entry functions could be folded into one, but in order to validate the idea it’s easier to make separate ones for each state. The pattern is easy to spot, on entry a delayed return event is registered (if there’s a previous one its replaced with a new), except for the steady state (sResting) where any delayed event is removed, and the turning states (sTurnRight and sTurnLeft) since those states immediately return to sTurning anyway.

Finally it also holds the handle for the delayed event:

58
59
60

private:
    int _delayed;
};

Now the main function for setting it all up is simple:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

int main(int argc, char **argv)
{
    QApplication app(argc, argv);
    QStateMachine m;
    MButtons b(m);
    b.show();
 
    QState *sResting = new QState(),
           *sOne = new QState(),
           *sTwo = new QState(),
           *sThree = new QState(),
           *sFour = new QState(),
           *sTurning = new QState(),
           *sTurnRight = new QState(),
           *sTurnLeft = new QState();
 
    m.addState(sResting);
    m.addState(sOne);
    m.addState(sTwo);
    m.addState(sThree);
    m.addState(sFour);
    m.addState(sTurning);
    m.addState(sTurnRight);
    m.addState(sTurnLeft);
    m.setInitialState(sResting);
 
    sResting->addTransition(&b._right, SIGNAL(clicked()), sOne);
    sResting->addTransition(&b._left, SIGNAL(clicked()), sOne);
    sOne->addTransition(&b._right, SIGNAL(clicked()), sTwo);
    sOne->addTransition(&b._left, SIGNAL(clicked()), sTwo);
    sOne->addTransition(new ReturnTransition(sResting));
    sTwo->addTransition(&b._right, SIGNAL(clicked()), sThree);
    sTwo->addTransition(&b._left, SIGNAL(clicked()), sThree);
    sTwo->addTransition(new ReturnTransition(sResting));
    sThree->addTransition(&b._right, SIGNAL(clicked()), sFour);
    sThree->addTransition(&b._left, SIGNAL(clicked()), sFour);
    sThree->addTransition(new ReturnTransition(sResting));
    sFour->addTransition(sTurning);
    sTurning->addTransition(&b._right, SIGNAL(clicked()), sTurnRight);
    sTurning->addTransition(&b._left, SIGNAL(clicked()), sTurnLeft);
    sTurning->addTransition(new ReturnTransition(sResting));
    sTurnRight->addTransition(sTurning);
    sTurnLeft->addTransition(sTurning);
 
    QObject::connect(sResting, SIGNAL(entered()), &b, SLOT(sRestingEntered()));
    QObject::connect(sOne, SIGNAL(entered()), &b, SLOT(sOneEntered()));
    QObject::connect(sTwo, SIGNAL(entered()), &b, SLOT(sTwoEntered()));
    QObject::connect(sThree, SIGNAL(entered()), &b, SLOT(sThreeEntered()));
    QObject::connect(sFour, SIGNAL(entered()), &b, SLOT(sFourEntered()));
    QObject::connect(sTurning, SIGNAL(entered()), &b, SLOT(sTurningEntered()));
    QObject::connect(sTurnRight, SIGNAL(entered()), &b, SLOT(sTurnRightEntered()));
    QObject::connect(sTurnLeft, SIGNAL(entered()), &b, SLOT(sTurnLeftEntered()));
 
    m.start();
 
    return(app.exec());
}

Conclusion and open questions

I’m fairly happy with the solution, but I’d be curious how other people, people more skilled in using Qt, would have solved the problem.

For a while I considered solving the skipping of four initial signals using a single state and counter, but I saw no obvious easy way to implement that, so I instead opted to use separate states. Slightly wasteful of resources, but not too bad, and simplicity is important. I’m very curious to find out if there’s a simply way to implement it using a single state.

Before the weekend I started looking at using Qt on Windows. More specifically I wanted to know whether this combination could be an option for a sub-project at work. We need to develop a program for the Windows desktop, and due to the overall context it would make sense to write it in C++ (that’s what we use for another part of the project). We already use both Eclipse and Visual Studio in the project, but I strongly prefer Eclipse, so I was hoping to be able to use it. However, it seems that the Qt developers strongly favour their own tool Qt Creator, though there are (outdated?) integrators for both Eclipse and Visual Studio. I’d rather avoid introducing a third IDE into a project—two is already one too many in my opinion. Anyway, I think I managed to find an acceptable configuration of Eclipse without using that old Qt integration plugin together with the MSVC (I was using the gratis version of MSVC for this).

Qt setup

I decided to install Qt into C:\QtSDK, and then I made the following permanent changes to the environment:

> set QTDIR=C:\QtSDK\Desktop\Qt\4.8.0\msvc2010
> set QMAKESPEC=%QTDIR%\mkspecs\win32-msvc2010
> set PATH=%PATH%;%QTDIR%\bin;C:\QtSDK\QtCreator\bin

Starting Eclipse so that it finds the compiler

It’s slightly disappointing that Eclipse happily lets one create MSVC project that isn’t buildable because it doesn’t know where the compiler is located. One easy way to remedy that seems to create a BAT file to create the proper environment for Eclipse:

@echo off
setlocal
call "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat"
start C:\Eclipse\Indigo\eclipse.exe
endlocal

Creating the project

Creating a “makefile” project in Eclipse is fairly straight forward; one needs a C/C++ project, of the makefile type, and make it empty too so that there isn’t any cruft in the way. Then add a single source file, e.g. main.cxx:

#include <iostream>
#include <Qt/QtGui>
 
int main(int argc, char **argv)
{
    std::cout << __FUNCTION__ << std::endl;
    QApplication app(argc, argv);
    return(app.exec());
}

And then a project file, e.g. Test.pro:

TEMPLATE = app
TARGET = 
DEPENDPATH += .
INCLUDEPATH += .
 
CONFIG += qt
 
HEADERS +=
SOURCES += main.cxx

After this use qmake to create the required makefile. I decided to use a subdirectory (_build) in the project, which qmake seems to have full support for:

> qmake ..\Test.pro

Setting up building from Eclipse

In the project properties modify the C/C++ Build settings for the Debug target. Instead of the default build command (which is make) one can use nmake, or even better jom:

• Build command: C:/QtSDK/QTCreator/bin/jom -f Makefile.Debug
• Build directory: ${workspace_loc:/Test}/_build

Then one can create a Release target, which differs only in that it builds using Makefile.Release.

Running qmake from inside Eclipse

It’s very convenient to be able to run qmake and re-generate the makefiles from inside Eclipse. One can set that up by adding an external tool:

• Location: C:\QtSDK\Desktop\Qt\4.8.0\msvc2010\bin\qmake.exe
• Working directory: ${workspace_loc:/Test}/_build
• Arguments: ../Test.pro

In closing

I plan to also have a look at the Qt Visual Studio Add-in, though I suspect we might be using the latest version of VS, which might cause trouble.

Suggestions for further integration with Eclipse would be most welcome, e.g. for forms and translations.

When using Gnome3 I was really impressed with the support for multiple screens. Then I switched to LXDE and was very disappointed in that desktop’s support for multiple screens. In fact so disappointed that I sat down and read the man-page for ‘randr’ and hacked up the following script:

#! /bin/bash
 
cmd=$1; shift
 
case $cmd in
    on)
        # turn on VGA1, auto-select size, right of laptop screen
        xrandr --output VGA1 --auto --right-of LVDS1
        ;;
    off)
        xrandr --output VGA1 --off
        ;;
    list)
        xrandr
        ;;
    *)
        echo "Commands: on, off, list"
esac

In my mind it’s vastly more usable than ‘lxrandr’

Ever since I heard the FLOSS weekly episode on 0MQ I’ve been looking for a reason to take a look at it. Well, to hell with reason, I’ll have a first look without any specific goal in mind.

I found a simple introduction to it in Nicholas Piël’s post ZeroMQ an introduction. The only issue was that it was based on Python, and Python2 at that. So here are my attempts at translating two of the clients to Haskell (using zeromq-haskell).

req-rep

Here’s the client in Python3 first:

1
2
3
4
5
6
7
8
9
10
11

import zmq
 
ctx = zmq.Context()
socket = ctx.socket(zmq.REQ)
socket.connect('tcp://127.0.0.1:5000')
 
for i in range(10):
    msg = "msg %s" % i
    socket.send_unicode(msg)
    print('Sending', msg)
    msg_in = socket.recv()

And here in Haskell:

1
2
3
4
5
6
7
8
9
10

import System.ZMQ
import Data.ByteString.Char8 as CBS
 
main = withContext 1 $ \ ctx -> withSocket ctx Req $ \ soc -> do
    connect soc "tcp://127.0.0.1:5000"
    let msgs = [pack ("msg " ++ show i) | i <- [0..9]]
    flip mapM_ msgs $ \ msg -> do
        send soc msg []
        CBS.putStrLn msg
        receive soc []

pub-sub

In Python3:

1
2
3
4
5
6
7
8
9
10

import zmq
 
ctx = zmq.Context()
socket = ctx.socket(zmq.SUB)
socket.connect('tcp://127.0.0.1:5000')
socket.setsockopt(zmq.SUBSCRIBE, b'sweden')
socket.setsockopt(zmq.SUBSCRIBE, b'denmark')
 
while True:
    print(socket.recv())

Haskell:

1
2
3
4
5
6
7
8
9

import System.ZMQ
import Control.Monad
import Data.ByteString.Char8 as CBS
 
main = withContext 1 $ \ ctx -> withSocket ctx Sub $ \ soc -> do
    connect soc "tcp://127.0.0.1:5000"
    subscribe soc "sweden"
    subscribe soc "denmark"
    forever $ receive soc [] >>= CBS.putStrLn

Two comments on the Haskell code here:

• I’m not sure why, but the Haskell client dies after receiving just a few messages (they are properly filtered though).
• The API for subscribe is a bit strange, it would make more sense if it took a ByteString rather than a String.

A little while ago shelltestrunner was announced on haskell-cafe. At the time I was slowly losing hope on ever getting decent test coverage in cblrepo using HUnit. Using something like shelltestrunner could be an easier and more workable solution, especially since what cblrepo needed most in the short term is a bit of integration testing.

shelltestrunner is basically just a tool that runs shell commands and compares output (both stdout and stderr) and the exit code. It’s also possible to provide data to be passed to the command on stdin. The documentation on the shelltestrunner home page is very good and accessible. There are only a few things that I’d like to add to it:

• Use the --with (`-w´) flag, it’s very handy to avoid littering the tests with long paths to the output of your build environment.
• There is no support for set-up and tear-down steps in the tests (in my opinion this would be a very nice addition to the tool) so anything needed to be set up for the actual tests, will itself have to be tests.
• There is no way to name tests (would be another good addition) so I found it crucial to organise tests into several files.

Since I’m now working a bit with embedded systems I thought I’d take a look at compiling for one of the ARM-based machines that QEMU supports. I settled for VersatilePB after finding this old-ish article. Rather optimistically I thought that maybe, just maybe things had change in a year and that the limitation of flash was removed. How wrong I was.

I did find an easier way to get it working, though with the limitation that Linux has to be started via tftpboot or some other network-based fashion. The patch looks like this:

--- u-boot.orig/src/u-boot-2011.12/include/configs/versatile.h
+++ u-boot/src/u-boot-2011.12/include/configs/versatile.h
@@ -31,6 +31,8 @@
 #ifndef __CONFIG_H
 #define __CONFIG_H

+#define CONFIG_ARCH_VERSATILE_QEMU
+
 /*
  * High Level Configuration Options
  * (easy to change)

Then just go ahead and modify the default boot argument (CONFIG_BOOTARGS in the same file) to your hearts content to minimise the amount of manual work for booting.

Having lived outside of Sweden for about a decade I’ve grown accustomed to non-Swedish keyboard layouts, first the US (as it’s widely used in The Netherlands) and later on the UK layout. Moving back to Sweden had me swearing over the layout used here within only a few days. The placement of “{[]}” is especially painful. Clearly the Swedish layout wasn’t designed for developers! Rather than go on muscle memory I decided to first attempt a small change to the X key mappings.

I found a good description of per-user XKB configuration after a bit of searching. Then I modified it slightly to fit better in my Arch-based LXDE system.

The XKB config

I started with removing all the configuration I’d previously put into /etc/X11/xorg.conf.d — if I’m to use per-user configuration then there should be no system-wide settings at all. Then I put the output of setxkbmap -print into ~/.xkb/maps/$(hostname) as a starting point. The main goal is to move the characters that requires awkward single-hand combinations with AltGr to slightly more comfortable mappings. After a bit of experimentation I settled on the following (which I put in ~/.xkb/symbols/sedev)

1
2
3
4
5
6
7
8
9
10

partial alphanumeric_keys
xkb_symbols "devkeys" {
    key <AD01> { [ q, Q, backslash ] };
    key <AD02> { [ w, W, asciitilde ] };
 
    key <AC01> { [ a, A, braceleft ] };
    key <AC02> { [ s, S, bracketleft ] };
    key <AC03> { [ d, D, bracketright ] };
    key <AC04> { [ f, F, braceright ] };
};

After setting it manually and verifying that the new mappings work I added it to my keymap, which ended up looking like this

1
2
3
4
5
6
7

xkb_keymap {
    xkb_keycodes  { include "evdev+aliases(qwerty)" };
    xkb_types     { include "complete" };
    xkb_compat    { include "complete" };
    xkb_symbols   { include "pc+se(nodeadkeys)+inet(evdev)+capslock(swapescape)+compose(paus)+sedev(devkeys)" };
    xkb_geometry  { include "pc(pc104)" };
};

Tying it together

Now all the remains is to load the new configuration on login. Based on madduck’s example I put the following into ~/.xprofile

1
2
3
4
5
6

# load XKB, if there is one
XKBDIR=${HOME}/.xkb
XKBMAPFILE=${XKBDIR}/keymap/$(hostname)
if [[ -f ${XKBMAPFILE} ]]; then
    xkbcomp -I${XKBDIR} ${XKBMAPFILE} ${DISPLAY}
fi

Now I just have to get used to using the new mappings.