Recently I began writing a tool to scrape some information off a web site for some off-line processing. After writing up the basics using TagSoup I showed what I had to a colleague. His first comment was “Can’t you use Parsec for that?” It took me a second to realise that he didn’t mean that I should write my own XML parser but rather that Parsec allows writing parsers of a list of anything. So I thought I’d see just what it’d take to create a parser for [Tag].
A look at the string parser shipped with Parsec offered a lot of inspiration.
First the basic type, TagParser:
type TagParser = GenParser Tag
The basic function of Parsec is tokenPrim, basically that’s what other basic parsers use. Taking a cue from the string parser implementation I defined a function called satisfy:
satisfy f = tokenPrim
show(\ pos t _-> updatePosTag pos t)(\ t ->if(f t)then Just t else Nothing)
The positioning in a list of tags simply an increase of column, irrespective of what tag is processed:
updatePosTag s _= incSourceColumn s 1
Now I have enough to create the first Tag parser—one that accepts a single instance of the specified kind:
tag t = satisfy (~== t)<?>show t
It’s important to stick the supplied tag on the right of (~==). See its documentation for why that is. The second parser is one that accepts any kind of tag:
anyTag = satisfy (const True)
So far so good. The next parser to implement is one that accepts any kind of tag out of a list of tags. Here I want to make use of the convenient behaviour of (~==) so I’ll need to implement a custom version of elem:
l `elemTag` r =or$ l `elemT` r
where
l `elemT` []=[False]
l `elemT` (r:rs)=(l ~== r) : l `elemT` rs
With that in place it’s easy to implement oneOf and noneOf:
Of course the big question is whether I’ll rewrite my original code using Parsec. Hmm, probably not in this case, but the next time I need to do some web page scraping it offers yet another option for doing it.
As Edward pointed out the code in the previous post expressions such as a .+. b .+. c are problematic. By expanding the expression and introducing a few parentheses it becomes apparent what the problem is:
fromIB $(toIB (fromIB $(toIB a) `addIB` (toIB b))) `addIB` (toIB c)
The problem is that inner fromIB. Unfortunately GHC doesn’t realise that the left-most sum could take on any type that is a CIB, the exact type doesn’t really matter since the result is passed to toIB anyway. It would be sort of cool if I could tell the compiler to prefer a specific CIB, basically a directive along the lines of “if in doubt, use Bytes”. I don’t think that’s possible in GHC. As it stands one would have to specify the type of the inner sum:
(a .+. b :: Bytes).+. c :: KBytes
One possible solution would be to remove the call to fromIB from the definition of .+. and instead require the user to call it explicitly:
fromIB $ a .+. b .+. c :: KBytes
I suppose it’s all right, but not quite as elegant as I had hoped.
Now on to the more interesting part of Edward’s comment. I needed quite a bit of clarification, but I now know what he is proposing, I even think I understand it
The general idea is to have the compiler choose the largest prefix that can represent the sum of two values without losing precision. The represention I used didn’t have the problem of ever losing precision, so I’ll change the representation to better show what Edward meant.
Now the prefixes are represented with a single integer (or rather a single instance of Num). This is easily done thanks to GeneralizedNewtypeDeriving.
newtype Bytes a = Bytes a
deriving(Eq,Show,Num)newtype KBytes a = KBytes a
deriving(Eq,Show,Num)newtype KiBytes a = KiBytes a
deriving(Eq,Show,Num)
Now I need a typeclass to bind together two types in a ‘lesser-than-or-equal’ relationship and provide a conversion function:
class LEq s u where
lower ::Num a => s a -> u a
Now the relation has to be implemented for the prefixes. In short the following says that Bytes is the less than both KBytes and KiBytes and that each prefix is less than or equal to itself:
instance LEq Bytes Bytes where
lower =idinstance LEq KBytes KBytes where
lower =idinstance LEq KiBytes KiBytes where
lower =idinstance LEq KBytes Bytes where
lower (KBytes k)= Bytes $ k *10^3instance LEq KiBytes Bytes where
lower (KiBytes ki)= Bytes $ ki *2^10
Now there’s a second relationship that is designed to relate two prefixes to a third one. Basically the third is the largest prefix that can be used to represent the sum of the other two without a loss of precision. One could say that the third is where the other two meet.
class(LEq s u, LEq t u)=> Meet s t u | s t -> u
I have to manually define where the different prefixes meet:
Now I can define addition and subtraction in terms of Meet instances:
(.+.)::(Meet s t u,Num a,Num(u a))=> s a -> t a -> u a
(.+.) a b =(lower a)+(lower b)(.-.)::(Meet s t u,Num a,Num(u a))=> s a -> t a -> u a
(.-.) a b =(lower a)-(lower b)
Finally I’ve arrived at the destination, but as so often I have to admit that the journey was at least half the fun.
*Prefixes>let i = KiBytes 4*Prefixes>let j = KBytes 3*Prefixes>let k = Bytes 900*Prefixes> i .+. j
Bytes 7096*Prefixes> i .+. i
KiBytes 8*Prefixes> i .+. j .+. k
Bytes 7996
I just realised that it’s been over a month since I posted something. All I can do is blame life With a daughter of 8 months and a recent switch of positions I just haven’t found time to play around really. Hopefully that’ll change now that the situation at work is “calming down” a bit.
Anyway, here’s a bit of code I played with today. A friend described a problem inside a tool related to reporting of free and used memory. Apparently, depending on the location in the code, memory could be stored in integers in bytes or in kilobytes (kB) or in kibibytes (Ki) or in mebibytes (Mi) etc. In every place the prefix would be ‘recorded’ in the variable name, resulting in a situation where a change of prefix used in one place would force a change of variable names in callers/callees. A bit of a mess. I thought I’d try to explore what Haskell’s type system could produce
First approach
My first thought was to create a data type for the prefixes:
data Size = Bytes Int| KB IntInt| KiBi IntIntderiving(Show,Eq)
Both for KB and KiBi I record the number of full KBs (Kis) and the remaining bytes. Then I thought that it’d be easiest to do all the calculations in bytes, which required a conversion function:
toBytes (KB i r)= Bytes (10^3* i + r)
toBytes (KiBi i r)= Bytes $2^10* i + r
toBytes b = b
After that it’s possible to make Size an instance of Num (incomplete, but the rest of the functions in Num are straight forward):
Unfortunately the result will always be in Bytes so a few other conversion functions are necessary:
toKB (Bytes i)=uncurry KB $divMod i (10^3)
toKB s = toKB $ toBytes s
toKiBi (Bytes i)=uncurry KiBi $divMod i (2^10)
toKiBi s = toKiBi $ toBytes s
Now it’s easy to create a type for bookeeping of memory and making it an instance of Num as well:
data MemUsage = Free Size | Used Size
deriving(Eq,Show)instanceNum MemUsage where(Free i)+(Free j)= Free $ i + j
(Used i)+(Used j)= Used $ i + j
(Free i)+(Used j)= Free $ i - j
(Used i)+(Free j)= Used $ i - j
(Free i)-(Free j)= Free $ i - j
(Used i)-(Used j)= Used $ i - j
(Free i)-(Used j)= Free $ i + j
(Used i)-(Free j)= Used $ i + j
On some level this ‘algebra’ for memory usage makes sense, though refinements are certainly possible e.g. . I suspect there are some interesting choices to be made regarding the remaining functions in Num, e.g. should negate turn free memory into used memory?
I find this solution somewhat lacking in the extensibility department—as the commonly available amount of memory grows (the code above is stuck in the 80’s it seems) changes have to be made to Size. The same goes for changes to support non-standard prefixes, probably not a common problem but It would be nice to have a solution that is “independently extensible”.
Second approach
Here my idea was to create one type per prefix, use a separate type for the calculations and use a type class for the conversions. Quite straightforward when put like that, though it took me a while to come up with it
Here is the internal representation used during calculations:
data IB = IB Intderiving(Eq,Show)
Simple enough. Then the class for conversions to and from IB:
class CIB a where
toIB :: a -> IB
fromIB :: IB -> a
No surprises there either. Now I want to add and subtract, but due to the definition of Num (it requires all arguments to be of the same type, e.g. the type of addition is (+) :: a -> a -> a) I’ll need two new functions for that (at least I don’t know a way around this):
(.+.)::(CIB a, CIB b, CIB c)=> a -> b -> c
i .+. j = fromIB $(toIB i) `addIB` (toIB j)where
addIB (IB i1)(IB i2)= IB (i1 + i2)(.-.)::(CIB a, CIB b, CIB c)=> a -> b -> c
i .-. j = fromIB $(toIB i) `subIB` (toIB j)where
subIB (IB i1)(IB i2)= IB (i1 - i2)
One interesting observation is that due to the type of .+. and .-. it is possible, and often required, to specify the type of the result, somewhat similar to HSH. Now the stage is set for the prefix types to be defined, starting with plain bytes:
data Bytes = Bytes Intderiving(Eq,Show)instance CIB Bytes where
toIB (Bytes i)= IB i
fromIB (IB i)= Bytes i
Similarly for kilobytes and kibibytes:
data KBytes = KBytes IntIntderiving(Eq,Show)instance CIB KBytes where
toIB (KBytes i j)= IB $ i *10^3+ j
fromIB (IB i)=uncurry KBytes $divMod i (10^3)data KiBytes = KiBytes IntIntderiving(Eq,Show)instance CIB KiBytes where
toIB (KiBytes i j)= IB $ i *2^10+ j
fromIB (IB i)=uncurry KiBytes $divMod i (2^10)
The data type for memory usage wrap the memory usage:
data MemUsage a = Free a | Used a
deriving(Eq,Show)
Again I can’t make MemUsage a an instance of Num, instead there are two new operators:
(~+~)::(CIB a, CIB b, CIB c)=> MemUsage a -> MemUsage b -> MemUsage c
(Free i)~+~(Free j)= Free $ i .+. j
(Used i)~+~(Used j)= Used $ i .+. j
(Free i)~+~(Used j)= Free $ i .-. j
(Used i)~+~(Free j)= Used $ i .-. j
(~-~)::(CIB a, CIB b, CIB c)=> MemUsage a -> MemUsage b -> MemUsage c
(Free i)~-~(Free j)= Free $ i .-. j
(Used i)~-~(Used j)= Used $ i .-. j
(Free i)~-~(Used j)= Free $ i .+. j
(Used i)~-~(Free j)= Used $ i .+. j
End thoughts
There is one thing I like about the first approach and one thing I don’t like. That both the types are instances of Num is something I like since I am interested in adding and subtracting memory, though arguably there are functions in Num that don’t make much sense for memory (e.g. multiplication and negation). The thing I don’t like is the need for explicit conversion functions (toKB and friends).
There are also one thing that I like about the second approach and one I don’t like. I would have liked to use the standard functions in Num, introducing new (type-specific) operators is something that I’m never particularly keen on. Though I suppose that implementing separate operators does emphasize that memory isn’t quite like regular integers. I really do like using Haskell’s own syntax to do the conversion, it’s more readable than having to call conversion functions.
That’s it! Comments and suggestions for improvements are always welcome of course.
![[Digg]](http://therning.org/magnus/wp-content/plugins/bookmarkify/digg.png)
![[Reddit]](http://therning.org/magnus/wp-content/plugins/bookmarkify/reddit.png)
