Incoherent mumblings
Posts tagged ‘python’
I bet a lot of people would consider this to be fairly obvious, but today, while solving the fourth of Google’s Treasure Hunt problems I realised just how similar in essence laziness+infinate lists in Haskell is to generators in Python. It’s just much easier to work with the former 
Dear lazyweb, I’ve been racking my brain trying to get information out of the following C struct in a nice way when using ctypes:
struct Foo { unsigned int length; char name[]; };
I wrote a little C function to get instance of the struct easily into python:
struct Foo * put_name(char *name) { struct Foo *foo = malloc(sizeof(struct Foo) + strlen(name)); foo->length = strlen(name); memcpy(foo->name, name, foo->length); return(foo); }
Now, how do I actually get the full contents of Foo.name in python? The only way I could think of that actually works is to create a dummy-struct type in order to get the length out and then use it to dynamically create a new sub-class of ctypes.Structure, then create an instance based on the address of what was returned. I think the following shows it pretty clearly:
class FOO(Structure): _fields_ = [('length', c_uint), ('name', c_char * 0)] liblib = cdll.LoadLibrary('./_build/liblib.so') c_put_name = liblib.put_name c_put_name.argtypes = [c_char_p] c_put_name.restype = POINTER(FOO) def put_name(str): f = c_put_name(str) K = type('FOO%s' % f.contents.length, (Structure,), {'_fields_' : [('length', c_uint), ('name', c_char * f.contents.length)]}) return K.from_address(addressof(f.contents))
I still think there ought to be some other way that ‘feels’ nicer. I mean the use of “short arrays” (declared as here with [], or zero size as supported by GCC, or the more portable array of size one) seems to be common enough to warrant some support from ctypes, right?
Any suggestions for improvements?
The other day I was talking to a mate and former colleague of mine, he’s been doing a lot of Java and C# before but recently he got hired by a small company to do Python work. Anyway he related a funny part of the interview where he said he’d done design patterns and they asked him to explain one that he’s used. He chose Decorator. After he was done explaining the interviewer commented that surely he meant Proxy. The interviewer was wrong and my mate suspects this might be something that’s common in the Python world due to the built-in support for function/method decorators in the language. I suspect he’s right. Anyway, he showed me what he was playing with and I couldn’t help but play a bit on my own afterwards.
Here’s the class of the core object, a simple self-explanatory piece of code:
![[Digg]](http://therning.org/magnus/wp-content/plugins/bookmarkify/digg.png)
![[Reddit]](http://therning.org/magnus/wp-content/plugins/bookmarkify/reddit.png)
class Writer(object):
def write(self, s):
print s
Here’s a not very exciting example of using it:
> w = Writer()
> w.write('hello')
hello
We want to decorate it by modifying the string passed to write in different ways. First here’s a base decorator class:
class WriterDecorator(object):
def __init__(self, wrappee):
self.wrappee = wrappee
def write(self, s):
self.wrappee.write(s)
Using it is straight forward, and still not very exciting:
> wd = WriterDecorator(w)
> wd.write('hello')
hello
The constructor requires a wrappee object and the implementation of write is straight forward. Strictly speaking this class is unnecessary, but it’s convenient once we implement “real” decorators. Here’s the first one, it converts the string to upper case before passing it on down the chain:
class UpperWriter(WriterDecorator):
def write(self, s):
self.wrappee.write(s.upper())
This is where it gets a little more exciting, not much though:
> uw = UpperWriter(w)
> uw.write('hello')
HELLO
Here’s a nice detail about Python that I’ve never reflected over myself—constructors are inherited in Python. Here’s another decorator, one that makes the string “shouty”:
class ShoutWriter(WriterDecorator):
def write(self, s):
self.wrappee.write('!'.join([t for t in s.split(' ') if t]) + '!')
Now it’s getting a little more interesting, because the decorators can be combined:
> sw1 = ShoutWriter(w)
> sw1.write('hello again')
hello!again!
> sw2 = ShoutWriter(uw)
> sw2.write('hello again')
HELLO!AGAIN!
Some of these combinations are more useful than others, and if they’re used very often then it might be worth creating a convenience class for them. Here’s one that I imagine could be useful if you’re a writer for The Register:
class YahooWriter(WriterDecorator):
def __init__(self, wrappee):
self.wrappee = UpperWriter(ShoutWriter(wrappee))
Using it is simple:
> yw = YahooWriter(w)
> yw.write('hello again')
HELLO!AGAIN!
Well, so far it’s been child’s play and I wouldn’t have bothered writing about this unless I took this a little further. I thought something was familiar about how the convenience class worked. I vaguely remembered reading something about super being harmful and there seemed to be similarities between behaviour described there and the desired behaviour when nesting decorators. Rewriting the basic decorator classes using super like this retains their behaviour:
class UpperWriter(WriterDecorator):
def write(self, s):
super(UpperWriter, self).write(s.upper())
class ShoutWriter(WriterDecorator):
def write(self, s):
super(ShoutWriter, self).write('!'.join([t for t in s.split(' ') if t]) + '!')
What this does though is allow implementing YahooWriter like this:
class YahooWriter(UpperWriter, ShoutWriter):
pass
I think that’s pretty cute.
Here’s where I have to stop though. I don’t know if this is even useful, is it? Maybe it has some serious draw-backs my inexperience and ignorance prevents me from seeing, does it? Has super been used like this somewhere? I’d love pointers to that code 
[Edited 16-06-2007 00:34 BST] Bloody hell, can’t believe I had a spelling error in the title all this time. Embarrassing really!
In the June issue of Linux Journal (#158) there’s an article on Lua in which the author compares it to Python. The article is only available to subscribers at the moment, so I decided to post my comment here as well.
“There is no need to worry about different types of integers.” There is non need to do that in Python either nowadays. The conversion between “regular integers” and long ones is done automagically. Take the following function:
def frac(n):
if n == 0: return 1
else: return n * frac(n - 1)
Evaluating frac(10) gives the result 3628800 while frac(30) returns 265252859812191058636308480000000L.
“You can add a float, long integer or a double to any other type of integer or number without a hitch in Lua. In contrast, doing this can cause programs written in Python 2.4 or older versions to crash.” Really? I’ve never heard of this problem. Do you have any references?
“In contrast to Python, Lua does not focus on 100% backward compatibility.” That Python would focus on 100% backward compatibility is just plain wrong. Code I’ve written broke when moving from version 2.4 to 2.5 due to changes in generators. This is the only time I’ve personally been bitten by it but every version of Python has broken something from earlier versions. AFAIU Python 3000 will break a whole lot of older Python programs.
“Unlike Python, global variables do not need to be declared.” They don’t need to be declared at all, not on a global level. If a function wants to access a global variable then it has to declare it as global. Contrast that to the following point.
“Exactly the opposite of Python, most variables in Lua are global by default, and you must declare the variable “local” to make it a local variable rather than assuming that all variables are local.” Variables in Python are local by default, which in my experience better reflect how variables are used by programmers.
“The for loop is a little more complex in Python than it is in Lua, because Python needs to know the key to be able to get the key’s value.” It’s possible to iterate over both key and value in Python as well:
h = {1:'hello', 2:'world'}
for k, v in h.iteritems():
print k, v
In general it’s a good article, it shows off some interesting points in Lua. In order to make a comparison between two languages it’s necessary to have very good knowledge of both languages, otherwise there’s a real risk that one language is the favourite simply because the author knows that one the best. This piece definitely suffers from that.
I believe it would have been better without the author’s personal opinions on aesthetics sprinkled across the piece.
A while ago I decided that in order to learn Haskell I should make an attempt to stop reaching for Python whenever I needed to solve a “small problem” and use Haskell instead. This morning I found a need to unescape URLs inside Vim and I didn’t catch myself until after I had written the following Python code:
#! /usr/bin/env python
import urllib
import sys
for l in sys.stdin:
print urllib.unquote(l),
After a little bit of cursing I started browsing Hoogle and after bit of searching I found the Network.URI module. My Haskell solution ended up looking like this:
module Main where
import Network.URI
main :: IO ()
main = interact $ unlines . (map unEscapeString) . lines
Short, sweet, and easy to understand, I think.
In case you want to go the other direction, i.e. to escape strings in Haskell then combining escapeURIString and isUnreserved is the right thing to do.