Interesting information about throttling on usage of posts/all. It all adds up to del.icio.us API being nearly useless for experimentation and interesting new uses of bookmarks. I’m just so happy there are other social bookmark sites that don’t put whimsical limitations on API usage.

More »

It’s just somewhat unfortunate that the name of the plugin doesn’t fit that well anymore.

More »

I received a report about some problems with epilicious, to which I responded with a pointer to my post about my past experiences. In response I got a suggestion to pick apart the “official” Firefox plugin for del.icio.us. It turned up some interesting details. AFAICS they are using the API in an undocumented way!

First of all every single request is a POST rather than a GET. The data they send is _user=<cookie>. That cookie is retrieved from Firefoxes cookie jar; it’s set after the user logs in at https://secure.del.icio.us/login. Apparently this allows them to use cookie:cookie as the username and password for Basic Authentication. Also, every request has src=ffbmext1.4.27 as an argument in the URI (1.4.27 is the current version of their plugin). There’s also an undocumented “function” for getting only the hashes of the bookmarks at https://api.del.icio.us/v1/posts/hashes?. The maybe most interesting detail is that they don’t seem to bother with the threat of throttling at all!

Hopeful that this might offer a way around the increasingly erratic behaviour of the API I played around with python. No luck though! A few hours of investigation, some quick hacking and long testing (once you’re throttled it takes a LONG time for del.icio.us to let you back in, sometimes more than 10 minutes!!) later I found myself not being able to hold back on profanities. The API usage description is a joke! Programming against something like that is little better than programming Windows based on MSDN

It seem the official plugin can get away with disregarding throttling because all interaction with del.icio.us happens with user interaction between each call. It takes a very fast user to trigger the throttling.

I can understand that del.icio.us wasn’t designed to be used a base for synchronising bookmarks, the throttling clearly shows that. However, one second between requests is not enough to avoid throttling and there is no indication on how long one remains in a throttled state.

This exercise has pushed me to put more energy into looking for alternative backends for epilicious. So far ma.gnolia seems good and I’d like to add support for Google bookmarks as well.

More »

So far it’s not quite complete, there is no GUI to change the backend store you’ll have to interact with GConf directly, other parts of the GUI lie by still referring to del.icio.us even though ma.gnolia.com is being used. However, the backend is working and the only problem so far seems to be that ma.gnolia.com returns badly formatted XML at times (still haven’t investigated it thoroughly to say anything definite).

I haven’t pushed the changes into any of my repos yet. The only way to get it is by using the epiphany-extension-epilicious-pre package in my APT repo. Debian Sid users can simply install the package, others will have to use the source.

More »

Short interlude here. del.icio.us asks of developers that programs using the del.icio.us API wait at least 1 second between requests. Otherwise requests will be throttled by responding with error code 503.

So, guessing a little I made my use of the API a little less aggressive. I waited a second for every request, not only after having been throttled. That solved the problem. All’s well again, right? No!!!

Yesterday I was playing around a bit with epilicious again. Moving to darcs and some refactoring. To test my changes I cleaned out my del.icio.us account using delbackup. It failed with a code 500. So, I modified the code, treating 500 in the same way as 503—wait a second. Then I received a 999 and I was locked out from del.icio.us and asked to “hold off for a few minutes and try again later, in a gentler fashion”. WTF?

I modified the code again, waiting 2 seconds for 503, 10 seconds for 500, and 15 seconds for 999ⁱ. Then I waited for the lock-out to end and I tried again. Locked out again! Tiring!

So, now I’ve modified the code again, now waiting 2 seconds before each request, 2 seconds if I ever get a 503, 10 seconds for a 500, and 15 seconds if I get a 999.

Now I’m instead waiting an awfully long time to get something done, and what happens then? I get locked out since I’m obviously not waiting long enough after receiving a 999.

I am clearly waiting long enough between requests! Has del.icio.us introduced some new arbitrary, undocumented limitation on using the API?

If anyone knows anything that I could stick as an option next to using del.icio.us I would love to hear about itⁱⁱ. Two requirements:

1. Web API, preferably as easy as del.icio.us API (even better, a compatible API)
2. Use https for connections

Dear lazyweb, any suggestions?

More »

1. It’s not as silly as it sounds to wait when receiving a 999. From some testing I concluded that receiving a 999 doesn’t result in being locked out immidiately.
2. I know of scuttle but I’m having problem finding basic information about it, e.g. what URI is the API on? I’m also not sure it provides API access over https

I’ve written a minimal Python module that offers exactly as much of the del.icio.us API as epilicious needs. If someone’s interested in extending it into a more full-fledged del.icio.us module—be my guest. You can find it in the development BZR repo (libepilicious/delicious.py).

I’ll be making another release of epilicious ASAP.

More »