Blog Archives

1. Using MapLibre GL in Tauri

   Knowing almost nothing about frontend development or GIS I wanted to be able to display something on a map on my own computer (desktop). Knowing a fair bit of Rust but not having a clue about anything else involved I decided the easiest way to do this was probably to …

2. Using Modern Linux Sockets

   Introduction

   On the surface of it the Linux socket API is pretty straight forward. However once you start to care about performance things get a bit more complicated, and documentation a bit more sparse.

   Because I thought it'd be fun to explore this a bit more I made a simple …

3. Experiments with an Email Protocol

   Background

   The primary way I use email is through notmuch and notmuch-emacs on my laptop, which reads email from a local maildir. Sending mail is done using the venerable sendmail(8) command, which is backed by nullmailer to send it out via SMTP to the relay server of my email …

4. Encrypted root on Debian with keyfile/keyscript

   I've recently set up another laptop with whith whole-disk encryption, including /boot. This is all fine --the debian-installer almost gets it right, you just need to edit /etc/default/grub to add GRUB_ENABLE_CRYPTODISK=y and then re-run grub-install on the target-- but once you get this working you end up …

5. A Container is an Erlang Process

   This post is a response to A Container Is A Function Call by Glyph. It is a good article and worth your time reading, and you might want to read it to follow here. On twitter I asserted the article recommends building a monolith while Glyph countered "On the contrary …

6. py.test sprint in Freiburg

   Testing is a really important part of Python development and picking the testing tool of choice is no light decision. Quite a few years ago I eventually decided py.test would be the best tool for this, a choice I have never regretted but has rather been re-enforced ever since …

7. Pylint and dynamically populated packages

   Python links the module namespace directly to the layout of the source locations on the filesystem. And this is mostly fine, certainly for applications. For libraries sometimes one might want to control the toplevel namespace or API more tightly. This also is mostly fine as one can just use private …

8. New pytest-timeout release

   At long last I have updated my pytest-timeout plugin. pytest-timeout is a plugin to py.test which will interrupt tests which are taking longer then a set time and dump the stack traces of all threads. This was initially developed in order to debug some some tests which would occasionally …

9. Designing binary/text APIs in a polygot py2/py3 world

   The general advice for handling text in an application is to use a so called unicode sandwich: that is decode bytes to unicode (text) as soon as receiving it, have everything internally handle unicode and then right at the boundary encode it back to bytes. Typically the boundaries where the …

10. Don't be scared of copyright

    It appears there is some arguments against putting copyright statements on the top of a file in free software or open source projects. Over at opensource.com Rich Bowen argues that it is counter productive and not in the community spirit (in this case talking about OpenStack). It seems to …

11. Setting up a Python development environment on Solaris 11

    Solaris is one of those things which tends to be a bit un-loved, it being baught by Oracle probably didn't help either. But regardless they've made a rather nice and modern OS out of it and it is used in a fair few places so you may need to support …

12. Inspecting un-imported modules using ast

    or

    Skipping modules in py.test using marks but no importing

    As I've said before, py.test is my favourite testing tool. One if it's features is that it allows you to mark tests with arbitrary "marks", built in ones are e.g. skipif, xfail etc. But py.test's extension …

13. Small Emacs tweaks impoving my Python coding

    Today I've spent a few minutes tweaking Emacs a little. The result is very simple yet makes a decent impact on usage.

    Firstly I remembered using the c-subword-mode a long time ago, I couldn't believe I never used that in Python. Turns out there is a more genericly named subword-mode …

14. Using __getattr__ and property

    Today I wasted a lot of time trying to figure out why a class using both a __getattr__ and a property mysteriously failed. The short version is: Make sure you don't raise an AttributeError in the property.fget()

    The start point of this horrible voyage was a class which looked …

15. Synchronising eventlets and threads

    Eventlet is an asynchronous network I/O framework which combines an event loop with greenlet based coroutines to provide a familiar blocking-like API to the developer. One of the reasons I like eventlet a lot is that the technology it builds on allows it's event loop to run inside a …

16. Creating subprocesses in new contracts on Solaris 10

    Solaris 10 introduced "contracts" for processes. You can read all about it in the contract(4) manpage but simply put it's a grouping of processes under another ID, and you can "monitor" these groups, e.g. be notified when a process in a group coredumps etc. This is actually one …

17. re.search() faster then re.match()

    This is a counter-intuitive discovery, in IPython:

    In [18]: expr = re.compile('foo')
    
    In [19]: %timeit expr.search('foobar')
    1000000 loops, best of 3: 453 ns per loop
    
    In [20]: %timeit expr.match('foobar')
    1000000 loops, best of 3: 638 ns per loop
    

    So now I'm left wondering why .match …

18. Storm and SQLite in-memory databases

    When using an SQLite in-memory databases in storm the different stores created from the same database are not modifying the same SQLite database. E.g.

    db = storm.locals.create_database('sqlite:')
    store1 = storm.locals.Store(db)
    store2 = storm.locals.Store(db)
    

    Here store1 and store2 will not refer to the same …

19. Finding the linux thread ID from within python using ctypes

    So I've got a multi-threaded application and suddenly I notice there's one thread running away and using all CPU. Not good, probably a loop gone wrong. But where? One way to find this is revert history in the VCS and keep trying it out till you find the bad commit …

20. Return inside with statement

    Somehow my brain seems to think there's a reason not to return inside a with statement, so rather then doing this:

    def foo():
        with ctx_manager:
            return bar()
    

    I always do:

    def foo():
        with ctx_manager:
             result = bar()
        return result
    

    No idea why nor where I think to have heard/read this …

21. Templating engine in python stdlib?

    I am a great proponent of the python standard library, I love having lots of tools in there and hate having to resort to thirdparty libs. This is why I was wondering if there could be a real templating engine in the stdlib some day? I'm not a great user …

22. Using Debian source format 3.0 (quilt) and svn-buildpackage

    Searching the svn-buildpackage manpage for the 3.0 (quilt) format I thought that it wasn't able to apply the patches in debian/patches during build time. Instead I was doing a horrible dance which looked something like svn-buildpackage --svn-export; cd ../build-area/...; debuild. Turns out I was completely wrong.

    svn-buildpackage doesn't …

23. Europython, threading and virtualenv

    I use threads

    correctly

    I do not use virtualenv

    and dont't want to

    Just needed to get that out of my system after europython, now mock me.

    PS: I should probably have done this as a lightening talk but that occurred to me too late.

24. py.test test generators and cached setup

    Recently I've been enjoying py.test's test function arguments, it takes a little getting used too but soon you find that it's quite likely a better way then the xUnit-style of setup/teardown. One slightly more advanced usage was using cached setup together with test generators however. While not …

25. Selectable queue

    Sometimes you'd want to use something like select.select() on Queues. If you do some searching for this it turns out that this question has been answered for multiprocessing Queues where you can simply use the real select on the underlying socket (IIRC), but for a good old fashioned Queue …

26. weakref and circular references: should I really care?

    While Python has a garbage collector pretty much whenever circular references are touched upon it is advised to use weak references or otherwise break the cycle. But should we really care? I'd like not to, it seem like something the platfrom (python vm) should just provide for me. Are all …

27. python-prctl

    There is sometimes a need to set the process name from within python, this would allow you to use something like pkill myapp rather then the process name of your application just being yet another "python". Bugs (which I'm now failing to find in Python's tracker) have been filed about …

28. Storm and sqlite locking

    The Storm ORM struggles with sqlite3's <http://docs.python.org/library/sqlite3.html> transaction behaviour as they explain in their source code. Looking at the implementation of .raw_execute() the side effect of their solution to this is that they start an explicit transaction on every statement that gets executed. Including …

29. Python optimisation has surprising side effects

    Here's something that surprised me:

    a = None
    def f():
        b = a
        return b
    def g():
        b = a
        a = 'foo'
        return b
    

    While f() is perfectly fine, g() raises an UnboundLocalError. This is because Python optimises access to local variables using the LOAD_FAST/STORE_FAST opcode, you can easily see why this …

30. Judging performance of python code

    Recently I"ve been messing around with python bytecode, as a result I now know approximately how the python virtual virtual machine works at a bytecode level. I've found this quite interesting but other then satisfying my own curiosity had no benefit from this knowledge. Until today I was wondering …

31. Hacking mock: Mock.assert_api(...)

    Mock is a great module to use in testing, I use it pretty much all the time. But one thing I have nerver felt great about is the syntax of it's call_args (and call_args_list): it is a 2-tuple of the positional arguments and the keyword arguments, e.g. (('arg1', 'arg2' …

32. Using lib2to3 in setup.py

    It seems that many people think you need distribute if you want to run 2to3 automatically in your setup.py. But personally I don't like setuptools (aka distribute) and hence don't like forcing this on users. No worries since plain old distutils supports this as well, but it simply appears …

33. Discovering basestring

    This may seem simple and trivial, but today I discovered the basestring type in Python. It is essentially a base type for str and unicode, which comes in handy when writing isinstance(foo, basestring) in test code for example.

    Strangely, despide PEP 237 mentioning the equivalent for int and long …

34. Registering callbacks: to quack like a duck or tell which bit will quack the duck way?

    When you have someplace where you register a set of callbacks you have a few options:

    • Use an interface-style API. I.e. you have a Mixin style class defining the methods expected to be present, probably just raising NotImplementedError. You then just pass in the entire object to the .register …

35. Pointer arithmetic in C

    Pointer arithmetic in C is great. The only downside is that it's only defined when pointing to an array (or an item just after the array). That is one giant downside.

    Allocate a chunk of memory and you can't use pointer arithmetic. So if you are building a linked list …

36. The master plan

    Catherine Devlin describes it very nicely:

    I finally understand Al-Qaeda's master plan, and it's freaking brilliant. [...] I'm just surprised that we're choosing to participate in the plan. I thought we were on opposite sides?

37. Terminology

    Why was it ever considered desirable to call a directory containing a __init__.py file a "package" rather then just "module". They are after all simply "modules containing other modules". It's not like that solved some sort of problem was it? But, as sad as that sometimes might be, we …

38. Installing Debian on a Sun Fire X2200 M2

    Most important thing first: the eLOM console lives on ttyS1 (9600n8). If that gets you going you do not need to read any more.

    This works pretty easily, but there's isn't much information about how this works if you don't like using a keyboard and a screen (having these things …

39. Decorators specific to a class

    My gut tells me it's horribly wrong but I am failing to formulate a decent argument, let me show an example what I mean (somewhat contrived):

    def deco(f):
        def wrapper(self, *args, **kw):
            with self.lock:
                f(self, *args, **kw)
    
    class Foo:
        def __init__(self):
            self.lock = threading.Lock …

40. Calling COM methods in C

    So Windows has this strange thing called COM. It allows you to share objects between unrelated processes and languages, a bit like CORBA in that sense, only very different. Anyway, if you'd like to get information out of this other Windows thing called WMI (which provides a lot of information …

41. Setting descriptors on modules

    This counts for one of the more crazy things I'd like to do with Python: insert an instance of a descriptor object into the class dict of a module.

    While you can get hold of the module type class by using the __class__ attribute of any module or use types …

42. Skipping slow test by default in py.test

    If you got a test suite that runs some very slow tests it might be troublesome to run those all the time. There is of course the "-k" option to py.test which allows you to selectively enable only a few tests. But what I really wanted was a way …

43. setuptools vs distribute

    Reinout van Rees:

    In case you heard of both setuptools and distribute: distribute fully replaces setuptools. Just use distribute. Setuptools is “maintained” (for various historically dubious values of “maintain”) by one person (whom all should applaud for creating the darn thing in the first place, btw!). Distribute is maintained by …

44. Dictionary entry

    innovative company

    noun 1 a company which can't afford patent lawyers *2 one of the lucky handful companies who are innovative and have money for patent lawyers *HELP Not to be confused with most companies lobying in patent litigation.

    PS: It's also possible to have companies that can't afford patent …

45. How to drive on a motorway

    If you are driving on a motorway where I am driving too, here are some rules you should follow. I think you should even follow them when I'm not around.

    1. Be sensible, this one overrules all the others
    2. Keep left unless overtaking
    3. KEEP LEFT UNLESS OVERTAKING
    4. KEEP LEFT UNLESS OVERTAKING …

46. How to make a crosslinked RS-232 cable

    Because it's easier then buying one

    When connecting two computes together (e.g. your laptop to a server or router) to get access to a console you need to use a crosslinked RS-232 cable, usually with two female DE-9 connectors these days. This cable is more commonly known as a …

47. On using .__getstate__()

    Do not, I repeat, do not define .__getstate__() idly.

    Thank you very much

48. Tuple unpacking goodness

    Todays pleasant surprise:

    >>> a = {'a': (0, 1), 'b': (2, 3)}
    >>> for k, (v1, v2) in a.iteritems():
    ...     print k, v1, v2
    ...
    a 0 1
    b 2 3
    >>>
    

    Nice!

49. The new faces of Europe

    I can't help but agree.

50. Finding memory leaks in extension modules

    After reading Ned Batchelder's post on his experience hunting a memory leak (which turned out to be a reference counting error) it occurred to me that even tough I have a script to check memory usage I should also really be checking reference counts with sys.gettotalrefcount(). And indeed, after …

51. New Python System Information release!

    I've just released a new version of PSI! PSI is a cross-platform Python package providing real-time access to processes and other miscellaneous system information such as architecture, boottime and filesystems. Among the highlights of this release are:

    Improved handling of time

    We now have our own object to represent time …

52. Synchronous classes in Python

    What I'd like to build is an object that when doing anything with it would first acquire a lock and release it when finished. It's a pattern I use fairly-regularly and I am getting bored of always manually defining a lock next to the other object and manually acquiring and …

53. Python modules and the GPL: I still don't get it

    I've never understood if you can use GPL python modules (or packages) in GPL-incompatibly licensed code. Today I re-read the GPL looking for this and am tempted to think yes, you can. The GPL says:

    1. You may copy and distribute verbatim copies of the Program's source code as you receive …

54. Delny 0.4.1

    A little while ago I released a new version of Delny (my python wrapper around Qhull for Delaunay triangulations), the main purpose to use numpy instead of numeric. Impressively enough people actually seemed to care and I got a few bug reports and hints for improvements.

    So I just released …

55. No permission to see Ubuntu bugs?

    So I'm looking at the release notes for Ubuntu 9.10 and am interested in the hibernation issue. Naturally I follow the link to the bug report.

    First I need to log in to launchpad. That's weird, since when do I need to log in simply to view bugs? But …

56. Cross platform shell scripts

    At work we have an rather elaborate collection of shell scripts that builds our softweare and all it's dependencies on about 7 UNIX variants as well as on Windows. Shell seemed like a good choice when we started to write those scripts: it is available on all hosts (using Cygwin …

57. Battery life

    Normally I'm quite happy with the 3h of battery life of my laptop, it covers all my disconnected time on trains etc. But it just doesn't cut it on a 10h flight, especially painful if my brain has loads of ideas to try out and things to do.

58. New Delny release

    A few days ago I got another of those two-a-year inquiries about Delny, the python wrapper around Qhull for Delaunay triangulations. But increadably indirectly it finally got me to do a new release, it's only been a few years since the last one! Nothing has changed really, the only difference …

59. Resuming an scp file transfer

    Sometimes you have to transfer data over dodgy links. And when you're transferring a large file this can hurt when it fails. So as explained elsewhere rsync can safe the day:

    $ scp host:remote_file local_file
    # now interrupt and resume with
    $ rsync --partial --progress --rsh=ssh host:remote_file local_file
    

    You need …

60. Should bare except statements be allowed in the Python stdlib?

    Firstly to clarify the terminology, this is bare except statement:

    try:
        ...
    except:
        ...
    

    And this is a non-bare except statement, but bear in mind the type of the exception that is caught can be anything:

    try:
        ...
    except Exception:
        ...
    

    The point is that both fragments are a catch-all exception handler, only the …

61. Pain

    I've spent the last 2 days trying to get a stack trace from a crashing python extension module in windows. And I still haven't figured it out. That's sooo very motivating.

    Give me GNU/Linux any day.

62. cifs why don't you follow unix conventions?

    Often it's nice to just have conventions and gentleman's agreements. But then sometimes someone doesn't and the harm is done, it's too hard to reverse.

    NFS has introduced the de-facto standard of using <host>:<path> for the device of a remote mount. Yes this break because you can have ":" in …

63. Letters on keyboard stop working

    Dear Lazy Web

    I have some Very weird behaviour where the lowercase letters "C" and "V" stop generating the Correct key press event after a short while (which is why I am typing them as upper Case here...). But only when using Compiz as window manager, when switching away from …

64. PyCon UK 2009

    So there's a PyCon UK 2009 Unconference. I wish I could go but have already planned to be on Pabbay and Mingulay in the Outer Hebrides, Scotland. So instead of hearing all about python I'll be clinging on to their amazing cliffs. Guess I'm not that bad off anyway...

    

65. How to bring a running python program under debugger control

    Of course pdb has already got functions to start a debugger in the middle of your program, most notably pdb.set_trace(). This however requires you to know where you want to start debugging, it also means you can't leave it in for production code.

    But I've always been envious of …

66. devenv.com via cygwin ssh (visual studio 2003)

    Integrating an automatic build on a windows host with the rest of your one-command cross-platform build system can be quite a pain. Luckily there is cygwin which makes things like ssh, bash, make etc all work on windows. It's great.

    The trick to building using visual studio from there is …

67. Importing modules in C extension modules

    It seems that if you need another module in a function of your extension module, the way modules in the standard library seem to solve this is like this:

    static PyObject *
    func(void)
    {
        PyObject *foo;
    
        foo = PyImport_ImportModuleNoBlock("foo");
        if (foo == NULL)
            return NULL;
        /* do stuff with foo */
        Py_DECREF(foo);
        return …

68. Singletons in a Python extension module

    If you want a singleton in a C extension module for CPython you basically have to do the same as when doing this in plain Python: the .__new__() method needs to return the same object each time it is called. Implementing this has a few catches though.

    static PyObject *
    MyType_new …

69. Voting and identification in the UK

    Last Thursday I had the pleasure of being able to vote for the local council as well as for the European Parliament. Since I'm a Belgian citizen living in the United Kingdom this involved convincing the officials at the voting office to re-read their instructions (at first they only allowed …

70. Raising exceptions in threads

    It's not simply raising an exception in a thread that I want. I want to raise an exception in a thread from another thread. It's like sending signals to threads, only signals in pyhon can only be delivered to the main thread (for portability). But Python has a other asynchronous …

71. New home for PSI

    PSI, the Python System Information package that pulls interesting information from the kernel and makes it available in a nice Python API, has a new home at bitbucket. This means that the source conde now lives inside a mercurial repository instead of subversion.

    This actually happened about a week ago …

72. Python System Information 0.3b1^W0.3b1.1

    Short summary: PSI is alive and we've just released the first beta of a much improved upcoming 0.3 release!

    Back in 2007 Chris Miles announced PSI - Python System Information, a Python extension module to provide access to some system information not normally available to Python. Most notably it allows …

73. Compiling applications in setup.py

    If you need to compile a random application in setup.py this is not that hard:

    cc = distutils.ccompiler.new_compiler()
    distutils.sysconfig.customize_compiler(cc)
    cc.link_executable(['test.c'], 'a.out')
    

    There is no need to create the object files explicitly with cc.compile() first if you have no need for …

74. What's new in Python 2.6: logging

    The What's New in Python 2.6 document is very good and contains loads of information, congrats for making it. But it misses out a nice addition to the logging module: LoggerAdapter Objects. It allows you to create a logger object with extra contextual information really easy:

    log = logging.LoggerAdapter …

75. Sun Fire T1000 not closet friendly

    Obviously the Sun Fire T1000 is a noisy machine, that's why it's sitting in the closet in the first place. But it has more shortcomings that make it a closet-unfriendly server:

    • It shuts down at 50 degrees Celsius. That means wrapping it in a duvet to damp the noise is …

76. Time, floating points and python modules handling time

    Some memory in the back of my mind tells I've once read a rant about storing time that argued you should never store time in floating point variables. My memory seems to think the rant convinced me and it does indeed seem better to store time in integers so that …

77. Reading a 64-bit core from a 32-bit proc on solaris?

    I'm trying to read some pointers from a 64-bit address space file (from /proc/pid/as) while running in a 32-bit process - all this on Solaris. To do this I'm using the transitional large file compilation environment, i.e. where you get both open()/pread() and open64()/pread64() since I …

78. Replacing the screen of your digital camera

    The screen of my Canon Digital Ixus 40 compact camera broke on my last trip (sport climbing in Geyikbayiri near Antalya, Turkey - if you climb: it's an amzing place!) which made me rather sad. But with the help of Ebay I found a new screen and have just managed to …

79. How to send emails

    Dear World

    Subject lines in emails are there for a reason.

    Please use them.

    Love

    Floris

80. Virtual memory size (VSZ) on AIX

    If you have ever looked at the ps(1) output on an AIX box and wondered why the virtual memory size (VSZ) is so small, often even smaller then the resident set size (RSS), then here's the answer: it's not the virtual memory size. It is actually the size of …

81. Closed source frustration

    Developing against a closed source libraries is jut painful and wastes your time. Function definitions are missing from the header files, so you get to declare them yourself from the manpages. Then you get random error messages even though you do everything just like the manpage tells you. Some dark …

82. ssh magic

    Dear Lazy Web

    If I write the following in my ~/.ssh/config:

    Host bartunnel
      HostKeyAlias bar
      HostName bar
      LocalForward 8822 foo:22
    
    Host barjump
      HostKeyAlias bar
      HostName localhost
      Port 8822
    

    Then I can connect to host bar via host foo (circumnavigating a firewall that stops me from going to bar …

83. Planning upgrades

    So Debian has finally released again. Congratulations. Time to start planning upgrades of servers then.

84. Compiling 32-bit Python on amd64

    If you ever feel the need to compile a 32-bit version of Python on a amd64 bit machine, this is how you do it on a Debian/Ubuntu system.

    Firstly you need the correct compiler stuff, this means you need gcc-multilib and 32-bit development libraries of at least libc. On …

85. sed for binary files: bbe

    While GNU sed takes care not to munge NULL characters this is not the POSIX standard, therefore it's no surprise some implementations don't manage to edit binary files properly (not to mention that many implementations will struggle with long lines). Hence the search for a binary sed:

    bbe is just …

86. Specifying the RUNPATH in sofiles

    When you create a shared object that depends on another shared object there are multiple ways to tell it where to look for the shared object. Two of these are by encoding (at linking time) a DT_RPATH and DT_RUNPATH entry in the .dynamic section. A third one is to use …

87. Resistance to change

    Why can C developers be happy with using config.h to get constants about where to look for configuration or data files for example, yet Python developers seem to refuse to think of any way they might want to support finding these files in a portable way?

88. FreeBSD on Virtualbox

    Virtualbox is my desktop virtualisation technology of choice most of the time and I wanted to have a play with FreeBSD. Seems they don't get along tough and you won't get a network connection.

    • Solution 1: Change the network adaptor in Virtualbox to PCnet-PCI II (instead of PCnet-PCI III). I've …

89. datetime.datetime.totimestamp()

    There exists a datetime.datetime.fromtimestamp() and many a time I've wondered why there is no .totimestamp() equivalent. The answer is that not all datetime objects can be represented by a timestamp. But if you know this restriction and want it anyway, this is the magic:

    time.mktime(datetime_object.timetuple …

90. Resident Set Size (RSS) from /proc/pid/stat

    Most UNIX-like systems have information about processes stored in /proc/$pid/ files, so does Linux.

    If you would want to get the Resident Set Size (RSS) of a process on Linux you could find this in a number of files:

    /proc/$pid/stat

    targetted for scanf(3)

    /proc/$pid/statm …

91. Porting a C extension module to py3k

    This was not that hard! First look at what everyone else has to say about this, nice aggregated on the python wiki. It covers a lot.

    Now for what I discovered on top of this:

    • When you're defining a new type self->ob_type doesn't exist anymore. This is a problem …

92. Generating source files in setup.py

    I have a Python extension module written in C and one of the weirder things it needs to do is have a number of module constants with their values which are defined (e.g. #define PREFIX_FOO 0x01 etc.) in an external C header file. All the defined names in that …

93. import skynet

    So who's going to make the skynet module?

94. Finding memory leaks in Python extension modules

    Valgrind is amazing, even if you've never used it before it's basic usage is really simple:

    $ valgrind --tool=memcheck --leak-check=full /usr/bin/python ./test.py
    

    This is enough to point you at the places in your extension module where you allocated stuff that didn't get freed later. This is …

95. gcc, LD_RUN_PATH & Solaris

    I actually started writing a long post about how linking against shared libraries works and what and how ld.so does, but that seemed to get very long. So here the short version that assumes pre-existing knowledge.

    If you want an RPATH in your DSOs (dynamic shared objects: executables and …

96. Mocking away the .__init__() method

    I wanted to test a method on a class that didn't really depend on a lot of other stuff off the class. Or rather what it did depend on I had already turned into Mock objects with appropriate .return_value and asserting with .called etc. Problem was that the .__init__() method …

97. Preserving modification times in subversion

    This is an oft requested feature apparently. But who know if and when it will be there. Usually things get done when someone has an itch to scratch, but given that no one has done this yet -but it's been requested for ages- it seems that it must either be …

98. Generative Tests

    Both nose and py.test allow you to write generative tests. These are basically generators that yield the actual test functions and their arguments. The test runners will then test all the yielded functions with their arguments.

    However I'm left wondering why this is better then just iterating in the …

99. Solaris equivalent of dpkg -S

    In the past I have spent quite some time trying to figure out what package did install a certain file on Solaris. What I wanted was really just the equivalent of dpkg -S /path/to/file. But those searches where not very fruitfull, even talking to more experienced Solaris admins …

100. Bluetooth on a Toshiba Tecra A9-127

     I got a Toshiba Tecra A9-127, listening to the model number of PTS52E-06002LEN as written on the back, from work to replace my dying old HP Compaq nx9030. As my laptop OS of choice is Ubuntu, currently in it's Hardy release, it's not completely coincidence that the hardware is almost …

101. Ripping videos from DVD

     Physical discs are a nuisance, I really just want to play what I want to watch in the room I want to watch it just streaming it over the wireless. This actually works wonderfully well. Unfortunately just copying the file structure of a DVD works and gives you DVD-quality video …

102. Discovery of the day

     Now I'm debating if I should really use that in production code...

103. Time to read standards

     Sometimes I like quotes out of context...

     Anyway, I don't think it really is an ambiguity in practice — only in the minds of those that have too much time to read standards documents.

     —Greg Ewing (on distutils-sig)

104. @x.setter syntax in Python 2.5

     The new property.setter syntax in Python 2.6 and 3.0 looks very clean to me. So I couldn't wait and wanted it in Python 2.5. With the help of comp.lang.python I finally got there (thanks especially to Arnaud Delobelle):

     _property = property
     
     class property(property):
         def …

105. (O)OXML and ISO voting processes

     Many have recently complained about ISO's voting processes, mainly how they need to be revised as currently it seems it's possible to buy yourself a standard given enough lobbyist and money.

     But part of this is ISO's trust in ECMA. ISO allows ECMA to submit standards for the fast-track process …

106. Shell history

     hehe

     flub@signy:~$ history | awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn |head
     79 nosetests
     74 ls
     46 cd
     36 ssh
     28 man
     22 apt-cache
     19 vi
     19 ack
     17 svn
     17 sudo
     flub@signy:~$
     

     It must be noted that most of my editing …

107. nose and ipython

     I wish there was a simple way to integrate nose and ipython. Something like a command line option not to catch exceptions would be sufficient I think, so that you could do:

     In [1]: %run tests/test_module.py --do-not-catch-exceptions
     

     Obviously you'd want a shorter option switch...

     Seems like Test.run …

108. One s3fs to rule them all?

     Sometimes I wish we could fast forward 6 or 12 months or so. Hopefully by then there will be just one s3fs that is mature and maintained. Right now it's hard to tell which one will turn out the best.

     Interstingly, of the ones that look like they could have …

109. GPL and python modules

     I should probably google this, but can't think of good keywords. So instead this is a "dear lazy web" post.

     Is importing a python module considered linking for the GPL? I.e. are you allowed to import a python module into an script or module that has a non-GPL-comptible license …

110. Cheating the KAME turtule

     This is shocking. Even though I have not yet used IPv6 I can see the the KAME turtle dance! I won't give you a direct link but SixXS privde an IPv6->IPv4 service as well as an IPv4->IPv6 service. Feels bad having used the 4->6 bit... </p>

111. OmniORB 4.1.1 in lenny!

     Something that happened about a week ago I think. But omniORB 4.1.1 finally make it into Debian testing aka lenny! This means that python-omniorb 3.1, which has been waiting for a while now, also make it into lenny!

     It has been stalled for ages since we had …

112. Compiling libtool examples

     This is more a reminder to myself, but here's how to compile the libtool examples:

     $ cd demo
     $ aclocal
     $ libtoolize --automake
     $ automake --add-missing
     $ autoconf
     $ ./configure
     $ make
     

     It's really how to bootstrap any GNU autotools using application I suppose.

113. My favourite package description

     This sentence in a package description is always great...

     You probably don't need it if you don't understand what this is all about.

     Brought to you today by nut-snmp but can be found in many other packages.

114. Another shell trick

     Let's start with the solution instead of the problem:

     myvar=$(myfunc) || exit $?
     

     Assuming myfunc is a function defined elsewhere in your shell script this will execute that function and assign it's output to myvar. However when myfunc fails, the entire assignment statement fails with the exit status of myfunc, so …

115. Creating a Debian ARM build host

     If you want/need to compile a program on ARM but are not lucky enough to have the hardware available QEMU can still help you. Aurélien Jarno has an excellent description of how to do this. It is still valid today, even if you want to install sid instead of …

116. Docstring folding

     In my last post I had a little rant about how python docstrings visually separate the function/method/class definition from the body. Mikko Ohtamaa raised a very good point though: the editor should fold it away.

     Since my editor of choice is Emacs, said to be more an operating …

117. Documenting functions/methods/classes

     There are several simple guidelines for writing functions or methods (guidelines that I like that is).

     • Keep them clean, don't intersperse them with comments. If they require more then one or two inline comments it's too complicated and you should restructure.
     • Keep them short, it should fit on one screen …

118. Paradox

     Creating a Facebook group to complain about the UK ID cards because:

     [...] An ID card would hold ALL personal information about you including biometrics, hospital records etc etc., a perfect target for ID thieves. [...]

     And you're doing this on Facebook?

119. Mild improvement for movie industry

     Until now every time I bought a DVD I found a few scary and angry leaflets in the box telling me how insanely bad I might be for buying pirated DVDs. They usually convince you that you are the worst person in the world for no particular reason. So just …

120. Rock climbing in Costa Daurada

     Just spent a long week sport climbing in Costa Daurada. We stayed in a house (with about 32 students and ex-students) near the village of Cornudella de Montsant which was an excellent location really close to Siurana and Arboli. Absolutely brilliant climbing and although my brain wanted to stay a …

121. Updated omniORB in Debian

     A while ago Thomas Girard started the Debian CORBA Team with the aim of maintaining omniORB and related packages in Debian. After a lot of work (and time!) we managed to get the packages of up to date versions in a good state fixing almost all bugs in Debian's BTS …

122. Seeing your browser's headers using python

     There's a very annoying website that won't send their CSS to my normal web browser (epiphany ) which makes it rather ugly. However when I use iceweasel the CSS gets applied. Since both browsers use exactly the same rendering engine, gecko, on my machine as far as I know, I thought …

123. Schneier: hammer, nail

     As Bruce Schneider says:

     If you remember, this was sold to the public as essential for fighting terrorism. It's already being misused.

124. Making Debian packages on an SMP machine

     When you have an SMP machine (including dual core CPUs - but I'm sure everyone knows that by now), you quickly learn to use the -jN flag to GNU make(1). N is the number of CPUs you have and it lets make(1) run that many jobs in parallel whenever …

125. How far does encryption get you?

     Finally, since a short while, all my hard disks are setup to use encrypted disks using LUKS. Works like a charm and you don't really notice it slow down. Unfortunately I am now left wondering what the point is, given that I currently live in the UK. Not that there …

126. Simple music players

     A while ago I bragged about my own music player. Basically I couldn't find the player I liked so started creating my own. It was the first time I wrote anything related to media or a GUI -good thing my basic requirement was simplicity- and was interesting to do, but …

127. There's a first time for everying

     flub@signy:~/src$ tar -xzf storm-0.10.tar.gz
     
     gzip: stdin: not in gzip format
     tar: Child returned status 1
     tar: Error exit delayed from previous errors
     flub@signy:~/src$ file storm-0.10.tar.gz
     storm-0.10.tar.gz: POSIX tar archive (GNU)
     flub@signy:~/src$ mv storm-0.10.tar …

128. EU in censor madness

     So the EU seems to think that the Internet should be filtered. The first step is blocking searches for bomb recipes.

     To any rational person it is obvious that this is wrong on more levels then I care to explain (others have done so already better then I can), but …

129. Capable Impressive System Crippled OS

     That's my new acronym for cisco. Seriously, I did expect a learning curve for IOS when I was ordering a cisco 877W, but this is just disappointing.

     No doubt IOS was once very amazing and had a very nice and clear concept and interface to the first few products it …

130. Source code to my music player

     Recently I said I created an increadably stupid, dump and boring GTK music player. I also offered to make the source code available in the very unlikely case someone else would find this thing interesting. I should have known to just do that right away, so here it is.

131. My very own music player

     Yes, I've contributed to the masses of music players out there. It is pleasantly easy using python and the pygtk and pygst modules.

     But why yet an other music player? Well it is, intentionally, increadably simple.

     • No library. Just a simple list of files, played sequentially.
     • No metadata. If you …

132. Encrypted hard disk: LVM on LUKS

     There are of course about a million other guides on how to use encrypted disks out there. However I did run into some trouble when trying this, so here mine. Specificly I address the issue of getting an encrypted root disk, with the root being on a Logical Volume Management …

133. Writing entropy

     Q:

     How long does it take to write a 120 GB disk full with random data?

     A:

     15 hours, 2 minutes and 27 seconds.

     Obviously this depends on the machine. For me it was going at just under 8 minutes per GB, others report around 5 minutes per GB. Also …

134. Debian is 14

     Happy brithday!

     And thanks for everyone involved all those years.

135. Gazpacho and libglade

     If you read my last post about using Gazpacho you should read the comment by Johan Dahlin too. He's one of the authors of Gazpacho and explains the libglade and save format issues^Wthings in Gazpacho nicely.

136. Making GUI applications with Gazpacho

     Earlier I have made some simple GUI applications using PyGTK and Glade, which is surprisingly easy. Now I have another small itch to scratch and have another go at some GUI app. Only this time I decided that the coolness of Gazpacho looked slightly nicer to me, so gave that …

137. Optimising python code by AST fiddling

     Obviously no real optimisation, just optimisation exactly like python -O [-O] would create. But in "normal" byte compiled files, i.e. in .pyc files instead of .pyo files.

     Why would I want to do that? Well, we really don't feel like shipping code with assert statements or if __debug__: ... bits …

138. How to choose a web framework?

     So I have a bit of organically growing information that is nice to represent on internal web pages in a semi-organised fashion. The first iteration just runs a few python scripts from some cron jobs that utilise SimpleTAL to create staticly served pages. SimpleTAL as templating language simply because we …

139. __slots__: worth the trouble?

     I find __slots__ annoying. They seem to have a lot of rules about their use and make things like a borgs rather awkward to use (I even prefer to use a singleton instead of a borg in new-style classes because of this).

     And all it does is safe some space …

140. IPC in the stdlib(?)

     Many interesting comments on my last musings about IPC. Each and every one of them look very nice and have their application area, go and have a look! These found a weak spot in my heart though:

     • Candygram is Erlang style message passing. Very stunning, but to my liking a …

141. IPC and the GIL

     As recently described excelently, threads are ugly beasts, waiting to get you when you let your guard down (and eventually you will). Yes, that means that I should really get my head round the asynchat module and stop using ThreadingMixIn with the SocketServer, but that's not the point of this …

142. What's in a number?

     Ok, I coudn't resist this. Someone told me (as much as writing a blog counts as telling) this is my lucky number: 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0. Someone else says it's magic. Lucky and/or magic numbers... hmm, sounds …

143. More roundup love

     As I've said before, roundup is an amazing issue tracker. Last time I wrote I had just implemented our internal bug tracker with all the weird requirements of a few developers. By now the incredibly flexible system, thanks to python, has been moulded into our complete project management tracker.

     This …

144. Campaigning for -delete

     Whenever find(1) is used to delete some files there are always a few variants that can be seen. They are generally combinations of -exec and -print piped to xargs. While demonstrating the increadible power of UNIX shells and pipes etc they don't appear very practical.

     Most implementations struggle with …

145. Debian Developers working together

     My quote of the day (from Josselin Mouette):

     BTW, who said flamewars are a nuisance to the Debian project? I've never seen developers as united as when Mr Schilling is around. With a few more people like him we would probably be always working together.

146. Backup script in 43 lines

     This is no more then my personal backup prefference at this moment. It is full of personal preferences and might not be what other people like. It also isn't written with portability in mind. But I think it is rather elegant.

     My use case: on Friday evening I get home …

147. Timers for profilers

     After some oh... and ah... experiences I have decided that the only timer that will give you correct results in a portable manner (accross POSIX platforms that is) is resource.getrusage(resource.RUSAGE_SELF) and os.times(). This is rather shocking, certainly after I then went on to discover that on …

148. Schizophrenic thoughts

     On one hand I use tab-completion on the command line all the time. I even go as far as creating my own -terribly incomplete- completion functions in bash for commands that I use often

     On the other hand, when programming I don't use tab-completion -maybe mostly because it's not so …

149. Roundup praise

     Roundup is just an amazing bug tracking system. It really is way more general then that, their own words for it are not bad: an issue tracking system for knowledge workers.

     Why I love it:

     • It's written in python
     • It has a poweful and flexible concept/abstraction of a database …

150. Writing applications as modules

     The Problem

     I recently had to write a few command line applications of the form command [options] args that did some stuff, maybe printed a few things on screen and exited with a certain exit code. Nothing weird here.

     These apps where part of a larger server system however and …

151. When web pages check web-broswer compatibility

     I don't trust banks and definately not online banking since their log in procedures are laugable, at least in the UK (no complaints about my Belgian bank, although not all Belgian banks are good either). But somehow I managed to get an account which only allows transactions to my own …

152. Warnings revisited (aka logging vs warnings)

     Earlier I posted about how to make warning messages from warnings.warn() look like fairly normal messages so they don't give you all the details about which line of code called them. The result was neat, but I found it a fairly crude way of doing it.

     There is another …

153. Sorry for the flooding...

     Apologies for the flooding of planet python.

154. Overriding formatting of warnings.warn()

     Today I needed to add the printing of warning messages to a command line tool written in Python. So Instead of writing things out using print, wich is not flexible enough and goes against my way to avoid user interaction deep inside functions and classes, I decided to use the …

155. Ubuntu & reportbug

     Lately I only appear to be writing about things I'm unhappy with and general moaning. Sorry for that. But here's another one of them:

     An important part of Debian is the BTS, it allows users to report bugs as most people will know is obviously very important. Quite a few …

156. mawk considered harmful

     Consider the following little awk script:

     /^--- [-[:alnum:].\/]+$/ { if ($2 !~ /[-[:alnum:].]+\/debian\//) print $2 }
     

     According the the O'Reilly's sed & awk book it is, as far as my knowledge goes, normal valid awk. It's intended to be used like such:

     ~$ zcat debian_package_version.diff.gz | awk -f script.awk
     

     The purpose is to …

157. site, eggs, etc.

     Before I complained about why .pth files are not processed in files on in your $PYTHONPATH. I got a few pointers in that regard, thanks for them. And now I know why it doesn't happen: you can import foo, bar; foo.do(); bar.harm() inside a .pth file. Since this …

158. Ubuntu & upstart

     Upstart is a new sysvinit </tt> replacement that Ubuntu wants to use. Scott James Remnant explained before it is innovating in that it is events based. Now he writes more about it, and one quote is this:

     So what are events and where do they come from? (Note that this …

159. Nuking your home directory

     /path/to/some/weird/directory$ rm -rf ~

     Yep, I've done that today. It sucks. 1.5 seconds after I did that I hit the power button in the hope the kernel wouldn't have flushed it's buffers yet (hm, that old trick might not work with journaling filesystems). Boot up in …

160. When essential tools break...

     I hate it when that happens. Current case:

     flub@coronation:/tmp/sandbox$ gpg --clearsign test
     gpg: selecting openpgp failed: unknown command
     gpg: signing failed: general error
     gpg: test: clearsign failed: general error
     flub@coronation:/tmp/sandbox$
     

     That's just to get desparate. Some quick poking around results in leading me to …

161. UTF-8 and Emacs

     While looking around in Manoj's Debian web-page/directory for his new Python policy (for how to package Python stuff for Debian) I stumbled, amongst other intresting things, across this cool UTF-8 demo file.

     To my surprise my web browser (Epiphany) shows nearly everything perfect, so does gedit. However my much …

162. site module musings

     The site package reads .pth files and inserts directories listed there in sys.path. However it only scans lib directories ( lib/python2.4/site-packages/ and lib/python/site-python/ on UNIX) starting with sys.prefix or sys.exec_prefix for these .pth files. Thus if you install a module distribution using the …

163. Python web frameworks

     It appears discussing web frameworks is popular currently. It also appears GvR has decided he likes Django and hopes it will become sort of the default.

     I've been looking at this Django and TurboGears things for a while unable to decide if I should move the website that I appear …

164. setup.py: setup(..., extra_path="foobar", ...)

     The extra_path keyword that you can pass to setup() when using the distutils for building and installing a package is completely undocumented as far as I know. I have no idea how I found it the first time (about a year ago) but now it was there and was not …

165. Today's WTF: sys.path in python != sys.path in ipython

     Spot the current directory ( <snip>/build/lib.linux-i686-2.4) in the path:

     <snip>/build/lib.linux-i686-2.4$ echo $PYTHONPATH
     /home/flub/lib/python:/home/flub/lib/python2.4:
     <snip>/build/lib.linux-i686-2.4$ python
     Python 2.4.3 (#2, Apr 27 2006, 14:43:58)
     [GCC 4.0.3 (Ubuntu …

166. GnuPG Smartcard on Debian Sarge

     There's a document on the GnuPG explaining how to set up your system to use a smartcard. There a few people around claiming that it works on a standard Debian Sarge by using udev. Up to today I have had no luck with that however, but that finally changed!

     I'll …

167. Dalai Lama visits Belgium

     Like I care.

     But China cares, that's more worrying. Dispite it not being an "official" visit (he was invited by some buddhists to open a new temple) he visited the prime minister and some other politicians. And hence the chinese ambassador says it will hurt the Belgian-China relationships as they …

168. Summer of Code 2006

     There must be about a million blog posts with this title by now. Anyway, here my own little addition, I've been meaning to write this for a while.

     I have not applied for SoC again althoug I was still eligible. There where intresting projects however, and in fact I'm sorry …

169. No feedback?

     In the beginning of this week I emailed my mentor to know what he thought of my project, if he was happy with the state etc. I do know from his blog that he's fairly busy these days, but still would enjoy some feedback. Also just in case my email …

170. One FIXME down...

     Profile.calibrate() now uses the correct timer instead of just the default timer (time.time()). In fact it still uses this timer when the user did not specify a specific timer (and why really would anyone want this? Appart form making me work one or two weeks more in the …

171. Magic

     As promised I worked on a proper installble package of hprof. That went quite smooth, I had some trouble figuring out how to create a .pth file with distutils, but after some googling I found some setup keyword that worked. Not that is it documented in the official documentation. The …

172. Small update

     This weeked I've been helping at home again building the patio outside. It's quite impressive to see things changing from a big pit of dirt in the morning to a nice surfaced area in the evening. If I had a digital camera I'd show you nice before and after pictures …

173. (Almost) Done!

     Just completed hpstats module, it's all in CVS. This means that theoretically the project is finished! Altho I expect to code another few days on it before the end of August, nothing can be perfect... leave alone the first time round!

     Along the list of TODO's is making a little …

174. Sleep is good

     Last night I got some problems with a function call. As far as I could see it was right but it's unit test kept failing. After a while I called it a day and went to sleep. Wake up this morning and have a look. Only to discover it was …

175. Nice progress

     Work has been progressing nicely the last few days. All the hacking has been going smoothly, which does not mean bug free - far from it! But I seem to have finally reached a good cycle of writing unit tests, implementing the features and refining that all. The scaffolding is in …

176. Ambiguous documentation

     pstats does not behave as the documentation says it does. I got to the stage that you can look through the output of Lib/test/test_profile.py (in Python distribution) and the output of that very same file but with a import hprofile as profile at the top instead of …

177. Clarifying the hstats module

     Some confusion seems to exist about the stats modules I'm writing. About a month and a bit ago I wrote hstats as a module to analyse hotshot profiling data. It was never my intention to make it compatible with pstats at all. It aimed at being usable to read profiling …

178. Why one should have a separate /boot partition: lessons learned

     Background

     The box (SPARC box to make matters more intresting) has two disks /dev/hda and /dev/hdb, they each have the same partion table appart from some free space at the end since two disks are never of the same size. All partitions are of type "RAID autodetec" and …

179. Cross platform developing

     Even when using Python this is easier said then done. One should be very carefull as details lies in tiny details. Wich I discovered to my shame.

     Normally I work on a desktop, this box listenst to the name of Ultra 10, so it is a UltraSparc architecture as developed …

180. Cleaning up?

     So last time I said I wasn't happy with hstats. Maybe that was a bit impulsive, but still valid. Kent Johnson kindly pointed out <http://bruynooghe.blogspot.com/2005/08/i-wrote-crap.html#112378135725654786> for me that was the wrong attitude. Maybe he's right (read: yeah, of course he's right), but …

181. I wrote crap

     hstats is crap. And I only wrote it a month ago...

     The good part of that observation is that it shows that I'm learning. ;-)

     So why really? Basically just it's desing. It has one single monolithic class. And within this class everything just happens like one would wirte precedural or …

182. Never ending quest

     On my never ending quest to find the best testing strategies I stumbled across this article. It is not too long and well worth reading. Explains some stuff about using mock objects and related testing strategies. All of this is of course XP influenced.

183. hotshot accepts user timer function

     After a long day it all works. It's checked in to CVS in the "hotshot_timer-branch". The way it works is that the object returned by _hotshot.profiler(filename) now has an additional method settimer(timer). The timer is any callable object that returns a number or a sequence of numbers …

184. Pointer to funcion

     Great fun these things. Certainly when the pointer to the function is stored inside a struc. The result is that my version of _hotshot now uses a void pointer in ProfilerObject as a function to get the time difference since last time it was called. For now it only goes …

185. Programming can be simple

     It really is clear that it has been over 10 months since I last wrote C. As I mentioned in my last post I got segmentation faults when I changed the version string in _hotshot.c. Today I started by figuring out that the version string gets indeed used at …

186. Double Ouch

     One

     To get a user supplied timer function in hotshot I need to start hacking on _hotshot.c. Being cautious I pull the file from python CVS, put it in my package tree, write a setup script, build the stuff before modifying everything -just to test build environment- and finally …

187. Seconds & nanoseconds

     Last night I struggled a lot with the calibration. In the end I fainted and gave up, went to bed and read a crap book for way too long -books are no good for me, even if they're crap I can't stop reading them. Today, after waking up way to …

188. The easy bits

     So I've written the easy methods and functions of the profile module wrapper hprofile today. Including their unittests.

     Tomorrow starts the harder work, the Profile.calibrate code. Maybe I shouldn't be too optimistic about how far I'll get with it. For what it's worth, here my current ideas of how …

189. If things do their job...

     Unit tests work. It's amazing...

     But good!

190. To hybernate or not to hybernate...

     That's the question

     Old days

     For years and years I always envied my father who never had to log out of his window manager at his work, he only had to lock his screen. This meant he could leave all his terminals and text editors open at night. The next …

191. Refactor Mercilessly

     Ok, I admit. Maybe I'm just using this as an excuse. I haven't really impressed myself with the amount of work done the last few day. But hey! This was my holiday after all.

     The result however is that I have revised the test module for the hstats module (instead …

192. Native hoshot statistics!

     Finally, after some refactoring, having cursed unittests and XP but in the end blessing them anyway. The result is there: the hstats module!

     It loads hotshot data 35% faster then hotshot.stats.load() as I reported earlier. It sorts the data, and prints it out nicely. Moreover the sorting and …

193. Progress...

     So I tried this XP approach of first creating unittests the last couple of days. Writing the tests goes soooo slow! I was a bit surprised on how easily I could write the real functions afterwards tho. Maybe it is good. But it requires lots of tinking in advace and …

194. Stats unittests

     That's what I've mostly done, been making and polishing unittests for the hstats module (or what's written of it. hstats btw, is the module that will create the statistics for hotshot, natively reading hotshot data. In the process one extra omission (not really a bug as such ;-)) got into the …

195. Project on Savannah

     So I got my project accepted on savannah. After some time of not being sure where it would be hosted, on savannah on it's own or on sourcefource in the Python project under the nondist/sandbox thing. Guess I'll stay with savannah now.

     It's under the project name pyprof. Not …

196. Paperwork rant

     This is going to be a rant.

     I faxed Google's paperwork really early after we got it (27th of June IIRC). Now we've been told that if we haven't received confirmation yet we should re-submit! The problem is that in the mean time I've moved houses in Southampton, UK where …

197. First success!

     Loading a profiling file dumped by hotshot goes *35% faster* than with hotshot's old method!

     Specifically on a Sun Ultra 10 with a TI UltraSparc IIi (Sabre) processor -872BogoMips if that tells anything- I loaded the profile dump of pystones in 139.8 seconds with my new code (hstats.Stats …

198. Deep or shallow frame stack?

     So I'm working on parsing the file created by hotshot. The question in the title relates to how the storing of the cumulative time has to happen. The problem is that every frame has to add it's time to the cumulative time of all the anchestors.

     Shallow frame stack

     Here …

199. Profiling code accounted to user code!?

     It appears hotshot does only take one time sample in the profiler callback. This means that all the time spent in hotshot's code is accounted to the user code!

     Mental note to self: remember this when I'm trying to increase accuracy later this summer!

200. Iterative learning

     Understanding Code

     I's amazing how little you (well, at least me!) understand the first time you read some (complex) code. Only after a few times you start to get the big picture and see how it all fits together. It's mostly a matter of understanding the underlying principles I guess …

201. Coding started

     Profile wrapper

     So the plan is to wrap hotshot so it can behave exactly as the profile module. It will be a bit tricky as some things of the old profile module don't make sense anymore in hotshot.

     Hotshot - Memory allocation

     So I have been learning a lot the last …

202. Software patents rejected!

     Finally, the european parliament managed to see the light and rejected the software patents directive. There is still hope.

     Also last night I found the Self-certifying Filesystem. I nerver even considered using plain NFS, but this looks very promising. Maybe tonight I'll end up trying it on my LAN. That …

203. hotshot

     So I spent today looking at the old profile module and hotshot. Learned a lot - but not enough yet.

     I quite like the idea of Brett to try and stick with hotshot and try to improve that. The main job would then be to write a wrapper to emulate profile …

204. Project for the next two months...

     So this is my blog for the SoC (Google Summer of Code), as recommended by the email I got from the Python Software Foundation.

     For those who don't know what I'm doing. It is listed on Python's Summer of Code wiki under "Profile Replacement".

     I haven't done very much yet …