Blog Archives

  1. Using Modern Linux Sockets

    Introduction

    On the surface of it the Linux socket API is pretty straight forward. However once you start to care about performance things get a bit more complicated, and documentation a bit more sparse.

    Because I thought it'd be fun to explore this a bit more I made a simple …

  2. Encrypted root on Debian with keyfile/keyscript

    By Floris Bruynooghe

    I've recently set up another laptop with whith whole-disk encryption, including /boot. This is all fine --the debian-installer almost gets it right, you just need to edit /etc/default/grub to add GRUB_ENABLE_CRYPTODISK=y and then re-run grub-install on the target-- but once you get this working you end up …

  3. py.test sprint in Freiburg

    By Floris Bruynooghe

    Testing is a really important part of Python development and picking the testing tool of choice is no light decision. Quite a few years ago I eventually decided py.test would be the best tool for this, a choice I have never regretted but has rather been re-enforced ever since …

  4. Pylint and dynamically populated packages

    By Floris Bruynooghe

    Python links the module namespace directly to the layout of the source locations on the filesystem. And this is mostly fine, certainly for applications. For libraries sometimes one might want to control the toplevel namespace or API more tightly. This also is mostly fine as one can just use private …

  5. New pytest-timeout release

    By Floris Bruynooghe

    At long last I have updated my pytest-timeout plugin. pytest-timeout is a plugin to py.test which will interrupt tests which are taking longer then a set time and dump the stack traces of all threads. This was initially developed in order to debug some some tests which would occasionally …

  6. Designing binary/text APIs in a polygot py2/py3 world

    By Floris Bruynooghe

    The general advice for handling text in an application is to use a so called unicode sandwich: that is decode bytes to unicode (text) as soon as receiving it, have everything internally handle unicode and then right at the boundary encode it back to bytes. Typically the boundaries where the …

  7. Don't be scared of copyright

    It appears there is some arguments against putting copyright statements on the top of a file in free software or open source projects. Over at opensource.com Rich Bowen argues that it is counter productive and not in the community spirit (in this case talking about OpenStack). It seems to …

  8. Small Emacs tweaks impoving my Python coding

    By Floris Bruynooghe

    Today I've spent a few minutes tweaking Emacs a little. The result is very simple yet makes a decent impact on usage.

    Firstly I remembered using the c-subword-mode a long time ago, I couldn't believe I never used that in Python. Turns out there is a more genericly named subword-mode …

  9. Using __getattr__ and property

    By Floris Bruynooghe

    Today I wasted a lot of time trying to figure out why a class using both a __getattr__ and a property mysteriously failed. The short version is: Make sure you don't raise an AttributeError in the property.fget()

    The start point of this horrible voyage was a class which looked …

  10. Synchronising eventlets and threads

    By Floris Bruynooghe

    Eventlet is an asynchronous network I/O framework which combines an event loop with greenlet based coroutines to provide a familiar blocking-like API to the developer. One of the reasons I like eventlet a lot is that the technology it builds on allows it's event loop to run inside a …

  11. Creating subprocesses in new contracts on Solaris 10

    Solaris 10 introduced "contracts" for processes. You can read all about it in the contract(4) manpage but simply put it's a grouping of processes under another ID, and you can "monitor" these groups, e.g. be notified when a process in a group coredumps etc. This is actually one …

  12. re.search() faster then re.match()

    By Floris Bruynooghe

    This is a counter-intuitive discovery, in IPython:

    In [18]: expr = re.compile('foo')
    
    In [19]: %timeit expr.search('foobar')
    1000000 loops, best of 3: 453 ns per loop
    
    In [20]: %timeit expr.match('foobar')
    1000000 loops, best of 3: 638 ns per loop
    

    So now I'm left wondering why .match …

  13. Storm and SQLite in-memory databases

    By Floris Bruynooghe

    When using an SQLite in-memory databases in storm the different stores created from the same database are not modifying the same SQLite database. E.g.

    db = storm.locals.create_database('sqlite:')
    store1 = storm.locals.Store(db)
    store2 = storm.locals.Store(db)
    

    Here store1 and store2 will not refer to the same …

  14. Finding the linux thread ID from within python using ctypes

    By Floris Bruynooghe

    So I've got a multi-threaded application and suddenly I notice there's one thread running away and using all CPU. Not good, probably a loop gone wrong. But where? One way to find this is revert history in the VCS and keep trying it out till you find the bad commit …

  15. Return inside with statement

    By Floris Bruynooghe

    Somehow my brain seems to think there's a reason not to return inside a with statement, so rather then doing this:

    def foo():
        with ctx_manager:
            return bar()
    

    I always do:

    def foo():
        with ctx_manager:
             result = bar()
        return result
    

    No idea why nor where I think to have heard/read this …

  16. Templating engine in python stdlib?

    By Floris Bruynooghe

    I am a great proponent of the python standard library, I love having lots of tools in there and hate having to resort to thirdparty libs. This is why I was wondering if there could be a real templating engine in the stdlib some day? I'm not a great user …

  17. Using Debian source format 3.0 (quilt) and svn-buildpackage

    By Floris Bruynooghe

    Searching the svn-buildpackage manpage for the 3.0 (quilt) format I thought that it wasn't able to apply the patches in debian/patches during build time. Instead I was doing a horrible dance which looked something like svn-buildpackage --svn-export; cd ../build-area/...; debuild. Turns out I was completely wrong.

    svn-buildpackage doesn't …

  18. Europython, threading and virtualenv

    By Floris Bruynooghe

    I use threads

    correctly

    I do not use virtualenv

    and dont't want to

    Just needed to get that out of my system after europython, now mock me.

    PS: I should probably have done this as a lightening talk but that occurred to me too late.

  19. weakref and circular references: should I really care?

    By Floris Bruynooghe

    While Python has a garbage collector pretty much whenever circular references are touched upon it is advised to use weak references or otherwise break the cycle. But should we really care? I'd like not to, it seem like something the platfrom (python vm) should just provide for me. Are all …

  20. python-prctl

    By Floris Bruynooghe

    There is sometimes a need to set the process name from within python, this would allow you to use something like pkill myapp rather then the process name of your application just being yet another "python". Bugs (which I'm now failing to find in Python's tracker) have been filed about …

  21. Storm and sqlite locking

    By Floris Bruynooghe

    The Storm ORM struggles with sqlite3's <http://docs.python.org/library/sqlite3.html> transaction behaviour as they explain in their source code. Looking at the implementation of .raw_execute() the side effect of their solution to this is that they start an explicit transaction on every statement that gets executed. Including …

  22. Python optimisation has surprising side effects

    By Floris Bruynooghe

    Here's something that surprised me:

    a = None
    def f():
        b = a
        return b
    def g():
        b = a
        a = 'foo'
        return b
    

    While f() is perfectly fine, g() raises an UnboundLocalError. This is because Python optimises access to local variables using the LOAD_FAST/STORE_FAST opcode, you can easily see why this …

  23. Judging performance of python code

    By Floris Bruynooghe

    Recently I"ve been messing around with python bytecode, as a result I now know approximately how the python virtual virtual machine works at a bytecode level. I've found this quite interesting but other then satisfying my own curiosity had no benefit from this knowledge. Until today I was wondering …

  24. Hacking mock: Mock.assert_api(...)

    By Floris Bruynooghe

    Mock is a great module to use in testing, I use it pretty much all the time. But one thing I have nerver felt great about is the syntax of it's call_args (and call_args_list): it is a 2-tuple of the positional arguments and the keyword arguments, e.g. (('arg1', 'arg2' …

  25. Using lib2to3 in setup.py

    By Floris Bruynooghe

    It seems that many people think you need distribute if you want to run 2to3 automatically in your setup.py. But personally I don't like setuptools (aka distribute) and hence don't like forcing this on users. No worries since plain old distutils supports this as well, but it simply appears …

  26. Discovering basestring

    By Floris Bruynooghe

    This may seem simple and trivial, but today I discovered the basestring type in Python. It is essentially a base type for str and unicode, which comes in handy when writing isinstance(foo, basestring) in test code for example.

    Strangely, despide PEP 237 mentioning the equivalent for int and long …

  27. Pointer arithmetic in C

    By Floris Bruynooghe

    Pointer arithmetic in C is great. The only downside is that it's only defined when pointing to an array (or an item just after the array). That is one giant downside.

    Allocate a chunk of memory and you can't use pointer arithmetic. So if you are building a linked list …

  28. The master plan

    By Floris Bruynooghe

    Catherine Devlin describes it very nicely:

    I finally understand Al-Qaeda's master plan, and it's freaking brilliant. [...] I'm just surprised that we're choosing to participate in the plan. I thought we were on opposite sides?
  29. Terminology

    By Floris Bruynooghe

    Why was it ever considered desirable to call a directory containing a __init__.py file a "package" rather then just "module". They are after all simply "modules containing other modules". It's not like that solved some sort of problem was it? But, as sad as that sometimes might be, we …

  30. Installing Debian on a Sun Fire X2200 M2

    By Floris Bruynooghe

    Most important thing first: the eLOM console lives on ttyS1 (9600n8). If that gets you going you do not need to read any more.

    This works pretty easily, but there's isn't much information about how this works if you don't like using a keyboard and a screen (having these things …

  31. Decorators specific to a class

    By Floris Bruynooghe

    My gut tells me it's horribly wrong but I am failing to formulate a decent argument, let me show an example what I mean (somewhat contrived):

    def deco(f):
        def wrapper(self, *args, **kw):
            with self.lock:
                f(self, *args, **kw)
    
    class Foo:
        def __init__(self):
            self.lock = threading.Lock …
  32. Calling COM methods in C

    By Floris Bruynooghe

    So Windows has this strange thing called COM. It allows you to share objects between unrelated processes and languages, a bit like CORBA in that sense, only very different. Anyway, if you'd like to get information out of this other Windows thing called WMI (which provides a lot of information …

  33. Setting descriptors on modules

    By Floris Bruynooghe

    This counts for one of the more crazy things I'd like to do with Python: insert an instance of a descriptor object into the class dict of a module.

    While you can get hold of the module type class by using the __class__ attribute of any module or use types …

  34. Skipping slow test by default in py.test

    By Floris Bruynooghe

    If you got a test suite that runs some very slow tests it might be troublesome to run those all the time. There is of course the "-k" option to py.test which allows you to selectively enable only a few tests. But what I really wanted was a way …

  35. setuptools vs distribute

    Last updated:
    By Floris Bruynooghe

    Reinout van Rees:

    In case you heard of both setuptools and distribute: distribute fully replaces setuptools. Just use distribute. Setuptools is “maintained” (for various historically dubious values of “maintain”) by one person (whom all should applaud for creating the darn thing in the first place, btw!). Distribute is maintained by …
  36. Dictionary entry

    By Floris Bruynooghe

    innovative company

    noun 1 a company which can't afford patent lawyers *2 one of the lucky handful companies who are innovative and have money for patent lawyers *HELP Not to be confused with most companies lobying in patent litigation.

    PS: It's also possible to have companies that can't afford patent …

  37. How to drive on a motorway

    By Floris Bruynooghe

    If you are driving on a motorway where I am driving too, here are some rules you should follow. I think you should even follow them when I'm not around.

    1. Be sensible, this one overrules all the others
    2. Keep left unless overtaking
    3. KEEP LEFT UNLESS OVERTAKING
    4. KEEP LEFT UNLESS OVERTAKING …
  38. How to make a crosslinked RS-232 cable

    By Floris Bruynooghe

    Because it's easier then buying one

    When connecting two computes together (e.g. your laptop to a server or router) to get access to a console you need to use a crosslinked RS-232 cable, usually with two female DE-9 connectors these days. This cable is more commonly known as a …

  39. Tuple unpacking goodness

    By Floris Bruynooghe

    Todays pleasant surprise:

    >>> a = {'a': (0, 1), 'b': (2, 3)}
    >>> for k, (v1, v2) in a.iteritems():
    ...     print k, v1, v2
    ...
    a 0 1
    b 2 3
    >>>
    

    Nice!

  40. Synchronous classes in Python

    By Floris Bruynooghe

    What I'd like to build is an object that when doing anything with it would first acquire a lock and release it when finished. It's a pattern I use fairly-regularly and I am getting bored of always manually defining a lock next to the other object and manually acquiring and …

  41. Python modules and the GPL: I still don't get it

    By Floris Bruynooghe

    I've never understood if you can use GPL python modules (or packages) in GPL-incompatibly licensed code. Today I re-read the GPL looking for this and am tempted to think yes, you can. The GPL says:

    1. You may copy and distribute verbatim copies of the Program's source code as you receive …
  42. Delny 0.4.1

    By Floris Bruynooghe

    A little while ago I released a new version of Delny (my python wrapper around Qhull for Delaunay triangulations), the main purpose to use numpy instead of numeric. Impressively enough people actually seemed to care and I got a few bug reports and hints for improvements.

    So I just released …

  43. Cross platform shell scripts

    By Floris Bruynooghe

    At work we have an rather elaborate collection of shell scripts that builds our softweare and all it's dependencies on about 7 UNIX variants as well as on Windows. Shell seemed like a good choice when we started to write those scripts: it is available on all hosts (using Cygwin …

  44. Battery life

    By Floris Bruynooghe

    Normally I'm quite happy with the 3h of battery life of my laptop, it covers all my disconnected time on trains etc. But it just doesn't cut it on a 10h flight, especially painful if my brain has loads of ideas to try out and things to do.

  45. New Delny release

    By Floris Bruynooghe

    A few days ago I got another of those two-a-year inquiries about Delny, the python wrapper around Qhull for Delaunay triangulations. But increadably indirectly it finally got me to do a new release, it's only been a few years since the last one! Nothing has changed really, the only difference …

  46. Resuming an scp file transfer

    By Floris Bruynooghe

    Sometimes you have to transfer data over dodgy links. And when you're transferring a large file this can hurt when it fails. So as explained elsewhere rsync can safe the day:

    $ scp host:remote_file local_file
    # now interrupt and resume with
    $ rsync --partial --progress --rsh=ssh host:remote_file local_file
    

    You need …

  47. Should bare except statements be allowed in the Python stdlib?

    By Floris Bruynooghe

    Firstly to clarify the terminology, this is bare except statement:

    try:
        ...
    except:
        ...
    

    And this is a non-bare except statement, but bear in mind the type of the exception that is caught can be anything:

    try:
        ...
    except Exception:
        ...
    

    The point is that both fragments are a catch-all exception handler, only the …

  48. Pain

    By Floris Bruynooghe

    I've spent the last 2 days trying to get a stack trace from a crashing python extension module in windows. And I still haven't figured it out. That's sooo very motivating.

    Give me GNU/Linux any day.

  49. cifs why don't you follow unix conventions?

    By Floris Bruynooghe

    Often it's nice to just have conventions and gentleman's agreements. But then sometimes someone doesn't and the harm is done, it's too hard to reverse.

    NFS has introduced the de-facto standard of using <host>:<path> for the device of a remote mount. Yes this break because you can have ":" in …

  50. Letters on keyboard stop working

    By Floris Bruynooghe

    Dear Lazy Web

    I have some Very weird behaviour where the lowercase letters "C" and "V" stop generating the Correct key press event after a short while (which is why I am typing them as upper Case here...). But only when using Compiz as window manager, when switching away from …

  51. How to bring a running python program under debugger control

    By Floris Bruynooghe

    Of course pdb has already got functions to start a debugger in the middle of your program, most notably pdb.set_trace(). This however requires you to know where you want to start debugging, it also means you can't leave it in for production code.

    But I've always been envious of …

  52. devenv.com via cygwin ssh (visual studio 2003)

    By Floris Bruynooghe

    Integrating an automatic build on a windows host with the rest of your one-command cross-platform build system can be quite a pain. Luckily there is cygwin which makes things like ssh, bash, make etc all work on windows. It's great.

    The trick to building using visual studio from there is …

  53. Importing modules in C extension modules

    By Floris Bruynooghe

    It seems that if you need another module in a function of your extension module, the way modules in the standard library seem to solve this is like this:

    static PyObject *
    func(void)
    {
        PyObject *foo;
    
        foo = PyImport_ImportModuleNoBlock("foo");
        if (foo == NULL)
            return NULL;
        /* do stuff with foo */
        Py_DECREF(foo);
        return …
  54. Singletons in a Python extension module

    By Floris Bruynooghe

    If you want a singleton in a C extension module for CPython you basically have to do the same as when doing this in plain Python: the .__new__() method needs to return the same object each time it is called. Implementing this has a few catches though.

    static PyObject *
    MyType_new …
  55. Voting and identification in the UK

    By Floris Bruynooghe

    Last Thursday I had the pleasure of being able to vote for the local council as well as for the European Parliament. Since I'm a Belgian citizen living in the United Kingdom this involved convincing the officials at the voting office to re-read their instructions (at first they only allowed …

  56. Raising exceptions in threads

    By Floris Bruynooghe

    It's not simply raising an exception in a thread that I want. I want to raise an exception in a thread from another thread. It's like sending signals to threads, only signals in pyhon can only be delivered to the main thread (for portability). But Python has a other asynchronous …

  57. New home for PSI

    By Floris Bruynooghe

    PSI, the Python System Information package that pulls interesting information from the kernel and makes it available in a nice Python API, has a new home at bitbucket. This means that the source conde now lives inside a mercurial repository instead of subversion.

    This actually happened about a week ago …

  58. Compiling applications in setup.py

    By Floris Bruynooghe

    If you need to compile a random application in setup.py this is not that hard:

    cc = distutils.ccompiler.new_compiler()
    distutils.sysconfig.customize_compiler(cc)
    cc.link_executable(['test.c'], 'a.out')
    

    There is no need to create the object files explicitly with cc.compile() first if you have no need for …

  59. Sun Fire T1000 not closet friendly

    By Floris Bruynooghe

    Obviously the Sun Fire T1000 is a noisy machine, that's why it's sitting in the closet in the first place. But it has more shortcomings that make it a closet-unfriendly server:

    • It shuts down at 50 degrees Celsius. That means wrapping it in a duvet to damp the noise is …
  60. Time, floating points and python modules handling time

    By Floris Bruynooghe

    Some memory in the back of my mind tells I've once read a rant about storing time that argued you should never store time in floating point variables. My memory seems to think the rant convinced me and it does indeed seem better to store time in integers so that …

  61. Reading a 64-bit core from a 32-bit proc on solaris?

    By Floris Bruynooghe

    I'm trying to read some pointers from a 64-bit address space file (from /proc/pid/as) while running in a 32-bit process - all this on Solaris. To do this I'm using the transitional large file compilation environment, i.e. where you get both open()/pread() and open64()/pread64() since I …

  62. Replacing the screen of your digital camera

    By Floris Bruynooghe

    The screen of my Canon Digital Ixus 40 compact camera broke on my last trip (sport climbing in Geyikbayiri near Antalya, Turkey - if you climb: it's an amzing place!) which made me rather sad. But with the help of Ebay I found a new screen and have just managed to …

  63. How to send emails

    By Floris Bruynooghe

    Dear World

    Subject lines in emails are there for a reason.

    Please use them.

    Love

    Floris

  64. Virtual memory size (VSZ) on AIX

    By Floris Bruynooghe

    If you have ever looked at the ps(1) output on an AIX box and wondered why the virtual memory size (VSZ) is so small, often even smaller then the resident set size (RSS), then here's the answer: it's not the virtual memory size. It is actually the size of …

  65. Closed source frustration

    Last updated:
    By Floris Bruynooghe

    Developing against a closed source libraries is jut painful and wastes your time. Function definitions are missing from the header files, so you get to declare them yourself from the manpages. Then you get random error messages even though you do everything just like the manpage tells you. Some dark …

  66. ssh magic

    By Floris Bruynooghe

    Dear Lazy Web

    If I write the following in my ~/.ssh/config:

    Host bartunnel
      HostKeyAlias bar
      HostName bar
      LocalForward 8822 foo:22
    
    Host barjump
      HostKeyAlias bar
      HostName localhost
      Port 8822
    

    Then I can connect to host bar via host foo (circumnavigating a firewall that stops me from going to bar …

  67. Planning upgrades

    By Floris Bruynooghe

    So Debian has finally released again. Congratulations. Time to start planning upgrades of servers then.

  68. Compiling 32-bit Python on amd64

    By Floris Bruynooghe

    If you ever feel the need to compile a 32-bit version of Python on a amd64 bit machine, this is how you do it on a Debian/Ubuntu system.

    Firstly you need the correct compiler stuff, this means you need gcc-multilib and 32-bit development libraries of at least libc. On …

  69. sed for binary files: bbe

    By Floris Bruynooghe

    While GNU sed takes care not to munge NULL characters this is not the POSIX standard, therefore it's no surprise some implementations don't manage to edit binary files properly (not to mention that many implementations will struggle with long lines). Hence the search for a binary sed:

    bbe is just …

  70. Specifying the RUNPATH in sofiles

    By Floris Bruynooghe

    When you create a shared object that depends on another shared object there are multiple ways to tell it where to look for the shared object. Two of these are by encoding (at linking time) a DT_RPATH and DT_RUNPATH entry in the .dynamic section. A third one is to use …

  71. Resistance to change

    By Floris Bruynooghe

    Why can C developers be happy with using config.h to get constants about where to look for configuration or data files for example, yet Python developers seem to refuse to think of any way they might want to support finding these files in a portable way?

  72. FreeBSD on Virtualbox

    By Floris Bruynooghe

    Virtualbox is my desktop virtualisation technology of choice most of the time and I wanted to have a play with FreeBSD. Seems they don't get along tough and you won't get a network connection.

    • Solution 1: Change the network adaptor in Virtualbox to PCnet-PCI II (instead of PCnet-PCI III). I've …
  73. datetime.datetime.totimestamp()

    By Floris Bruynooghe

    There exists a datetime.datetime.fromtimestamp() and many a time I've wondered why there is no .totimestamp() equivalent. The answer is that not all datetime objects can be represented by a timestamp. But if you know this restriction and want it anyway, this is the magic:

    time.mktime(datetime_object.timetuple …
  74. Porting a C extension module to py3k

    By Floris Bruynooghe

    This was not that hard! First look at what everyone else has to say about this, nice aggregated on the python wiki. It covers a lot.

    Now for what I discovered on top of this:

    • When you're defining a new type self->ob_type doesn't exist anymore. This is a problem …

  75. Generating source files in setup.py

    By Floris Bruynooghe

    I have a Python extension module written in C and one of the weirder things it needs to do is have a number of module constants with their values which are defined (e.g. #define PREFIX_FOO 0x01 etc.) in an external C header file. All the defined names in that …

  76. Finding memory leaks in Python extension modules

    By Floris Bruynooghe

    Valgrind is amazing, even if you've never used it before it's basic usage is really simple:

    $ valgrind --tool=memcheck --leak-check=full /usr/bin/python ./test.py
    

    This is enough to point you at the places in your extension module where you allocated stuff that didn't get freed later. This is …

  77. gcc, LD_RUN_PATH & Solaris

    By Floris Bruynooghe

    I actually started writing a long post about how linking against shared libraries works and what and how ld.so does, but that seemed to get very long. So here the short version that assumes pre-existing knowledge.

    If you want an RPATH in your DSOs (dynamic shared objects: executables and …

  78. Mocking away the .__init__() method

    By Floris Bruynooghe

    I wanted to test a method on a class that didn't really depend on a lot of other stuff off the class. Or rather what it did depend on I had already turned into Mock objects with appropriate .return_value and asserting with .called etc. Problem was that the .__init__() method …

  79. Preserving modification times in subversion

    By Floris Bruynooghe

    This is an oft requested feature apparently. But who know if and when it will be there. Usually things get done when someone has an itch to scratch, but given that no one has done this yet -but it's been requested for ages- it seems that it must either be …

  80. Generative Tests

    By Floris Bruynooghe

    Both nose and py.test allow you to write generative tests. These are basically generators that yield the actual test functions and their arguments. The test runners will then test all the yielded functions with their arguments.

    However I'm left wondering why this is better then just iterating in the …

  81. Solaris equivalent of dpkg -S

    By Floris Bruynooghe

    In the past I have spent quite some time trying to figure out what package did install a certain file on Solaris. What I wanted was really just the equivalent of dpkg -S /path/to/file. But those searches where not very fruitfull, even talking to more experienced Solaris admins …

  82. Bluetooth on a Toshiba Tecra A9-127

    By Floris Bruynooghe

    I got a Toshiba Tecra A9-127, listening to the model number of PTS52E-06002LEN as written on the back, from work to replace my dying old HP Compaq nx9030. As my laptop OS of choice is Ubuntu, currently in it's Hardy release, it's not completely coincidence that the hardware is almost …

  83. Ripping videos from DVD

    By Floris Bruynooghe

    Physical discs are a nuisance, I really just want to play what I want to watch in the room I want to watch it just streaming it over the wireless. This actually works wonderfully well. Unfortunately just copying the file structure of a DVD works and gives you DVD-quality video …

  84. Time to read standards

    By Floris Bruynooghe

    Sometimes I like quotes out of context...

    Anyway, I don't think it really is an ambiguity in practice — only in the minds of those that have too much time to read standards documents.

    —Greg Ewing (on distutils-sig)

  85. @x.setter syntax in Python 2.5

    By Floris Bruynooghe

    The new property.setter syntax in Python 2.6 and 3.0 looks very clean to me. So I couldn't wait and wanted it in Python 2.5. With the help of comp.lang.python I finally got there (thanks especially to Arnaud Delobelle):

    _property = property
    
    class property(property):
        def …
  86. (O)OXML and ISO voting processes

    By Floris Bruynooghe

    Many have recently complained about ISO's voting processes, mainly how they need to be revised as currently it seems it's possible to buy yourself a standard given enough lobbyist and money.

    But part of this is ISO's trust in ECMA. ISO allows ECMA to submit standards for the fast-track process …

  87. Shell history

    By Floris Bruynooghe

    hehe

    flub@signy:~$ history | awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn |head
    79 nosetests
    74 ls
    46 cd
    36 ssh
    28 man
    22 apt-cache
    19 vi
    19 ack
    17 svn
    17 sudo
    flub@signy:~$
    

    It must be noted that most of my editing …

  88. nose and ipython

    By Floris Bruynooghe

    I wish there was a simple way to integrate nose and ipython. Something like a command line option not to catch exceptions would be sufficient I think, so that you could do:

    In [1]: %run tests/test_module.py --do-not-catch-exceptions
    

    Obviously you'd want a shorter option switch...

    Seems like Test.run …

  89. One s3fs to rule them all?

    By Floris Bruynooghe

    Sometimes I wish we could fast forward 6 or 12 months or so. Hopefully by then there will be just one s3fs that is mature and maintained. Right now it's hard to tell which one will turn out the best.

    Interstingly, of the ones that look like they could have …

  90. GPL and python modules

    By Floris Bruynooghe

    I should probably google this, but can't think of good keywords. So instead this is a "dear lazy web" post.

    Is importing a python module considered linking for the GPL? I.e. are you allowed to import a python module into an script or module that has a non-GPL-comptible license …

  91. Cheating the KAME turtule

    By Floris Bruynooghe

    This is shocking. Even though I have not yet used IPv6 I can see the the KAME turtle dance! I won't give you a direct link but SixXS privde an IPv6->IPv4 service as well as an IPv4->IPv6 service. Feels bad having used the 4->6 bit... </p>

  92. OmniORB 4.1.1 in lenny!

    By Floris Bruynooghe

    Something that happened about a week ago I think. But omniORB 4.1.1 finally make it into Debian testing aka lenny! This means that python-omniorb 3.1, which has been waiting for a while now, also make it into lenny!

    It has been stalled for ages since we had …

  93. Compiling libtool examples

    By Floris Bruynooghe

    This is more a reminder to myself, but here's how to compile the libtool examples:

    $ cd demo
    $ aclocal
    $ libtoolize --automake
    $ automake --add-missing
    $ autoconf
    $ ./configure
    $ make
    

    It's really how to bootstrap any GNU autotools using application I suppose.

  94. My favourite package description

    By Floris Bruynooghe

    This sentence in a package description is always great...

    You probably don't need it if you don't understand what this is all about.

    Brought to you today by nut-snmp but can be found in many other packages.

  95. Another shell trick

    By Floris Bruynooghe

    Let's start with the solution instead of the problem:

    myvar=$(myfunc) || exit $?
    

    Assuming myfunc is a function defined elsewhere in your shell script this will execute that function and assign it's output to myvar. However when myfunc fails, the entire assignment statement fails with the exit status of myfunc, so …

  96. Creating a Debian ARM build host

    By Floris Bruynooghe

    If you want/need to compile a program on ARM but are not lucky enough to have the hardware available QEMU can still help you. Aurélien Jarno has an excellent description of how to do this. It is still valid today, even if you want to install sid instead of …

  97. Docstring folding

    By Floris Bruynooghe

    In my last post I had a little rant about how python docstrings visually separate the function/method/class definition from the body. Mikko Ohtamaa raised a very good point though: the editor should fold it away.

    Since my editor of choice is Emacs, said to be more an operating …

  98. Documenting functions/methods/classes

    By Floris Bruynooghe

    There are several simple guidelines for writing functions or methods (guidelines that I like that is).

    • Keep them clean, don't intersperse them with comments. If they require more then one or two inline comments it's too complicated and you should restructure.
    • Keep them short, it should fit on one screen …
  99. Paradox

    By Floris Bruynooghe

    Creating a Facebook group to complain about the UK ID cards because:

    [...] An ID card would hold ALL personal information about you including biometrics, hospital records etc etc., a perfect target for ID thieves. [...]

    And you're doing this on Facebook?

  100. Mild improvement for movie industry

    By Floris Bruynooghe

    Until now every time I bought a DVD I found a few scary and angry leaflets in the box telling me how insanely bad I might be for buying pirated DVDs. They usually convince you that you are the worst person in the world for no particular reason. So just …

  101. Rock climbing in Costa Daurada

    By Floris Bruynooghe

    Just spent a long week sport climbing in Costa Daurada. We stayed in a house (with about 32 students and ex-students) near the village of Cornudella de Montsant which was an excellent location really close to Siurana and Arboli. Absolutely brilliant climbing and although my brain wanted to stay a …

  102. Updated omniORB in Debian

    By Floris Bruynooghe

    A while ago Thomas Girard started the Debian CORBA Team with the aim of maintaining omniORB and related packages in Debian. After a lot of work (and time!) we managed to get the packages of up to date versions in a good state fixing almost all bugs in Debian's BTS …

  103. Seeing your browser's headers using python

    By Floris Bruynooghe

    There's a very annoying website that won't send their CSS to my normal web browser (epiphany ) which makes it rather ugly. However when I use iceweasel the CSS gets applied. Since both browsers use exactly the same rendering engine, gecko, on my machine as far as I know, I thought …

  104. Schneier: hammer, nail

    By Floris Bruynooghe

    As Bruce Schneider says:

    If you remember, this was sold to the public as essential for fighting terrorism. It's already being misused.
  105. Making Debian packages on an SMP machine

    By Floris Bruynooghe

    When you have an SMP machine (including dual core CPUs - but I'm sure everyone knows that by now), you quickly learn to use the -jN flag to GNU make(1). N is the number of CPUs you have and it lets make(1) run that many jobs in parallel whenever …

  106. How far does encryption get you?

    By Floris Bruynooghe

    Finally, since a short while, all my hard disks are setup to use encrypted disks using LUKS. Works like a charm and you don't really notice it slow down. Unfortunately I am now left wondering what the point is, given that I currently live in the UK. Not that there …

  107. Simple music players

    By Floris Bruynooghe

    A while ago I bragged about my own music player. Basically I couldn't find the player I liked so started creating my own. It was the first time I wrote anything related to media or a GUI -good thing my basic requirement was simplicity- and was interesting to do, but …

  108. There's a first time for everying

    By Floris Bruynooghe
    flub@signy:~/src$ tar -xzf storm-0.10.tar.gz
    
    gzip: stdin: not in gzip format
    tar: Child returned status 1
    tar: Error exit delayed from previous errors
    flub@signy:~/src$ file storm-0.10.tar.gz
    storm-0.10.tar.gz: POSIX tar archive (GNU)
    flub@signy:~/src$ mv storm-0.10.tar …
  109. EU in censor madness

    By Floris Bruynooghe

    So the EU seems to think that the Internet should be filtered. The first step is blocking searches for bomb recipes.

    To any rational person it is obvious that this is wrong on more levels then I care to explain (others have done so already better then I can), but …

  110. Capable Impressive System Crippled OS

    By Floris Bruynooghe

    That's my new acronym for cisco. Seriously, I did expect a learning curve for IOS when I was ordering a cisco 877W, but this is just disappointing.

    No doubt IOS was once very amazing and had a very nice and clear concept and interface to the first few products it …

  111. Source code to my music player

    By Floris Bruynooghe

    Recently I said I created an increadably stupid, dump and boring GTK music player. I also offered to make the source code available in the very unlikely case someone else would find this thing interesting. I should have known to just do that right away, so here it is.

  112. My very own music player

    By Floris Bruynooghe

    Yes, I've contributed to the masses of music players out there. It is pleasantly easy using python and the pygtk and pygst modules.

    But why yet an other music player? Well it is, intentionally, increadably simple.

    • No library. Just a simple list of files, played sequentially.
    • No metadata. If you …
  113. Encrypted hard disk: LVM on LUKS

    By Floris Bruynooghe

    There are of course about a million other guides on how to use encrypted disks out there. However I did run into some trouble when trying this, so here mine. Specificly I address the issue of getting an encrypted root disk, with the root being on a Logical Volume Management …

  114. Writing entropy

    By Floris Bruynooghe

    Q:

    How long does it take to write a 120 GB disk full with random data?

    A:

    15 hours, 2 minutes and 27 seconds.

    Obviously this depends on the machine. For me it was going at just under 8 minutes per GB, others report around 5 minutes per GB. Also …

  115. Debian is 14

    By Floris Bruynooghe

    Happy brithday!

    And thanks for everyone involved all those years.

  116. Gazpacho and libglade

    By Floris Bruynooghe

    If you read my last post about using Gazpacho you should read the comment by Johan Dahlin too. He's one of the authors of Gazpacho and explains the libglade and save format issues^Wthings in Gazpacho nicely.

  117. Making GUI applications with Gazpacho

    By Floris Bruynooghe

    Earlier I have made some simple GUI applications using PyGTK and Glade, which is surprisingly easy. Now I have another small itch to scratch and have another go at some GUI app. Only this time I decided that the coolness of Gazpacho looked slightly nicer to me, so gave that …

  118. Optimising python code by AST fiddling

    By Floris Bruynooghe

    Obviously no real optimisation, just optimisation exactly like python -O [-O] would create. But in "normal" byte compiled files, i.e. in .pyc files instead of .pyo files.

    Why would I want to do that? Well, we really don't feel like shipping code with assert statements or if __debug__: ... bits …

  119. How to choose a web framework?

    By Floris Bruynooghe

    So I have a bit of organically growing information that is nice to represent on internal web pages in a semi-organised fashion. The first iteration just runs a few python scripts from some cron jobs that utilise SimpleTAL to create staticly served pages. SimpleTAL as templating language simply because we …

  120. __slots__: worth the trouble?

    By Floris Bruynooghe

    I find __slots__ annoying. They seem to have a lot of rules about their use and make things like a borgs rather awkward to use (I even prefer to use a singleton instead of a borg in new-style classes because of this).

    And all it does is safe some space …

  121. IPC in the stdlib(?)

    By Floris Bruynooghe

    Many interesting comments on my last musings about IPC. Each and every one of them look very nice and have their application area, go and have a look! These found a weak spot in my heart though:

    • Candygram is Erlang style message passing. Very stunning, but to my liking a …
  122. IPC and the GIL

    By Floris Bruynooghe

    As recently described excelently, threads are ugly beasts, waiting to get you when you let your guard down (and eventually you will). Yes, that means that I should really get my head round the asynchat module and stop using ThreadingMixIn with the SocketServer, but that's not the point of this …

  123. What's in a number?

    By Floris Bruynooghe

    Ok, I coudn't resist this. Someone told me (as much as writing a blog counts as telling) this is my lucky number: 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0. Someone else says it's magic. Lucky and/or magic numbers... hmm, sounds …

  124. More roundup love

    By Floris Bruynooghe

    As I've said before, roundup is an amazing issue tracker. Last time I wrote I had just implemented our internal bug tracker with all the weird requirements of a few developers. By now the incredibly flexible system, thanks to python, has been moulded into our complete project management tracker.

    This …

  125. Campaigning for -delete

    By Floris Bruynooghe

    Whenever find(1) is used to delete some files there are always a few variants that can be seen. They are generally combinations of -exec and -print piped to xargs. While demonstrating the increadible power of UNIX shells and pipes etc they don't appear very practical.

    Most implementations struggle with …

  126. Debian Developers working together

    By Floris Bruynooghe

    My quote of the day (from Josselin Mouette):

    BTW, who said flamewars are a nuisance to the Debian project? I've never seen developers as united as when Mr Schilling is around. With a few more people like him we would probably be always working together.
  127. Backup script in 43 lines

    By Floris Bruynooghe

    This is no more then my personal backup prefference at this moment. It is full of personal preferences and might not be what other people like. It also isn't written with portability in mind. But I think it is rather elegant.

    My use case: on Friday evening I get home …

  128. Timers for profilers

    By Floris Bruynooghe

    After some oh... and ah... experiences I have decided that the only timer that will give you correct results in a portable manner (accross POSIX platforms that is) is resource.getrusage(resource.RUSAGE_SELF) and os.times(). This is rather shocking, certainly after I then went on to discover that on …

  129. Schizophrenic thoughts

    By Floris Bruynooghe

    On one hand I use tab-completion on the command line all the time. I even go as far as creating my own -terribly incomplete- completion functions in bash for commands that I use often

    On the other hand, when programming I don't use tab-completion -maybe mostly because it's not so …

  130. Roundup praise

    By Floris Bruynooghe

    Roundup is just an amazing bug tracking system. It really is way more general then that, their own words for it are not bad: an issue tracking system for knowledge workers.

    Why I love it:

    • It's written in python
    • It has a poweful and flexible concept/abstraction of a database …
  131. Writing applications as modules

    By Floris Bruynooghe

    The Problem

    I recently had to write a few command line applications of the form command [options] args that did some stuff, maybe printed a few things on screen and exited with a certain exit code. Nothing weird here.

    These apps where part of a larger server system however and …

  132. When web pages check web-broswer compatibility

    By Floris Bruynooghe

    I don't trust banks and definately not online banking since their log in procedures are laugable, at least in the UK (no complaints about my Belgian bank, although not all Belgian banks are good either). But somehow I managed to get an account which only allows transactions to my own …

  133. Warnings revisited (aka logging vs warnings)

    By Floris Bruynooghe

    Earlier I posted about how to make warning messages from warnings.warn() look like fairly normal messages so they don't give you all the details about which line of code called them. The result was neat, but I found it a fairly crude way of doing it.

    There is another …

  134. Overriding formatting of warnings.warn()

    By Floris Bruynooghe

    Today I needed to add the printing of warning messages to a command line tool written in Python. So Instead of writing things out using print, wich is not flexible enough and goes against my way to avoid user interaction deep inside functions and classes, I decided to use the …

  135. Ubuntu & reportbug

    By Floris Bruynooghe

    Lately I only appear to be writing about things I'm unhappy with and general moaning. Sorry for that. But here's another one of them:

    An important part of Debian is the BTS, it allows users to report bugs as most people will know is obviously very important. Quite a few …

  136. mawk considered harmful

    By Floris Bruynooghe

    Consider the following little awk script:

    /^--- [-[:alnum:].\/]+$/ { if ($2 !~ /[-[:alnum:].]+\/debian\//) print $2 }
    

    According the the O'Reilly's sed & awk book it is, as far as my knowledge goes, normal valid awk. It's intended to be used like such:

    ~$ zcat debian_package_version.diff.gz | awk -f script.awk
    

    The purpose is to …

  137. site, eggs, etc.

    By Floris Bruynooghe

    Before I complained about why .pth files are not processed in files on in your $PYTHONPATH. I got a few pointers in that regard, thanks for them. And now I know why it doesn't happen: you can import foo, bar; foo.do(); bar.harm() inside a .pth file. Since this …

  138. Nuking your home directory

    By Floris Bruynooghe

    /path/to/some/weird/directory$ rm -rf ~

    Yep, I've done that today. It sucks. 1.5 seconds after I did that I hit the power button in the hope the kernel wouldn't have flushed it's buffers yet (hm, that old trick might not work with journaling filesystems). Boot up in …

  139. When essential tools break...

    By Floris Bruynooghe

    I hate it when that happens. Current case:

    flub@coronation:/tmp/sandbox$ gpg --clearsign test
    gpg: selecting openpgp failed: unknown command
    gpg: signing failed: general error
    gpg: test: clearsign failed: general error
    flub@coronation:/tmp/sandbox$
    

    That's just to get desparate. Some quick poking around results in leading me to …

  140. UTF-8 and Emacs

    By Floris Bruynooghe

    While looking around in Manoj's Debian web-page/directory for his new Python policy (for how to package Python stuff for Debian) I stumbled, amongst other intresting things, across this cool UTF-8 demo file.

    To my surprise my web browser (Epiphany) shows nearly everything perfect, so does gedit. However my much …

  141. site module musings

    By Floris Bruynooghe

    The site package reads .pth files and inserts directories listed there in sys.path. However it only scans lib directories ( lib/python2.4/site-packages/ and lib/python/site-python/ on UNIX) starting with sys.prefix or sys.exec_prefix for these .pth files. Thus if you install a module distribution using the …

  142. Python web frameworks

    By Floris Bruynooghe

    It appears discussing web frameworks is popular currently. It also appears GvR has decided he likes Django and hopes it will become sort of the default.

    I've been looking at this Django and TurboGears things for a while unable to decide if I should move the website that I appear …

  143. setup.py: setup(..., extra_path="foobar", ...)

    By Floris Bruynooghe

    The extra_path keyword that you can pass to setup() when using the distutils for building and installing a package is completely undocumented as far as I know. I have no idea how I found it the first time (about a year ago) but now it was there and was not …

  144. Today's WTF: sys.path in python != sys.path in ipython

    By Floris Bruynooghe

    Spot the current directory ( <snip>/build/lib.linux-i686-2.4) in the path:

    <snip>/build/lib.linux-i686-2.4$ echo $PYTHONPATH
    /home/flub/lib/python:/home/flub/lib/python2.4:
    <snip>/build/lib.linux-i686-2.4$ python
    Python 2.4.3 (#2, Apr 27 2006, 14:43:58)
    [GCC 4.0.3 (Ubuntu …
  145. GnuPG Smartcard on Debian Sarge

    By Floris Bruynooghe

    There's a document on the GnuPG explaining how to set up your system to use a smartcard. There a few people around claiming that it works on a standard Debian Sarge by using udev. Up to today I have had no luck with that however, but that finally changed!

    I'll …

  146. Dalai Lama visits Belgium

    By Floris Bruynooghe

    Like I care.

    But China cares, that's more worrying. Dispite it not being an "official" visit (he was invited by some buddhists to open a new temple) he visited the prime minister and some other politicians. And hence the chinese ambassador says it will hurt the Belgian-China relationships as they …

  147. Summer of Code 2006

    By Floris Bruynooghe

    There must be about a million blog posts with this title by now. Anyway, here my own little addition, I've been meaning to write this for a while.

    I have not applied for SoC again althoug I was still eligible. There where intresting projects however, and in fact I'm sorry …

  148. No feedback?

    By Floris Bruynooghe

    In the beginning of this week I emailed my mentor to know what he thought of my project, if he was happy with the state etc. I do know from his blog that he's fairly busy these days, but still would enjoy some feedback. Also just in case my email …

  149. One FIXME down...

    By Floris Bruynooghe

    Profile.calibrate() now uses the correct timer instead of just the default timer (time.time()). In fact it still uses this timer when the user did not specify a specific timer (and why really would anyone want this? Appart form making me work one or two weeks more in the …

  150. Magic

    By Floris Bruynooghe

    As promised I worked on a proper installble package of hprof. That went quite smooth, I had some trouble figuring out how to create a .pth file with distutils, but after some googling I found some setup keyword that worked. Not that is it documented in the official documentation. The …

  151. Small update

    By Floris Bruynooghe

    This weeked I've been helping at home again building the patio outside. It's quite impressive to see things changing from a big pit of dirt in the morning to a nice surfaced area in the evening. If I had a digital camera I'd show you nice before and after pictures …

  152. (Almost) Done!

    By Floris Bruynooghe

    Just completed hpstats module, it's all in CVS. This means that theoretically the project is finished! Altho I expect to code another few days on it before the end of August, nothing can be perfect... leave alone the first time round!

    Along the list of TODO's is making a little …

  153. Sleep is good

    By Floris Bruynooghe

    Last night I got some problems with a function call. As far as I could see it was right but it's unit test kept failing. After a while I called it a day and went to sleep. Wake up this morning and have a look. Only to discover it was …

  154. Nice progress

    By Floris Bruynooghe

    Work has been progressing nicely the last few days. All the hacking has been going smoothly, which does not mean bug free - far from it! But I seem to have finally reached a good cycle of writing unit tests, implementing the features and refining that all. The scaffolding is in …

  155. Ambiguous documentation

    By Floris Bruynooghe

    pstats does not behave as the documentation says it does. I got to the stage that you can look through the output of Lib/test/test_profile.py (in Python distribution) and the output of that very same file but with a import hprofile as profile at the top instead of …

  156. Clarifying the hstats module

    By Floris Bruynooghe

    Some confusion seems to exist about the stats modules I'm writing. About a month and a bit ago I wrote hstats as a module to analyse hotshot profiling data. It was never my intention to make it compatible with pstats at all. It aimed at being usable to read profiling …

  157. Why one should have a separate /boot partition: lessons learned

    By Floris Bruynooghe

    Background

    The box (SPARC box to make matters more intresting) has two disks /dev/hda and /dev/hdb, they each have the same partion table appart from some free space at the end since two disks are never of the same size. All partitions are of type "RAID autodetec" and …

  158. Cross platform developing

    By Floris Bruynooghe

    Even when using Python this is easier said then done. One should be very carefull as details lies in tiny details. Wich I discovered to my shame.

    Normally I work on a desktop, this box listenst to the name of Ultra 10, so it is a UltraSparc architecture as developed …

  159. Cleaning up?

    By Floris Bruynooghe

    So last time I said I wasn't happy with hstats. Maybe that was a bit impulsive, but still valid. Kent Johnson kindly pointed out <http://bruynooghe.blogspot.com/2005/08/i-wrote-crap.html#112378135725654786> for me that was the wrong attitude. Maybe he's right (read: yeah, of course he's right), but …

  160. I wrote crap

    By Floris Bruynooghe

    hstats is crap. And I only wrote it a month ago...

    The good part of that observation is that it shows that I'm learning. ;-)

    So why really? Basically just it's desing. It has one single monolithic class. And within this class everything just happens like one would wirte precedural or …

  161. Never ending quest

    By Floris Bruynooghe

    On my never ending quest to find the best testing strategies I stumbled across this article. It is not too long and well worth reading. Explains some stuff about using mock objects and related testing strategies. All of this is of course XP influenced.

  162. hotshot accepts user timer function

    By Floris Bruynooghe

    After a long day it all works. It's checked in to CVS in the "hotshot_timer-branch". The way it works is that the object returned by _hotshot.profiler(filename) now has an additional method settimer(timer). The timer is any callable object that returns a number or a sequence of numbers …

  163. Pointer to funcion

    By Floris Bruynooghe

    Great fun these things. Certainly when the pointer to the function is stored inside a struc. The result is that my version of _hotshot now uses a void pointer in ProfilerObject as a function to get the time difference since last time it was called. For now it only goes …

  164. Programming can be simple

    By Floris Bruynooghe

    It really is clear that it has been over 10 months since I last wrote C. As I mentioned in my last post I got segmentation faults when I changed the version string in _hotshot.c. Today I started by figuring out that the version string gets indeed used at …

  165. Double Ouch

    By Floris Bruynooghe

    One

    To get a user supplied timer function in hotshot I need to start hacking on _hotshot.c. Being cautious I pull the file from python CVS, put it in my package tree, write a setup script, build the stuff before modifying everything -just to test build environment- and finally …

  166. Seconds & nanoseconds

    By Floris Bruynooghe

    Last night I struggled a lot with the calibration. In the end I fainted and gave up, went to bed and read a crap book for way too long -books are no good for me, even if they're crap I can't stop reading them. Today, after waking up way to …

  167. The easy bits

    By Floris Bruynooghe

    So I've written the easy methods and functions of the profile module wrapper hprofile today. Including their unittests.

    Tomorrow starts the harder work, the Profile.calibrate code. Maybe I shouldn't be too optimistic about how far I'll get with it. For what it's worth, here my current ideas of how …

  168. To hybernate or not to hybernate...

    By Floris Bruynooghe

    That's the question

    Old days

    For years and years I always envied my father who never had to log out of his window manager at his work, he only had to lock his screen. This meant he could leave all his terminals and text editors open at night. The next …

  169. Refactor Mercilessly

    By Floris Bruynooghe

    Ok, I admit. Maybe I'm just using this as an excuse. I haven't really impressed myself with the amount of work done the last few day. But hey! This was my holiday after all.

    The result however is that I have revised the test module for the hstats module (instead …

  170. Native hoshot statistics!

    By Floris Bruynooghe

    Finally, after some refactoring, having cursed unittests and XP but in the end blessing them anyway. The result is there: the hstats module!

    It loads hotshot data 35% faster then hotshot.stats.load() as I reported earlier. It sorts the data, and prints it out nicely. Moreover the sorting and …

  171. Progress...

    By Floris Bruynooghe

    So I tried this XP approach of first creating unittests the last couple of days. Writing the tests goes soooo slow! I was a bit surprised on how easily I could write the real functions afterwards tho. Maybe it is good. But it requires lots of tinking in advace and …

  172. Stats unittests

    By Floris Bruynooghe

    That's what I've mostly done, been making and polishing unittests for the hstats module (or what's written of it. hstats btw, is the module that will create the statistics for hotshot, natively reading hotshot data. In the process one extra omission (not really a bug as such ;-)) got into the …

  173. Project on Savannah

    By Floris Bruynooghe

    So I got my project accepted on savannah. After some time of not being sure where it would be hosted, on savannah on it's own or on sourcefource in the Python project under the nondist/sandbox thing. Guess I'll stay with savannah now.

    It's under the project name pyprof. Not …

  174. Paperwork rant

    By Floris Bruynooghe

    This is going to be a rant.

    I faxed Google's paperwork really early after we got it (27th of June IIRC). Now we've been told that if we haven't received confirmation yet we should re-submit! The problem is that in the mean time I've moved houses in Southampton, UK where …

  175. First success!

    By Floris Bruynooghe

    Loading a profiling file dumped by hotshot goes *35% faster* than with hotshot's old method!

    Specifically on a Sun Ultra 10 with a TI UltraSparc IIi (Sabre) processor -872BogoMips if that tells anything- I loaded the profile dump of pystones in 139.8 seconds with my new code (hstats.Stats …

  176. Deep or shallow frame stack?

    By Floris Bruynooghe

    So I'm working on parsing the file created by hotshot. The question in the title relates to how the storing of the cumulative time has to happen. The problem is that every frame has to add it's time to the cumulative time of all the anchestors.

    Shallow frame stack

    Here …

  177. Profiling code accounted to user code!?

    By Floris Bruynooghe

    It appears hotshot does only take one time sample in the profiler callback. This means that all the time spent in hotshot's code is accounted to the user code!

    Mental note to self: remember this when I'm trying to increase accuracy later this summer!

  178. Iterative learning

    By Floris Bruynooghe

    Understanding Code

    I's amazing how little you (well, at least me!) understand the first time you read some (complex) code. Only after a few times you start to get the big picture and see how it all fits together. It's mostly a matter of understanding the underlying principles I guess …

  179. Coding started

    By Floris Bruynooghe

    Profile wrapper

    So the plan is to wrap hotshot so it can behave exactly as the profile module. It will be a bit tricky as some things of the old profile module don't make sense anymore in hotshot.

    Hotshot - Memory allocation

    So I have been learning a lot the last …

  180. Software patents rejected!

    By Floris Bruynooghe

    Finally, the european parliament managed to see the light and rejected the software patents directive. There is still hope.

    Also last night I found the Self-certifying Filesystem. I nerver even considered using plain NFS, but this looks very promising. Maybe tonight I'll end up trying it on my LAN. That …

  181. hotshot

    By Floris Bruynooghe

    So I spent today looking at the old profile module and hotshot. Learned a lot - but not enough yet.

    I quite like the idea of Brett to try and stick with hotshot and try to improve that. The main job would then be to write a wrapper to emulate profile …

  182. Project for the next two months...

    By Floris Bruynooghe

    So this is my blog for the SoC (Google Summer of Code), as recommended by the email I got from the Python Software Foundation.

    For those who don't know what I'm doing. It is listed on Python's Summer of Code wiki under "Profile Replacement".

    I haven't done very much yet …