Knowing almost nothing about frontend development or GIS I wanted to be able to display something on a map on my own computer (desktop). Knowing a fair bit of Rust but not having a clue about anything else involved I decided the easiest way to do this was probably to …
All Articles
-
-
Using Modern Linux Sockets
Introduction
On the surface of it the Linux socket API is pretty straight forward. However once you start to care about performance things get a bit more complicated, and documentation a bit more sparse.
Because I thought it'd be fun to explore this a bit more I made a simple …
-
Experiments with an Email Protocol
Background
The primary way I use email is through notmuch and notmuch-emacs on my laptop, which reads email from a local maildir. Sending mail is done using the venerable sendmail(8) command, which is backed by nullmailer to send it out via SMTP to the relay server of my email …
-
Encrypted root on Debian with keyfile/keyscript
I've recently set up another laptop with whith whole-disk encryption, including /boot. This is all fine --the debian-installer almost gets it right, you just need to edit /etc/default/grub to add GRUB_ENABLE_CRYPTODISK=y and then re-run grub-install on the target-- but once you get this working you end up …
-
A Container is an Erlang Process
This post is a response to A Container Is A Function Call by Glyph. It is a good article and worth your time reading, and you might want to read it to follow here. On twitter I asserted the article recommends building a monolith while Glyph countered "On the contrary …
-
py.test sprint in Freiburg
Testing is a really important part of Python development and picking the testing tool of choice is no light decision. Quite a few years ago I eventually decided py.test would be the best tool for this, a choice I have never regretted but has rather been re-enforced ever since …
-
Pylint and dynamically populated packages
Python links the module namespace directly to the layout of the source locations on the filesystem. And this is mostly fine, certainly for applications. For libraries sometimes one might want to control the toplevel namespace or API more tightly. This also is mostly fine as one can just use private …
-
New pytest-timeout release
At long last I have updated my pytest-timeout plugin. pytest-timeout is a plugin to py.test which will interrupt tests which are taking longer then a set time and dump the stack traces of all threads. This was initially developed in order to debug some some tests which would occasionally …
-
Designing binary/text APIs in a polygot py2/py3 world
The general advice for handling text in an application is to use a so called unicode sandwich: that is decode bytes to unicode (text) as soon as receiving it, have everything internally handle unicode and then right at the boundary encode it back to bytes. Typically the boundaries where the …
-
Don't be scared of copyright
It appears there is some arguments against putting copyright statements on the top of a file in free software or open source projects. Over at opensource.com Rich Bowen argues that it is counter productive and not in the community spirit (in this case talking about OpenStack). It seems to …
-
Setting up a Python development environment on Solaris 11
Solaris is one of those things which tends to be a bit un-loved, it being baught by Oracle probably didn't help either. But regardless they've made a rather nice and modern OS out of it and it is used in a fair few places so you may need to support …
-
Inspecting un-imported modules using ast
or
Skipping modules in py.test using marks but no importing
As I've said before, py.test is my favourite testing tool. One if it's features is that it allows you to mark tests with arbitrary "marks", built in ones are e.g. skipif, xfail etc. But py.test's extension …
-
Small Emacs tweaks impoving my Python coding
Today I've spent a few minutes tweaking Emacs a little. The result is very simple yet makes a decent impact on usage.
Firstly I remembered using the c-subword-mode a long time ago, I couldn't believe I never used that in Python. Turns out there is a more genericly named subword-mode …
-
Using __getattr__ and property
Today I wasted a lot of time trying to figure out why a class using both a __getattr__ and a property mysteriously failed. The short version is: Make sure you don't raise an AttributeError in the property.fget()
The start point of this horrible voyage was a class which looked …
-
Synchronising eventlets and threads
-
Creating subprocesses in new contracts on Solaris 10
Solaris 10 introduced "contracts" for processes. You can read all about it in the contract(4) manpage but simply put it's a grouping of processes under another ID, and you can "monitor" these groups, e.g. be notified when a process in a group coredumps etc. This is actually one …
-
re.search() faster then re.match()
This is a counter-intuitive discovery, in IPython:
In [18]: expr = re.compile('foo') In [19]: %timeit expr.search('foobar') 1000000 loops, best of 3: 453 ns per loop In [20]: %timeit expr.match('foobar') 1000000 loops, best of 3: 638 ns per loop
So now I'm left wondering why .match …
-
Storm and SQLite in-memory databases
When using an SQLite in-memory databases in storm the different stores created from the same database are not modifying the same SQLite database. E.g.
db = storm.locals.create_database('sqlite:') store1 = storm.locals.Store(db) store2 = storm.locals.Store(db)
Here store1 and store2 will not refer to the same …
-
Finding the linux thread ID from within python using ctypes
So I've got a multi-threaded application and suddenly I notice there's one thread running away and using all CPU. Not good, probably a loop gone wrong. But where? One way to find this is revert history in the VCS and keep trying it out till you find the bad commit …
-
Return inside with statement
Somehow my brain seems to think there's a reason not to return inside a with statement, so rather then doing this:
def foo(): with ctx_manager: return bar()
I always do:
def foo(): with ctx_manager: result = bar() return result
No idea why nor where I think to have heard/read this …
-
Templating engine in python stdlib?
I am a great proponent of the python standard library, I love having lots of tools in there and hate having to resort to thirdparty libs. This is why I was wondering if there could be a real templating engine in the stdlib some day? I'm not a great user …
-
Using Debian source format 3.0 (quilt) and svn-buildpackage
Searching the svn-buildpackage manpage for the 3.0 (quilt) format I thought that it wasn't able to apply the patches in debian/patches during build time. Instead I was doing a horrible dance which looked something like svn-buildpackage --svn-export; cd ../build-area/...; debuild. Turns out I was completely wrong.
svn-buildpackage doesn't …
-
Europython, threading and virtualenv
I use threads
correctly
I do not use virtualenv
and dont't want to
Just needed to get that out of my system after europython, now mock me.
PS: I should probably have done this as a lightening talk but that occurred to me too late.
-
py.test test generators and cached setup
Recently I've been enjoying py.test's test function arguments, it takes a little getting used too but soon you find that it's quite likely a better way then the xUnit-style of setup/teardown. One slightly more advanced usage was using cached setup together with test generators however. While not …
-
Selectable queue
Sometimes you'd want to use something like select.select() on Queues. If you do some searching for this it turns out that this question has been answered for multiprocessing Queues where you can simply use the real select on the underlying socket (IIRC), but for a good old fashioned Queue …
-
weakref and circular references: should I really care?
While Python has a garbage collector pretty much whenever circular references are touched upon it is advised to use weak references or otherwise break the cycle. But should we really care? I'd like not to, it seem like something the platfrom (python vm) should just provide for me. Are all …
-
python-prctl
There is sometimes a need to set the process name from within python, this would allow you to use something like pkill myapp rather then the process name of your application just being yet another "python". Bugs (which I'm now failing to find in Python's tracker) have been filed about …
-
Storm and sqlite locking
The Storm ORM struggles with sqlite3's <http://docs.python.org/library/sqlite3.html> transaction behaviour as they explain in their source code. Looking at the implementation of .raw_execute() the side effect of their solution to this is that they start an explicit transaction on every statement that gets executed. Including …
-
Python optimisation has surprising side effects
Here's something that surprised me:
a = None def f(): b = a return b def g(): b = a a = 'foo' return b
While f() is perfectly fine, g() raises an UnboundLocalError. This is because Python optimises access to local variables using the LOAD_FAST/STORE_FAST opcode, you can easily see why this …
-
Judging performance of python code
Recently I"ve been messing around with python bytecode, as a result I now know approximately how the python virtual virtual machine works at a bytecode level. I've found this quite interesting but other then satisfying my own curiosity had no benefit from this knowledge. Until today I was wondering …
Page 1 / 7 »