boodebr: boodebr.util.locking

Module boodebr.util.locking

boodebr : boodebr.util : boodebr.util.locking

This module provides multiprocess/multithread safe cooperative(¹) locking semantics.

boodebr.util.locking provides two public functions.

open_locked() Open a file for read+write access with an exclusive lock.
Locker() Lock a name in a caller-defined namespace.

open_locked() uses Locker() internally. However, Locker() is provided separately here since it can be useful on its own.

^{1. Cooperative locking is the only style of locking that is supported across platforms. "Cooperative" means that nothing prevents another process/thread from overwriting the locked object. Cooperative locking works by having all processes/threads use the same API, i.e. agreeing on locking rules for a given object.}

Classes

CannotGetLock(Exception)

Raised by open_locked() when unable to acquire lock.

LockerFailed(Exception)

Raised by Locker.lock() when unable to acquire lock.

Functions

open_locked(filename, timeout=10, lockdir=None, expire=300, method='mkdir')

Open the given filename for READ+WRITE access with an exclusive lock (cooperative locking). File will be created if it does not exist and positioned at start of file if it does.

Inputs

filename: File to open, will be created if it does not exist.
timeout: Wait timeout seconds to succeed before raising CannotGetLock.
lockdir: Directory to place locking information (defaults to path of filename).

expire: An expiration time in seconds. Locks older than this may be automatically removed to avoid deadlocks.

method: The lowlevel method to use for lock creation. Choices are: 'mkdir': Use mkdir() to create lock. 'opencreate': Use open(os.O_CREAT..) 'fake': Fake locking (no locking at all). Mainly for test purposes.

Returns file object.

It is strongly recommended that you call .close() on the returned fileobj rather than letting it go out of scope and assuming it will be deleted, since there is no guarantee about when that will happen.

Locker(objname, lockdir, ns='file', timeout=10, expire=300, method='mkdir')

Create an opaque multiprocess and multithreading safe cooperative lock. Does not lock a file or any concrete object, but rather creates a lock on a name within a caller-defined namespace.

Inputs:

objname

Name of object to be locked.
Although objname can be any string, I recommend not using file pathnames, since multiple pathnames may be aliased to the same file under some operating systems. A recommended usage when using filenames is: lockdir, objname = os.path.split(filename)
It is up to the caller to ensure the objname is invariant across multiple calls.

lockdir

A directory in which to store locking information. Will be created if it does not exist. Note that ns:objname must be unique within lockdir.

ns

The object namespace. This is any freeform string. This allows you to have e.g. (ns='one', objname='foo') and (ns='two', objname='foo') be two distinct objects.

timeout

How long to wait on lock before raising LockerFailed (seconds)

expire

Expiration time on lock (seconds). Previously created locks may be broken if they are older than this.

method

The lowlevel method to use for lock creation. Choices are:

'mkdir': Use mkdir() to create lock.
'opencreate': Use open(os.O_CREAT..) to create lock.
'fake': Fake locking, mainly for test purposes.

Public API:

lock = Locker(..)
lock.lock()
del lock

You should always explicitly delete locks - do not count on Python to autodelete them when they go out of scope.

Usage:

# Closed form
lock = Locker('abc', '.')
# use nested try..finally for Python < 2.5
try:
    try:
        lock.lock()
        .. do locked operations ..
    except LockerFailed:
        .. locking failed, handle error ..
finally:
    del lock

# Open form
lock = Locker('abc', '.')
try:
    lock.lock()
except LockerFailed:
    .. locking failed, handle error ..
    del lock
    return

.. continue with locked operation ...
.. eventually doing ..
del lock