Quickstart

Containers

The provided containers: tdict, tlist, and tset serve as drop-in replacements for pythons dict, list, and set types that are Transactionable and Packable (more info below). Furthermore, all keys in adict that are valid attribute names, can be treated as attributes.

A few examples:

from humpack import adict, tdict, tlist, tset
from humpack import json_pack, json_unpack
from humpack import AbortTransaction

d = adict({'apple':1, 'orange':10, 'pear': 3})
d.apple += 10
d.update({'non-det banana':tset({2,3,7}), 'orange': None})
del d.pear
assert d.apple == 11 and 2 in d['non-det banana'] and 'pear' not in d
options = tlist(d.keys())
options.sort()
first = options[0]
assert first == 'apple'
d.order = options

json_d = json_pack(d)
assert isinstance(json_d, str)

d.begin() # starts a transaction (tracking all changes)
assert options.in_transaction()

d['non-det banana'].discard(7)
d.cherry = 4.2
assert 'cherry' in d and len(d['non-det banana']) == 2
d['order'].extend(['grape', 'lemon', 'apricot'])
assert 'grape' in options
del d.order[0]
del d['orange']
d.order.sort()
assert options[0] == 'apricot'

d.abort()
assert 'cherry' not in d and 7 in d['non-det banana']
assert 'grape' not in options

with d:
    assert d['non-det banana'].in_transaction()
    d.clear()
    assert len(d) == 0
    d.melon = 100j
    assert 'melon' in d and d['melon'].real == 0
    raise AbortTransaction

assert 'melon' not in d

assert json_pack(d) == json_d
assert sum(d['non-det banana']) == sum(json_unpack(json_d)['non-det banana'])

with d:
    assert 'cherry' not in d
    d.cherry = 5
    # automatically commits transaction on exiting the context if no exception is thrown

assert 'cherry' in d

When starting with data in standard python, it can be converted to using the “t” series counter parts using containerify.

from humpack import containerify
from humpack import AbortTransaction

x = {'one': 1, 1:2, None: ['hello', 123j, {1,3,4,5}]}

d = containerify(x)

assert len(x) == len(d)
assert len(x[None]) == len(d[None])
assert x['one'] == d.one
with d:
    assert d[None][-1].in_transaction()
    del d.one
    d.two = 2
    d[None][-1].add(1000)
    assert d['two'] == 2 and 'one' not in d and sum(d[None][-1]) > 1000
    raise AbortTransaction
assert 1000 not in d[None][-1] and 'one' in d and 'two' not in d

Finally, there are a few useful containers which don’t have explicit types in standard python are also provided including heaps and stacks: theap and tstack.

Packing (serialization)

To serializing an object into a human-readable, json compliant format, this library implements packing and unpacking. When an object is packed, it can still be read (and manipulated, although that not recommended), converted to a valid json string, or encrypted/decrypted (see the Security section below). However for an obejct to be packable it and all of it’s submembers (recursively) must either be primitives (int, float, str, bool, None) or registered as a Packable, which can be done

Packing and unpacking is primarily done using the pack and unpack functions, however, several higher level functions are provided to combine packing and unpacking with other common features in object serialization. For custom classes to be Packable, they must implement three methods: __pack__, __create__, __unpack__ (for more info see the documentation for Packable). When implementing these methods, all members of the objects that should be packed/unpacked, must use pack_member and unpack_member to avoid reference loops.

from humpack import pack, unpack

x = {'one': 1, 1:2, None: ['hello', 123j, {1,3,4,5}]}

p = pack(x) # several standard python types are already packable
assert isinstance(p, dict)
deepcopy_x = unpack(p)
assert repr(x) == repr(deepcopy_x)

from humpack import json_pack, json_unpack # Convert to/from json string

j = json_pack(x)
assert isinstance(j, str)
deepcopy_x = json_unpack(j)
assert repr(x) == repr(deepcopy_x)


from humpack import save_pack, load_pack # Save/load packed object to disk as json file
import os, tempfile

fd, path = tempfile.mkstemp()
try:
    with open(path, 'w') as tmp:
        save_pack(x, tmp)
    with open(path, 'r') as tmp:
        deepcopy_x = load_pack(tmp)
finally:
    os.remove(path)
assert repr(x) == repr(deepcopy_x)

For examples of how to any types can registered to be Packable or objects can be wrapped in Packable wrappers, see the humpack/common.py and humpack/wrappers.py scripts.

Transactions

For examples of how Transactionable objects behave see the “Containers” section above.

To enable transactions for a class, it must be a subclass of Transactionable and implement the four required functions: begin, in_transaction, commit, and abort. Assuming these functions are implemented as specified (see documentation), you can manipulate instances of these classes in a transaction and then roll back all the changes by aborting the transaction.

One important thing to note with subclassing Transactionable: any members of instances of Transactionable subclasses should be checked for if they are also Transactionable, and if so, they the call should be delegated. In the example below, Account has to take into account that its attribute user could be Transactionable.

from humpack import Transactionable

class Account(Transactionable):
    def __init__(self, user, balance=0.):
        super().__init__()
        self._in_transaction = False
        self._shadow_user = None

        self.user = user
        self.balance = balance

    def change(self, delta):

        if self.balance + delta < 0.:
            raise ValueError
        self.balance += delta

    def begin(self):
        # FIRST: begin the transaction in self
        self._shadow_user = self.user.copy(), self.balance # Assuming `user` can be shallow copied with `copy()`
        self._in_transaction = True

        # THEN: begin transactions in any members that are Transactionable
        if isinstance(self.user, Transactionable):
            self.user.begin()

        # To be extra safe, you could also check `self.balance`, but we'll assume it's always a primitive (eg. float)

    def in_transaction(self):
        return self._in_transaction

    def commit(self):
        # FIRST: commit the transaction in self
        self._in_transaction = False
        self._shadow_user = None

        # THEN: commit transactions in any members that are Transactionable
        if isinstance(self.user, Transactionable):
            self.user.commit()

    def abort(self):
        # FIRST: abort the transaction in self
        if self.in_transaction(): # Note that this call only has an effect if self was in a transaction.
            self.user, self.balance = self._shadow_user

        self._in_transaction = False
        self._shadow_user = None

        # THEN: abort transactions in any members that are Transactionable
        if isinstance(self.user, Transactionable):
            self.user.abort()

Optionally, for a more pythonic implementation, you can use try/except statements instead of type checking with isinstance.

Security

There are a few high-level cryptography routines. Nothing special, just meant to make integration in larger projects simple and smooth.