Functional Python

Composing Functions

I’ve preferred a functional style over a class style for my data transfers, because it’s easier to test and easier to reuse ideas. The missing parts in Python that makes this easier is a compose function. Here’s one I’ve used:

def compose2(g, f):
    def h(*a, **kw):
        result = wrap(g, *a, **kw)
        return wrap(f, result)
    h.__name__ = "%s -> %s" % (g.__name__, f.__name__)
    h.__doc__ = f.__doc__
    h.__parent__ = f
    f.__parent__ = g
    return h

def compose(*fs):
    """
    Create a function from a set of functions, each output is the
    input of the succeeding function.

    Each function is submitted to wrap, which logs, caches, and handles errors.

    The functions are called in the order they are listed, for example:

        d = compose(a, b, c)

    Is similar to:

        c(b(a()))

    This can be called with parameters to d:

        d(42)

    Which is similar to:

        c(b(a(42)))

    """
    return reduce(compose2, fs)

Then, I use functools.partial and compose to make functions that don’t depend on system state to run. The wrap function referenced logs function calls, handles errors consistently, and uses caching…which is a longer conversation.

From David Richards on Utah Data Engineering Slack