toofishes.net

Python cached property decorator

Edit: Reddit and Simon in the comments below are a lot smarter than me and came up with something much better than my original clever solution. Read a lot more about it there, and I’ve replaced the code below with what was posted.

The @property decorator in Python is very handy when you make an object that requires some additional work to fetch attributes, but want it to behave like any other object. One drawback is that this work is repeated each time you fetch the property; e.g. if I call object.fake_property multiple times my decorated method will get called multiple times. I had several of these methods on a class, and they were doing lxml xPath lookups each time, so it would be great to memoize or cache the result the first time and return that on all subsequent calls.

There was a post to the python-ideas mailing list around 20 months ago regarding caching properties, but nothing conclusive came out of it. Additionally, the method proposed there had a rather large problem- because the cache was kept on the decorator itself, it never got freed, which would cause significant memory leakage in a running web application. In my case, it would have kept around several XML element objects- not good.

So here you are: a @cached_propery decorator that acts exactly like @property, except calling it additional times on the same Python object will retrieve the cached value instead. Values are cached per-instance, and on the instance itself, so if you reconstruct another “identical” object, it will not be using the same cache. This helps keep the TTL of objects down in a long-running environment.

class cached_property(object):
    '''A read-only @property that is only evaluated once. The value is cached
    on the object itself rather than the function or class; this should prevent
    memory leakage.'''
    def __init__(self, fget, doc=None):
        self.fget = fget
        self.__doc__ = doc or fget.__doc__
        self.__name__ = fget.__name__
        self.__module__ = fget.__module__

    def __get__(self, obj, cls):
        if obj is None:
            return self
        obj.__dict__[self.__name__] = result = self.fget(obj)
        return result

And the somewhat amusing discussion that convinced me this wasn’t my worst idea ever:

Dan: i coded up a horrendous decorator today, and by horrendous i think it might actually be clever.
Dusty: it’s not.
Dan: take a look, you decide.
Dusty: oh. I’ve written similar decorators, but it never crossed my mind to extend property. That’s pretty sweet, actually.
Dan: i didn’t know if it could be done at first. i had seen the chained “@property\n@cached” deal, but that seems uglyish, so i thought i’d give this a shot.
Dan: property lookups are really mapping to xpath xml lookups where i used this, so this is nice if you know you are going to hit them more than once.
Dan: see? not horrendous :P
Dusty: no, I think it’s nifty.

Tags

See Also