5.3 Special Methods

A class may define or inherit special methods (i.e., methods whose names begin and end with double underscores). Each special method relates to a specific operation. Python implicitly invokes a special method whenever you perform the related operation on an instance object. In most cases, the method's return value is the operation's result, and attempting an operation when its related method is not present raises an exception. Throughout this section, I will point out the cases in which these general rules do not apply. In the following, x is the instance of class C on which you perform the operation, and y is the other operand, if any. The formal argument self of each method also refers to instance object x.

5.3.1 General-Purpose Special Methods

Some special methods relate to general-purpose operations. A class that defines or inherits these methods allows its instances to control such operations. These operations can be divided into the following categories:

Initialization and finalization

An instance can control its initialization (a frequent need) via special method _ _init_ _, and/or its finalization (a rare need) via _ _del_ _.

Representation as string

An instance can control how Python represents it as a string via special methods _ _repr_ _, _ _str_ _, and _ _unicode_ _.

Comparison, hashing, and use in a Boolean context

An instance can control how it compares with other objects (methods _ _lt_ _ and _ _cmp_ _), how dictionaries use it as a key (_ _hash_ _), and whether it evaluates to true or false in Boolean contexts (_ _nonzero_ _).

Attribute reference, binding, and unbinding

An instance can control access to its attributes (reference, binding, unbinding) by defining special methods _ _getattribute_ _, _ _getattr_ _, _ _setattr_ _, and _ _delattr_ _.

Callable instances

An instance is callable, just like a function object, if it has the special method _ _call_ _.

The rest of this section documents the general-purpose special methods.

_ _call_ _(self[,args...])

When you call x([args...]), Python translates the operation into a call to x._ _call_ _([args...]). The formal arguments for the call operation are the same as for the _ _call_ _ method, minus the first argument. The first argument, conventionally called self, refers to x, and Python supplies it implicitly and automatically, just as in any other call to a bound method.

_ _cmp_ _(self,other)

Any comparison, when its specific special method (_ _lt_ _, _ _gt_ _, etc.) is absent or returns NotImplemented, calls x._ _cmp_ _(y) instead, as do built-in function cmp(x,y) and the sort method of list objects. _ _cmp_ _ should return -1 if x is less than y, 0 if x is equal to y, or 1 if x is greater than y. When _ _cmp_ _ is also absent, order comparisons (<, <=, >, >=) raise exceptions. Equality comparisons (= =, !=), in this case, become identity checks: x= =y evaluates id(x)= =id(y) (i.e., x is y).

_ _del_ _(self)

Just before x disappears because of garbage collection, Python calls x._ _del_ _( ) to let x finalize itself. If _ _del_ _ is absent, Python performs no special finalization upon garbage-collecting x (this is the usual case, as very few classes need to define _ _del_ _). Python ignores the return value of _ _del_ _. Python performs no implicit call to _ _del_ _ methods of class C's superclasses. C._ _del_ _ must explicitly perform any needed finalization.

For example, when class C has a base class B to finalize, the code in C._ _del_ _ must call B._ _del_ _(self) (or better, for new-style classes, super(C, self)._ _del_ _( )). _ _del_ _ is generally not the best approach when you need timely and guaranteed finalization. For such needs, use the try/finally statement covered in Chapter 6.

_ _delattr_ _(self,name)

At every request to unbind attribute x.y (typically, a del statement del x.y), Python calls x._ _delattr_ _('y'). All the considerations discussed for _ _setattr_ _ also apply to _ _delattr_ _. Python ignores the return value of _ _delattr_ _. If _ _delattr_ _ is absent, Python usually translates del x.y into del x._ _dict_ _['y'].

_ _eq_ _(self,other)
_ _ge_ _(self,other)
_ _gt_ _(self,other)
_ _le_ _(self,other)
_ _lt_ _(self,other)
_ _ne_ _(self,other)

Comparisons x= =y, x>=y, x>y, x<=y, x<y, and x!=y, respectively, call the special methods listed here, which should return False or True (in Python 2.2.1 and later; 0 or 1 in Python 2.2, 2.1, and earlier). Each method may return NotImplemented to tell Python to handle the comparison in alternative ways (e.g., Python may then try y>x in lieu of x<y).

_ _getattr_ _(self,name)

When attribute x.y is accessed but not found by the usual procedure (i.e., where AttributeError would normally be raised), Python calls x._ _getattr_ _('y') instead. Python does not call _ _getattr_ _ for attributes found by normal means (i.e., as keys in x._ _dict_ _ or via x._ _class_ _). If you want Python to call _ _getattr_ _ on every attribute reference, keep the attributes elsewhere (e.g., in another dictionary referenced by an attribute with a private name, as shown earlier in this chapter), or else write a new-style class and override _ _getattribute_ _ instead. _ _getattr_ _ should raise AttributeError if it cannot find y.

_ _getattribute_ _(self,name)

At every request to access attribute x.y, if x is an instance of new-style class C, Python calls x._ _getattribute_ _('y'), which must obtain and return the attribute value or else raise AttributeError. The normal semantics of attribute access (using x._ _dict_ _, C._ _slots_ _, C's class attributes, x._ _getattr_ _) are all due to object._ _getattribute_ _.

If class C overrides _ _getattribute_ _, it must implement all of the attribute access semantics it wants to offer. Most often, the most convenient way to implement attribute access semantics is by delegating (e.g., calling object._ _getattribute_ _(self, ...) as part of the operation of your override of _ _getattribute_ _). Note that a class that overrides _ _getattribute_ _ makes attribute access on instances of the class quite slow, since your overriding code is called on every such attribute access.

_ _hash_ _(self)

The hash(x) built-in function call, and using x as a dictionary key (typically, D[x] where D is a dictionary), call x._ _hash_ _( ). _ _hash_ _ must return a 32-bit int such that x= =y implies hash(x)= =hash(y), and must always return the same value for a given object.

When _ _hash_ _ is absent, hash(x) and using x as a dictionary key call id(x) instead, as long as _ _cmp_ _ and _ _eq_ _ are also absent.

Any x such that hash(x) returns a result, rather than raising an exception, is known as a hashable object. When _ _hash_ _ is absent, but _ _cmp_ _ or _ _eq_ _ is present, hash(x) and using x as a dictionary key raise an exception. In this case, x is not hashable and cannot be a dictionary key.

You normally define _ _hash_ _ only for immutable objects that also define _ _cmp_ _ and/or _ _eq_ _. Note that, if there exists any y such that x= =y, even if y is of a different type, and both x and y are hashable, you must ensure that hash(x)= =hash(y).

_ _init_ _(self[,args...])

When a call C([args...]) creates instance x of class C Python calls x._ _init_ _([args...]) to let x initialize itself. If _ _init_ _ is absent, you must call class C without arguments, C( ), and x has no instance-specific attributes upon creation (note that _ _init_ _ is never absent for a new-style class, since such a class inherits _ _init_ _ from object unless it redefines it). _ _init_ _ must return None. Python performs no implicit call to _ _init_ _ methods of class C's superclasses. C._ _init_ _ must explicitly perform any needed initialization. For example, when class C has a base class B to initialize without arguments, the code in C._ _init_ _ must explicitly call B._ _init_ _(self) (or better, for new-style classes, call super(C, self)._ _init_ _( )).

_ _new_ _(cls[,args...])

When you call C([args...]) and C is a new-style class, Python will obtain the new instance x that you are creating by invoking C._ _new_ _(C,[args...]). _ _new_ _ is a static method that every new-style class has (often simply inheriting it from object) and it can return any value x. In other words, _ _new_ _ is not constrained to returning a new instance of C, although normally it is expected to do so. If, and only if, the value x that _ _new_ _ returns is indeed an instance of C (whether a new or previously existing one), Python continues after calling _ _new_ _ by implicitly calling _ _init_ _ on x.

_ _nonzero_ _(self)

When evaluating x as true or false (see Section 4.2.6), for example on a call to bool(x) in Python 2.2.1 and later, Python calls x._ _nonzero_ _( ), which should return True or False. When _ _nonzero_ _ is not present, Python calls _ _len_ _ instead, and takes x as false when x._ _len_ _( ) returns 0. When neither _ _nonzero_ _ nor _ _len_ _ is present, Python always takes x as true.

_ _repr_ _(self)

The repr(x) built-in function call, the `x` expression form, and the interactive interpreter (when x is the result of an expression statement) call x._ _repr_ _( ) to obtain an official, complete string representation of x. If _ _repr_ _ is absent, Python uses a default string representation. _ _repr_ _ should return a string with unambiguous information on x. Ideally, when feasible, the string should be an expression such that eval(repr(x))= =x.

_ _setattr_ _(self, name, value)

At every request to bind attribute x.y (typically, an assignment statement x.y=value), Python calls x._ _setattr_ _('y',value). Python always calls _ _setattr_ _ for any attribute binding on x; a major difference from _ _getattr_ _ (_ _setattr_ _ is closer to new-style classes' _ _getattribute_ _ in this sense). To avoid recursion, when x._ _setattr_ _ binds x's attributes, it must modify x._ _dict_ _ directly (e.g., by x._ _dict_ _[name]=value), or better, for a new-style class, delegate (e.g., call super(C, x)._ _setattr_ _('y',value)). Python ignores the return value of _ _setattr_ _. If _ _setattr_ _ is absent, Python usually translates x.y=z into x._ _dict_ _['y']=z.

_ _str_ _(self)

The str(x) built-in type and the print x statement call x._ _str_ _( ) to obtain an informal, concise string representation of x. If _ _str_ _ is absent, Python calls x._ _repr_ _ instead. _ _str_ _ should return a conveniently human-readable string, even if it entails some approximation.

_ _unicode_ _(self)

The unicode(x) built-in type call, in Python 2.2 and later, invokes x._ _unicode_ _( ), if present, in preference to x._ _str_ _( ). If a class supplies both special methods _ _unicode_ _ and _ _str_ _, the two should return equivalent strings (of Unicode and plain string type respectively).

5.3.2 Special Methods for Containers

An instance can be a container (either a sequence or a mapping, but not both, as they are mutually exclusive concepts). For maximum usefulness, containers should provide not just special methods _ _getitem_ _, _ _setitem_ _, _ _delitem_ _, _ _len_ _, _ _contains_ _, and _ _iter_ _, but also a few non-special methods, as discussed in the following sections.

5.3.2.1 Sequences

In each item access special method, a sequence that has L items should accept any integer key, such that 0<=key<L. For compatibility with built-in sequences, a negative index key, 0>key>=-L, should be equivalent to key+L. When key has an invalid type, the method should raise TypeError. When key is a value of a valid type, but out of range, the method should raise IndexError. In Python 2.1, and also in later Python versions for classes that do not define _ _iter_ _, the for statement relies on these requirements, as do built-in functions that take sequences as arguments.

A sequence should also allow concatenation by + and repetition by *. A sequence should therefore have special methods _ _add_ _, _ _mul_ _, _ _radd_ _, and _ _rmul_ _, covered in Section 5.3.3 later in this chapter. Mutable sequences should also have _ _iadd_ _ and _ _imul_ _, and the non-special methods covered in Section 4.6.4.3: append, count, index, insert, extend, pop, remove, reverse, and sort.

5.3.2.2 Mappings

A mapping's item access special methods should raise KeyError, rather than IndexError, when they receive an invalid key argument value of a valid type. A mapping should define the non-special methods covered in Section 4.7.3: copy, get, has_key, items, keys, values, iteritems, iterkeys, and itervalues. Special method _ _iter_ _ should be equivalent to iterkeys. A mutable mapping should also define methods clear, popitem, setdefault, and update.

5.3.2.3 Sets

Sets, scheduled to be introduced in Python 2.3, can be seen as rather peculiar kinds of containers—containers that are neither sequences nor mappings, and cannot be indexed, but do have a length (number of elements) and are iterable. Unfortunately, the interface of sets (and even the final decision about introducing them in Python 2.3) is still not stable as of this writing. Therefore, I do not consider sets in this book.

5.3.2.4 Container slicing

When you reference, bind, or unbind a slicing such as x[i:j] or x[i:j:k] on a container x, Python calls x's applicable item access special method, passing as key an object of a built-in type called a slice object. A slice object has attributes start, stop, and step. Each attribute is None if the corresponding value is omitted in the slice syntax. For example, del x[:3] calls x._ _delitem_ _(y), and y is a slice object such that y.stop is 3, y.start is None, and y.step is None. It is up to container object x to appropriately interpret the slice object argument passed to x's special methods.

Some built-in types, such as list and tuple, define now-deprecated special methods _ _getslice_ _, _ _setslice_ _, and _ _delslice_ _. For an instance x of such a type, slicing x with only one colon, as in x[i:j], calls a slice-specific special method. Slicing x with two colons, as in x[i:j:k], calls an item access special method with a slice object argument. For example:

class C:
    def _ _getslice_ _(self, i, j): print 'getslice', i, j
    def _ _getitem_ _(self, index): print 'getitem', index
x = C(  )
x[12:34]
x[56:78:9]

The first slicing calls x._ _getslice_ _(12,34), and the second calls x._ _getitem_ _(slice(56,78,9)). It's best to avoid defining the slice-specific special methods in your classes, but you may need to override them if your class subclasses list or tuple and you want to provide special functionality when an instance of your class is sliced. Note that built-in sequences do not yet support slicing with two colons up to Python 2.2: this functionality is scheduled to be introduced in Python 2.3.

5.3.2.5 Container methods

Special methods _ _getitem_ _, _ _setitem_ _, _ _delitem_ _, _ _iter_ _, _ _len_ _, and _ _contains_ _ expose container functionality.

_ _contains_ _(self,item)

The Boolean test y in x calls x._ _contains_ _(y). When x is a sequence, _ _contains_ _ should return True when y equals the value of an item in the sequence. When x is a mapping, _ _contains_ _ should return True when y equals the value of a key in the mapping. Otherwise, _ _contains_ _ should return False. If _ _contains_ _ is absent, Python performs y in x as follows, taking time proportional to len(x):

for z in x:
    if y=  =z: return True
return False

_ _delitem_ _(self,key)

For a request to unbind an item or slice of x (typically del x[key]), Python will call x._ _delitem_ _(key). A container x should have _ _delitem_ _ only if x is mutable, so that items (and possibly slices) can be removed.

_ _getitem_ _(self,key)

When x[key] is accessed (i.e., when container x is indexed or sliced), Python calls x._ _getitem_ _(key). All containers should have _ _getitem_ _.

_ _iter_ _(self)

For a request to loop on all items of x (typically for item in x), Python calls x._ _iter_ _( ) to obtain an iterator on x. The built-in function iter(x) also calls x._ _iter_ _( ). If _ _iter_ _ is absent and x is a sequence, iter(x) synthesizes and returns an iterator object that wraps x and returns x[0], x[1], and so on, until one of these item accesses raises IndexError to indicate the end of the sequence.

_ _len_ _(self)

The len(x) built-in function call, and other built-in functions that need to know how many items are in container x, call x._ _len_ _( ). _ _len_ _ should return an int, the number of items in x. Python also calls x._ _len_ _( ) to evaluate x in a Boolean context, if _ _nonzero_ _ is absent. Absent _ _nonzero_ _, a container is taken as false if and only if the container is empty (i.e., the container's length is 0).

_ _setitem_ _(self,key,value)

For a request to bind an item or slice of x (typically an assignment x[key]=value), Python calls x._ _setitem_ _(key,value). A container x should have _ _setitem_ _ only if x is mutable, so that items, and possibly slices, can be added and/or rebound.

5.3.3 Special Methods for Numeric Objects

An instance may support numeric operations by means of many special methods. Some classes that are not numbers also support some of the following special methods, in order to overload operators such as + and *. For example, sequences should have special methods _ _add_ _, _ _mul_ _, _ _radd_ _, and _ _rmul_ _, as mentioned earlier in this chapter.

_ _abs_ _(self)
_ _invert_ _(self)
_ _neg_ _(self)
_ _pos_ _(self)

Unary operators abs(x), ~x, -x, and +x, respectively, call these methods.

_ _add_ _(self,other)
_ _div_ _(self,other)
_ _floordiv_ _(self,other)
_ _mod_ _(self,other)
_ _mul_ _(self,other)
_ _sub_ _(self,other)
_ _truediv_ _(self,other)

Operators x+y, x/y, x//y, x%y, x*y, x-y, and x/y, respectively, call these methods. The operator / calls _ _truediv_ _, if present, instead of _ _div_ _, in the situations where division is non-truncating, as covered in Section 4.5.2.

_ _and_ _(self,other)
_ _lshift_ _(self,other)
_ _or_ _(self,other)
_ _rshift_ _(self,other)
_ _xor_ _(self,other)

Operators x&y, x<<y, x|y, x>>y, and x^y, respectively, call these methods.

_ _coerce_ _(self,other)

For any numeric operation with two operands x and y, Python invokes x._ _coerce_ _(y). _ _coerce_ _ should return a pair with x and y converted to acceptable types. _ _coerce_ _ returns None when it cannot perform the conversion. In such cases, Python will call y._ _coerce_ _(x). This special method is now deprecated: new Python classes should not implement it, but instead deal with whatever types they can accept directly in the special methods of the relevant numeric operations. However, if a class does supply _ _coerce_ _, Python still calls it for backward compatibility.

_ _complex_ _(self)
_ _float_ _(self)
_ _int_ _(self)
_ _long_ _(self)

Built-in types complex(x), float(x), int(x), and long(x), respectively, call these methods.

_ _divmod_ _(self,other)

Built-in function divmod(x,y) calls x._ _divmod_ _(y). _ _divmod_ _ should return a pair (quotient,remainder) equal to (x//y,x%y).

_ _hex_ _(self)
_ _oct_ _(self)

Built-in function hex(x) calls x._ _hex_ _( ). Built-in function oct(x) calls x._ _oct_ _( ). Each of these special methods should return a string representing the value of x, in base 16 and 8 respectively.

_ _iadd_ _(self,other)
_ _idiv_ _(self,other)
_ _ifloordiv_ _(self,other)
_ _imod_ _(self,other)
_ _imul_ _(self,other)
_ _isub_ _(self,other)
_ _itruediv_ _(self,other)

The augmented assignments x+=y, x/=y, x//=y, x%=y, x*=y, x-=y, and x/=y, respectively, call these methods. Each method should modify x in-place and return self. Define these methods when x is mutable (i.e., when x can change in-place).

_ _iand_ _(self,other)
_ _ilshift_ _(self,other)
_ _ior_ _(self,other)
_ _irshift_ _(self,other)
_ _ixor_ _(self,other)

Augmented assignments x&=y, x<<=y, x|=y, x>>=y, and x^=y, respectively, call these methods. Each method should modify x in-place and return self.

_ _ipow_ _(self,other)

Augmented assignment x**=y calls x._ _ipow_ _(y). _ _ipow_ _ should modify x in-place and return self.

_ _pow_ _(self,other[,modulo])

x**y and pow(x,y) both call x._ _pow_ _(y), while pow(x,y,z) calls x._ _pow_ _(y,z). x._ _pow_ _(y,z) should return a value equal to the expression x._ _pow_ _(y)%z.

_ _radd_ _(self,other)
_ _rdiv_ _(self,other)
_ _rmod_ _(self,other)
_ _rmul_ _(self,other)
_ _rsub_ _(self,other)

Operators y+x, y/x, y%x, y*x, and y-x, respectively, call these methods when y doesn't have a needed method _ _add_ _, _ _div_ _, and so on.

_ _rand_ _(self,other)
_ _rlshift_ _(self,other)
_ _ror_ _(self,other)
_ _rrshift_ _(self,other)
_ _rxor_ _(self,other)

Operators y&x, y<<x, y|x, y>>x, and y^x, respectively, call these methods when y doesn't have needed method _ _and_ _, _ _lshift_ _, and so on.

_ _rdivmod_ _(self,other)

Built-in function divmod(y,x) calls x._ _rdivmod_ _(y) when y doesn't have _ _divmod_ _. _ _rdivmod_ _ should return a pair (remainder,quotient).

_ _rpow_ _(self,other)

y**x and pow(y,x) call x._ _rpow_ _(y), when y doesn't have _ _pow_ _. There is no three-argument form in this case.