c++boost.gif (8819 bytes)Special Method and Operator Support

Overview

Boost.Python supports all of the standard special method names supported by real Python class instances except __complex__ (more on the reasons below). In addition, it can quickly and easily expose suitable C++ functions and operators as Python operators. The following categories of special method names are supported:

Basic Customization

Python provides a number of special operators for basic customization of a class. Only a brief description is provided below; more complete documentation can be found here.

__init__(self)
Initialize the class instance. For extension classes not subclassed in Python, __init__ is defined by
    my_class.def(boost::python::constructor<...>())
(see section "A Simple Example Using Boost.Python").

__del__(self)
Called when the extension instance is about to be destroyed. For extension classes not subclassed in Python, __del__ is always defined automatically by means of the class' destructor.
__repr__(self)
Create a string representation from which the object can be reconstructed.
__str__(self)
Create a string representation which is suitable for printing.
__lt__(self, other)
__le__(self, other)
__eq__(self, other)
__ne__(self, other)
__gt__(self, other)
__ge__(self, other)
Rich Comparison methods. New in Python 2.1. See Rich Comparisons.
__cmp__(self, other)
Three-way compare function. See Rich Comparisons.
__hash__(self)
Called for the key object for dictionary operations, and by the built-in function hash(). Should return a 32-bit integer usable as a hash value for dictionary operations (only allowed if __cmp__ is also defined)
__nonzero__(self)
called if the object is used as a truth value (e.g. in an if statement)
__call__ (self[, args...])
Called when the instance is ``called'' as a function; if this method is defined, x(arg1, arg2, ...) is a shorthand for x.__call__(arg1, arg2, ...).
If we have a suitable C++ function that supports any of these features, we can export it like any other function, using its Python special name. For example, suppose that class Foo provides a string conversion function:
std::string to_string(Foo const& f)
{
    std::ostringstream s;
    s << f;
    return s.str();
}
This function would be wrapped like this:
boost::python::class_builder<Foo> foo_class(my_module, "Foo");
foo_class.def(&to_string, "__str__");
Note that Boost.Python also supports automatic wrapping of __str__ and __cmp__. This is explained in the next section and the Table of Automatically Wrapped Methods.

Numeric Operators

Numeric operators can be exposed manually, by defing C++ [member] functions that support the standard Python numeric protocols. This is the same basic technique used to expose to_string() as __str__() above, and is covered in detail below. Boost.Python also supports automatic wrapping of numeric operators whenever they have already been defined in C++.

Exposing C++ Operators Automatically

Supose we wanted to expose a C++ class BigNum which supports addition. That is, in C++ we can write:

BigNum a, b, c;
...
c = a + b;

To enable the same functionality in Python, we first wrap the BigNum class as usual:

boost::python::class_builder<BigNum> bignum_class(my_module, "BigNum");
bignum_class.def(boost::python::constructor<>());
...
Then we export the addition operator like this:
bignum_class.def(boost::python::operators<boost::python::op_add>());
Since BigNum also supports subtraction, multiplication, and division, we want to export those also. This can be done in a single command by ``or''ing the operator identifiers together (a complete list of these identifiers and the corresponding operators can be found in the Table of Automatically Wrapped Methods):
bignum_class.def(boost::python::operators<(boost::python::op_sub | boost::python::op_mul | boost::python::op_div)>());
[Note that the or-expression must be enclosed in parentheses.]

This form of operator definition can be used to wrap unary and homogeneous binary operators (a homogeneous operator has left and right operands of the same type). Now suppose that our C++ library also supports addition of BigNums and plain integers:

BigNum a, b;
int i;
...
a = b + i;
a = i + b;
To wrap these heterogeneous operators, we need to specify a different type for one of the operands. This is done using the right_operand and left_operand templates:
bignum_class.def(boost::python::operators<boost::python::op_add>(), boost::python::right_operand<int>());
bignum_class.def(boost::python::operators<boost::python::op_add>(), boost::python::left_operand<int>());
Boost.Python uses overloading to register several variants of the same operation (more on this in the context of coercion). Again, several operators can be exported at once:
bignum_class.def(boost::python::operators<(boost::python::op_sub | boost::python::op_mul | boost::python::op_div)>(),
                 boost::python::right_operand<int>());
bignum_class.def(boost::python::operators<(boost::python::op_sub | boost::python::op_mul | boost::python::op_div)>(), 
                 boost::python::left_operand<int>());
The type of the operand not mentioned is taken from the class being wrapped. In our example, the class object is bignum_class, and thus the other operand's type is ``BigNum const&''. You can override this default by explicitly specifying a type in the operators template:
bignum_class.def(boost::python::operators<boost::python::op_add, BigNum>(), boost::python::right_operand<int>());

Note that automatic wrapping uses the expression ``left + right'' and can be used uniformly regardless of whether the C++ operators are supplied as free functions

BigNum operator+(BigNum, BigNum)
or as member functions
BigNum::operator+(BigNum).

For the Python built-in functions pow() and abs(), there is no corresponding C++ operator. Instead, automatic wrapping attempts to wrap C++ functions of the same name. This only works if those functions are known in namespace python. On some compilers (e.g. MSVC) it might be necessary to add a using declaration prior to wrapping:

namespace boost { namespace python { 
  using my_namespace::pow;
  using my_namespace::abs;
}

Wrapping Numeric Operators Manually

In some cases, automatic wrapping of operators may be impossible or undesirable. Suppose, for example, that the modulo operation for BigNums is defined by a set of functions called mod():

BigNum mod(BigNum const& left, BigNum const& right);
BigNum mod(BigNum const& left, int right);
BigNum mod(int left, BigNum const& right);

For automatic wrapping of the modulo function, operator%() would be needed. Therefore, the mod()-functions must be wrapped manually. That is, we have to export them explicitly with the Python special name "__mod__":

bignum_class.def((BigNum (*)(BigNum const&, BigNum const&))&mod, "__mod__");
bignum_class.def((BigNum (*)(BigNum const&, int))&mod, "__mod__");

The third form of mod() (with int as left operand) cannot be wrapped directly. We must first create a function rmod() with the operands reversed:

BigNum rmod(BigNum const& right, int left)
{
    return mod(left, right);
}
This function must be wrapped under the name "__rmod__" (standing for "reverse mod"):
bignum_class.def(&rmod,  "__rmod__");
Many of the possible operator names can be found in the Table of Automatically Wrapped Methods. Special treatment is necessary to export the ternary pow operator.

Automatic and manual wrapping can be mixed arbitrarily. Note that you cannot overload the same operator for a given extension class on both ``int'' and ``float'', because Python implicitly converts these types into each other. Thus, the overloaded variant found first (be it ``int`` or ``float'') will be used for either of the two types.

Inplace Operators

Boost.Python can also be used to expose inplace numeric operations (i.e., += and so forth). These operators must be wrapped manually, as described in the previous section. For example, suppose the class BigNum has an operator+=:

BigNum& operator+= (BigNum const& right);
This can be exposed by first writing a wrapper function:
BigNum& iadd (BigNum& self, const BigNum& right)
{
  return self += right;
}
and then exposing the wrapper with
bignum_class.def(&iadd, "__iadd__");

Coercion

Plain Python can only execute operators with identical types on the left and right hand side. If it encounters an expression where the types of the left and right operand differ, it tries to coerce these types to a common type before invoking the actual operator. Implementing good coercion functions can be difficult if many type combinations must be supported.

Boost.Python solves this problem the same way that C++ does: with overloading. This technique drastically simplifies the code neccessary to support operators: you just register operators for all desired type combinations, and Boost.Python automatically ensures that the correct function is called in each case; there is no need for user-defined coercion functions. To enable operator overloading, Boost.Python provides a standard coercion which is implicitly registered whenever automatic operator wrapping is used.

If you wrap all operator functions manually, but still want to use operator overloading, you have to register the standard coercion function explicitly:

// this is not necessary if automatic operator wrapping is used
bignum_class.def_standard_coerce();
If you encounter a situation where you absolutely need a customized coercion, you can still define the "__coerce__" operator manually. The signature of a coercion function should look like one of the following (the first is the safest):
boost::python::tuple custom_coerce(boost::python::reference left, boost::python::reference right);
boost::python::tuple custom_coerce(PyObject* left, PyObject* right);
PyObject* custom_coerce(PyObject* left, PyObject* right);
The resulting tuple must contain two elements which represent the values of left and right converted to the same type. Such a function is wrapped as usual:
// this must be called before any use of automatic operator  
// wrapping or a call to some_class.def_standard_coerce()
some_class.def(&custom_coerce, "__coerce__");
Note that the standard coercion (defined by use of automatic operator wrapping on a class_builder or a call to class_builder::def_standard_coerce()) will never be applied if a custom coercion function has been registered. Therefore, in your coercion function you should call
boost::python::standard_coerce(left, right);
for all cases that you don't want to handle yourself.

The Ternary pow() Operator

In addition to the usual binary pow(x, y) operator (meaning xy), Python also provides a ternary variant that implements xy mod z, presumably using a more efficient algorithm than concatenation of power and modulo operators. Automatic operator wrapping can only be used with the binary variant. Ternary pow() must always be wrapped manually. For a homgeneous ternary pow(), this is done as usual:

BigNum power(BigNum const& first, BigNum const& second, BigNum const& modulus);
typedef BigNum (ternary_function1)(const BigNum&, const BigNum&, const BigNum&);
...
bignum_class.def((ternary_function1)&power,  "__pow__");
If you want to support this function with non-uniform argument types, wrapping is a little more involved. Suppose you have to wrap:
BigNum power(BigNum const& first, int second, int modulus);
BigNum power(int first, BigNum const& second, int modulus);
BigNum power(int first, int second, BigNum const& modulus);
The first variant can be wrapped as usual:
typedef BigNum (ternary_function2)(const BigNum&, int, int);
bignum_class.def((ternary_function2)&power,  "__pow__");
In the second variant, however, BigNum appears only as second argument, and in the last one it's the third argument. These functions must be presented to Boost.Python such that that the BigNum argument appears in first position:
BigNum rpower(BigNum const& second, int first, int modulus)
{
    return power(first, second, modulus);
}

BigNum rrpower(BigNum const& modulus, int first, int second)
{
    return power(first, second, modulus);
}

These functions must be wrapped under the names "__rpow__" and "__rrpow__" respectively:

bignum_class.def((ternary_function2)&rpower,  "__rpow__");
bignum_class.def((ternary_function2)&rrpower,  "__rrpow__");
Note that "__rrpow__" is an extension not present in plain Python.

Table of Automatically Wrapped Methods

Boost.Python can automatically wrap the following special methods:

Python Operator Name Python Expression C++ Operator Id C++ Expression Used For Automatic Wrapping
with cpp_left = from_python(left, type<Left>()),
cpp_right = from_python(right, type<Right>()),
and cpp_oper = from_python(oper, type<Oper>())
__add__, __radd__ left + right op_add cpp_left + cpp_right
__sub__, __rsub__ left - right op_sub cpp_left - cpp_right
__mul__, __rmul__ left * right op_mul cpp_left * cpp_right
__div__, __rdiv__ left / right op_div cpp_left / cpp_right
__mod__, __rmod__ left % right op_mod cpp_left % cpp_right
__divmod__, __rdivmod__ (quotient, remainder)
= divmod(left, right)
op_divmod cpp_left / cpp_right
cpp_left % cpp_right
__pow__, __rpow__ pow(left, right)
(binary power)
op_pow pow(cpp_left, cpp_right)
__rrpow__ pow(left, right, modulo)
(ternary power modulo)
no automatic wrapping, special treatment required
__lshift__, __rlshift__ left << right op_lshift cpp_left << cpp_right
__rshift__, __rrshift__ left >> right op_rshift cpp_left >> cpp_right
__and__, __rand__ left & right op_and cpp_left & cpp_right
__xor__, __rxor__ left ^ right op_xor cpp_left ^ cpp_right
__or__, __ror__ left | right op_or cpp_left | cpp_right
__cmp__, __rcmp__ cmp(left, right)

See Rich Comparisons.
op_cmp cpp_left < cpp_right 
cpp_right < cpp_left
__lt__
__le__
__eq__
__ne__
__gt__
__ge__
left < right
left <= right
left == right
left != right
left > right
left >= right
See Rich Comparisons
op_lt
op_le
op_eq
op_ne
op_gt
op_ge
cpp_left < cpp_right 
cpp_left <= cpp_right 
cpp_left == cpp_right 
cpp_left != cpp_right 
cpp_left > cpp_right 
cpp_left >= cpp_right 
__neg__ -oper  (unary negation) op_neg -cpp_oper
__pos__ +oper  (identity) op_pos +cpp_oper
__abs__ abs(oper)  (absolute value) op_abs abs(cpp_oper)
__invert__ ~oper  (bitwise inversion) op_invert ~cpp_oper
__int__ int(oper)  (integer conversion) op_int long(cpp_oper)
__long__ long(oper) 
(infinite precision integer conversion)
op_long PyLong_FromLong(cpp_oper)
__float__ float(oper)  (float conversion) op_float double(cpp_oper)
__str__ str(oper)  (string conversion) op_str std::ostringstream s; s << oper;
__coerce__ coerce(left, right) usually defined automatically, otherwise special treatment required

Sequence and Mapping Operators

Sequence and mapping operators let wrapped objects behave in accordance to Python's iteration and access protocols. These protocols differ considerably from the ones found in C++. For example, Python's typical iteration idiom looks like

for i in S:
while in C++ one writes
for (iterator i = S.begin(), end = S.end(); i != end; ++i)

One could try to wrap C++ iterators in order to carry the C++ idiom into Python. However, this does not work very well because

  1. It leads to non-uniform Python code (wrapped sequences support a usage different from Python built-in sequences) and
  2. Iterators (e.g. std::vector::iterator) are often implemented as plain C++ pointers which are problematic for any automatic wrapping system.

It is a better idea to support the standard Python sequence and mapping protocols for your wrapped containers. These operators have to be wrapped manually because there are no corresponding C++ operators that could be used for automatic wrapping. The Python documentation lists the relevant container operators. In particular, expose __getitem__, __setitem__ and remember to raise the appropriate Python exceptions (PyExc_IndexError for sequences, PyExc_KeyError for mappings) when the requested item is not present.

In the following example, we expose std::map<std::size_t,std::string>:

typedef std::map<std::size_t, std::string> StringMap;

// A helper function for dealing with errors. Throw a Python exception
// if p == m.end().
void throw_key_error_if_end(
        const StringMap& m, 
        StringMap::const_iterator p, 
        std::size_t key)
{
    if (p == m.end())
    {
        PyErr_SetObject(PyExc_KeyError, boost::python::converters::to_python(key));
        boost::python::throw_error_already_set();
    }
}

// Define some simple wrapper functions which match the Python  protocol
// for __getitem__, __setitem__, and __delitem__.  Just as in Python, a
// free function with a ``self'' first parameter makes a fine class method.

const std::string& get_item(const StringMap& self, std::size_t key)
{
    const StringMap::const_iterator p = self.find(key);
    throw_key_error_if_end(self, p, key);
    return p->second;
}

// Sets the item corresponding to key in the map.
void StringMapPythonClass::set_item(StringMap& self, std::size_t key, const std::string& value)
{
    self[key] = value;
}

// Deletes the item corresponding to key from the map.
void StringMapPythonClass::del_item(StringMap& self, std::size_t key)
{
    const StringMap::iterator p = self.find(key);
    throw_key_error_if_end(self, p, key);
    self.erase(p);
}

class_builder<StringMap> string_map(my_module, "StringMap");
string_map.def(boost::python::constructor<>());
string_map.def(&StringMap::size, "__len__");
string_map.def(get_item, "__getitem__");
string_map.def(set_item, "__setitem__");
string_map.def(del_item, "__delitem__");

Then in Python:

>>> m = StringMap()
>>> m[1]
Traceback (innermost last):
  File "<stdin>", line 1, in ?
KeyError: 1
>>> m[1] = 'hello'
>>> m[1]
'hello'
>>> del m[1]
>>> m[1]            # prove that it's gone
Traceback (innermost last):
  File "<stdin>", line 1, in ?
KeyError: 1
>>> del m[2]
Traceback (innermost last):
  File "<stdin>", line 1, in ?
KeyError: 2
>>> len(m)
0
>>> m[0] = 'zero'
>>> m[1] = 'one'
>>> m[2] = 'two'
>>> m[3] = 'three'
>>> len(m)
4

Customized Attribute Access

Just like built-in Python classes, Boost.Python extension classes support special the usual attribute access methods __getattr__, __setattr__, and __delattr__. Because writing these functions can be tedious in the common case where the attributes being accessed are known statically, Boost.Python checks the special names

to provide functional access to the attribute <name>. This facility can be used from C++ or entirely from Python. For example, the following shows how we can implement a ``computed attribute'' in Python:
>>> class Range(AnyBoost.PythonExtensionClass):
...    def __init__(self, start, end):
...        self.start = start
...        self.end = end
...    def __getattr__length__(self):
...        return self.end - self.start
...
>>> x = Range(3, 9)
>>> x.length
6

Direct Access to Data Members

Boost.Python uses the special __xxxattr__<name>__ functionality described above to allow direct access to data members through the following special functions on class_builder<> and extension_class<>:

Note that the first two functions, used alone, may produce surprising behavior. For example, when def_getter() is used, the default functionality for setattr() and delattr() remains in effect, operating on items in the extension instance's name-space (i.e., its __dict__). For that reason, you'll usually want to stick with def_readonly and def_read_write.

For example, to expose a std::pair<int,long> we might write:

typedef std::pair<int,long> Pil;
int first(const Pil& x) { return x.first; }
long second(const Pil& x) { return x.second; }
   ...
my_module.def(first, "first");
my_module.def(second, "second");

class_builder<Pil> pair_int_long(my_module, "Pair");
pair_int_long.def(boost::python::constructor<>());
pair_int_long.def(boost::python::constructor<int,long>());
pair_int_long.def_read_write(&Pil::first, "first");
pair_int_long.def_read_write(&Pil::second, "second");

Now your Python class has attributes first and second which, when accessed, actually modify or reflect the values of corresponding data members of the underlying C++ object. Now in Python:

>>> x = Pair(3,5)
>>> x.first
3
>>> x.second
5
>>> x.second = 8
>>> x.second
8
>>> second(x) # Prove that we're not just changing the instance __dict__
8

And what about __complex__?

That, dear reader, is one problem we don't know how to solve. The Python source contains the following fragment, indicating the special-case code really is hardwired:

/* XXX Hack to support classes with __complex__ method */
if (PyInstance_Check(r)) { ...

Next: A Peek Under the Hood Previous: Inheritance Up: Top

© Copyright David Abrahams and Ullrich Köthe 2000. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided ``as is'' without express or implied warranty, and with no claim as to its suitability for any purpose.

Updated: Nov 26, 2000