Skip to content

Commit e2fa664

Browse files
committed
Some small fixes
1 parent 2d5926d commit e2fa664

File tree

3 files changed

+77
-49
lines changed

3 files changed

+77
-49
lines changed

Doc/extending/index.rst

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,11 @@ Recommended third party tools
2626
=============================
2727

2828
This guide only covers the basic tools for creating extensions provided
29-
as part of this version of CPython. Third party tools like Cython,
30-
``cffi``, SWIG and Numba offer both simpler and more sophisticated
31-
approaches to creating C and C++ extensions for Python.
29+
as part of this version of CPython. Third party tools like
30+
`Cython <http://cython.org/>`_, `cffi <https://cffi.readthedocs.io>`_,
31+
`SWIG <http://www.swig.org>`_ and `Numba <https://numba.pydata.org/>`_
32+
offer both simpler and more sophisticated approaches to creating C and C++
33+
extensions for Python.
3234

3335
.. seealso::
3436

Doc/extending/newtypes.rst

Lines changed: 51 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,9 @@ implement a handful of these.
2222
As you probably expect by now, we're going to go over this and give more
2323
information about the various handlers. We won't go in the order they are
2424
defined in the structure, because there is a lot of historical baggage that
25-
impacts the ordering of the fields; be sure your type initialization keeps the
26-
fields in the right order! It's often easiest to find an example that includes
27-
all the fields you need (even if they're initialized to ``0``) and then change
28-
the values to suit your new type. ::
25+
impacts the ordering of the fields. It's often easiest to find an example
26+
that includes the fields you need and then change the values to suit your new
27+
type. ::
2928

3029
const char *tp_name; /* For printing */
3130

@@ -443,17 +442,23 @@ these in the :file:`Objects` directory of the Python source distribution. ::
443442
hashfunc tp_hash;
444443

445444
This function, if you choose to provide it, should return a hash number for an
446-
instance of your data type. Here is a moderately pointless example::
445+
instance of your data type. Here is a simple example::
447446

448447
static Py_hash_t
449448
newdatatype_hash(newdatatypeobject *obj)
450449
{
451450
Py_hash_t result;
452-
result = obj->obj_UnderlyingDatatypePtr->size;
453-
result = result * 3;
451+
result = obj->some_size + 32767 * obj->some_number;
452+
if (result == -1)
453+
result = -2;
454454
return result;
455455
}
456456

457+
:c:type:`Py_hash_t` is a signed integer type with a platform-varying width.
458+
Returning ``-1`` from :c:member:`~PyTypeObject.tp_hash` indicates an error,
459+
which is why you should be careful to avoid returning it when hash computation
460+
is successful, as seen above.
461+
457462
::
458463

459464
ternaryfunc tp_call;
@@ -505,25 +510,33 @@ Here is a toy ``tp_call`` implementation::
505510
These functions provide support for the iterator protocol. Both handlers
506511
take exactly one parameter, the instance for which they are being called,
507512
and return a new reference. In the case of an error, they should set an
508-
exception and return *NULL*.
509-
510-
Any :term:`iterable` object must implement the ``tp_iter`` handler, which
511-
must return an :term:`iterator` object. For collections (such as lists and
512-
tuples) which can support multiple iterators which do not interfere with
513-
each other, a new iterator should be created and returned by each call to
514-
``tp_iter``. Objects which can only be iterated over once (usually due to
515-
side effects of iteration, for example file objects) can implement
516-
``tp_iter`` by returning a new reference to themselves, and should also
517-
implement the ``tp_iternext`` handler.
518-
519-
Any :term:`iterator` object should implement both ``tp_iter`` and ``tp_iternext``.
520-
The ``tp_iter`` handler should return a new reference to the iterator. The
521-
``tp_iternext`` handler should return a new reference to the next object in the
522-
iteration, if there is one. If the iteration has reached the end,
523-
``tp_iternext`` may return *NULL* without setting an exception, or it may
524-
also set :exc:`StopIteration` (while still returning *NULL*); avoiding
525-
the exception can yield slightly better performance. If an actual error occurs,
526-
it should always set an exception and return *NULL*.
513+
exception and return *NULL*. :c:member:`~PyTypeObject.tp_iter` corresponds
514+
to the Python :meth:`__iter__` method, while :c:member:`~PyTypeObject.tp_iternext`
515+
corresponds to the Python :meth:`~iterator.__next__` method.
516+
517+
Any :term:`iterable` object must implement the :c:member:`~PyTypeObject.tp_iter`
518+
handler, which must return an :term:`iterator` object. Here the same guidelines
519+
apply as for Python classes:
520+
521+
* For collections (such as lists and tuples) which can support multiple
522+
independent iterators, a new iterator should be created and returned by
523+
each call to :c:member:`~PyTypeObject.tp_iter`.
524+
* Objects which can only be iterated over once (usually due to side effects of
525+
iteration, such as file objects) can implement :c:member:`~PyTypeObject.tp_iter`
526+
by returning a new reference to themselves -- and should also therefore
527+
implement the :c:member:`~PyTypeObject.tp_iternext` handler.
528+
529+
Any :term:`iterator` object should implement both :c:member:`~PyTypeObject.tp_iter`
530+
and :c:member:`~PyTypeObject.tp_iternext`. An iterator's
531+
:c:member:`~PyTypeObject.tp_iter` handler should return a new reference
532+
to the iterator. Its :c:member:`~PyTypeObject.tp_iternext` handler should
533+
return a new reference to the next object in the iteration, if there is one.
534+
If the iteration has reached the end, :c:member:`~PyTypeObject.tp_iternext`
535+
may return *NULL* without setting an exception, or it may set
536+
:exc:`StopIteration` *in addition* to returning *NULL*; avoiding
537+
the exception can yield slightly better performance. If an actual error
538+
occurs, :c:member:`~PyTypeObject.tp_iternext` should always set an exception
539+
and return *NULL*.
527540

528541

529542
.. _weakref-support:
@@ -535,14 +548,19 @@ One of the goals of Python's weak reference implementation is to allow any type
535548
to participate in the weak reference mechanism without incurring the overhead on
536549
performance-critical objects (such as numbers).
537550

551+
.. seealso::
552+
Documentation for the :mod:`weakref` module.
553+
538554
For an object to be weakly referencable, the extension type must do two things:
539555

540556
#. Include a :c:type:`PyObject\*` field in the C object structure dedicated to
541-
the weak reference mechanism; it must be initialized to *NULL* by the object's
542-
constructor.
557+
the weak reference mechanism. The object's constructor should leave it
558+
*NULL* (which is automatic when using the default
559+
:c:member:`~PyTypeObject.tp_alloc`).
543560

544-
#. Set the :c:member:`~PyTypeObject.tp_weaklistoffset` member of the C type
545-
object to the offset of the aforementioned field in the C object structure.
561+
#. Set the :c:member:`~PyTypeObject.tp_weaklistoffset` type member
562+
to the offset of the aforementioned field in the C object structure,
563+
so that the interpreter knows how to access and modify that field.
546564

547565
Concretely, here is how a trivial object structure would be augmented
548566
with the required field::
@@ -572,10 +590,6 @@ non-*NULL*::
572590
Py_TYPE(self)->tp_free((PyObject *) self);
573591
}
574592

575-
.. seealso::
576-
The :mod:`weakref` module allows creating weak references to instances
577-
of types that implement the protocol described above.
578-
579593

580594
More Suggestions
581595
----------------
@@ -586,9 +600,9 @@ then search the C source files for ``tp_`` plus the function you want
586600
(for example, ``tp_richcompare``). You will find examples of the function
587601
you want to implement.
588602

589-
When you need to verify that an object is an instance of the type you are
590-
implementing, use the :c:func:`PyObject_TypeCheck` function. A sample of its use
591-
might be something like the following::
603+
When you need to verify that an object is a concrete instance of the type you
604+
are implementing, use the :c:func:`PyObject_TypeCheck` function. A sample of
605+
its use might be something like the following::
592606

593607
if (!PyObject_TypeCheck(some_object, &MyType)) {
594608
PyErr_SetString(PyExc_TypeError, "arg #1 not a mything");

Doc/extending/newtypes_tutorial.rst

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,12 @@ This sort of thing can only be explained by example, so here's a minimal, but
4040
complete, module that defines a new type named :class:`Noddy` inside a C
4141
extension module :mod:`noddy`:
4242

43+
.. note::
44+
What we're showing here is the traditional way of defining *static*
45+
extension types. It should be adequate for most uses. The C API also
46+
allows defining heap-allocated extension types using the
47+
:c:func:`PyType_FromSpec` function, which isn't covered in this tutorial.
48+
4349
.. literalinclude:: ../includes/noddy.c
4450

4551
Now that's quite a bit to take in at once, but hopefully bits will seem familiar
@@ -61,7 +67,7 @@ The first bit is::
6167

6268
This is what a Noddy object will contain. ``PyObject_HEAD`` is mandatory
6369
at the start of each object struct and defines a field called ``ob_base``
64-
of type c:type:`PyObject`, containing a pointer to a type object and a
70+
of type :c:type:`PyObject`, containing a pointer to a type object and a
6571
reference count (these can be accessed using the macros :c:macro:`Py_REFCNT`
6672
and :c:macro:`Py_TYPE` respectively). The reason for the macro is to
6773
abstract away the layout and to enable additional fields in debug builds.
@@ -70,9 +76,9 @@ abstract away the layout and to enable additional fields in debug builds.
7076
There is no semicolon above after the :c:macro:`PyObject_HEAD` macro.
7177
Be wary of adding one by accident: some compilers will complain.
7278

73-
Of course, objects generally include other information in addition to
74-
the standard ``PyObject_HEAD`` boilerplate; for example, here is the
75-
definition for standard Python floats::
79+
Of course, objects generally store additional data besides the standard
80+
``PyObject_HEAD`` boilerplate; for example, here is the definition for
81+
standard Python floats::
7682

7783
typedef struct {
7884
PyObject_HEAD
@@ -90,6 +96,11 @@ The second bit is the definition of the type object. ::
9096
.tp_new = PyType_GenericNew,
9197
};
9298

99+
.. note::
100+
We recommend using C99-style designated initializers as above, to
101+
avoid listing all the :c:type:`PyTypeObject` fields that you don't care
102+
about and also to avoid caring about the fields' declaration order.
103+
93104
The actual definition of :c:type:`PyTypeObject` in :file:`object.h` has
94105
many more :ref:`fields <type-structs>` than the definition above. The
95106
remaining fields will be filled with zeros by the C compiler, and it's
@@ -701,23 +712,24 @@ participate in cycles::
701712
return 0;
702713
}
703714

704-
Notice the use the :c:func:`Py_CLEAR` macro. It is the recommended, safe,
705-
way of clearing data attributes of arbitrary types while decrementing
715+
Notice the use of the :c:func:`Py_CLEAR` macro. It is the recommended and safe
716+
way to clear data attributes of arbitrary types while decrementing
706717
their reference counts. If you were to call :c:func:`Py_XDECREF` instead
707718
on the attribute before setting it to *NULL*, there is a possibility
708719
that the attribute's destructor would call back into code that reads the
709-
attribute again (especially if there is a reference cycle).
720+
attribute again (*especially* if there is a reference cycle).
710721

711722
.. note::
712-
You could still emulate :c:func:`Py_CLEAR` by writing::
723+
You could emulate :c:func:`Py_CLEAR` by writing::
713724

714725
PyObject *tmp;
715726
tmp = self->first;
716727
self->first = NULL;
717728
Py_XDECREF(tmp);
718729

719730
Nevertheless, it is much easier and less error-prone to always
720-
call :c:func:`Py_CLEAR` when wanting to delete an attribute.
731+
use :c:func:`Py_CLEAR` when deleting an attribute. Don't
732+
try to micro-optimize at the expense of robustness!
721733

722734
The deallocator ``Noddy_dealloc`` may call arbitrary code when clearing
723735
attributes. It means the circular GC can be triggered inside the function.

0 commit comments

Comments
 (0)