| Upgrade guide |
| ############# |
| |
| This is a companion guide to the :doc:`changelog`. While the changelog briefly |
| lists all of the new features, improvements and bug fixes, this upgrade guide |
| focuses only the subset which directly impacts your experience when upgrading |
| to a new version. But it goes into more detail. This includes things like |
| deprecated APIs and their replacements, build system changes, general code |
| modernization and other useful information. |
| |
| |
| v2.2 |
| ==== |
| |
| Deprecation of the ``PYBIND11_PLUGIN`` macro |
| -------------------------------------------- |
| |
| ``PYBIND11_MODULE`` is now the preferred way to create module entry points. |
| The old macro emits a compile-time deprecation warning. |
| |
| .. code-block:: cpp |
| |
| // old |
| PYBIND11_PLUGIN(example) { |
| py::module m("example", "documentation string"); |
| |
| m.def("add", [](int a, int b) { return a + b; }); |
| |
| return m.ptr(); |
| } |
| |
| // new |
| PYBIND11_MODULE(example, m) { |
| m.doc() = "documentation string"; // optional |
| |
| m.def("add", [](int a, int b) { return a + b; }); |
| } |
| |
| |
| New API for defining custom constructors and pickling functions |
| --------------------------------------------------------------- |
| |
| The old placement-new custom constructors have been deprecated. The new approach |
| uses ``py::init()`` and factory functions to greatly improve type safety. |
| |
| Placement-new can be called accidentally with an incompatible type (without any |
| compiler errors or warnings), or it can initialize the same object multiple times |
| if not careful with the Python-side ``__init__`` calls. The new-style custom |
| constructors prevent such mistakes. See :ref:`custom_constructors` for details. |
| |
| .. code-block:: cpp |
| |
| // old -- deprecated (runtime warning shown only in debug mode) |
| py::class<Foo>(m, "Foo") |
| .def("__init__", [](Foo &self, ...) { |
| new (&self) Foo(...); // uses placement-new |
| }); |
| |
| // new |
| py::class<Foo>(m, "Foo") |
| .def(py::init([](...) { // Note: no `self` argument |
| return new Foo(...); // return by raw pointer |
| // or: return std::make_unique<Foo>(...); // return by holder |
| // or: return Foo(...); // return by value (move constructor) |
| })); |
| |
| Mirroring the custom constructor changes, ``py::pickle()`` is now the preferred |
| way to get and set object state. See :ref:`pickling` for details. |
| |
| .. code-block:: cpp |
| |
| // old -- deprecated (runtime warning shown only in debug mode) |
| py::class<Foo>(m, "Foo") |
| ... |
| .def("__getstate__", [](const Foo &self) { |
| return py::make_tuple(self.value1(), self.value2(), ...); |
| }) |
| .def("__setstate__", [](Foo &self, py::tuple t) { |
| new (&self) Foo(t[0].cast<std::string>(), ...); |
| }); |
| |
| // new |
| py::class<Foo>(m, "Foo") |
| ... |
| .def(py::pickle( |
| [](const Foo &self) { // __getstate__ |
| return py::make_tuple(f.value1(), f.value2(), ...); // unchanged |
| }, |
| [](py::tuple t) { // __setstate__, note: no `self` argument |
| return new Foo(t[0].cast<std::string>(), ...); |
| // or: return std::make_unique<Foo>(...); // return by holder |
| // or: return Foo(...); // return by value (move constructor) |
| } |
| )); |
| |
| For both the constructors and pickling, warnings are shown at module |
| initialization time (on import, not when the functions are called). |
| They're only visible when compiled in debug mode. Sample warning: |
| |
| .. code-block:: none |
| |
| pybind11-bound class 'mymodule.Foo' is using an old-style placement-new '__init__' |
| which has been deprecated. See the upgrade guide in pybind11's docs. |
| |
| |
| Stricter enforcement of hidden symbol visibility for pybind11 modules |
| --------------------------------------------------------------------- |
| |
| pybind11 now tries to actively enforce hidden symbol visibility for modules. |
| If you're using either one of pybind11's :doc:`CMake or Python build systems |
| <compiling>` (the two example repositories) and you haven't been exporting any |
| symbols, there's nothing to be concerned about. All the changes have been done |
| transparently in the background. If you were building manually or relied on |
| specific default visibility, read on. |
| |
| Setting default symbol visibility to *hidden* has always been recommended for |
| pybind11 (see :ref:`faq:symhidden`). On Linux and macOS, hidden symbol |
| visibility (in conjunction with the ``strip`` utility) yields much smaller |
| module binaries. `CPython's extension docs`_ also recommend hiding symbols |
| by default, with the goal of avoiding symbol name clashes between modules. |
| Starting with v2.2, pybind11 enforces this more strictly: (1) by declaring |
| all symbols inside the ``pybind11`` namespace as hidden and (2) by including |
| the ``-fvisibility=hidden`` flag on Linux and macOS (only for extension |
| modules, not for embedding the interpreter). |
| |
| .. _CPython's extension docs: https://docs.python.org/3/extending/extending.html#providing-a-c-api-for-an-extension-module |
| |
| The namespace-scope hidden visibility is done automatically in pybind11's |
| headers and it's generally transparent to users. It ensures that: |
| |
| * Modules compiled with different pybind11 versions don't clash with each other. |
| |
| * Some new features, like ``py::module_local`` bindings, can work as intended. |
| |
| The ``-fvisibility=hidden`` flag applies the same visibility to user bindings |
| outside of the ``pybind11`` namespace. It's now set automatic by pybind11's |
| CMake and Python build systems, but this needs to be done manually by users |
| of other build systems. Adding this flag: |
| |
| * Minimizes the chances of symbol conflicts between modules. E.g. if two |
| unrelated modules were statically linked to different (ABI-incompatible) |
| versions of the same third-party library, a symbol clash would be likely |
| (and would end with unpredictable results). |
| |
| * Produces smaller binaries on Linux and macOS, as pointed out previously. |
| |
| Within pybind11's CMake build system, ``pybind11_add_module`` has always been |
| setting the ``-fvisibility=hidden`` flag in release mode. From now on, it's |
| being applied unconditionally, even in debug mode and it can no longer be opted |
| out of with the ``NO_EXTRAS`` option. The ``pybind11::module`` target now also |
| adds this flag to it's interface. The ``pybind11::embed`` target is unchanged. |
| |
| The most significant change here is for the ``pybind11::module`` target. If you |
| were previously relying on default visibility, i.e. if your Python module was |
| doubling as a shared library with dependents, you'll need to either export |
| symbols manually (recommended for cross-platform libraries) or factor out the |
| shared library (and have the Python module link to it like the other |
| dependents). As a temporary workaround, you can also restore default visibility |
| using the CMake code below, but this is not recommended in the long run: |
| |
| .. code-block:: cmake |
| |
| target_link_libraries(mymodule PRIVATE pybind11::module) |
| |
| add_library(restore_default_visibility INTERFACE) |
| target_compile_options(restore_default_visibility INTERFACE -fvisibility=default) |
| target_link_libraries(mymodule PRIVATE restore_default_visibility) |
| |
| |
| Local STL container bindings |
| ---------------------------- |
| |
| Previous pybind11 versions could only bind types globally -- all pybind11 |
| modules, even unrelated ones, would have access to the same exported types. |
| However, this would also result in a conflict if two modules exported the |
| same C++ type, which is especially problematic for very common types, e.g. |
| ``std::vector<int>``. :ref:`module_local` were added to resolve this (see |
| that section for a complete usage guide). |
| |
| ``py::class_`` still defaults to global bindings (because these types are |
| usually unique across modules), however in order to avoid clashes of opaque |
| types, ``py::bind_vector`` and ``py::bind_map`` will now bind STL containers |
| as ``py::module_local`` if their elements are: builtins (``int``, ``float``, |
| etc.), not bound using ``py::class_``, or bound as ``py::module_local``. For |
| example, this change allows multiple modules to bind ``std::vector<int>`` |
| without causing conflicts. See :ref:`stl_bind` for more details. |
| |
| When upgrading to this version, if you have multiple modules which depend on |
| a single global binding of an STL container, note that all modules can still |
| accept foreign ``py::module_local`` types in the direction of Python-to-C++. |
| The locality only affects the C++-to-Python direction. If this is needed in |
| multiple modules, you'll need to either: |
| |
| * Add a copy of the same STL binding to all of the modules which need it. |
| |
| * Restore the global status of that single binding by marking it |
| ``py::module_local(false)``. |
| |
| The latter is an easy workaround, but in the long run it would be best to |
| localize all common type bindings in order to avoid conflicts with |
| third-party modules. |
| |
| |
| Negative strides for Python buffer objects and numpy arrays |
| ----------------------------------------------------------- |
| |
| Support for negative strides required changing the integer type from unsigned |
| to signed in the interfaces of ``py::buffer_info`` and ``py::array``. If you |
| have compiler warnings enabled, you may notice some new conversion warnings |
| after upgrading. These can be resolved using ``static_cast``. |
| |
| |
| Deprecation of some ``py::object`` APIs |
| --------------------------------------- |
| |
| To compare ``py::object`` instances by pointer, you should now use |
| ``obj1.is(obj2)`` which is equivalent to ``obj1 is obj2`` in Python. |
| Previously, pybind11 used ``operator==`` for this (``obj1 == obj2``), but |
| that could be confusing and is now deprecated (so that it can eventually |
| be replaced with proper rich object comparison in a future release). |
| |
| For classes which inherit from ``py::object``, ``borrowed`` and ``stolen`` |
| were previously available as protected constructor tags. Now the types |
| should be used directly instead: ``borrowed_t{}`` and ``stolen_t{}`` |
| (`#771 <https://github.com/pybind/pybind11/pull/771>`_). |
| |
| |
| Stricter compile-time error checking |
| ------------------------------------ |
| |
| Some error checks have been moved from run time to compile time. Notably, |
| automatic conversion of ``std::shared_ptr<T>`` is not possible when ``T`` is |
| not directly registered with ``py::class_<T>`` (e.g. ``std::shared_ptr<int>`` |
| or ``std::shared_ptr<std::vector<T>>`` are not automatically convertible). |
| Attempting to bind a function with such arguments now results in a compile-time |
| error instead of waiting to fail at run time. |
| |
| ``py::init<...>()`` constructor definitions are also stricter and now prevent |
| bindings which could cause unexpected behavior: |
| |
| .. code-block:: cpp |
| |
| struct Example { |
| Example(int &); |
| }; |
| |
| py::class_<Example>(m, "Example") |
| .def(py::init<int &>()); // OK, exact match |
| // .def(py::init<int>()); // compile-time error, mismatch |
| |
| A non-``const`` lvalue reference is not allowed to bind to an rvalue. However, |
| note that a constructor taking ``const T &`` can still be registered using |
| ``py::init<T>()`` because a ``const`` lvalue reference can bind to an rvalue. |
| |
| v2.1 |
| ==== |
| |
| Minimum compiler versions are enforced at compile time |
| ------------------------------------------------------ |
| |
| The minimums also apply to v2.0 but the check is now explicit and a compile-time |
| error is raised if the compiler does not meet the requirements: |
| |
| * GCC >= 4.8 |
| * clang >= 3.3 (appleclang >= 5.0) |
| * MSVC >= 2015u3 |
| * Intel C++ >= 15.0 |
| |
| |
| The ``py::metaclass`` attribute is not required for static properties |
| --------------------------------------------------------------------- |
| |
| Binding classes with static properties is now possible by default. The |
| zero-parameter version of ``py::metaclass()`` is deprecated. However, a new |
| one-parameter ``py::metaclass(python_type)`` version was added for rare |
| cases when a custom metaclass is needed to override pybind11's default. |
| |
| .. code-block:: cpp |
| |
| // old -- emits a deprecation warning |
| py::class_<Foo>(m, "Foo", py::metaclass()) |
| .def_property_readonly_static("foo", ...); |
| |
| // new -- static properties work without the attribute |
| py::class_<Foo>(m, "Foo") |
| .def_property_readonly_static("foo", ...); |
| |
| // new -- advanced feature, override pybind11's default metaclass |
| py::class_<Bar>(m, "Bar", py::metaclass(custom_python_type)) |
| ... |
| |
| |
| v2.0 |
| ==== |
| |
| Breaking changes in ``py::class_`` |
| ---------------------------------- |
| |
| These changes were necessary to make type definitions in pybind11 |
| future-proof, to support PyPy via its ``cpyext`` mechanism (`#527 |
| <https://github.com/pybind/pybind11/pull/527>`_), and to improve efficiency |
| (`rev. 86d825 <https://github.com/pybind/pybind11/commit/86d825>`_). |
| |
| 1. Declarations of types that provide access via the buffer protocol must |
| now include the ``py::buffer_protocol()`` annotation as an argument to |
| the ``py::class_`` constructor. |
| |
| .. code-block:: cpp |
| |
| py::class_<Matrix>("Matrix", py::buffer_protocol()) |
| .def(py::init<...>()) |
| .def_buffer(...); |
| |
| 2. Classes which include static properties (e.g. ``def_readwrite_static()``) |
| must now include the ``py::metaclass()`` attribute. Note: this requirement |
| has since been removed in v2.1. If you're upgrading from 1.x, it's |
| recommended to skip directly to v2.1 or newer. |
| |
| 3. This version of pybind11 uses a redesigned mechanism for instantiating |
| trampoline classes that are used to override virtual methods from within |
| Python. This led to the following user-visible syntax change: |
| |
| .. code-block:: cpp |
| |
| // old v1.x syntax |
| py::class_<TrampolineClass>("MyClass") |
| .alias<MyClass>() |
| ... |
| |
| // new v2.x syntax |
| py::class_<MyClass, TrampolineClass>("MyClass") |
| ... |
| |
| Importantly, both the original and the trampoline class are now specified |
| as arguments to the ``py::class_`` template, and the ``alias<..>()`` call |
| is gone. The new scheme has zero overhead in cases when Python doesn't |
| override any functions of the underlying C++ class. |
| `rev. 86d825 <https://github.com/pybind/pybind11/commit/86d825>`_. |
| |
| The class type must be the first template argument given to ``py::class_`` |
| while the trampoline can be mixed in arbitrary order with other arguments |
| (see the following section). |
| |
| |
| Deprecation of the ``py::base<T>()`` attribute |
| ---------------------------------------------- |
| |
| ``py::base<T>()`` was deprecated in favor of specifying ``T`` as a template |
| argument to ``py::class_``. This new syntax also supports multiple inheritance. |
| Note that, while the type being exported must be the first argument in the |
| ``py::class_<Class, ...>`` template, the order of the following types (bases, |
| holder and/or trampoline) is not important. |
| |
| .. code-block:: cpp |
| |
| // old v1.x |
| py::class_<Derived>("Derived", py::base<Base>()); |
| |
| // new v2.x |
| py::class_<Derived, Base>("Derived"); |
| |
| // new -- multiple inheritance |
| py::class_<Derived, Base1, Base2>("Derived"); |
| |
| // new -- apart from `Derived` the argument order can be arbitrary |
| py::class_<Derived, Base1, Holder, Base2, Trampoline>("Derived"); |
| |
| |
| Out-of-the-box support for ``std::shared_ptr`` |
| ---------------------------------------------- |
| |
| The relevant type caster is now built in, so it's no longer necessary to |
| include a declaration of the form: |
| |
| .. code-block:: cpp |
| |
| PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>) |
| |
| Continuing to do so won’t cause an error or even a deprecation warning, |
| but it's completely redundant. |
| |
| |
| Deprecation of a few ``py::object`` APIs |
| ---------------------------------------- |
| |
| All of the old-style calls emit deprecation warnings. |
| |
| +---------------------------------------+---------------------------------------------+ |
| | Old syntax | New syntax | |
| +=======================================+=============================================+ |
| | ``obj.call(args...)`` | ``obj(args...)`` | |
| +---------------------------------------+---------------------------------------------+ |
| | ``obj.str()`` | ``py::str(obj)`` | |
| +---------------------------------------+---------------------------------------------+ |
| | ``auto l = py::list(obj); l.check()`` | ``py::isinstance<py::list>(obj)`` | |
| +---------------------------------------+---------------------------------------------+ |
| | ``py::object(ptr, true)`` | ``py::reinterpret_borrow<py::object>(ptr)`` | |
| +---------------------------------------+---------------------------------------------+ |
| | ``py::object(ptr, false)`` | ``py::reinterpret_steal<py::object>(ptr)`` | |
| +---------------------------------------+---------------------------------------------+ |
| | ``if (obj.attr("foo"))`` | ``if (py::hasattr(obj, "foo"))`` | |
| +---------------------------------------+---------------------------------------------+ |
| | ``if (obj["bar"])`` | ``if (obj.contains("bar"))`` | |
| +---------------------------------------+---------------------------------------------+ |