Commit Graph

1150 Commits

Author SHA1 Message Date
William S Fulton 2f00a7dc07 Use heap types for builtin wrappers
Closes #3196
2025-06-20 23:04:35 +01:00
William S Fulton 4a2f3d8ec4 tp_dictoffset is not available when using Py_LIMITED_API
Heap types and Py_LIMITED_API cannot support __tp_dictoffset__.
As this tp_dictoffset slot is only used by the builtin wrappers
(-builtin), builtin wrappers and Py_LIMITED_API will require python-3.9
or later.

Issue #3196
2025-06-20 07:31:07 +01:00
William S Fulton 1844d2efff Restore Stable ABI for < python-3.9
Compilation fix as tp_dictoffset is not available when defining
Py_LIMITED_ABI.

This still needs addressing in the case of multiple modules, mixing
builtin modules and non-builtin (proxy classes) modules in one python
interpreter session.

Add python 3.8 -builtin to GHA
2025-06-18 19:29:48 +01:00
William S Fulton e6283e7552 Python heap types fix for < python-3.9
structmember.h inclusion moved for external runtime to work without
requiring users to include structmember.h and the Py_READONLY and
Py_ST_PYSSIZE_T definitions. Including Python.h remains the requirement
for using the external runtime (Examples/python/external_runtime).

Note that stddef.h is not always included by structmember.h

Rename dict to swigdict in PySwigObject for easier identification.

SWIG_HEAPTYPES definition is moved to support both external runtime
and Python Stable ABI.
2025-06-18 19:29:43 +01:00
Petr Viktorin 80292de98f Move dictoffset from derived classes to SwigPyObject for builtin wrappers
Issue #3196

Fixes "TypeError: multiple bases have instance lay-out conflict"
for python-3.11 and earlier for multiple inheritance, see 22d9fda0 and 4908133e.
Fix is achieved by moving __dictoffset__ to the single common base class SwigPyObject.
2025-06-17 08:27:27 +01:00
William S Fulton f3672e2d6f Py_LIMITED_API support in SwigPyObject_Check for -builtin
Simplify implementation for -builtin, which does not need a fallback to
use strcmp as PyType_IsSubtype() 'just works' even when using multiple
modules (I think perhaps because SwigPyObject_stype->clientdata->pytype is
common across modules due to the implementation in SwigPyObject_Type()).

In the non-builtin case SwigPyObject is not a base type and usage is
different and when multiple modules are being used, SwigPyObject_Type()
returns two implementations of SwigPyObject which is solved in a hacky
way by comparing the types as strings when the pointer comparison fails.

Add import_callback test - tests %import and %callback to exercise
all of SwigPyPacked_Check(). Python only - the main callback example is
not widely tested and needs work in most of the other languages.
2025-06-09 18:36:13 +01:00
William S Fulton 9ec4c6f1aa Py_LIMITED_API support in SwigPyBuiltin_SetMetaType for -builtin
Also SWIG_HEAPTYPE redefinition warning fix in test-suite
2025-06-09 18:35:49 +01:00
William S Fulton 080093306a Remove pyname_compat.i 2025-06-07 11:48:45 +01:00
William S Fulton 170f3fc065 Python type creation functions refactor
Rename to use naming conventions used in the Python C code base and
typically used in extensions when creating types:

SwigPyObject_type -> SwigPyObject_Type
SwigPyPacked_type -> SwigPyPacked_Type
swig_varlink_type -> SwigVarLink_Type

to also match other type creating functions SwigPyObjectType_Type
and SwigPyStaticVar_Type.

Add SwigVarLink singleton creation to init function for thread safety.
2025-06-07 10:50:39 +01:00
William S Fulton 7df18106fc Add SwigPyObjectType and SwigPyStaticVar to the swig runtime module
for default case (they have just been added for -builtin and
SWIG_HEAPTYPES in previous couple of commits).

These are only used by -builtin and were missed when when moving
SwigPyObject, SwigPyPacked and SwigVarLink were added to the swig
runtime module. Should fix potential warnings for these two types:

  DeprecationWarning: builtin type has no __module__ attribute

although this has not been seen in the test-suite.
2025-06-07 10:17:09 +01:00
William S Fulton f4776dcb94 SWIG_HEAPTYPES for SwigPyStaticVar and add it to the runtime module
Renamed from swig_static_var_getset_descriptor, used by -builtin only.
2025-06-07 10:17:03 +01:00
William S Fulton 0c6c9fd476 Cosmetic whitespace formatting around PY_VERSION_HEX 2025-06-02 19:06:43 +01:00
William S Fulton bf297d608c SWIG_HEAPTYPES for SwigPyObjectType and add it to the runtime module
Used by -builtin only.
2025-06-02 19:06:26 +01:00
William S Fulton d06b54f7aa More graceful error in failures calling back_reference
Python error instead of abort due to assert failure in python-3.8 and
earlier versions which fail using -builtin.
For li_std_vector_back_reference testcase when defining SWIG_HEAPTYPES.
2025-05-31 08:57:35 +01:00
William S Fulton 081682a89e Show ref count before final decrement when using SWIG_REFCNT_DEBUG
Otherwise garbage might be shown if the PyObject is deleted
2025-05-28 07:21:28 +01:00
William S Fulton fde28cb9cb Replace use of Python internal ob_type with public API Py_TYPE 2025-05-24 09:54:25 +01:00
William S Fulton 05c439d1dd Add SWIG_PYTHON_OLD_BUFFER_PROTOCOL fragments for pybuffer typemaps
The pybuffer typemaps use a new fragment for issuing the helpful warning
to help clarify when the old Buffer Protocol can be used. This prevents
it being issued when the typemaps are not used and makes it clearer in
the generated code when the old Protocol Buffer is being used.

Define SWIG_NO_PYTHON_OLD_BUFFER_PROTOCOL to override the choice of when
to define SWIG_PYTHON_OLD_BUFFER_PROTOCOL.
2025-05-23 15:17:50 +01:00
William S Fulton 8710367043 Remove error which might break builds if pybuffer typemaps are not actually used 2025-05-23 15:17:50 +01:00
Julien Schueller f2a3e5306c Python/buffer: Warn about old protocol removal
Closes #3059
2025-05-23 15:17:50 +01:00
Julien Schueller fc2d3fbf5b Python/buffer: Prefer new protocol with Stable ABI
Use the new buffer protocol from stable abi 3.11
2025-05-23 15:17:50 +01:00
William S Fulton 53f4751ffa Fix undefined behaviour in directorout typemaps for void * and const char *&
if the same wrapped function is called more than once. Note that using returning
pointers in directors is still full of traps and not recommended. As such,
warning SWIGWARN_TYPEMAP_DIRECTOROUT_PTR (473) continues to be issued.

The debug Python interpreter failed 3 director testcases which have been
modified in this patch to work around unrecommended director usage
returning from director methods. These are C strings and std::string_view.

A Pyton reference count leak is chosen over undefined behaviour for
returning std::string_view.
2025-05-14 22:10:39 +01:00
William S Fulton 53453d51ec Rename Python swigvarlink type to SwigVarLink
With this type having just moved into a module, let's also
make a change to use correct Python naming conventions for the
name of the type.
2025-05-13 21:49:37 +01:00
William S Fulton 7ecbb7103d Add SwigPyObject, SwigPyPacked, swigvarlink to the swig runtime module
Also add SWIG_REFCNT_DEBUG for debugging reference counts when the main
module is destroyed - there's some work to be done here on reference
counts!
2025-05-13 21:49:37 +01:00
William S Fulton 49a7d2b0c6 Python DeprecationWarning fixes
DeprecationWarning fixes when using 'python -Walways' or if using one of the types being warned about:

  DeprecationWarning: builtin type SwigPyPacked has no __module__ attribute
  DeprecationWarning: builtin type SwigPyObject has no __module__ attribute
  DeprecationWarning: builtin type swigvarlink has no __module__ attribute

The warning was present if code was generated without using -builtin in 4.3.x.
The warning was only present if using the Python limited API in 4.2.1 and earlier.

These 3 types were not in any module at all. Now they are in the swig runtime module.
Docs to follow.

Issue #2881
Issue #3061
Issue #3160
2025-05-13 21:46:49 +01:00
William S Fulton d2056de196 Fix the max number of elements in the STL container wrappers for Python.
Maximum number is now PY_SSIZE_T_MAX instead of INT_MAX as Python uses
Py_ssize_t instead of int for sizes (since python-2.5).
2025-04-30 19:16:27 +01:00
William S Fulton 54596b44f1 Fix maximum size of strings being marshalled from C/C++ into Python
This has been changed from INT_MAX to PY_SSIZE_T_MAX in line
with the Python C API maximum string length handling change in python-2.5.

Closes #3134
2025-04-30 19:07:36 +01:00
William S Fulton f91a9cb8e4 Use SWIG_NOEXCEPT for deprecated dynamic exception specifications
SWIG_NOEXCEPT is generated instead of throw() which is deprecated in
c++11. If c++11 or later is being used, then this macro expands to
noexcept instead of throw().

Affects director code only.

Also fix up some testcase to not use throw() when using c++11 or later.
Tested with clang and -Wdeprecated-dynamic-exception-spec as gcc
doesn't seem to warn for this deprecation.

Closes #3027
2025-04-29 22:41:09 +01:00
klaus spanderen a75b7b8b9c Added support for Python free threading (aka no-gil) 2025-04-16 19:00:48 +01:00
William S Fulton 56afa6e6de Fix Python limited API and external runtime
Partially revert 500d3e1470.

Now structmember.h is only included if needed to avoid the global
pollution that this header introduces.

Closes #3067
2025-04-10 08:13:04 +01:00
Vadim Zeitlin 892be2b802 Fix crash when using directors in Python without limited API
Call correct functions when unlocking/destroying a mutex implemented
using Python API: they were accidentally exchanged in 21f1c923c (Make
directors implementation for Python work with limited API, 2019-12-23)
resulting in crashes when using directors and threads together.

Closes #2889, #3112.
2025-01-28 15:11:28 +01:00
William S Fulton 1b9d30398d Improved swig_type_info static initialisation 2024-10-20 16:25:16 +01:00
William S Fulton d06382ec2a Fix seg fault passing invalid unicode values to std::string
Problem reported as starting with python-3.13.
Due to error not being cleared before calling Python C APIs.

The alternative could have been to ensure SWIG_AsCharPtrAndSize
never returns with a pending error (not that easy).

Closes #3051
2024-10-20 14:19:24 +01:00
Olly Betts 500d3e1470 [python] Only include structmember.h if we need it
This header pollutes the namespace and can cause clashes with the
API being wrapped so it's helpful to only include it when we actually
need to (we actually never need to currently, but will if/when
SWIG_HEAPTYPES is fixed to work with -builtin).

See #2350
2024-10-19 13:58:11 +13:00
William S Fulton 42b9e254f0 Use Py_ prefixed names Py_T_PYSSIZET and Py_READONLY
Prefer these over their globally polluting old names
2024-10-14 08:49:27 +01:00
William S Fulton 05c1f48166 Only include structmember.h for newer Python versions
structmember.h is only needed for python-3.11 and earlier.
Confines the global namespace pollution that it contains to just the older Python versions.
2024-10-11 19:07:08 +01:00
William S Fulton 56314f811d Keep to lowercase convention for PY_VERSION_HEX 2024-10-11 19:01:34 +01:00
William S Fulton 0d70a7ba0e Only use heaptypes for non-builtin and python>=3.3 2024-09-29 00:30:04 +01:00
William S Fulton 03a001a5ce Partial revert "Resolve warnings about __module__ attribute missing for SwigPyPacked"
This reverts commit 7911683101.
but keeps the SWIG_name generation into the f_runtime rather than f_headers.

This change was breaking tests that use SwigPyPacked (callbacks/function
pinters) eg callback test when using Py_LIMITED_API.
2024-09-28 19:20:42 +01:00
William S Fulton ad036e98d7 Prettier generated python C/C++ code 2024-09-28 12:03:42 +01:00
William S Fulton 98e3b6c44e Use HEAPTYPES for python-3.3 and later (non-builtin) 2024-09-28 09:05:55 +01:00
Clinton Stimpson 7911683101 Resolve warnings about __module__ attribute missing for SwigPyPacked 2024-09-27 22:37:27 -06:00
Clinton Stimpson 5a6bc0cc1d Allow deriving from SwigPyObject 2024-09-27 08:21:44 -06:00
William S Fulton da03674a77 Code style fixes 2024-09-25 22:13:11 +01:00
Clinton Stimpson f121300b6b Use more portable PyType_FromSpec with Python 3.4+
This makes progress towards building with Py_LIMITED_API.
If using -builtin, PyType_FromSpec is enabled with Python 3.9+
2024-09-25 19:18:26 +01:00
Clinton Stimpson dd97c20f40 Make functions to inialize python types. 2024-09-25 19:00:27 +01:00
William S Fulton 9bf4842002 C++ reference errors when passing in a 'NULL' change of behaviour.
Most languages now use "NullReferenceError" in the error message
where they previously used "ValueError". Also exception changes:

Guile:    "swig-null-reference-error" instead of "swig-value-error"
MzScheme: "swig-null-reference-error" instead of "swig-value-error"
PHP:      zend_ce_type_error instead of zend_ce_value_error
Python:   Consistently raises TypeError instead of a mix of ValueError
          and TypeError.
Ruby:     Consistently raises NullReferenceError instead of a mix of
          ArgumentError and NullReferenceErrorError.

The consistent raising of a TypeError instead of ValueError for Python
ensures that incorrectly passing 'None' into a C++ reference argument
will correctly convert the error into a NotImplemented error for
the rich comparisons implementations per PEP 207. Fixes #2987

Note that the li_constraints checking implementation for the NONNULL
typemap for pointers also makes the same error change from
SWIG_ValueError to SWIG_NullReferenceError.

The D typemaps use SWIG_DNullReferenceException instead of
SWIG_DIllegalArgumentException, although this ultimately has no change
as the same D Exception is still thrown.
2024-09-14 13:03:36 +01:00
William S Fulton 5ac5d90f97 Add forward class definition for std::filesystem::path class
The namespace is missing for the path class, so this provides
more type information to fix this problem.

Closes #2993
2024-08-19 18:33:10 +01:00
Alexander Shadchin c3e30c5734 Fix missing-field-initializers warning with Py3.13 (#2968)
Introduced https://github.com/python/cpython/pull/114900
2024-07-21 09:03:09 +01:00
William S Fulton 93887e5715 Merge branch 'github-win-ci'
* github-win-ci:
  No need to test so many versions of Python
  More cosmetic tweaks
  Cosmetic corrections - MinGW-w64
  Revert PYTHON_LIB change
  fromdos consistency for removing CR
  Show notest in name
  Restore appveyor testing
  Follow @wsfulton feedback
  Remove win32 userenv library.
  Replace 'cmd' with 'powershell'. As powershell stop on error. With 'Install MSYS2', as 'pacman' works with cmd,  add checks after each command for errors. Revert Windows 'SWIG_LIB'. SWIG Main, check SWIG_LIB environment for  null and empty string. Build with MING w64 compiler. Configure improve Windows python 3, try python-config first.
  Add "Machine Info" to CMAKE-WIN workflows. Replace here-documents and tabs with multiple echo lines. Add comments.
  As we add a new GitHub Windows test in ".github/workflows/win_ci.yml"
  Add windows actions using GitHub. Update Windows document. Small update in configuration. Remove SWIG_LIB_SET, windows should use the same value. Add better striping for multiple test in common make file. Add library path to dynamic python linking,  MSVC need to find the windows library linking file (*.lib). For other GCC, it does not change.

Closes issue #2813
2024-07-15 19:18:00 +01:00
Erez Geva cede516ae9 Follow @wsfulton feedback
Add 'PYLIB' and 'PY3LIB' with '-L' in configure.

Revert python 3 probing in configure.ac
Add ruby with MingW64 instead.

Signed-off-by: Erez Geva <ErezGeva2@gmail.com>
2024-07-10 12:51:23 +02:00