[lldb] Use Py_InitializeFromConfig with Python >= 3.8 (NFC) #114112

JDevlieghere · 2024-10-29T18:24:54Z

This fixes the deprecation warning for Py_SetPythonHome, which was deprecated in Python 3.11. With this patch, when building against Python 3.8 or later, we now use Py_InitializeFromConfig instead.

Fixes #113475

This fixes the deprecation warning for Py_SetPythonHome, which was deprecated in Python 3.11. With this patch, when building against Python 3.8 or later, we now use Py_InitializeFromConfig instead. Fixes llvm#113475

JDevlieghere · 2024-10-29T18:25:04Z

CC @R-Goc

llvmbot · 2024-10-29T18:25:30Z

@llvm/pr-subscribers-lldb

Author: Jonas Devlieghere (JDevlieghere)

Changes

This fixes the deprecation warning for Py_SetPythonHome, which was deprecated in Python 3.11. With this patch, when building against Python 3.8 or later, we now use Py_InitializeFromConfig instead.

Fixes #113475

Full diff: https://github.com/llvm/llvm-project/pull/114112.diff

1 Files Affected:

(modified) lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp (+40-27)

diff --git a/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp b/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
index 7cc38da6a6a94b..d7734c5e199058 100644
--- a/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
+++ b/lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp
@@ -92,7 +92,38 @@ namespace {
 struct InitializePythonRAII {
 public:
   InitializePythonRAII() {
-    InitializePythonHome();
+#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 8) || (PY_MAJOR_VERSION > 3)
+    PyConfig config;
+    PyConfig_InitPythonConfig(&config);
+#endif
+
+#if LLDB_EMBED_PYTHON_HOME
+    typedef wchar_t *str_type;
+    static str_type g_python_home = []() -> str_type {
+      const char *lldb_python_home = LLDB_PYTHON_HOME;
+      const char *absolute_python_home = nullptr;
+      llvm::SmallString<64> path;
+      if (llvm::sys::path::is_absolute(lldb_python_home)) {
+        absolute_python_home = lldb_python_home;
+      } else {
+        FileSpec spec = HostInfo::GetShlibDir();
+        if (!spec)
+          return nullptr;
+        spec.GetPath(path);
+        llvm::sys::path::append(path, lldb_python_home);
+        absolute_python_home = path.c_str();
+      }
+      size_t size = 0;
+      return Py_DecodeLocale(absolute_python_home, &size);
+    }();
+    if (g_python_home != nullptr) {
+#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 8)
+      PyConfig_SetBytesString(&config, &config.home, g_python_home);
+#else
+      Py_SetPythonHome(g_python_home);
+#endif
+    }
+#endif
 
     // The table of built-in modules can only be extended before Python is
     // initialized.
@@ -117,15 +148,22 @@ struct InitializePythonRAII {
       PyImport_AppendInittab("_lldb", LLDBSwigPyInit);
     }
 
+#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 8) || (PY_MAJOR_VERSION > 3)
+  config.install_signal_handlers = 0;
+  Py_InitializeFromConfig(&config);
+  PyConfig_Clear(&config);
+  InitializeThreadsPrivate();
+#else
 // Python < 3.2 and Python >= 3.2 reversed the ordering requirements for
 // calling `Py_Initialize` and `PyEval_InitThreads`.  < 3.2 requires that you
 // call `PyEval_InitThreads` first, and >= 3.2 requires that you call it last.
-#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 2) || (PY_MAJOR_VERSION > 3)
+#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 2)
     Py_InitializeEx(0);
     InitializeThreadsPrivate();
 #else
     InitializeThreadsPrivate();
     Py_InitializeEx(0);
+#endif
 #endif
   }
 
@@ -142,31 +180,6 @@ struct InitializePythonRAII {
   }
 
 private:
-  void InitializePythonHome() {
-#if LLDB_EMBED_PYTHON_HOME
-    typedef wchar_t *str_type;
-    static str_type g_python_home = []() -> str_type {
-      const char *lldb_python_home = LLDB_PYTHON_HOME;
-      const char *absolute_python_home = nullptr;
-      llvm::SmallString<64> path;
-      if (llvm::sys::path::is_absolute(lldb_python_home)) {
-        absolute_python_home = lldb_python_home;
-      } else {
-        FileSpec spec = HostInfo::GetShlibDir();
-        if (!spec)
-          return nullptr;
-        spec.GetPath(path);
-        llvm::sys::path::append(path, lldb_python_home);
-        absolute_python_home = path.c_str();
-      }
-      size_t size = 0;
-      return Py_DecodeLocale(absolute_python_home, &size);
-    }();
-    if (g_python_home != nullptr) {
-      Py_SetPythonHome(g_python_home);
-    }
-#endif
-  }
 
   void InitializeThreadsPrivate() {
 // Since Python 3.7 `Py_Initialize` calls `PyEval_InitThreads` inside itself,

github-actions · 2024-10-29T18:28:18Z

✅ With the latest revision this PR passed the C/C++ code formatter.

medismailben

LGTM

R-Goc · 2024-10-29T22:46:56Z

Looks good.

DavidSpickett · 2024-10-30T09:51:02Z

lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp

 // Python < 3.2 and Python >= 3.2 reversed the ordering requirements for
 // calling `Py_Initialize` and `PyEval_InitThreads`.  < 3.2 requires that you
 // call `PyEval_InitThreads` first, and >= 3.2 requires that you call it last.
-#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 2) || (PY_MAJOR_VERSION > 3)
+#if (PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION >= 2)


Doesn't this now do the wrong thing for a hypothetical 4.x? I thought that's what the || (PY_MAJOR_VERSION > 3) was there for.

#if PY_VERSION_HEX>=0x03020000 ?

@DavidSpickett The hypothetical Python 4 case is covered above and goes down the Py_InitializeFromConfig path.

@labath Happy to use that but everything else in this file uses the same check I have here, so I'll simplify the defines in a separate patch/PR.

Yes you're right I was struggling to read the nested ifdefs. This 3.2 code will be removed once the 20 branch happens so it's fine as is.

lldb/source/Plugins/ScriptInterpreter/Python/ScriptInterpreterPython.cpp

DavidSpickett

LGTM

…114112)" This reverts commit 45f420e.

…114290) Reverts #114112 because this triggers a compile error: ``` no known conversion from 'str_type' (aka 'wchar_t *') to 'const char *' for 3rd argument 221 | PyAPI_FUNC(PyStatus) PyConfig_SetBytesString( | ^ 222 | PyConfig *config, 223 | wchar_t **config_str, 224 | const char *str); | ~~~~~~~~~~~~~~~ 1 error generated. ```

) This fixes the deprecation warning for Py_SetPythonHome, which was deprecated in Python 3.11. With this patch, when building against Python 3.8 or later, we now use Py_InitializeFromConfig instead. Fixes llvm#113475

This fixes the deprecation warning for Py_SetPythonHome, which was deprecated in Python 3.11. With this patch, when building against Python 3.8 or later, we now use Py_InitializeFromConfig instead. Fixes #113475

…lvm#114290) Reverts llvm#114112 because this triggers a compile error: ``` no known conversion from 'str_type' (aka 'wchar_t *') to 'const char *' for 3rd argument 221 | PyAPI_FUNC(PyStatus) PyConfig_SetBytesString( | ^ 222 | PyConfig *config, 223 | wchar_t **config_str, 224 | const char *str); | ~~~~~~~~~~~~~~~ 1 error generated. ```

) This fixes the deprecation warning for Py_SetPythonHome, which was deprecated in Python 3.11. With this patch, when building against Python 3.8 or later, we now use Py_InitializeFromConfig instead. Fixes llvm#113475

…lvm#114290) Reverts llvm#114112 because this triggers a compile error: ``` no known conversion from 'str_type' (aka 'wchar_t *') to 'const char *' for 3rd argument 221 | PyAPI_FUNC(PyStatus) PyConfig_SetBytesString( | ^ 222 | PyConfig *config, 223 | wchar_t **config_str, 224 | const char *str); | ~~~~~~~~~~~~~~~ 1 error generated. ```

) This fixes the deprecation warning for Py_SetPythonHome, which was deprecated in Python 3.11. With this patch, when building against Python 3.8 or later, we now use Py_InitializeFromConfig instead. Fixes llvm#113475

[lldb] Use Py_InitializeFromConfig with Python >= 3.8 (NFC)

c584381

This fixes the deprecation warning for Py_SetPythonHome, which was deprecated in Python 3.11. With this patch, when building against Python 3.8 or later, we now use Py_InitializeFromConfig instead. Fixes llvm#113475

JDevlieghere requested review from DavidSpickett and medismailben October 29, 2024 18:24

llvmbot added the lldb label Oct 29, 2024

Fix formatting

35d76ac

medismailben approved these changes Oct 29, 2024

View reviewed changes

DavidSpickett reviewed Oct 30, 2024

View reviewed changes

Add missing PY_MAJOR_VERSION > 3 in LLDB_EMBED_PYTHON_HOME

97b6b53

DavidSpickett approved these changes Oct 30, 2024

View reviewed changes

JDevlieghere merged commit 45f420e into llvm:main Oct 30, 2024
7 checks passed

JDevlieghere deleted the issue-113475 branch October 30, 2024 15:41

JDevlieghere added a commit that referenced this pull request Oct 30, 2024

Revert "[lldb] Use Py_InitializeFromConfig with Python >= 3.8 (NFC) (#…

6d6e0db

…114112)" This reverts commit 45f420e.

JDevlieghere mentioned this pull request Oct 30, 2024

Revert "[lldb] Use Py_InitializeFromConfig with Python >= 3.8 (NFC)" #114290

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[lldb] Use Py_InitializeFromConfig with Python >= 3.8 (NFC) #114112

[lldb] Use Py_InitializeFromConfig with Python >= 3.8 (NFC) #114112

Uh oh!

JDevlieghere commented Oct 29, 2024

Uh oh!

JDevlieghere commented Oct 29, 2024

Uh oh!

llvmbot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024 •

edited

Loading

Uh oh!

medismailben left a comment

Uh oh!

R-Goc commented Oct 29, 2024

Uh oh!

DavidSpickett Oct 30, 2024

Uh oh!

labath Oct 30, 2024

Uh oh!

JDevlieghere Oct 30, 2024

Uh oh!

DavidSpickett Oct 30, 2024

Uh oh!

Uh oh!

DavidSpickett left a comment

Uh oh!

Uh oh!

Uh oh!

[lldb] Use Py_InitializeFromConfig with Python >= 3.8 (NFC) #114112

[lldb] Use Py_InitializeFromConfig with Python >= 3.8 (NFC) #114112

Uh oh!

Conversation

JDevlieghere commented Oct 29, 2024

Uh oh!

JDevlieghere commented Oct 29, 2024

Uh oh!

llvmbot commented Oct 29, 2024

Uh oh!

github-actions bot commented Oct 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

medismailben left a comment

Choose a reason for hiding this comment

Uh oh!

R-Goc commented Oct 29, 2024

Uh oh!

DavidSpickett Oct 30, 2024

Choose a reason for hiding this comment

Uh oh!

labath Oct 30, 2024

Choose a reason for hiding this comment

Uh oh!

JDevlieghere Oct 30, 2024

Choose a reason for hiding this comment

Uh oh!

DavidSpickett Oct 30, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DavidSpickett left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 29, 2024 •

edited

Loading