Skip to content

Updated files to work with Python 3 #26296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

Updated files to work with Python 3 #26296

wants to merge 5 commits into from

Conversation

tachoknight
Copy link
Contributor

With Python 2 going out of support on January 1st, 2020, Python 3 will be the "standard" version going forward, and certain Linux distributions (e.g. Fedora) will use Python 3 as the default. These changes make Swift buildable using Python 3.

I understand that this pull request is probably too soon; I wanted to make the changes available for when it's decided to move forward with using Python 3 with Swift.

Copy link
Contributor

@drodriguez drodriguez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to see these merged, specially if they are compatible with Python 2 at the same time. However, I think someone from Apple should give the final go or no go with this, because macOS might be the last OS left not jumping into the Python 3 train.

@swift-ci please test

/cc @gwynne

@@ -61,7 +61,7 @@ line_pattern = re.compile(

def _make_line_map(target_filename, stream=None):
"""
>>> from StringIO import StringIO
>>> from io import StringIO
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not work in Python 2, I'm afraid.

You can see in https://github.com/apple/swift/pull/23038/files#diff-cca357e5f9d41d8ba84f741ccf60fe55R64 how I solved it.

from SwiftIntTypes import all_integer_types, int_max_bits, should_define_truncating_bit_pattern_init
from SwiftFloatingPointTypes import getFtoIBounds
import SwiftIntTypes
import SwiftFloatingPointTypes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn’t need these changes back in January. Are you sure they are needed?

(The change with the // is needed, only the import ones werent't needed)

@BenHetherington
Copy link

I'm not aware if there's been any more pushes towards Python 3 lately, but just to update the macOS concern above: macOS Catalina now seems to include python3 by default, and python (2.7) outputs a warning that it'll be removed in the future.

WARNING: Python 2.7 is not recommended. 
This version is included in macOS for compatibility with legacy software. 
Future versions of macOS will not include Python 2.7. 
Instead, it is recommended that you transition to using 'python3' from within Terminal.

So hopefully this will encourage us to support Python 3 in the near future (if not full-on transitioning away from Python 2).

@tachoknight
Copy link
Contributor Author

I added an additional commit to fix pulling sub-projects when using Python 3. Fedora is all-in on Python 3 now and adding it as a community build isn't possible, especially when I can't even download the source. :)

@hlopko
Copy link
Contributor

hlopko commented Feb 11, 2020

Hi all, what's the status of this PR? @tachoknight do you plan to rebase it and push it through? Thank you!

@tachoknight
Copy link
Contributor Author

@hlopko Not sure what you mean; we're still waiting for someone from Apple to merge it.

@hlopko
Copy link
Contributor

hlopko commented Feb 12, 2020

I interpreted (maybe incorrectly?) that 'Conflicting files' section in the CI status means the PR needs to be rebased? And by pushing-through I meant pinging Apple folks :)

@RLovelett
Copy link
Contributor

@tachoknight I do think this branch has grown out of date and cannot be merged as is. A merge or rebase from master might be in order.

Another thing that might need to be considered in all of this PEP 394. Basically it says that runtime distributors are not required to provide /usr/bin/python. Even if they do they can symlink /usr/bin/python to python3 or python2.

I bring this up because some of the CMake steps that check for PythonInterp only look for /usr/bin/python. On Ubuntu 20.04 (at least right now) there is no /usr/bin/python and the build breaks. The short-term work around for me was just to use update-alternatives to provide a link from python3 to /usr/bin/python.

The PEP 394/CMake changes might be out of scope for this change. I just was interested in your thoughts since you've spent more time thinking about this than I have.

@tachoknight
Copy link
Contributor Author

@RLovelett I've been keeping my patches up-to-date for the Releases for Fedora, but yeah, I haven't tried to build master with them lately. In regards to your mention of /usr/bin/python, that's true and I have to do all that before executing the build script.

# Conflicts:
#	stdlib/public/core/Mirrors.swift.gyb
#	utils/gyb_syntax_support/Node.py
#	utils/gyb_syntax_support/__init__.py
@tachoknight
Copy link
Contributor Author

Unless anyone has any objections, I'm going to close this PR and replace it with two others; one that makes it possible to checkout master using Python 3 that also works with Python 2, and another that includes all the Python 3 adjustments.

@RLovelett
Copy link
Contributor

I've used the patches (with some modifications) to build 5.2 on Ubuntu 20.04. So first off I want to say thank you.

I personally see no problem with that. Unfortunately I'm not a Python expert but is it possible that both sets of patches remain Python 2 backwards compatible?

@tachoknight
Copy link
Contributor Author

@RLovelett The checkout part is backwards compatible, and I will see about keeping the bulk of the other ones Python 2 compatible as well.

@RLovelett
Copy link
Contributor

Thank you for all the effort you've been putting into maintaining these patches.

Also for the effort you put into making and getting the Swift package into the official Fedora repo list. That is no small feat. It never ceases to amuse me that while Ubuntu is the official supported distro it has no package and Fedora does. Again kudos.

@RLovelett
Copy link
Contributor

RLovelett commented Mar 26, 2020

One last thing @tachoknight one of the things I had to do to get GYB working on Ubuntu 20.04 was patch CMake to use the FindPython module. You might not need that or maybe you have a better way to do it. I just thought I'd share it.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index ca6c310aecd..f0e4e7b8553 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -1,4 +1,4 @@
-cmake_minimum_required(VERSION 3.4.3)
+cmake_minimum_required(VERSION 3.12.4)
 
 # TODO: Fix RPATH usage to be CMP0068 compliant
 # Disable Policy CMP0068 for CMake 3.9
@@ -881,7 +881,7 @@ if(NOT CMAKE_SYSTEM_NAME STREQUAL "Darwin")
   endif()
 endif()
 
-find_package(PythonInterp REQUIRED)
+find_package(Python COMPONENTS Interpreter REQUIRED)
 
 #
 # Find optional dependencies.
diff --git a/cmake/modules/SwiftHandleGybSources.cmake b/cmake/modules/SwiftHandleGybSources.cmake
index f8dc984ea0c..37f2dec6039 100644
--- a/cmake/modules/SwiftHandleGybSources.cmake
+++ b/cmake/modules/SwiftHandleGybSources.cmake
@@ -1,7 +1,7 @@
 include(SwiftAddCustomCommandTarget)
 include(SwiftSetIfArchBitness)
 
-find_package(Python2 COMPONENTS Interpreter REQUIRED)
+find_package(Python COMPONENTS Interpreter REQUIRED)
 
 # Create a target to process single gyb source with the 'gyb' tool.
 #
@@ -60,7 +60,7 @@ function(handle_gyb_source_single dependency_out_var_name)
       COMMAND
           "${CMAKE_COMMAND}" -E make_directory "${dir}"
       COMMAND
-          "$<TARGET_FILE:Python2::Interpreter>" "${gyb_tool}" ${SWIFT_GYB_FLAGS} ${GYB_SINGLE_FLAGS} -o "${GYB_SINGLE_OUTPUT}.tmp" "${GYB_SINGLE_SOURCE}"
+          "$<TARGET_FILE:Python::Interpreter>" "${gyb_tool}" ${SWIFT_GYB_FLAGS} ${GYB_SINGLE_FLAGS} -o "${GYB_SINGLE_OUTPUT}.tmp" "${GYB_SINGLE_SOURCE}"
       COMMAND
           "${CMAKE_COMMAND}" -E copy_if_different "${GYB_SINGLE_OUTPUT}.tmp" "${GYB_SINGLE_OUTPUT}"
       COMMAND

@tachoknight
Copy link
Contributor Author

Awesome, thanks @RLovelett! Also, the first, simpler PR is here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants