Skip to content

[stdlib] Set PYTHONIOENCODING when building the stdlib #68218

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 31, 2023

Conversation

bnbarham
Copy link
Contributor

Ubuntu 18.04 seems to be defaulting to 'ascii', causing an error when using line-directive combined with the utf-8 diagnostic output used when early swift syntax is enabled.

Also fix up a validation-test while I'm here.

Ubuntu 18.04 seems to be defaulting to 'ascii', causing an error when
using line-directive combined with the utf-8 diagnostic output used when
early swift syntax is enabled.
This was disabled due to:
```
Unable to decode the command from the command line:
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 13-20: surrogates not allowed
```

Note that this error isn't in `line-directive`, but in Python decoding
surrogates from the command line. Not sure why that's not allowed, but
printing from within a script works so change to that.

We already set `PYTHONIOENCODING=UTF8` in the environment in
`test/lit.cfg`, which makes sure stdout is UTF8 as opposed to ASCII.

Resolves rdar://92613094.
@bnbarham bnbarham requested review from compnerd and etcwilde August 30, 2023 18:57
@bnbarham
Copy link
Contributor Author

@swift-ci Please Build Toolchain Ubuntu 18.04

@bnbarham
Copy link
Contributor Author

@swift-ci please test

Copy link
Member

@etcwilde etcwilde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. Hope that works.

PYTHONIOENCODING is already specified in `lit.cfg`. The change that
added this also added the `<`, which is what actually made this pass -
PYTHONIOENCODING only overrides the encoding of stdin/stdout/stderr.
@bnbarham bnbarham requested a review from artemcm as a code owner August 30, 2023 20:41
@bnbarham bnbarham requested a review from xymus August 30, 2023 20:41
@bnbarham
Copy link
Contributor Author

@xymus for the last commit

@bnbarham
Copy link
Contributor Author

@swift-ci please smoke test

Copy link
Contributor

@xymus xymus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to set LC_ALL and others instead, but setting this env var worked fine for the test.

@bnbarham
Copy link
Contributor Author

@swift-ci please smoke test Linux platform

@bnbarham bnbarham merged commit d431e03 into swiftlang:main Aug 31, 2023
@bnbarham bnbarham deleted the set-python-encoding branch August 31, 2023 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants