Skip to content

bpo-39406: Implement os.putenv() with setenv() if available #18095

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

bpo-39406: Implement os.putenv() with setenv() if available #18095

wants to merge 2 commits into from

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Jan 21, 2020

If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.

If setenv() is available, posix_putenv_garbage state is not needed.

https://bugs.python.org/issue39406

If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.

Renane posix_putenv_garbage to posix_putenv_dict in posixmodule.c.
On Windows or if setenv() is available, posix_putenv_dict is no
longer needed.
@vstinner
Copy link
Member Author

I updated the PR to use SetEnvironmentVariable() on Windows.

@serhiy-storchaka: Would you mind to review the updated PR?

@vstinner
Copy link
Member Author

On Windows, I removed the explicit check on _MAX_ENV:

    if (size > _MAX_ENV) {
        PyErr_Format(PyExc_ValueError,
                     "the environment variable is longer than %u characters",
                     _MAX_ENV);
        goto error;
    }

On my Windows 10 VM, _MAX_ENV is equal to 32767, but SetEnvironmentVariableW() seems to accept way longer values!

>>> value='x' * (32767); os.environ['long'] = value; os.environ['long'] == value, len(value)    
(True, 32767)
>>> value='x' * (32767+10); os.environ['long'] = value; os.environ['long'] == value, len(value) 
(True, 32777)
>>> value='abcd' * (32767); os.environ['long'] = value; os.environ['long'] == value, len(value) 
(True, 131068)

But I got ERROR_NO_SYSTEM_RESOURCES (error 1450: "Insufficient system resources exist to complete the requested service") for a value of 1 GB:

>>> import os
>>> value='x'*(2**30)
>>> os.putenv('long', value)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [WinError 1450] Ressources système insuffisantes pour terminer le service demandé

The limit seems to be the memory rather than an hardcoded limit. A "shorter" value (64 MB) is fine:

>>> import os
>>> value='x'*(2**26)
>>> os.putenv('long', value)

/* putenv() requires to manage the memory manually: use a Python dict
object mapping name (bytes) to env (bytes) where env is a bytes string
like "name=value\0". */
PyObject *posix_putenv_dict;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why rename it? This increases the size of the diff.

}
#else /* MS_WINDOWS */
#elif defined(HAVE_PUTENV)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#elif defined(HAVE_PUTENV)
#elif defined(HAVE_SETENV) || defined(HAVE_PUTENV)

Py_RETURN_NONE;
}
#endif /* MS_WINDOWS */
#endif /* HAVE_PUTENV */
#endif /* HAVE_PUTENV */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#endif /* HAVE_PUTENV */
#endif /* HAVE_SETENV || HAVE_PUTENV */

@vstinner
Copy link
Member Author

Oh, a test fail on Windows:


2020-01-21T10:50:49.1283745Z ======================================================================
2020-01-21T10:50:49.1283887Z ERROR: test_unset_error (test.test_os.EnvironTests)
2020-01-21T10:50:49.1284029Z ----------------------------------------------------------------------
2020-01-21T10:50:49.1284166Z Traceback (most recent call last):
2020-01-21T10:50:49.1284296Z   File "D:\a\cpython\cpython\lib\test\support\__init__.py", line 695, in wrapper
2020-01-21T10:50:49.1284429Z     return func(*args, **kw)
2020-01-21T10:50:49.1284504Z   File "D:\a\cpython\cpython\lib\test\test_os.py", line 962, in test_unset_error
2020-01-21T10:50:49.1284628Z     self.assertRaises(ValueError, os.environ.__delitem__, key)
2020-01-21T10:50:49.1284764Z   File "D:\a\cpython\cpython\lib\unittest\case.py", line 799, in assertRaises
2020-01-21T10:50:49.1284895Z     return context.handle('assertRaises', args, kwargs)
2020-01-21T10:50:49.1285989Z   File "D:\a\cpython\cpython\lib\unittest\case.py", line 202, in handle
2020-01-21T10:50:49.1286157Z     callable_obj(*args, **kwargs)
2020-01-21T10:50:49.1286268Z   File "D:\a\cpython\cpython\lib\os.py", line 691, in __delitem__
2020-01-21T10:50:49.1286377Z     raise KeyError(key) from None
2020-01-21T10:50:49.1295463Z KeyError: 

@vstinner
Copy link
Member Author

I made other changes in the meanwhile which introduced conflicts. So I wrote a new PR instead: PR #18128.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants