Skip to content

Commit c364ee8

Browse files
committed
Fix Stream.readuntil with non-bytes buffer objects
PR python#16429 introduced support for an iterable of separators in Stream.readuntil. Since bytes-like types are themselves iterable, this can introduce ambiguities in deciding whether the argument is an iterator of separators or a singleton separator. In python#16429, only 'bytes' was considered a singleton, but this will break code that passes other buffer object types. The Python library docs don't indicate what separator types were permitted in Python <=3.12, but comments in typeshed indicate that it would work with types that implement the buffer protocol and provide a len(). To keep those cases working the way they did before, I've changed the detection logic to consider any instance of collections.abc.Buffer as a singleton separator. There may still be corner cases where this doesn't do what the user wants e.g. a numpy array of byte strings will implement the buffer protocol and hence be treated as a singleton; but at least those corner cases should behave the same in 3.13 as they did in 3.12. Relates to python#81322.
1 parent ed785c0 commit c364ee8

File tree

2 files changed

+9
-2
lines changed

2 files changed

+9
-2
lines changed

Lib/asyncio/streams.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
'StreamReader', 'StreamWriter', 'StreamReaderProtocol',
33
'open_connection', 'start_server')
44

5-
import collections
5+
import collections.abc
66
import socket
77
import sys
88
import warnings
@@ -597,7 +597,7 @@ async def readuntil(self, separator=b'\n'):
597597
the shortest possible separator is considered to be the one that
598598
matched.
599599
"""
600-
if isinstance(separator, bytes):
600+
if isinstance(separator, collections.abc.Buffer):
601601
separator = [separator]
602602
else:
603603
# Makes sure shortest matches wins, and supports arbitrary iterables

Lib/test/test_asyncio/test_streams.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -512,6 +512,13 @@ def test_readuntil_multi_separator_negative_offset(self):
512512
self.assertEqual(b'dataZA', data)
513513
self.assertEqual(b'aaa', stream._buffer)
514514

515+
def test_readuntil_bytearray(self):
516+
stream = asyncio.StreamReader(loop=self.loop)
517+
stream.feed_data(b'some data\r\n')
518+
data = self.loop.run_until_complete(stream.readuntil(bytearray(b'\r\n')))
519+
self.assertEqual(b'some data\r\n', data)
520+
self.assertEqual(b'', stream._buffer)
521+
515522
def test_readexactly_zero_or_less(self):
516523
# Read exact number of bytes (zero or less).
517524
stream = asyncio.StreamReader(loop=self.loop)

0 commit comments

Comments
 (0)