Skip to content

PHPLIB-598: Clean up cursor usages to account for iterator changes #801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 20, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -435,17 +435,13 @@ axes:
display_name: Driver Version
values:
- id: "lowest-supported"
display_name: "1.8.1"
display_name: "1.9.0"
variables:
DRIVER_VERSION: "1.8.1"
DRIVER_VERSION: "1.9.0"
- id: "latest-stable"
display_name: "1.9-stable"
variables:
DRIVER_VERSION: "stable"
- id: "1.8-dev"
display_name: "1.8-dev"
variables:
DRIVER_BRANCH: "v1.8"
- id: "1.9-dev"
display_name: "1.9-dev"
variables:
Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"php": "^7.0 || ^8.0",
"ext-hash": "*",
"ext-json": "*",
"ext-mongodb": "^1.8.1",
"ext-mongodb": "^1.9.0",
"jean85/pretty-package-versions": "^1.2",
"symfony/polyfill-php80": "^1.19"
},
Expand Down
59 changes: 28 additions & 31 deletions docs/tutorial/tailable-cursor.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Overview
When the driver executes a query or command (e.g.
:manual:`aggregate </reference/command/aggregate>`), results from the operation
are returned via a :php:`MongoDB\\Driver\\Cursor <class.mongodb-driver-cursor>`
object. The Cursor class implements PHP's :php:`Traversable <traversable>`
object. The Cursor class implements PHP's :php:`Iterator <iterator>`
interface, which allows it to be iterated with ``foreach`` and interface with
any PHP functions that work with :php:`iterables <types.iterable>`. Similar to
result objects in other database drivers, cursors in MongoDB only support
Expand All @@ -35,20 +35,21 @@ While normal cursors can be iterated once with ``foreach``, that approach will
not work with tailable cursors. When ``foreach`` is used with a tailable cursor,
the loop will stop upon reaching the end of the initial result set. Attempting
to continue iteration on the cursor with a second ``foreach`` would throw an
exception, since PHP attempts to rewind the cursor.
exception, since PHP attempts to rewind the cursor. Therefore, reading from a
tailable cursor will require direct usage of the :php:`Iterator <iterator>` API.

In order to continuously read from a tailable cursor, we will need to wrap the
Cursor object with an :php:`IteratorIterator <iteratoriterator>`. This will
allow us to directly control the cursor's iteration (e.g. call ``next()``),
avoid inadvertently rewinding the cursor, and decide when to wait for new
results or stop iteration entirely.
.. note::

Before version 1.9.0 of the ``ext-mongodb`` extension, the cursor class does
not implement the :php:`Iterator <iterator>` interface. To manually iterate
a cursor using the method below, it must first be wrapped with an
:php:`IteratorIterator <iteratoriterator>`.

Wrapping a Normal Cursor
------------------------
Manually Iterating a Normal Cursor
----------------------------------

Before looking at how a tailable cursor can be wrapped with
:php:`IteratorIterator <iteratoriterator>`, we'll start by examining how the
class interacts with a normal cursor.
Before looking at how a tailable cursor can be iterated, we'll start by
examining how the ``Iterator`` methods interact with a normal cursor.

The following example finds five restaurants and uses ``foreach`` to view the
results:
Expand All @@ -74,7 +75,7 @@ not occurred, the iterator then advances to the next position, control jumps
back to the validity check, and the loop continues.

With the inner workings of ``foreach`` under our belt, we can now translate the
preceding example to use IteratorIterator:
preceding example to use the Iterator methods directly:

.. code-block:: php

Expand All @@ -84,28 +85,26 @@ preceding example to use IteratorIterator:

$cursor = $collection->find([], ['limit' => 5]);

$iterator = new IteratorIterator($cursor);
$cursor->rewind();

$iterator->rewind();

while ($iterator->valid()) {
$document = $iterator->current();
while ($cursor->valid()) {
$document = $cursor->current();
var_dump($document);
$iterator->next();
$cursor->next();
}

.. note::

Calling ``$iterator->next()`` after the ``while`` loop naturally ends would
Calling ``$cursor->next()`` after the ``while`` loop naturally ends would
throw an exception, since all results on the cursor have been exhausted.

The purpose of this example is simply to demonstrate the functional equivalence
between ``foreach`` and manual iteration with PHP's :php:`Iterator <iterator>`
API. For normal cursors, there is little reason to use IteratorIterator instead
of a concise ``foreach`` loop.
API. For normal cursors, there is little reason to manually iterate results
instead of a concise ``foreach`` loop.

Wrapping a Tailable Cursor
--------------------------
Iterating a Tailable Cursor
---------------------------

In order to demonstrate a tailable cursor in action, we'll need two scripts: a
"producer" and a "consumer". The producer script will create a new capped
Expand Down Expand Up @@ -154,7 +153,7 @@ If you execute this consumer script, you'll notice that it quickly exhausts all
results in the capped collection and then terminates. We cannot add a second
``foreach``, as that would throw an exception when attempting to rewind the
cursor. This is a ripe use case for directly controlling the iteration process
using :php:`IteratorIterator <iteratoriterator>`.
using the :php:`Iterator <iterator>` interface.

.. code-block:: php

Expand All @@ -167,17 +166,15 @@ using :php:`IteratorIterator <iteratoriterator>`.
'maxAwaitTimeMS' => 100,
]);

$iterator = new IteratorIterator($cursor);

$iterator->rewind();
$cursor->rewind();

while (true) {
if ($iterator->valid()) {
$document = $iterator->current();
if ($cursor->valid()) {
$document = $cursor->current();
printf("Consumed document created at: %s\n", $document->createdAt);
}

$iterator->next();
$cursor->next();
}

Much like the ``foreach`` example, this version on the consumer script will
Expand Down
8 changes: 3 additions & 5 deletions src/GridFS/ReadableStream.php
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

namespace MongoDB\GridFS;

use IteratorIterator;
use MongoDB\Driver\CursorInterface;
use MongoDB\Exception\InvalidArgumentException;
use MongoDB\GridFS\Exception\CorruptFileException;
use stdClass;
Expand Down Expand Up @@ -48,7 +48,7 @@ class ReadableStream
/** @var integer */
private $chunkOffset = 0;

/** @var IteratorIterator|null */
/** @var CursorInterface|null */
private $chunksIterator;

/** @var CollectionWrapper */
Expand Down Expand Up @@ -315,9 +315,7 @@ private function initBufferFromNextChunk()
*/
private function initChunksIterator()
{
$cursor = $this->collectionWrapper->findChunksByFileId($this->file->_id, $this->chunkOffset);

$this->chunksIterator = new IteratorIterator($cursor);
$this->chunksIterator = $this->collectionWrapper->findChunksByFileId($this->file->_id, $this->chunkOffset);
$this->chunksIterator->rewind();
}
}
8 changes: 4 additions & 4 deletions src/Model/CachingIterator.php
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class CachingIterator implements Countable, Iterator
/** @var array */
private $items = [];

/** @var IteratorIterator */
/** @var Iterator */
private $iterator;

/** @var boolean */
Expand All @@ -53,14 +53,14 @@ class CachingIterator implements Countable, Iterator
/**
* Initialize the iterator and stores the first item in the cache. This
* effectively rewinds the Traversable and the wrapping IteratorIterator.
* Additionally, this mimics behavior of the SPL iterators and allows users
* to omit an explicit call * to rewind() before using the other methods.
* Additionally, this mimics behavior of the SPL iterators and allows users
* to omit an explicit call to rewind() before using the other methods.
*
* @param Traversable $traversable
*/
public function __construct(Traversable $traversable)
{
$this->iterator = new IteratorIterator($traversable);
$this->iterator = $traversable instanceof Iterator ? $traversable : new IteratorIterator($traversable);

$this->iterator->rewind();
$this->storeCurrentItem();
Expand Down
4 changes: 2 additions & 2 deletions src/Model/CallbackIterator.php
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,12 @@ class CallbackIterator implements Iterator
/** @var Closure */
private $callback;

/** @var IteratorIterator */
/** @var Iterator */
private $iterator;

public function __construct(Traversable $traversable, Closure $callback)
{
$this->iterator = new IteratorIterator($traversable);
$this->iterator = $traversable instanceof Iterator ? $traversable : new IteratorIterator($traversable);
$this->callback = $callback;
}

Expand Down
5 changes: 2 additions & 3 deletions tests/Collection/CrudSpecFunctionalTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

namespace MongoDB\Tests\Collection;

use IteratorIterator;
use LogicException;
use MongoDB\BulkWriteResult;
use MongoDB\Collection;
Expand Down Expand Up @@ -105,8 +104,8 @@ public function provideSpecificationTests()
private function assertEquivalentCollections($expectedCollection, $actualCollection)
{
$mi = new MultipleIterator(MultipleIterator::MIT_NEED_ANY);
$mi->attachIterator(new IteratorIterator($expectedCollection->find()));
$mi->attachIterator(new IteratorIterator($actualCollection->find()));
$mi->attachIterator($expectedCollection->find());
$mi->attachIterator($actualCollection->find());

foreach ($mi as $documents) {
list($expectedDocument, $actualDocument) = $documents;
Expand Down
5 changes: 2 additions & 3 deletions tests/GridFS/SpecFunctionalTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
namespace MongoDB\Tests\GridFS;

use DateTime;
use IteratorIterator;
use LogicException;
use MongoDB\BSON\Binary;
use MongoDB\BSON\ObjectId;
Expand Down Expand Up @@ -121,8 +120,8 @@ public function provideSpecificationTests()
private function assertEquivalentCollections($expectedCollection, $actualCollection, $actualResult)
{
$mi = new MultipleIterator(MultipleIterator::MIT_NEED_ANY);
$mi->attachIterator(new IteratorIterator($expectedCollection->find()));
$mi->attachIterator(new IteratorIterator($actualCollection->find()));
$mi->attachIterator($expectedCollection->find());
$mi->attachIterator($actualCollection->find());

foreach ($mi as $documents) {
list($expectedDocument, $actualDocument) = $documents;
Expand Down
18 changes: 8 additions & 10 deletions tests/Operation/FindFunctionalTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

namespace MongoDB\Tests\Operation;

use IteratorIterator;
use MongoDB\Driver\BulkWrite;
use MongoDB\Driver\ReadPreference;
use MongoDB\Operation\CreateCollection;
Expand Down Expand Up @@ -183,40 +182,39 @@ public function testMaxAwaitTimeMS()

$operation = new Find($databaseName, $cappedCollectionName, [], ['cursorType' => Find::TAILABLE_AWAIT, 'maxAwaitTimeMS' => $maxAwaitTimeMS]);
$cursor = $operation->execute($this->getPrimaryServer());
$it = new IteratorIterator($cursor);

/* The initial query includes the one and only document in its result
* batch, so we should not expect a delay. */
$startTime = microtime(true);
$it->rewind();
$cursor->rewind();
$duration = microtime(true) - $startTime;
$this->assertLessThan($pivot, $duration);

$this->assertTrue($it->valid());
$this->assertSameDocument(['_id' => 1], $it->current());
$this->assertTrue($cursor->valid());
$this->assertSameDocument(['_id' => 1], $cursor->current());

/* Advancing again takes us to the last document of the result batch,
* but still should not issue a getMore */
$startTime = microtime(true);
$it->next();
$cursor->next();
$duration = microtime(true) - $startTime;
$this->assertLessThan($pivot, $duration);

$this->assertTrue($it->valid());
$this->assertSameDocument(['_id' => 2], $it->current());
$this->assertTrue($cursor->valid());
$this->assertSameDocument(['_id' => 2], $cursor->current());

/* Now that we've reached the end of the initial result batch, advancing
* again will issue a getMore. Expect to wait at least maxAwaitTimeMS,
* since no new documents should be inserted to wake up the server's
* query thread. Also ensure we don't wait too long (server default is
* one second). */
$startTime = microtime(true);
$it->next();
$cursor->next();
$duration = microtime(true) - $startTime;
$this->assertGreaterThan($pivot, $duration);
$this->assertLessThan(0.5, $duration);

$this->assertFalse($it->valid());
$this->assertFalse($cursor->valid());
}

public function testReadPreferenceWithinTransaction()
Expand Down
3 changes: 1 addition & 2 deletions tests/SpecTests/FunctionalTestCase.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
namespace MongoDB\Tests\SpecTests;

use ArrayIterator;
use IteratorIterator;
use LogicException;
use MongoDB\Collection;
use MongoDB\Driver\Server;
Expand Down Expand Up @@ -106,7 +105,7 @@ protected function assertOutcomeCollectionData(array $expectedDocuments, $result

$mi = new MultipleIterator(MultipleIterator::MIT_NEED_ANY);
$mi->attachIterator(new ArrayIterator($expectedDocuments));
$mi->attachIterator(new IteratorIterator($outcomeCollection->find([], ['sort' => ['_id' => 1]])));
$mi->attachIterator($outcomeCollection->find([], ['sort' => ['_id' => 1]]));

foreach ($mi as $documents) {
list($expectedDocument, $actualDocument) = $documents;
Expand Down
16 changes: 7 additions & 9 deletions tests/SpecTests/PrimaryStepDownSpecTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

namespace MongoDB\Tests\SpecTests;

use IteratorIterator;
use MongoDB\Client;
use MongoDB\Collection;
use MongoDB\Driver\Command;
Expand Down Expand Up @@ -205,12 +204,11 @@ public function testGetMoreIteration()
// Start a find operation on the collection with a batch size of 2, and retrieve the first batch of results.
$cursor = $this->collection->find([], ['batchSize' => 2]);

$iterator = new IteratorIterator($cursor);
$iterator->rewind();
$this->assertTrue($iterator->valid());
$cursor->rewind();
$this->assertTrue($cursor->valid());

$iterator->next();
$this->assertTrue($iterator->valid());
$cursor->next();
$this->assertTrue($cursor->valid());

$totalConnectionsCreated = $this->getTotalConnectionsCreated();

Expand All @@ -235,14 +233,14 @@ public function testGetMoreIteration()
$events = [];
$observer = new CommandObserver();
$observer->observe(
function () use ($iterator) {
$iterator->next();
function () use ($cursor) {
$cursor->next();
},
function ($event) use (&$events) {
$events[] = $event;
}
);
$this->assertTrue($iterator->valid());
$this->assertTrue($cursor->valid());
$this->assertCount(1, $events);
$this->assertSame('getMore', $events[0]['started']->getCommandName());

Expand Down
3 changes: 1 addition & 2 deletions tests/UnifiedSpecTests/CollectionData.php
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
namespace MongoDB\Tests\UnifiedSpecTests;

use ArrayIterator;
use IteratorIterator;
use MongoDB\Client;
use MongoDB\Driver\ReadConcern;
use MongoDB\Driver\ReadPreference;
Expand Down Expand Up @@ -75,7 +74,7 @@ public function assertOutcome(Client $client)

$mi = new MultipleIterator(MultipleIterator::MIT_NEED_ANY);
$mi->attachIterator(new ArrayIterator($this->documents));
$mi->attachIterator(new IteratorIterator($cursor));
$mi->attachIterator($cursor);

foreach ($mi as $i => $documents) {
list($expectedDocument, $actualDocument) = $documents;
Expand Down