Skip to content

Add transaction support #1904

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 51 commits into from
Closed
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
e54408a
add transaction support, requires mongodb version V4.0 or more and de…
klinson Jan 8, 2020
9a01d23
format code
klinson Jan 8, 2020
64b4156
add mongodb(replica set) environment into docker configuration
klinson Jan 9, 2020
34b1a98
must be compatible with Illuminate\Database\Connection::rollBack($toL…
klinson Jan 9, 2020
84850a9
add transaction usage into readme.md
klinson Jan 9, 2020
bedaaa8
update .travis.yml
klinson Jan 10, 2020
b030914
update database.conf
klinson Jan 13, 2020
163b41d
update .travis.yml
klinson Jan 13, 2020
92c1e07
update docker-compose.yml
klinson Jan 13, 2020
caa61ea
Update src/Jenssegers/Mongodb/Connection.php
klinson Jan 17, 2020
7d96c3e
fixbug Connection.php
klinson Feb 7, 2020
776492d
Merge remote-tracking branch 'remotes/upstream/master'
klinson Feb 7, 2020
10bda05
add transaction into README.md
klinson Feb 7, 2020
85a0be8
add replset mongodb tag
klinson Feb 7, 2020
b39251c
Remove travis from PR please
klinson Feb 9, 2020
fe61056
remove author tag and comments in Chinese language
klinson Feb 13, 2020
3029e36
improved function docblocks of transaction
klinson Feb 13, 2020
4ce523f
merge update
klinson Feb 15, 2020
44cb7a7
Merge remote-tracking branch 'upstream/master'
klinson Jun 28, 2020
7221417
add another testsuite for transactional tests in phpunit.xml
klinson Jun 28, 2020
1b90047
optimized code
klinson Jul 1, 2020
a5d0858
update ci and tests configs
klinson Jul 21, 2020
add9516
update tests configs
klinson Jul 21, 2020
14b3ad7
update tests configs
klinson Jul 21, 2020
0c22e4a
update ci.yml
klinson Jul 21, 2020
0b840db
update tests
klinson Jul 21, 2020
cf88a03
update tests
klinson Jul 21, 2020
2664bd5
update tests
klinson Jul 21, 2020
44bd081
update tests
klinson Jul 21, 2020
850d034
Merge branch 'master' into master
klinson Jul 21, 2020
83f45c9
delete links of docker-compose.yaml
klinson Jul 23, 2020
5eb6f42
Merge remote-tracking branch 'origin/master'
klinson Jul 23, 2020
84186f1
Merge branch 'master' into master
divine Jul 23, 2020
7741acb
optimize code
klinson Jul 23, 2020
72dbdcb
Merge remote-tracking branch 'origin/master'
klinson Jul 23, 2020
c14bd44
Merge branch 'master' into master
divine Aug 28, 2020
5773597
Merge branch 'master' into master
divine Sep 9, 2020
9e0cfd2
Merge branch 'master' into master
divine Sep 16, 2020
b2d1740
Merge branch 'master' into master
divine Oct 14, 2020
8cc01d5
Merge branch 'master' into master
klinson Nov 20, 2020
dafee61
remove testInsertWithId
klinson Nov 20, 2020
0fd27ba
update build-ci.yml
klinson Nov 23, 2020
6e89c8b
add return info to phpdoc
klinson Nov 23, 2020
5645fc7
format code
klinson Feb 19, 2021
9cd8bb2
update README.md
klinson Feb 19, 2021
551c184
Merge remote-tracking branch 'upstream/master'
klinson Feb 19, 2021
49307e2
remove excess $option
klinson Mar 27, 2021
59020fb
divide testUpdate on two methods testUpdateWithRollback and testUpda…
klinson Mar 27, 2021
97f9b4b
update ci part
klinson Mar 27, 2021
9c90125
update ci part
klinson Mar 27, 2021
041d02b
Merge remote-tracking branch 'upstream/master'
klinson Mar 27, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,11 @@ cache:

install:
- docker version
- sudo pip install docker-compose
- sudo curl -L "https://github.com/docker/compose/releases/download/1.25.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
- docker-compose version
- docker-compose build --build-arg PHP_VERSION=${PHP_VERSION}
- docker-compose run --rm tests composer install --no-interaction

script:
- docker-compose run --rm tests ./vendor/bin/phpunit --coverage-clover ./clover.xml
- docker-compose up -d
- docker-compose exec tests ./vendor/bin/phpunit
5 changes: 5 additions & 0 deletions MongoReplsetDockerFile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
FROM mongo

ADD mongo-replset-init.sh /usr/local/bin/

RUN chmod +x /usr/local/bin/mongo-replset-init.sh
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Table of contents
* [Eloquent](#eloquent)
* [Optional: Alias](#optional-alias)
* [Query Builder](#query-builder)
* [Transaction](#transaction)
* [Schema](#schema)
* [Extensions](#extensions)
* [Troubleshooting](#troubleshooting)
Expand Down Expand Up @@ -241,6 +242,26 @@ $user = DB::connection('mongodb')->collection('users')->get();

Read more about the query builder on http://laravel.com/docs/queries

Transaction
-------------
Transaction requires mongodb version V4.0 or more and deployment replica sets or sharded clusters.

Transaction supports create/insert,update,delete,etc operation.

Transaction supports infinite-level nested transactions, but outside transaction rollbacks do not affect the commit of inside transactions.
```php
DB::beginTransaction();

User::create(['name' => 'klinson', 'age' => 20, 'title' => 'admin']);

DB::transaction(function () {
    DB::collection('users')->where('name', 'klinson')->update(['age' => 20]);
});

DB::rollBack();
//DB::commit();
```

Schema
------

Expand Down
48 changes: 48 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ services:
depends_on:
- mongodb
- mysql
- mongo1
- mongo2
- mongo3
stdin_open: true
tty: true

mysql:
container_name: mysql
Expand All @@ -30,3 +35,46 @@ services:
- 27017:27017
logging:
driver: none

mongo1:
hostname: mongo1
container_name: mongo1
image: mongo
restart: always
expose:
- 27017
ports:
- 27018:27017
command: '--quiet --bind_ip_all --replSet rs0'

mongo2:
hostname: mongo2
container_name: mongo2
image: mongo
restart: always
expose:
- 27017
ports:
- 27019:27017
command: '--quiet --bind_ip_all --replSet rs0'

mongo3:
hostname: mongo3
container_name: mongo3
image: mongo
restart: always
expose:
- 27017
ports:
- 27020:27017
command: '--quiet --bind_ip_all --replSet rs0'

mongoreplret:
build:
context: .
dockerfile: MongoReplsetDockerFile
depends_on:
- mongo1
- mongo2
- mongo3
entrypoint: ["sh", "-c", "mongo-replset-init.sh"]
23 changes: 23 additions & 0 deletions mongo-replset-init.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash

echo "prepare rs initiating"

check_db_status() {
mongo1=$(mongo --host mongo1 --port 27017 --eval "db.stats().ok" | tail -n1 | grep -E '(^|\s)1($|\s)')
mongo2=$(mongo --host mongo2 --port 27017 --eval "db.stats().ok" | tail -n1 | grep -E '(^|\s)1($|\s)')
mongo3=$(mongo --host mongo3 --port 27017 --eval "db.stats().ok" | tail -n1 | grep -E '(^|\s)1($|\s)')
if [[ $mongo1 == 1 ]] && [[ $mongo2 == 1 ]] && [[ $mongo3 == 1 ]]; then
init_rs
else
check_db_status
fi
}

init_rs() {
ret=$(mongo --host mongo1 --port 27017 --eval "rs.initiate({ _id: 'rs0', members: [{ _id: 0, host: 'mongo1:27017' }, { _id: 1, host: 'mongo2:27017' }, { _id: 2, host: 'mongo3:27017' } ] })" > /dev/null 2>&1)
}

check_db_status > /dev/null 2>&1

echo "rs initiating finished"
exit 0
83 changes: 83 additions & 0 deletions src/Jenssegers/Mongodb/Connection.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
use Illuminate\Database\Connection as BaseConnection;
use Illuminate\Support\Arr;
use MongoDB\Client;
use MongoDB\Driver\ReadConcern;
use MongoDB\Driver\ReadPreference;
use MongoDB\Driver\WriteConcern;

class Connection extends BaseConnection
{
Expand All @@ -20,6 +23,9 @@ class Connection extends BaseConnection
*/
protected $connection;

protected $session_key; // sessions会话列表当前会话数组key 随机生成
protected $sessions = []; // 会话列表

/**
* Create a new database connection instance.
* @param array $config
Expand Down Expand Up @@ -254,4 +260,81 @@ public function __call($method, $parameters)
{
return call_user_func_array([$this->db, $method], $parameters);
}

/**
* create a session and start a transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* create a session and start a transaction in session
* Create a session and start a transaction in session

*
* In version 4.0, MongoDB supports multi-document transactions on replica sets.
* In version 4.2, MongoDB introduces distributed transactions, which adds support for multi-document transactions on sharded clusters and incorporates the existing support for multi-document transactions on replica sets.
* To use transactions on MongoDB 4.2 deployments(replica sets and sharded clusters), clients must use MongoDB drivers updated for MongoDB 4.2.
*
* @see https://docs.mongodb.com/manual/core/transactions/
* @author klinson <[email protected]>
*/
public function beginTransaction()
{
$this->session_key = uniqid();
$this->sessions[$this->session_key] = $this->connection->startSession();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the session from the connection instance is applied automatically to all operations sent through the query builder, I'd refrain from starting multiple sessions. Looking at the transaction logic used for PDO, the first call to beginTransaction will start a new transaction, while subsequent only create save points (or no-op if those are not supported). I generally believe this to be a better alternative that should be used instead.


$this->sessions[$this->session_key]->startTransaction([
'readPreference' => new ReadPreference(ReadPreference::RP_PRIMARY),
'writeConcern' => new WriteConcern(1),
'readConcern' => new ReadConcern(ReadConcern::LOCAL)
Comment on lines +311 to +313
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would avoid hardcoding these options this way, especially with the given values. There is no prior art for passing options (as most methods in the query builder don't accept any options), but an optional $transactionOptions parameter for this method would be more sensible than hardcoding these values here.

]);
}

/**
* commit transaction in this session and close this session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* commit transaction in this session and close this session
* Commit transaction in this session and close this session

* @author klinson <[email protected]>
*/
public function commit()
{
if ($session = $this->getSession()) {
$session->commitTransaction();
$this->setLastSession();
}
}

/**
* rollback transaction in this session and close this session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* rollback transaction in this session and close this session
* Rollback transaction in this session and close this session

* @author klinson <[email protected]>
*/
public function rollBack($toLevel = null)
{
if ($session = $this->getSession()) {
$session->abortTransaction();
$this->setLastSession();
}
}

/**
* close this session and get last session key to session_key
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* close this session and get last session key to session_key
* Close this session and get last session key to session_key

* Why do it ? Because nested transactions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Why do it ? Because nested transactions
* Required due to nested transactions

* @author klinson <[email protected]>
*/
protected function setLastSession()
{
if ($session = $this->getSession()) {
$session->endSession();
unset($this->sessions[$this->session_key]);
if (empty($this->sessions)) {
$this->session_key = null;
} else {
end($this->sessions);
$this->session_key = key($this->sessions);
}
}
}

/**
* get now session if it has session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* get now session if it has session
* Get session if it exists

* @return \MongoDB\Driver\Session|null
* @author klinson <[email protected]>
*/
public function getSession()
{
return $this->sessions[$this->session_key] ?? null;
? $this->sessions[$this->session_key]
: null;
}
}
68 changes: 58 additions & 10 deletions src/Jenssegers/Mongodb/Query/Builder.php
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,7 @@ public function getFresh($columns = [])
}
}
}

// The _id field is mandatory when using grouping.
if ($group && empty($group['_id'])) {
$group['_id'] = null;
Expand Down Expand Up @@ -338,6 +338,11 @@ public function getFresh($columns = [])
$options = array_merge($options, $this->options);
}

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}

// Execute aggregation
$results = iterator_to_array($this->collection->aggregate($pipeline, $options));

Expand All @@ -348,13 +353,15 @@ public function getFresh($columns = [])
// Return distinct results directly
$column = isset($this->columns[0]) ? $this->columns[0] : '_id';

// Execute distinct
if ($wheres) {
$result = $this->collection->distinct($column, $wheres);
} else {
$result = $this->collection->distinct($column);
$options = [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$options = [];

Please remove.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@klinson can you remove this line?

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}

// Execute distinct
$result = $this->collection->distinct($column, $wheres ?: [], $options);

return $this->useCollections ? new Collection($result) : $result;
} // Normal query
else {
Expand Down Expand Up @@ -397,6 +404,11 @@ public function getFresh($columns = [])
$options = array_merge($options, $this->options);
}

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}

// Execute query and get MongoCursor
$cursor = $this->collection->find($wheres, $options);

Expand Down Expand Up @@ -561,7 +573,12 @@ public function insert(array $values)
}

// Batch insert
$result = $this->collection->insertMany($values);
$options = [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$options = [];

Please remove.

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}
$result = $this->collection->insertMany($values, $options);

return (1 == (int) $result->isAcknowledged());
}
Expand All @@ -571,7 +588,12 @@ public function insert(array $values)
*/
public function insertGetId(array $values, $sequence = null)
{
$result = $this->collection->insertOne($values);
$options = [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Common code for many methods, better move this logic in special method used in many places

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@klinson see on this comment, please

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$options = [];

Please remove.

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}
$result = $this->collection->insertOne($values, $options);

if (1 == (int) $result->isAcknowledged()) {
if ($sequence === null) {
Expand All @@ -592,6 +614,10 @@ public function update(array $values, array $options = [])
if (!Str::startsWith(key($values), '$')) {
$values = ['$set' => $values];
}
// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}

return $this->performUpdate($values, $options);
}
Expand All @@ -614,6 +640,11 @@ public function increment($column, $amount = 1, array $extra = [], array $option
$query->orWhereNotNull($column);
});

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}

return $this->performUpdate($query, $options);
}

Expand Down Expand Up @@ -673,7 +704,14 @@ public function delete($id = null)
}

$wheres = $this->compileWheres();
$result = $this->collection->DeleteMany($wheres);

$options = [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$options = [];

Please remove.

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}

$result = $this->collection->DeleteMany($wheres, $options);
if (1 == (int) $result->isAcknowledged()) {
return $result->getDeletedCount();
}
Expand All @@ -698,7 +736,12 @@ public function from($collection, $as = null)
*/
public function truncate()
{
$result = $this->collection->drop();
$options = [];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$options = [];

Please remove.

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

Copy link

@ymgsmz ymgsmz Jan 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exist exists

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}
$result = $this->collection->drop($options);

return (1 == (int) $result->ok);
}
Expand Down Expand Up @@ -826,6 +869,11 @@ protected function performUpdate($query, array $options = [])
$options['multiple'] = true;
}

// if transaction in session
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// if transaction in session
// Check if transaction exist in session

if ($session = $this->connection->getSession()) {
$options['session'] = $session;
}

$wheres = $this->compileWheres();
$result = $this->collection->UpdateMany($wheres, $query, $options);
if (1 == (int) $result->isAcknowledged()) {
Expand Down
1 change: 1 addition & 0 deletions tests/TestCase.php
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ protected function getEnvironmentSetUp($app)
$app['config']->set('database.connections.mongodb', $config['connections']['mongodb']);
$app['config']->set('database.connections.mongodb2', $config['connections']['mongodb']);
$app['config']->set('database.connections.dsn_mongodb', $config['connections']['dsn_mongodb']);
$app['config']->set('database.connections.mongodb_replset', $config['connections']['mongodb_replset']);

$app['config']->set('auth.model', 'User');
$app['config']->set('auth.providers.users.model', 'User');
Expand Down
Loading