Merge pull request #364 from seperman/dev

v6.2.3

Author: seperman

AUTHORS.md

@@ -48,3 +48,5 @@ Authors in order of the timeline of their contributions:
 - [dtorres-sf](https://github.com/dtorres-sf) for the fix for diffing using iterable_compare_func with nested objects.
 - [Enric Pou](https://github.com/epou) for bug fix of ValueError when using Decimal 0.x
 - [Uwe Fladrich](https://github.com/uwefladrich) for fixing bug when diff'ing non-sequence iterables
+- [Michal Ozery-Flato](https://github.com/michalozeryflato) for setting equal_nan=ignore_nan_inequality in the call for np.array_equal
+- [martin-kokos](https://github.com/martin-kokos) for using Pytest's tmp_path fixture instead of /tmp/

CHANGELOG.md

@@ -1,5 +1,9 @@
 # DeepDiff Change log
 
+- v6-2-3
+    - Switching to Orjson for serialization to improve the performance.
+    - Setting `equal_nan=ignore_nan_inequality` in the call for `np.array_equal`
+    - Using Pytest's tmp_path fixture instead of `/tmp/`
 - v6-2-2
     - Enum test fix for python 3.11
     - Adding support for dateutils rrules

MANIFEST.in

@@ -9,7 +9,11 @@ include pytest.ini
 include *.py
 recursive-include docs/ *.rst
 recursive-include docs/ *.png
-recursive-include tests *.py
+recursive-include tests *.csv
 recursive-include tests *.json
+recursive-include tests *.pickle
+recursive-include tests *.py
+recursive-include tests *.toml
+recursive-include tests *.yaml
 global-exclude __pycache__
 global-exclude *.py[co]

README.md

@@ -1,4 +1,4 @@
-# DeepDiff v 6.2.2
+# DeepDiff v 6.2.3
 
 ![Downloads](https://img.shields.io/pypi/dm/deepdiff.svg?style=flat)
 ![Python Versions](https://img.shields.io/pypi/pyversions/deepdiff.svg?style=flat)
@@ -14,10 +14,12 @@
 
 Tested on Python 3.7+ and PyPy3.
 
-- **[Documentation](https://zepworks.com/deepdiff/6.2.2/)**
+- **[Documentation](https://zepworks.com/deepdiff/6.2.3/)**
 
 ## What is new?
 
+Please check the [ChangeLog](CHANGELOG.md) file for the detailed information.
+
 DeepDiff 6-2-0
 
 - Major improvement in the diff report for lists when items are all hashable and the order of items is important.
@@ -69,13 +71,13 @@ Note: if you want to use DeepDiff via commandline, make sure to run `pip install
 
 DeepDiff gets the difference of 2 objects.
 
-> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/6.2.2/diff.html)
-> - The full documentation of all modules can be found on <https://zepworks.com/deepdiff/6.2.2/>
+> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/6.2.3/diff.html)
+> - The full documentation of all modules can be found on <https://zepworks.com/deepdiff/6.2.3/>
 > - Tutorials and posts about DeepDiff can be found on <https://zepworks.com/tags/deepdiff/>
 
 ## A few Examples
 
-> Note: This is just a brief overview of what DeepDiff can do. Please visit <https://zepworks.com/deepdiff/6.2.2/> for full documentation.
+> Note: This is just a brief overview of what DeepDiff can do. Please visit <https://zepworks.com/deepdiff/6.2.3/> for full documentation.
 
 ### List difference ignoring order or duplicates
 
@@ -279,8 +281,8 @@ Example:
 ```
 
 
-> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/6.2.2/diff.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.2/>
+> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/6.2.3/diff.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.3/>
 
 
 # Deep Search
@@ -312,17 +314,17 @@ And you can pass all the same kwargs as DeepSearch to grep too:
 {'matched_paths': {"root['somewhere']": 'around'}, 'matched_values': {"root['long']": 'somewhere'}}
 ```
 
-> - Please take a look at the [DeepSearch docs](https://zepworks.com/deepdiff/6.2.2/dsearch.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.2/>
+> - Please take a look at the [DeepSearch docs](https://zepworks.com/deepdiff/6.2.3/dsearch.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.3/>
 
 # Deep Hash
 (New in v4-0-0)
 
 DeepHash is designed to give you hash of ANY python object based on its contents even if the object is not considered hashable!
 DeepHash is supposed to be deterministic in order to make sure 2 objects that contain the same data, produce the same hash.
 
-> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/6.2.2/deephash.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.2/>
+> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/6.2.3/deephash.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.3/>
 
 Let's say you have a dictionary object.
 
@@ -370,8 +372,8 @@ Which you can write as:
 At first it might seem weird why DeepHash(obj)[obj] but remember that DeepHash(obj) is a dictionary of hashes of all other objects that obj contains too.
 
 
-> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/6.2.2/deephash.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.2/>
+> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/6.2.3/deephash.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.3/>
 
 
 # Using DeepDiff in unit tests
@@ -451,11 +453,11 @@ Thank you!
 
 How to cite this library (APA style):
 
-    Dehpour, S. (2022). DeepDiff (Version 6.2.2) [Software]. Available from https://github.com/seperman/deepdiff.
+    Dehpour, S. (2022). DeepDiff (Version 6.2.3) [Software]. Available from https://github.com/seperman/deepdiff.
 
 How to cite this library (Chicago style):
 
-    Dehpour, Sep. 2022. DeepDiff (version 6.2.2).
+    Dehpour, Sep. 2022. DeepDiff (version 6.2.3).
 
 # Authors
 

deepdiff/__init__.py

@@ -1,6 +1,6 @@
 """This module offers the DeepDiff, DeepSearch, grep, Delta and DeepHash classes."""
 # flake8: noqa
-__version__ = '6.2.2'
+__version__ = '6.2.3'
 import logging
 
 if __name__ == '__main__':

deepdiff/diff.py

@@ -1366,7 +1366,7 @@ def _diff_numpy_array(self, level, parents_ids=frozenset(), local_tree=None):
         if not self.ignore_order_func(level):
             # fast checks
             if self.significant_digits is None:
-                if np.array_equal(level.t1, level.t2):
+                if np.array_equal(level.t1, level.t2, equal_nan=self.ignore_nan_inequality):
                     return  # all good
             else:
                 try:

deepdiff/serialization.py

@@ -3,6 +3,7 @@
 import io
 import os
 import json
+import orjson
 import uuid
 import logging
 import re  # NOQA
@@ -181,7 +182,7 @@ def to_json(self, default_mapping=None, **kwargs):
             '{"type_changes": {"root": {"old_type": "A", "new_type": "B", "old_value": "obj A", "new_value": "obj B"}}}'
         """
         dic = self.to_dict(view_override=TEXT_VIEW)
-        return json.dumps(dic, default=json_convertor_default(default_mapping=default_mapping), **kwargs)
+        return json_dumps(dic, default_mapping=default_mapping, **kwargs)
 
     def to_dict(self, view_override=None):
         """
@@ -410,7 +411,7 @@ def load_path_content(path, file_type=None):
         file_type = path.split('.')[-1]
     if file_type == 'json':
         with open(path, 'r') as the_file:
-            content = json.load(the_file)
+            content = json_loads(the_file.read())
     elif file_type in {'yaml', 'yml'}:
         if yaml is None:  # pragma: no cover.
             raise ImportError('Pyyaml needs to be installed.')  # pragma: no cover.
@@ -474,7 +475,8 @@ def save_content_to_path(content, path, file_type=None, keep_backup=True):
 def _save_content(content, path, file_type, keep_backup=True):
     if file_type == 'json':
         with open(path, 'w') as the_file:
-            content = json.dump(content, the_file)
+            content = json_dumps(content)
+            the_file.write(content)
     elif file_type in {'yaml', 'yml'}:
         if yaml is None:  # pragma: no cover.
             raise ImportError('Pyyaml needs to be installed.')  # pragma: no cover.
@@ -504,8 +506,15 @@ def _save_content(content, path, file_type, keep_backup=True):
     return content
 
 
+def _serialize_decimal(value):
+    if value.as_tuple().exponent == 0:
+        return int(value)
+    else:
+        return float(value)
+
+
 JSON_CONVERTOR = {
-    decimal.Decimal: float,
+    decimal.Decimal: _serialize_decimal,
     ordered_set.OrderedSet: list,
     type: lambda x: x.__name__,
     bytes: lambda x: x.decode('utf-8'),
@@ -552,7 +561,10 @@ def json_dumps(item, default_mapping=None, **kwargs):
     but the output it makes is a byte object and Postgres couldn't directly use it without
     encoding to str. So I switched back to json.
     """
-    return json.dumps(item, default=json_convertor_default(default_mapping=default_mapping), **kwargs)
+    return orjson.dumps(
+        item,
+        default=json_convertor_default(default_mapping=default_mapping),
+        **kwargs).decode(encoding='utf-8')
 
 
 json_loads = partial(json.loads, cls=JSONDecoder)

docs/authors.rst

@@ -61,7 +61,11 @@ Authors in order of the timeline of their contributions:
 -  `Enric Pou <https://github.com/epou>`__ for bug fix of ValueError
    when using Decimal 0.x
 - `Uwe Fladrich <https://github.com/uwefladrich>`__ for fixing bug when diff'ing non-sequence iterables
-
+-  `Michal Ozery-Flato <https://github.com/michalozeryflato>`__ for
+   setting equal_nan=ignore_nan_inequality in the call for
+   np.array_equal
+-  `martin-kokos <https://github.com/martin-kokos>`__ for using Pytest’s
+   tmp_path fixture instead of /tmp/
 
 .. _Sep Dehpour (Seperman): http://www.zepworks.com
 .. _Victor Hahn Castell: http://hahncastell.de

docs/changelog.rst

@@ -5,6 +5,13 @@ Changelog
 
 DeepDiff Changelog
 
+-  v6-2-3
+
+   -  Switching to Orjson for serialization to improve the performance.
+   -  Setting ``equal_nan=ignore_nan_inequality`` in the call for
+      ``np.array_equal``
+   -  Using Pytest’s tmp_path fixture instead of ``/tmp/``
+
 -  v6-2-2
 
    -  Enum test fix for python 3.11

docs/conf.py

@@ -60,9 +60,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '6.2.2'
+version = '6.2.3'
 # The full version, including alpha/beta/rc tags.
-release = '6.2.2'
+release = '6.2.3'
 
 load_dotenv(override=True)
 DOC_VERSION = os.environ.get('DOC_VERSION', version)

docs/index.rst

@@ -4,7 +4,7 @@
    contain the root `toctree` directive.
 
 
-DeepDiff 6.2.2 documentation!
+DeepDiff 6.2.3 documentation!
 =============================
 
 *****************

requirements.txt

@@ -1 +1,2 @@
 ordered-set>=4.0.2,<4.2.0
+orjson

setup.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 6.2.2
+current_version = 6.2.3
 commit = True
 tag = True
 tag_name = {new_version}

setup.py

@@ -10,7 +10,7 @@
 if os.environ.get('USER', '') == 'vagrant':
     del os.link
 
-version = '6.2.2'
+version = '6.2.3'
 
 
 def get_reqs(filename):

tests/test_command.py

@@ -44,8 +44,8 @@ def test_cli_cant_find_file(self):
         ('t1.pickle', 't2.pickle', {}, 0),
         ('t1.yaml', 't2.yaml', {}, 0),
     ])
-    def test_deeppatch_command(self, t1, t2, args, expected_exit_code):
-        t1_copy_path = f'/tmp/{t1}'
+    def test_deeppatch_command(self, t1, t2, args, expected_exit_code, tmp_path):
+        t1_copy_path = os.path.join(tmp_path, t1)
         t1 = os.path.join(FIXTURES_DIR, t1)
         t2 = os.path.join(FIXTURES_DIR, t2)
         copyfile(t1, t1_copy_path)
@@ -54,7 +54,7 @@ def test_deeppatch_command(self, t1, t2, args, expected_exit_code):
         assert delta_pickled.exit_code == expected_exit_code
 
         if expected_exit_code == 0:
-            delta_path = '/tmp/delta.pickle'
+            delta_path = os.path.join(tmp_path, 'delta.pickle')
             with open(delta_path, 'wb') as the_file:
                 the_file.write(delta_pickled.stdout_bytes)
 

tests/test_delta.py

@@ -84,35 +84,35 @@ def test_multiple_delta(self):
 
         assert t1 + delta1 + delta2 == t3
 
-    def test_delta_dump_and_read1(self):
+    def test_delta_dump_and_read1(self, tmp_path):
         t1 = [1, 2]
         t2 = [1, 2, 3, 5]
         diff = DeepDiff(t1, t2)
-        path = '/tmp/delta_test.delta'
+        path = os.path.join(tmp_path, 'delta_test.delta')
         with open(path, 'wb') as the_file:
             Delta(diff).dump(the_file)
         delta = Delta(delta_path=path)
         os.remove(path)
         assert delta + t1 == t2
 
-    def test_delta_dump_and_read2(self):
+    def test_delta_dump_and_read2(self, tmp_path):
         t1 = [1, 2]
         t2 = [1, 2, 3, 5]
         diff = DeepDiff(t1, t2)
         delta_content = Delta(diff).dumps()
-        path = '/tmp/delta_test2.delta'
+        path = os.path.join('tmp_path, delta_test2.delta')
         with open(path, 'wb') as the_file:
             the_file.write(delta_content)
         delta = Delta(delta_path=path)
         os.remove(path)
         assert delta + t1 == t2
 
-    def test_delta_dump_and_read3(self):
+    def test_delta_dump_and_read3(self, tmp_path):
         t1 = [1, 2]
         t2 = [1, 2, 3, 5]
         diff = DeepDiff(t1, t2)
         delta_content = Delta(diff).dumps()
-        path = '/tmp/delta_test2.delta'
+        path = os.path.join('tmp_path, delta_test2.delta')
         with open(path, 'wb') as the_file:
             the_file.write(delta_content)
         with pytest.raises(ValueError) as excinfo:

tests/test_serialization.py

@@ -318,8 +318,10 @@ def test_pretty_form_method(self, expected, verbose_level):
     @pytest.mark.parametrize('test_num, value', [
         (1, {'10': None}),
         (2, {"type_changes": {"root": {"old_type": None, "new_type": list, "new_value": ["你好", 2, 3, 5]}}}),
+        (3, {'10': Decimal(2017)}),
+        (4, Decimal(2017.1)),
     ])
     def test_json_dumps_and_loads(self, test_num, value):
         serialized = json_dumps(value)
         back = json_loads(serialized)
-        assert value == back, f"test_json_dumps_and_loads tesst #{test_num} failed"
+        assert value == back, f"test_json_dumps_and_loads test #{test_num} failed"