Fix for diffing using iterable_compare_func with nested objects.

README.md

@@ -1,4 +1,4 @@
-# DeepDiff v 5.8.1
+# DeepDiff v 5.8.2
 
 ![Downloads](https://img.shields.io/pypi/dm/deepdiff.svg?style=flat)
 ![Python Versions](https://img.shields.io/pypi/pyversions/deepdiff.svg?style=flat)
@@ -14,10 +14,13 @@
 
 Tested on Python 3.6+ and PyPy3.
 
-- **[Documentation](https://zepworks.com/deepdiff/5.8.1/)**
+- **[Documentation](https://zepworks.com/deepdiff/5.8.2/)**
 
 ## What is new?
 
+DeepDiff 5-8-2
+Fixing dependency for Py3.6
+
 DeepDiff 5-8-1 includes bug fixes:
 - Fixed test suite for 32bit systems (https://github.com/seperman/deepdiff/issues/302) by [Louis-Philippe Véronneau](https://github.com/baldurmen)
 - Fixed the issue when using `ignore_order=True` and `group_by` simultaneously
@@ -67,13 +70,13 @@ Note: if you want to use DeepDiff via commandline, make sure to run `pip install
 
 DeepDiff gets the difference of 2 objects.
 
-> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/5.8.1/diff.html)
-> - The full documentation of all modules can be found on <https://zepworks.com/deepdiff/5.8.1/>
+> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/5.8.2/diff.html)
+> - The full documentation of all modules can be found on <https://zepworks.com/deepdiff/5.8.2/>
 > - Tutorials and posts about DeepDiff can be found on <https://zepworks.com/tags/deepdiff/>
 
 ## A few Examples
 
-> Note: This is just a brief overview of what DeepDiff can do. Please visit <https://zepworks.com/deepdiff/5.8.1/> for full documentation.
+> Note: This is just a brief overview of what DeepDiff can do. Please visit <https://zepworks.com/deepdiff/5.8.2/> for full documentation.
 
 ### List difference ignoring order or duplicates
 
@@ -277,8 +280,8 @@ Example:
 ```
 
 
-> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/5.8.1/diff.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/5.8.1/>
+> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/5.8.2/diff.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/5.8.2/>
 
 
 # Deep Search
@@ -310,17 +313,17 @@ And you can pass all the same kwargs as DeepSearch to grep too:
 {'matched_paths': {"root['somewhere']": 'around'}, 'matched_values': {"root['long']": 'somewhere'}}
 ```
 
-> - Please take a look at the [DeepSearch docs](https://zepworks.com/deepdiff/5.8.1/dsearch.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/5.8.1/>
+> - Please take a look at the [DeepSearch docs](https://zepworks.com/deepdiff/5.8.2/dsearch.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/5.8.2/>
 
 # Deep Hash
 (New in v4-0-0)
 
 DeepHash is designed to give you hash of ANY python object based on its contents even if the object is not considered hashable!
 DeepHash is supposed to be deterministic in order to make sure 2 objects that contain the same data, produce the same hash.
 
-> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/5.8.1/deephash.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/5.8.1/>
+> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/5.8.2/deephash.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/5.8.2/>
 
 Let's say you have a dictionary object.
 
@@ -368,8 +371,8 @@ Which you can write as:
 At first it might seem weird why DeepHash(obj)[obj] but remember that DeepHash(obj) is a dictionary of hashes of all other objects that obj contains too.
 
 
-> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/5.8.1/deephash.html)
-> - The full documentation can be found on <https://zepworks.com/deepdiff/5.8.1/>
+> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/5.8.2/deephash.html)
+> - The full documentation can be found on <https://zepworks.com/deepdiff/5.8.2/>
 
 
 # Using DeepDiff in unit tests
@@ -449,11 +452,11 @@ Thank you!
 
 How to cite this library (APA style):
 
-    Dehpour, S. (2022). DeepDiff (Version 5.8.1) [Software]. Available from https://github.com/seperman/deepdiff.
+    Dehpour, S. (2022). DeepDiff (Version 5.8.2) [Software]. Available from https://github.com/seperman/deepdiff.
 
 How to cite this library (Chicago style):
 
-    Dehpour, Sep. 2022. DeepDiff (version 5.8.1).
+    Dehpour, Sep. 2022. DeepDiff (version 5.8.2).
 
 # Authors
 

deepdiff/__init__.py

@@ -1,6 +1,6 @@
 """This module offers the DeepDiff, DeepSearch, grep, Delta and DeepHash classes."""
 # flake8: noqa
-__version__ = '5.8.1'
+__version__ = '5.8.2'
 import logging
 
 if __name__ == '__main__':

deepdiff/delta.py

@@ -263,13 +263,21 @@ def _del_elem(self, parent, parent_to_obj_elem, parent_to_obj_action,
     def _do_iterable_item_added(self):
         iterable_item_added = self.diff.get('iterable_item_added', {})
         iterable_item_moved = self.diff.get('iterable_item_moved')
+
+        # First we need to create a placeholder for moved items.
+        # This will then get replaced below after we go through added items.
+        # Without this items can get double added because moved store the new_value and does not need item_added replayed
         if iterable_item_moved:
-            added_dict = {v["new_path"]: v["value"] for k, v in iterable_item_moved.items()}
+            added_dict = {v["new_path"]: None for k, v in iterable_item_moved.items()}
             iterable_item_added.update(added_dict)
 
         if iterable_item_added:
             self._do_item_added(iterable_item_added, insert=True)
 
+        if iterable_item_moved:
+            added_dict = {v["new_path"]: v["value"] for k, v in iterable_item_moved.items()}
+            self._do_item_added(added_dict, insert=False)
+
     def _do_dictionary_item_added(self):
         dictionary_item_added = self.diff.get('dictionary_item_added')
         if dictionary_item_added:

deepdiff/diff.py

@@ -709,7 +709,7 @@ def _diff_iterable_in_order(self, level, parents_ids=frozenset(), _original_type
                     x,
                     y,
                     child_relationship_class=child_relationship_class,
-                    child_relationship_param=i)
+                    child_relationship_param=j)
                 self._diff(next_level, parents_ids_added)
 
     def _diff_str(self, level):

docs/conf.py

@@ -60,9 +60,9 @@
 # built documents.
 #
 # The short X.Y version.
-version = '5.8.1'
+version = '5.8.2'
 # The full version, including alpha/beta/rc tags.
-release = '5.8.1'
+release = '5.8.2'
 
 load_dotenv(override=True)
 DOC_VERSION = os.environ.get('DOC_VERSION', version)

docs/index.rst

@@ -4,7 +4,7 @@
    contain the root `toctree` directive.
 
 
-DeepDiff 5.8.1 documentation!
+DeepDiff 5.8.2 documentation!
 =============================
 
 *****************
@@ -31,6 +31,11 @@ The DeepDiff library includes the following modules:
 What is New
 ***********
 
+DeepDiff 5-8-2
+--------------
+
+Fixing dependency for Py3.6
+
 New In DeepDiff 5-8-1
 ---------------------
 

requirements.txt

@@ -1 +1 @@
-ordered-set>=4.1.0,<4.2.0
+ordered-set>=4.0.2,<4.2.0

setup.cfg

@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 5.8.1
+current_version = 5.8.2
 commit = True
 tag = True
 tag_name = {new_version}

setup.py

@@ -10,7 +10,7 @@
 if os.environ.get('USER', '') == 'vagrant':
     del os.link
 
-version = '5.8.1'
+version = '5.8.2'
 
 
 def get_reqs(filename):

tests/fixtures/compare_func_result1.json

@@ -6,11 +6,11 @@
     "root['Cars'][3]['production']"
   ],
   "values_changed": {
-    "root['Cars'][0]['dealers'][1]['quantity']": {
+    "root['Cars'][2]['dealers'][0]['quantity']": {
       "new_value": 50,
       "old_value": 20
     },
-    "root['Cars'][2]['model_numbers'][2]": {
+    "root['Cars'][1]['model_numbers'][2]": {
       "new_value": 3,
       "old_value": 4
     },
@@ -20,20 +20,20 @@
     }
   },
   "iterable_item_added": {
-    "root['Cars'][0]['dealers'][1]": {
+    "root['Cars'][2]['dealers'][1]": {
       "id": 200,
       "address": "200 Fake St",
       "quantity": 10
     },
-    "root['Cars'][2]['model_numbers'][3]": 4,
+    "root['Cars'][1]['model_numbers'][3]": 4,
     "root['Cars'][0]": {
       "id": "7",
       "make": "Toyota",
       "model": "8Runner"
     }
   },
   "iterable_item_removed": {
-    "root['Cars'][0]['dealers'][0]": {
+    "root['Cars'][2]['dealers'][0]": {
       "id": 103,
       "address": "103 Fake St",
       "quantity": 50

tests/test_delta.py

@@ -1478,7 +1478,7 @@ def test_compare_func_with_duplicates_removed(self):
         t2 = [{'id': 3, 'val': 3}, {'id': 2, 'val': 2}, {'id': 1, 'val': 3}]
         ddiff = DeepDiff(t1, t2, iterable_compare_func=self.compare_func, verbose_level=2)
         expected = {
-            'values_changed': {"root[0]['val']": {'new_value': 3, 'old_value': 1}},
+            'values_changed': {"root[2]['val']": {'new_value': 3, 'old_value': 1}},
             'iterable_item_removed': {'root[2]': {'id': 1, 'val': 3}},
             'iterable_item_moved': {
                 'root[0]': {'new_path': 'root[2]', 'value': {'id': 1, 'val': 3}},
@@ -1495,7 +1495,7 @@ def test_compare_func_with_duplicates_added(self):
         t2 = [{'id': 1, 'val': 1}, {'id': 2, 'val': 2}, {'id': 1, 'val': 3}, {'id': 3, 'val': 3}]
         ddiff = DeepDiff(t1, t2, iterable_compare_func=self.compare_func, verbose_level=2)
         expected = {
-            'values_changed': {"root[2]['val']": {'new_value': 1, 'old_value': 3}},
+            'values_changed': {"root[0]['val']": {'new_value': 1, 'old_value': 3}},
             'iterable_item_added': {'root[2]': {'id': 1, 'val': 3}},
             'iterable_item_moved': {
                 'root[2]': {'new_path': 'root[0]', 'value': {'id': 1, 'val': 1}},
@@ -1526,3 +1526,72 @@ def test_compare_func_path_specific(self):
         delta = Delta(ddiff)
         recreated_t2 = t1 + delta
         assert t2 == recreated_t2
+
+    def test_compare_func_nested_changes(self):
+        t1 = {
+            "TestTable": [
+                {
+                    "id": "022fb580-800e-11ea-a361-39b3dada34b5",
+                    "name": "Max",
+                    "NestedTable": [
+                        {
+                            "id": "022fb580-800e-11ea-a361-39b3dada34a6",
+                            "NestedField": "Test Field"
+                        }
+                    ]
+                },
+                {
+                    "id": "022fb580-800e-11ea-a361-12354656532",
+                    "name": "Bob",
+                    "NestedTable": [
+                        {
+                            "id": "022fb580-800e-11ea-a361-39b3dada34c7",
+                            "NestedField": "Test Field 2"
+                        },
+                    ]
+                },
+            ]
+        }
+        t2 = {"TestTable": [
+            {
+                "id": "022fb580-800e-11ea-a361-12354656532",
+                "name": "Bob (Changed Name)",
+                "NestedTable": [
+                    {
+                        "id": "022fb580-800e-11ea-a361-39b3dada34c7",
+                        "NestedField": "Test Field 2 (Changed Nested Field)"
+                    },
+                    {
+                        "id": "new id",
+                        "NestedField": "Test Field 3"
+                    },
+                    {
+                        "id": "newer id",
+                        "NestedField": "Test Field 4"
+                    },
+                ]
+            },
+            {
+                "id": "adding_some_random_id",
+                "name": "New Name",
+                "NestedTable": [
+                    {
+                        "id": "random_nested_id_added",
+                        "NestedField": "New Nested Field"
+                    },
+                    {
+                        "id": "random_nested_id_added2",
+                        "NestedField": "New Nested Field2"
+                    },
+                    {
+                        "id": "random_nested_id_added3",
+                        "NestedField": "New Nested Field43"
+                    },
+                ]
+            }
+        ]}
+
+        ddiff = DeepDiff(t1, t2, iterable_compare_func=self.compare_func, verbose_level=2)
+        delta = Delta(ddiff)
+        recreated_t2 = t1 + delta
+        assert t2 == recreated_t2