Skip to content

fix: deepcopy fail for coping model_instance config #613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 1, 2024

Conversation

goasleep
Copy link
Contributor

link #609
When copying the configuration with model_instance, an error may arise because model_instance is typically an uncopyable object. To fix it, I attempted to use safe_deepcopy to replace the internal deepcopy during the copying of the LLM configuration.

The safe_deepcopy function first attempts to use the internal deepcopy, ensuring it doesn't disrupt the original usage. If that fails, it resorts to alternative copying logic, such as shallow copying. In the worst-case scenario, it returns the original object.

Note:

If there are any new graphs using deepcopy in future pull requests, special attention should be paid to copying the LLM configuration correctly.

Other Considerations:

This pull request may not represent the best practice for resolving the issue, but it offers a quick fix. Here are some alternative approaches that I believe are superior:

  • Using Pydantic Class Objects: Define the configuration with a Pydantic class object and implement a custom deepcopy method. This approach requires significant refactoring but is more elegant.
  • Replacing Internal Deepcopy: Utilize safe_deepcopy to replace the internal deepcopy. However, this might introduce unforeseen issues.
  • Alternative Deepcopy Packages: Consider using other deepcopy packages, but I have yet to find a suitable option.

@goasleep goasleep changed the title fix: fix deepcopy fail for coping model_instance config fix: deepcopy fail for coping model_instance config Aug 31, 2024
@goasleep goasleep force-pushed the feature/add_copy_tool branch 2 times, most recently from deba404 to d1926a4 Compare August 31, 2024 04:42
@goasleep
Copy link
Contributor Author

goasleep commented Aug 31, 2024

In addition, there are any formattor about this project? seem I cannot find it

from typing import Any, Dict, Optional


def safe_deepcopy(obj: Any, memo: Optional[Dict[int, Any]] = None) -> Any:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why memo and not a more standard cache?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I deleted it because I found it not very necessary and may cause other problems. BTW,memo is the second parameter name of deepcopy so I use it.

pass

# If all else fails, return the original object
return obj
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defaulting to returning the original object might be a bad idea for a number of reasons, if misused (e.g., assuming on the outside the copy always works)

would either remove it, throwing an error if even normal copying doesn't go through, or adding a parameter strict or something regulating the behavior

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I change it and raise custom error DeepCopyError. and I also add pydantic object copy deep function because I found out that he didn’t support it before.

@goasleep goasleep force-pushed the feature/add_copy_tool branch from d1926a4 to 36818b1 Compare August 31, 2024 09:39
@goasleep
Copy link
Contributor Author

wait a moment. find new problem

@VinciGit00
Copy link
Collaborator

What's the problem?

@goasleep
Copy link
Contributor Author

goasleep commented Sep 1, 2024

What's the problem?

I found there is message missing when use copy(deep=True) in the pydantic model v1. You can see as below. Maybe this is pydanticv1 bug. so I try to use shallow copy to avoid it when deepcopy failed.

image

@VinciGit00
Copy link
Collaborator

Now is ok?

@goasleep
Copy link
Contributor Author

goasleep commented Sep 1, 2024

Now is ok?

yeah. Now is ok.

@VinciGit00 VinciGit00 merged commit afdf524 into ScrapeGraphAI:main Sep 1, 2024
2 checks passed
Copy link

github-actions bot commented Sep 1, 2024

🎉 This issue has been resolved in version 1.16.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@VinciGit00
Copy link
Collaborator

thank you!

Copy link

github-actions bot commented Sep 2, 2024

🎉 This issue has been resolved in version 1.16.0-beta.4 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants