Skip to content

Fix for GenerateScraperGraph #215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 11, 2024

Conversation

epage480
Copy link
Contributor

Created a fix that allows examples/openai/script_generator_openai.py to run successfully.

I made 3 major changes:

  1. Added a new parameter "parse_html" to the node_config for ParseNode. If set to False it will give the unparsed html, which is useful for creating parsing scripts. Default behavior is identical to before.
  2. Removed the RAG node from the ScriptCreatorGraph, RAG for HTML is a non-trivial task and was not being done well.
  3. script_creator_graph.py contains a lot of dead or non-functional code, it appears someone attempted to use RAG unsuccessfully.

While ScriptCreatorGraph will fail for websites with a high token count, these changes will allow at least some functionality and a starting point for expansion.

@epage480 epage480 marked this pull request as ready for review May 11, 2024 00:00
@VinciGit00 VinciGit00 merged commit 1fa77e5 into ScrapeGraphAI:pre/beta May 11, 2024
Copy link

🎉 This PR is included in version 0.11.0-beta.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

Copy link

🎉 This PR is included in version 0.11.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants