Skip to content

bugfix: ScriptCreatorGraph verbose settings + RobotsNode handling model IDs #273

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 20, 2024

Conversation

JGalego
Copy link
Contributor

@JGalego JGalego commented May 20, 2024

This PR fixes bugs found while testing #272

ScriptGraphCreator: Missing verbose attribute

The ScriptCreatorGraph class tries to access the verbose attribute

https://github.com/VinciGit00/Scrapegraph-ai/blob/f1a25233d650010e1932e0ab80938079a22a296d/scrapegraphai/graphs/script_creator_graph.py#L68

before it is set

https://github.com/VinciGit00/Scrapegraph-ai/blob/f1a25233d650010e1932e0ab80938079a22a296d/scrapegraphai/graphs/abstract_graph.py#L59

Example:

python examples/bedrock/script_generator_bedrock.py

Before:

AttributeError: 'ScriptCreatorGraph' object has no attribute 'verbose'

After:

source: https://perinim.github.io/projects
from bs4 import BeautifulSoup

import requests

url = "https://perinim.github.io/projects"
response = requests.get(url)
soup = BeautifulSoup(response.content, "html.parser")

projects = soup.find_all("div", class_="card hoverable")

for project in projects:
    title = project.find("h4", class_="card-title").text
    description = project.find("p", class_="card-text").text
    print(f"Title: {title}")
    print(f"Description: {description}")
    print()
         node_name  total_tokens  prompt_tokens  ...  successful_requests  total_cost_USD  exec_time
0            Fetch             0              0  ...                    0             0.0   8.533663
1            Parse             0              0  ...                    0             0.0   0.318253
2  GenerateScraper             0              0  ...                    1             0.0   3.118265
3     TOTAL RESULT             0              0  ...                    1             0.0  11.970181

[4 rows x 7 columns]

RobotsNode: Missing model_name

Amazon Bedrock uses model IDs, not model names.

https://github.com/VinciGit00/Scrapegraph-ai/blob/9e92b03913245c32b6cec0ea7fb41d82a759d0c3/scrapegraphai/nodes/robots_node.py#L99-L104

Example:

python examples/bedrock/custom_graph_bedrock.py

Before:

AttributeError: 'Bedrock' object has no attribute 'model_name'

After:

--- Executing Robots Node ---
(Scraping this website is allowed)
--- Executing Fetch Node ---
--- Executing Parse Node ---
--- Executing RAG Node ---
--- (updated chunks metadata) ---
--- (tokens compressed and vector stored) ---
--- Executing GenerateAnswer Node ---
Processing chunks: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 3971.88it/s]
{
    "articles": [
        "Rotary Pendulum RL",
        "DQN Implementation from scratch",
        "Multi Agents HAED",
        "Wireless ESC for Modular Drones"
    ]
}

@VinciGit00 VinciGit00 merged commit c8c3201 into ScrapeGraphAI:pre/beta May 20, 2024
Copy link

🎉 This PR is included in version 1.5.0-beta.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Copy link

🎉 This PR is included in version 1.5.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants