Skip to content

Commit d76badd

Browse files
authored
Merge pull request #239 from mayurdb/deepScrapeFix
2 parents 218b8ed + a458ec4 commit d76badd

File tree

1 file changed

+10
-2
lines changed

1 file changed

+10
-2
lines changed

scrapegraphai/nodes/search_link_node.py

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -71,8 +71,16 @@ def execute(self, state: dict) -> dict:
7171
prompt_relevant_links = """
7272
You are a website scraper and you have just scraped the following content from a website.
7373
Content: {content}
74-
You are now asked to find all relevant links from the extracted webpage content related
75-
to prompt {user_prompt}. Only pick links which are valid and relevant
74+
75+
You are now tasked with identifying all hyper links within the content that are potentially
76+
relevant to the user task: {user_prompt}
77+
78+
Assume relevance broadly, including any links that might be related or potentially useful
79+
in relation to the task.
80+
81+
Please list only valid URLs and make sure to err on the side of inclusion if it's uncertain
82+
whether the content at the link is directly relevant.
83+
7684
Output only a list of relevant links in the format:
7785
[
7886
"link1",

0 commit comments

Comments
 (0)