Skip to content

Commit 19b27bb

Browse files
committed
feat(burr): first burr integration and docs
1 parent 819f071 commit 19b27bb

File tree

7 files changed

+30
-541
lines changed

7 files changed

+30
-541
lines changed

docs/source/scrapers/graph_config.rst

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,38 @@ Some interesting ones are:
1111
- `max_results`: The maximum number of results to be fetched from the search engine. Useful in `SearchGraph`.
1212
- `output_path`: The path where the output files will be saved. Useful in `SpeechGraph`.
1313
- `loader_kwargs`: A dictionary with additional parameters to be passed to the `Loader` class, such as `proxy`.
14+
- `burr_kwargs`: A dictionary with additional parameters to enable `Burr` graphical user interface.
1415
- `max_images`: The maximum number of images to be analyzed. Useful in `OmniScraperGraph` and `OmniSearchGraph`.
1516

17+
Burr Integration
18+
^^^^^^^^^^^^^^^^
19+
20+
`Burr` is an open source python library that allows the creation and management of state machine applications. Discover more about it `here <https://github.com/DAGWorks-Inc/burr>`_.
21+
It is possible to enable a local hosted webapp to visualize the scraping pipelines and the data flow.
22+
First, we need to install the `burr` library as follows:
23+
24+
.. code-block:: bash
25+
26+
pip install scrapegraphai[burr]
27+
28+
and then run the graphical user interface as follows:
29+
30+
.. code-block:: bash
31+
32+
burr
33+
34+
To log your graph execution in the platform, you need to set the `burr_kwargs` parameter in the graph configuration as follows:
35+
36+
.. code-block:: python
37+
38+
graph_config = {
39+
"llm":{...},
40+
"burr_kwargs": {
41+
"project_name": "test-scraper",
42+
"app_instance_id":"some_id",
43+
}
44+
}
45+
1646
Proxy Rotation
1747
^^^^^^^^^^^^^^
1848

examples/openai/search_graph_burr.py

Lines changed: 0 additions & 50 deletions
This file was deleted.

examples/openai/smart_scraper_burr.py

Lines changed: 0 additions & 112 deletions
This file was deleted.
-48.9 KB
Binary file not shown.
-28.5 KB
Binary file not shown.

0 commit comments

Comments
 (0)