Skip to content

allignment #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,15 @@ name: Upload Python Package
on:
release:
types: [published]
paths:
- 'scrapegraph-py/**'

jobs:
deploy:

runs-on: ubuntu-latest
# Only run if scrapegraph-py has changes
if: contains(github.event.release.body, 'scrapegraph-py/')

steps:
- uses: actions/checkout@v4
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,15 @@ on:
branches:
- main
- pre/*
paths:
- 'scrapegraph-py/**'

jobs:
build:
name: Build
runs-on: ubuntu-latest
# Only run if scrapegraph-py has changes
if: contains(github.event.head_commit.modified, 'scrapegraph-py/') || contains(github.event.head_commit.added, 'scrapegraph-py/') || contains(github.event.head_commit.removed, 'scrapegraph-py/')
steps:
- name: Install git
run: |
Expand Down
6 changes: 0 additions & 6 deletions CHANGELOG.MD

This file was deleted.

21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2024 ScrapeGraphAI

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
86 changes: 86 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# 🌐 ScrapeGraph AI SDKs

[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![Python SDK](https://img.shields.io/badge/Python_SDK-Latest-blue)](https://github.com/ScrapeGraphAI/scrapegraph-sdk/tree/main/scrapegraph-py)
[![JavaScript SDK](https://img.shields.io/badge/JavaScript_SDK-Latest-yellow)](https://github.com/ScrapeGraphAI/scrapegraph-sdk/tree/main/scrapegraph-js)
[![Documentation](https://img.shields.io/badge/Documentation-Latest-green)](https://scrapegraphai.com/docs)

Official SDKs for the ScrapeGraph AI API - Intelligent web scraping powered by AI. Extract structured data from any webpage with natural language prompts.

The credits can be bougth [here](https://scrapegraphai.com)!

## 🚀 Quick Links

- [Python SDK Documentation](scrapegraph-py/README.md)
- [JavaScript SDK Documentation](scrapegraph-js/README.md)
- [API Documentation](https://scrapegraphai.com/docs)
- [Website](https://scrapegraphai.com)

## 📦 Installation

### Python
```bash
pip install scrapegraph-py
```

### JavaScript
```bash
npm install scrapegraph-js
```

## 🎯 Core Features

- 🤖 **AI-Powered Extraction**: Use natural language to describe what data you want
- 📊 **Structured Output**: Get clean, structured data with optional schema validation
- 🔄 **Multiple Formats**: Extract data as JSON, Markdown, or custom schemas
- ⚡ **High Performance**: Concurrent processing and automatic retries
- 🔒 **Enterprise Ready**: Production-grade security and rate limiting

## 🛠️ Available Endpoints

### 🔍 SmartScraper
Extract structured data from any webpage using natural language prompts.

### 📝 Markdownify
Convert any webpage into clean, formatted markdown.

### 💻 LocalScraper
Extract information from a local HTML file using AI.


## 🌟 Key Benefits

- 📝 **Natural Language Queries**: No complex selectors or XPath needed
- 🎯 **Precise Extraction**: AI understands context and structure
- 🔄 **Adaptive Scraping**: Works with dynamic and static content
- 📊 **Schema Validation**: Ensure data consistency with Pydantic/TypeScript
- ⚡ **Async Support**: Handle multiple requests efficiently

## 💡 Use Cases

- 🏢 **Business Intelligence**: Extract company information and contacts
- 📊 **Market Research**: Gather product data and pricing
- 📰 **Content Aggregation**: Convert articles to structured formats
- 🔍 **Data Mining**: Extract specific information from multiple sources
- 📱 **App Integration**: Feed clean data into your applications

## 📖 Documentation

For detailed documentation and examples, visit:
- [Python SDK Guide](scrapegraph-py/README.md)
- [JavaScript SDK Guide](scrapegraph-js/README.md)
- [API Documentation](https://scrapegraphai.com/docs)

## 💬 Support & Feedback

- 📧 Email: [email protected]
- 💻 GitHub Issues: [Create an issue](https://github.com/ScrapeGraphAI/scrapegraph-sdk/issues)
- 🌟 Feature Requests: [Request a feature](https://github.com/ScrapeGraphAI/scrapegraph-sdk/issues/new)

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

Made with ❤️ by [ScrapeGraph AI](https://scrapegraphai.com)
137 changes: 0 additions & 137 deletions readme.md

This file was deleted.

22 changes: 11 additions & 11 deletions scrapegraph-js/CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,23 +17,23 @@ diverse, inclusive, and healthy community.
Examples of behavior that contributes to a positive environment for our
community include:

* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes,
- Demonstrating empathy and kindness toward other people
- Being respectful of differing opinions, viewpoints, and experiences
- Giving and gracefully accepting constructive feedback
- Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience
* Focusing on what is best not just for us as individuals, but for the
- Focusing on what is best not just for us as individuals, but for the
overall community

Examples of unacceptable behavior include:

* The use of sexualized language or imagery, and sexual attention or
- The use of sexualized language or imagery, and sexual attention or
advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email
- Trolling, insulting or derogatory comments, and personal or political attacks
- Public or private harassment
- Publishing others' private information, such as a physical or email
address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a
- Other conduct which could reasonably be considered inappropriate in a
professional setting

## Enforcement Responsibilities
Expand Down Expand Up @@ -106,7 +106,7 @@ Violating these terms may lead to a permanent ban.
### 4. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior, harassment of an
standards, including sustained inappropriate behavior, harassment of an
individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within
Expand Down
7 changes: 0 additions & 7 deletions scrapegraph-js/LICENSE

This file was deleted.

Loading
Loading