Skip to content

Commit 3c9f76d

Browse files
authored
Merge pull request #7 from ScrapeGraphAI/js-readme-standardization
fix: readme js sdk
2 parents ce53675 + 88a2f50 commit 3c9f76d

File tree

3 files changed

+135
-100
lines changed

3 files changed

+135
-100
lines changed

β€Žscrapegraph-js/package.json

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"name": "scrapegraph-sdk",
2+
"name": "scrapegraph-js",
33
"author": "ScrapeGraphAI",
44
"version": "0.0.1",
55
"description": "Scrape and extract structured data from a webpage using ScrapeGraph AI.",
@@ -23,7 +23,8 @@
2323
"module": "index.js",
2424
"type": "module",
2525
"dependencies": {
26-
"axios": "^1.6.0"
26+
"axios": "^1.6.0",
27+
"zod": "^3.23.8"
2728
},
2829
"devDependencies": {
2930
"dotenv": "^16.4.5"

β€Žscrapegraph-js/readme.md

Lines changed: 129 additions & 94 deletions
Original file line numberDiff line numberDiff line change
@@ -1,150 +1,185 @@
1-
# ScrapeGraph JS SDK
1+
# 🌐 ScrapeGraph JavaScript SDK
22

3-
A JavaScript SDK for interacting with the ScrapeGraph AI API. This SDK provides easy-to-use functions for web scraping, managing credits, and submitting feedback.
3+
[![npm version](https://badge.fury.io/js/scrapegraph-js.svg)](https://badge.fury.io/js/scrapegraph-js)
4+
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
5+
[![Build Status](https://github.com/ScrapeGraphAI/scrapegraph-sdk/actions/workflows/ci.yml/badge.svg)](https://github.com/ScrapeGraphAI/scrapegraph-sdk/actions)
6+
[![Documentation Status](https://img.shields.io/badge/docs-latest-brightgreen.svg)](https://docs.scrapegraphai.com)
47

5-
## Installation
8+
Official JavaScript/TypeScript SDK for the ScrapeGraph AI API - Smart web scraping powered by AI.
69

7-
Install the package using npm:
10+
## πŸš€ Features
811

9-
```bash
10-
npm install scrapegraph-js
11-
```
12+
- ✨ Smart web scraping with AI
13+
- πŸ”„ Fully asynchronous design
14+
- πŸ” Detailed error handling
15+
- ⚑ Automatic retries and logging
16+
- πŸ” Secure API authentication
1217

13-
## Usage
18+
## πŸ“¦ Installation
1419

15-
> [!WARNING]
16-
> Remember not to write API keys directly in the code; instead, store them securely in `.env` files.
20+
Install the package using npm or yarn:
1721

18-
First, import the required functions:
22+
```bash
23+
# Using npm
24+
npm i scrapegraph-js
1925

20-
```javascript
21-
import { smartScraper, getSmartScraperRequest, getCredits, sendFeedback } from 'scrapegraph-sdk';
26+
# Using yarn
27+
yarn add scrapegraph-js
2228
```
2329

24-
### Scraping Websites
2530

26-
#### Basic scraping
31+
## πŸ”§ Quick Start
32+
33+
> **Note**: Store your API keys securely in environment variables. Use `.env` files and libraries like `dotenv` to load them into your app.
34+
35+
### Basic Example
2736

2837
```javascript
29-
import { smartScraper } from 'scrapegraph-sdk';
38+
import { smartScraper } from 'scrapegraph-js';
39+
import 'dotenv/config';
3040

31-
const apiKey = process.env.SGAI_APIKEY;
32-
const url = 'https://scrapegraphai.com';
41+
// Initialize variables
42+
const apiKey = process.env.SGAI_APIKEY; // Set your API key as an environment variable
43+
const websiteUrl = 'https://example.com';
3344
const prompt = 'What does the company do?';
3445

35-
try {
36-
const response = await smartScraper(apiKey, url, prompt);
37-
console.log(response);
38-
} catch (error) {
39-
console.error(error);
40-
}
46+
(async () => {
47+
try {
48+
const response = await smartScraper(apiKey, websiteUrl, prompt);
49+
console.log(response.result);
50+
} catch (error) {
51+
console.error('Error:', error);
52+
}
53+
})();
4154
```
4255

43-
#### Scraping with custom output schema
56+
## 🎯 Examples
4457

45-
```javascript
46-
import { smartScraper } from 'scrapegraph-sdk';
58+
### Scraping Websites
4759

48-
const apiKey = 'your_api_key';
49-
const url = 'https://scrapegraphai.com';
50-
const prompt = 'What does the company do?';
51-
const schema = //TODO
52-
53-
try {
54-
const response = await smartScraper(apiKey, url, prompt, schema);
55-
console.log(response);
56-
} catch (error) {
57-
console.error(error);
58-
}
60+
#### Basic Scraping
61+
62+
```javascript
63+
import { smartScraper } from 'scrapegraph-js';
64+
65+
const apiKey = 'your-api-key';
66+
const url = 'https://example.com';
67+
const prompt = 'Extract the main heading and description.';
68+
69+
(async () => {
70+
try {
71+
const response = await smartScraper(apiKey, url, prompt);
72+
console.log(response.result);
73+
} catch (error) {
74+
console.error('Error:', error);
75+
}
76+
})();
5977
```
6078

61-
### Checking Credits
79+
#### Scraping with Custom Output Schema
6280

6381
```javascript
64-
import { getCredist } from 'scrapegraph-sdk';
82+
//TODO
83+
```
6584

66-
const apiKey = 'your_api_key';
85+
### Checking API Credits
6786

68-
try {
69-
const myCredit = await getCredits(apiKey);
70-
console.log(myCredit)
71-
} catch (error) {
72-
console.error(error)
73-
}
87+
```javascript
88+
import { getCredits } from 'scrapegraph-js';
89+
90+
const apiKey = 'your-api-key';
91+
92+
(async () => {
93+
try {
94+
const credits = await getCredits(apiKey);
95+
console.log('Available credits:', credits);
96+
} catch (error) {
97+
console.error('Error fetching credits:', error);
98+
}
99+
})();
74100
```
75101

76102
### Submitting Feedback
77103

78104
```javascript
79-
import { sendFeedback } from 'scrapegraph-sdk';
105+
import { sendFeedback } from 'scrapegraph-js';
80106

81-
const apiKey = 'your_api_key';
107+
const apiKey = 'your-api-key';
82108
const requestId = '16a63a80-c87f-4cde-b005-e6c3ecda278b';
83109
const rating = 5;
84-
const feedbackMessage = 'This is a test feedback message.';
85-
86-
try {
87-
const feedback_response = await sendFeedback(apiKey, requestId, rating, feedbackMessage);
88-
console.log(feedback_response);
89-
} catch (error) {
90-
console.error(error)
91-
}
110+
const feedbackText = 'This is a test feedback message.';
111+
112+
(async () => {
113+
try {
114+
const response = await sendFeedback(apiKey, requestId, rating, feedbackText);
115+
console.log('Feedback response:', response);
116+
} catch (error) {
117+
console.error('Error sending feedback:', error);
118+
}
119+
})();
92120
```
93121

94-
## API Reference
95-
96-
### scrape(apiKey, url[, options])
122+
## πŸ“š Documentation
97123

98-
Scrapes a website and returns the extracted data.
124+
For detailed documentation, visit [docs.scrapegraphai.com](https://docs.scrapegraphai.com)
99125

100-
Parameters:
101-
- `apiKey` (string): Your ScrapeGraph AI API key
102-
- `url` (string): The URL to scrape
103-
- `options` (object, optional):
104-
- `elements` (array): Specific elements to extract
105-
- `wait_for` (string): CSS selector to wait for before scraping
106-
- `javascript` (boolean): Enable JavaScript rendering
126+
## πŸ› οΈ Development
107127

108-
### credits(apiKey)
128+
### Setup
109129

110-
Retrieves your current credit balance.
130+
1. Clone the repository:
131+
```bash
132+
git clone https://github.com/ScrapeGraphAI/scrapegraph-sdk.git
133+
cd scrapegraph-sdk/scrapegraph-js
134+
```
111135

112-
Parameters:
113-
- `apiKey` (string): Your ScrapeGraph AI API key
136+
2. Install dependencies:
137+
```bash
138+
npm install
139+
```
114140

115-
### feedback(apiKey, requestId, rating, feedbackText)
141+
3. Run linting and testing:
142+
```bash
143+
npm run lint
144+
npm test
145+
```
116146

117-
Submits feedback for a scraping request.
147+
### Running Tests
118148

119-
Parameters:
120-
- `apiKey` (string): Your ScrapeGraph AI API key
121-
- `requestId` (string): The request ID from the scrape response
122-
- `rating` (number): Rating score
123-
- `feedbackText` (string) (optional): Feedback message
149+
```bash
150+
# Run all tests
151+
npm test
124152

125-
## Error Handling
153+
# Run tests with coverage
154+
npm run test:coverage
155+
```
126156

127-
All functions return javascript `Error` object with imformation. In case of errors, the response will include error details:
157+
## πŸ“ License
128158

129-
// TODO error list
159+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
130160

131-
```javascript
132-
{
133-
"statusCode": 400,
134-
"title": "HTTP error occurred"
135-
"details": "Error details",
136-
137-
}
138-
```
161+
## 🀝 Contributing
139162

140-
## License
163+
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
141164

142-
MIT
165+
1. Fork the repository
166+
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
167+
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
168+
4. Push to the branch (`git push origin feature/AmazingFeature`)
169+
5. Open a Pull Request
143170

144-
## Support
171+
## πŸ”— Links
145172

146-
For support, please visit [ScrapeGraph AI Documentation](https://sgai-api.onrender.com/docs).
173+
- [Website](https://scrapegraphai.com)
174+
- [Documentation](https://scrapegraphai.com/documentation)
175+
- [GitHub](https://github.com/ScrapeGraphAI/scrapegraph-sdk)
147176

177+
## πŸ’¬ Support
148178

179+
- πŸ“§ Email: [email protected]
180+
- πŸ’» GitHub Issues: [Create an issue](https://github.com/ScrapeGraphAI/scrapegraph-sdk/issues)
181+
- 🌟 Feature Requests: [Request a feature](https://github.com/ScrapeGraphAI/scrapegraph-sdk/issues/new)
149182

183+
---
150184

185+
Made with ❀️ by [ScrapeGraph AI](https://scrapegraphai.com)

β€Žscrapegraph-py/README.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -155,10 +155,9 @@ Contributions are welcome! Please feel free to submit a Pull Request. For major
155155

156156
## πŸ”— Links
157157

158-
- [Website](https://scrapegraphai.com)
159-
- [Documentation](https://docs.scrapegraphai.com)
160-
- [API Reference](https://docs.scrapegraphai.com/api)
161-
- [GitHub](https://github.com/ScrapeGraphAI/scrapegraph-sdk)
158+
- [Website](https://scrapegraphai.com)
159+
- [Documentation](https://scrapegraphai.com/documentation)
160+
- [GitHub](https://github.com/ScrapeGraphAI/scrapegraph-sdk)
162161

163162
## πŸ’¬ Support
164163

0 commit comments

Comments
Β (0)