Skip to content

feat: v2 documentation with versioned navigation and updated SDKs #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
VinciGit00 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: v2 documentation with versioned navigation and updated SDKs #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
494 changes: 291 additions & 203 deletions docs.json
View file
Open in desktop

Large diffs are not rendered by default.

71 changes: 36 additions & 35 deletions install.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Installation
description: 'Install and get started with ScrapeGraphAI SDKs'
description: 'Install and get started with ScrapeGraphAI v2 SDKs'
---

## Prerequisites
Expand All @@ -22,10 +22,10 @@ from scrapegraph_py import Client

client = Client(api_key="your-api-key-here")

# Scrape a website
response = client.smartscraper(
website_url="https://scrapegraphai.com",
user_prompt="Extract information about the company"
# Extract data from a website
response = client.extract(
url="https://scrapegraphai.com",
prompt="Extract information about the company"
)
print(response)
```
Expand All @@ -40,6 +40,8 @@ For more advanced usage, see the [Python SDK documentation](/sdks/python).

## JavaScript SDK

Requires **Node.js >= 22**.

Install using npm, pnpm, yarn, or bun:

```bash
Expand All @@ -59,20 +61,16 @@ bun add scrapegraph-js
**Usage:**

```javascript
import { smartScraper } from "scrapegraph-js";
import scrapegraphai from "scrapegraph-js";

const apiKey = "your-api-key-here";
const sgai = scrapegraphai({ apiKey: "your-api-key-here" });

const response = await smartScraper(apiKey, {
website_url: "https://scrapegraphai.com",
user_prompt: "What does the company do?",
});
const { data } = await sgai.extract(
"https://scrapegraphai.com",
{ prompt: "What does the company do?" }
);

if (response.status === "error") {
console.error("Error:", response.error);
} else {
console.log(response.data.result);
}
console.log(data);
```

<Note>
Expand All @@ -85,18 +83,18 @@ For more advanced usage, see the [JavaScript SDK documentation](/sdks/javascript

## Key Concepts

### SmartScraper
### Extract (formerly SmartScraper)
Extract specific information from any webpage using AI. Provide a URL and a prompt describing what you want to extract. [Learn more](/services/smartscraper)

### SearchScraper
Search and extract information from multiple web sources using AI. Start with just a prompt - SearchScraper will find relevant websites and extract the information you need. [Learn more](/services/searchscraper)
### Search (formerly SearchScraper)
Search and extract information from multiple web sources using AI. Start with just a query - Search will find relevant websites and extract the information you need. [Learn more](/services/searchscraper)

### Scrape
Convert any webpage into markdown, HTML, screenshot, or branding format. Replaces the previous Markdownify endpoint with additional output formats. [Learn more](/services/scrape)

### SmartCrawler
AI-powered extraction for any webpage with crawl capabilities. Automatically navigate and extract data from multiple pages. [Learn more](/services/smartcrawler)

### Markdownify
Convert any webpage into clean, formatted markdown. Perfect for content aggregation and processing. [Learn more](/services/markdownify)

### Structured Output with Schemas
Both SDKs support structured output using schemas:
- **Python**: Use Pydantic models
Expand All @@ -119,34 +117,37 @@ class CompanyInfo(BaseModel):
industry: str = Field(description="Industry sector")

client = Client(api_key="your-api-key")
result = client.smartscraper(
website_url="https://scrapegraphai.com",
user_prompt="Extract company information",
response = client.extract(
url="https://scrapegraphai.com",
prompt="Extract company information",
output_schema=CompanyInfo
)
print(result)
print(response)
```

### JavaScript Example

```javascript
import { smartScraper } from "scrapegraph-js";
import scrapegraphai from "scrapegraph-js";
import { z } from "zod";

const sgai = scrapegraphai({ apiKey: "your-api-key" });

const CompanySchema = z.object({
company_name: z.string().describe("The company name"),
companyName: z.string().describe("The company name"),
description: z.string().describe("Company description"),
website: z.string().url().describe("Company website URL"),
industry: z.string().describe("Industry sector"),
});

const apiKey = "your-api-key";
const response = await smartScraper(apiKey, {
website_url: "https://scrapegraphai.com",
user_prompt: "Extract company information",
output_schema: CompanySchema,
});
console.log(response.data.result);
const { data } = await sgai.extract(
"https://scrapegraphai.com",
{
prompt: "Extract company information",
schema: CompanySchema,
}
);
console.log(data);
```

---
Expand Down
Loading