Skip to main content

Quickstart

Get Pry running locally and scrape your first page in under five minutes.

1. Start Pry with Docker Compose​

The recommended way to run Pry is Docker Compose. It starts the API plus the FlareSolverr sidecar (needed for Cloudflare bypass) automatically.

git clone https://git.rugmunch.io/RugMunchMedia/pryscraper.git
cd pryscraper

# Copy the env template (optional for local use)
cp .env.example .env

docker compose up -d

:::note Ports

Pry publishes on host port 8005 → container port 8002. FlareSolverr publishes on host port 8192 → container port 8191.

From outside the container (host, SDK, CLI) always use http://localhost:8005. Port 8002 is only reachable if you docker exec into the running container.

:::

Verify the service is up:

curl http://localhost:8005/health

2. First scrape — curl​

curl -X POST http://localhost:8005/v1/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "formats": ["markdown"]}'

Response contains success: true and the scraped content under data:

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
    "metadata": {
      "title": "Example Domain",
      "status_code": 200,
      "method_used": "direct"
    }
  }
}

3. First scrape — Python SDK​

Install the SDK and use the async client:

from pry_sdk import PryCrawl

async def main():
    mc = PryCrawl("http://localhost:8005")
    result = await mc.scrape("https://example.com")
    print(result["data"]["markdown"])

import asyncio
asyncio.run(main())

Prefer a synchronous script? Use PryCrawlSync:

from pry_sdk import PryCrawlSync

mc = PryCrawlSync("http://localhost:8005")
result = mc.scrape("https://example.com")
print(result["data"]["markdown"])

4. Try the CLI​

If you installed Pry via pip (not Docker), the CLI is available directly:

# Scrape a URL to clean markdown
pry open https://example.com

# Scrape with JSON extraction against a schema
pry open https://store.com/product --json --schema product.json

# Crawl a site
pry crawl https://docs.com --max-pages 20 -o data.json

5. API key note​

Out of the box (no PRY_API_KEY set), Pry is loopback-only: every request from a non-loopback source is rejected with 401. That means a keyless instance is never exposed to the internet, even if the Docker port mapping is public.

When you set PRY_API_KEY, every request — loopback or remote — must send:

Authorization: Bearer <your-key>

Generate a strong key:

python -c "import secrets; print(secrets.token_urlsafe(48))"

See API Overview → Authentication for the full fail-closed policy.

What's next​