2 posts tagged with "Scraping"

Today we're announcing Pry: an adaptive web scraping and browser automation API that refuses to break. Pry pairs a 15-tier anti-bot escalation engine with 109 store-ready Apify Actors and an x402 crypto pay-per-call payment lane — so you can run it yourself, rent it per call, or let AI agents pay for it autonomously.

What is Pry?​

Pry is a self-hostable scraping engine with a clean REST API and Python SDK. Point it at a URL and it returns clean markdown, structured extraction, screenshots, PDFs, or LLM-ready corpora — across 15 data domains including e-commerce, SEO, crypto market data, jobs, real estate, and compliance screening.

pip install "pry-scraper @ git+https://codeberg.org/RugMunchMedia/pryscraper.git"

from pry import Pry

client = Pry(api_key="your-key")  # you set the key — fail-closed auth
result = client.scrape("https://example.com/pricing", render=True)
print(result.markdown)

Because it's self-hosted, you own the data and the infrastructure. There are no per-call fees in the self-hosted lane — you bring your own proxy pool and API keys, and Pry handles the hard part: getting past the bot defenses.

The 15-tier anti-bot engine​

Most scrapers have one strategy. When it fails, the scrape fails. Pry escalates through 15 adaptive tiers — from plain HTTP, through header and TLS fingerprint rotation, browser rendering, stealth browser profiles, and CAPTCHA-solving, up to distributed residential-proxy strategies. Each tier is tried in order; the engine learns which tier a target requires and remembers it.

The practical effect: a scrape that would break a single-strategy scraper instead costs a few extra seconds, not a failure. The stealth module is licensed under BSL 1.1 (free for personal, academic, and non-commercial use; converts to MIT in 2029) while the core engine is MIT.

109 store-ready Apify Actors​

If you'd rather not run infrastructure at all, Pry ships as 109 store-ready Apify Actors — from universal-scraper and site-crawler to price-watcher, lead-harvester, sanctions-screener, and niche monitors like EV charging stations and AI model pricing.

Every actor uses pay-per-event (PPE) billing: you're charged per successfully processed item, never for idle time. Pricing is set by an economics model that verifies margins against real infrastructure costs — break-even math, not vibes. Browse the catalog in the Apify Actors docs.

x402: pay per call, across 7 chains​

The third lane is x402 — the HTTP 402 Payment Required protocol that lets clients (especially AI agents) pay for API calls directly from a crypto wallet. No account, no API key: in the x402 lane, the payment is the credential.

Pry accepts payments across seven chains — Base, Ethereum, Arbitrum, Optimism, Polygon (EVM USDC/USDT), Solana (SPL), and TRON (TRC-20) — each with its own CAIP-2-keyed treasury wallet. Settlement is verified on-chain via EIP-3009 transferWithAuthorization, Permit2, or an external facilitator.

curl https://api.pryscraper.com/v1/x402/pricing
# {"scrape": {"price_usd": 0.001, ...}, "crawl": {"price_usd": 0.01, ...}}

An agent that needs one page pays one tenth of a cent. An agent that needs a thousand pays for a bulk crawl. Read the full x402 pay-per-call guide.

Self-host vs cloud hosting​

Self-host if you care about data ownership or need deep customization. Subscribe if you'd rather not run proxies and browsers. Either way, the x402 lane is available per-call with no subscription at all.

Get started​

• Read the quick start
• Browse the 109 Apify Actors
• Wire up x402 payments
• Source code: Codeberg

Questions or commercial licensing: [email protected].

Pry ships 109 store-ready Apify Actors. Hand-writing 109 actor packages — each with spec files, a Dockerfile, pay-per-event wiring, and a store listing — would be a maintenance nightmare. So we didn't. Every actor is generated from a single declarative spec by the open-source Actor Factory.

One spec, a complete actor​

An ActorSpec describes what the actor does and how it's priced. The factory generates everything else:

from pathlib import Path

from actor_factory import ActorSpec, PricingEvent, generate_actor

spec = ActorSpec(
    name="price-watcher",
    title="Price Watcher",
    description="Track e-commerce prices with change detection.",
    version="1.0",
    pricing_events=(
        PricingEvent(
            name="price-captured",
            title="Price captured",
            description="One product price captured.",
            price_usd=0.001,
        ),
    ),
)
generate_actor(spec, Path("actors/price-watcher"))

The output is a complete .actor package: Apify spec files, Dockerfile, PPE runtime that emits events only on successful processing, and a store-ready README. Generated actors pass Apify's automated quality gates on the first publish.

PPE pricing with an economics model​

Pay-per-event pricing is easy to get wrong: price too high and nobody runs the actor; too low and every run loses money. The factory includes an economics model that computes break-even pricing from real inputs — proxy bandwidth, browser minutes, LLM tokens, and Apify's platform fees — then applies a margin.

Two synthetic events (apify-actor-start, apify-default-dataset-item) are billed by the platform automatically; the model accounts for them so the listed per-item prices stay honest. The result: prices from $0.001 to $0.003 per event, verified against cost, not guessed.

The 15 new market-gap actors​

The latest batch targets niches with no first-class Apify coverage — places where demand exists but the store has nothing good:

• App store ranks — Apple App Store + Google Play rankings, reviews, ASO signals
• AI model pricing — LLM API price tracking across providers
• GPU cloud prices — GPU instance pricing across cloud providers
• EV charging stations — locations, pricing, availability
• Insurance rates — public quote and rate signals
• Ticket resale — secondary-market event ticket price monitoring
• …plus market-intel, pricing/rate, and data-pipeline niches

Each of these went through the same pipeline: define the spec, let the economics model set PPE prices, generate, and publish. Total authoring cost per actor: one spec file.

Generate and publish​

The full loop:

# 1. Define your spec (see above), then generate
python -m actor_factory.cli generate actors/price-watcher

# 2. Test locally
cd actors/price-watcher && apify run

# 3. Publish to the Apify Store
apify push

Because every actor shares the same generated skeleton, a fix to the PPE runtime or the anti-bot tier selection propagates to the whole catalog with a regeneration — 109 actors, one source of truth.

Build your own​

The Actor Factory is open source under the Pry repo. If you need an actor we don't ship, define a spec and generate it — or email us and we'll add it to the catalog.

• Apify Actors docs
• Actor Factory source