npm install @hyperbrowser/sdk dotenv

yarn add @hyperbrowser/sdk dotenv

pip install hyperbrowser python-dotenv

uv add hyperbrowser python-dotenv

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const scrapeResult = await client.scrape.startAndWait({
    url: "https://example.com",
  });
  console.log("Scrape result:", scrapeResult);
};

main();

import asyncio
import os
from dotenv import load_dotenv
from hyperbrowser import AsyncHyperbrowser as Hyperbrowser
from hyperbrowser.models import StartScrapeJobParams

# Load environment variables from .env file
load_dotenv()

# Initialize Hyperbrowser client
client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


async def main():
    # Start scraping and wait for completion
    scrape_result = await client.scrape.start_and_wait(
        StartScrapeJobParams(url="https://example.com")
    )
    print("Scrape result:", scrape_result)


if __name__ == "__main__":
    asyncio.run(main())

To view more details check out the Scrape page.

Welcome to Hyperbrowser, the Internet for AI. Get started in minutes with browser automation, web scraping, data extraction, workflow automation, and AI agents.

Get Started in Two Simple Steps

1. Create an Account: Sign up at the dashboard

2. Get Your API Key: Find it on your dashboard

Now, you are ready to explore the many different ways in which you can use Hyperbrowser!

Hyperbrowser provides SDKs in Node and Python.

Welcome to Hyperbrowser, the Internet for AI. Hyperbrowser is the next-generation platform empowering AI agents and enabling effortless, scalable browser automation. Built specifically for AI developers, it eliminates the headaches of local infrastructure and performance bottlenecks, allowing you to focus entirely on building your solutions, rather getting gummed up on browser problems.

Whether you're training AI agents to navigate the web, collecting data for model fine-tuning, testing applications, or simply scraping data, Hyperbrowser lets you launch and manage browser sessions with ease—no complicated setup required. Our platform provides streamlined solutions for all your web scraping needs, from single-page extraction to comprehensive site crawling.

Why Developers Choose Hyperbrowser:

• Instant Scalability - Deploy hundreds of AI agent browser sessions in seconds without infrastructure complexity

• Powerful APIs - Purpose-built APIs for managing sessions, training environments, scraping/crawling sites, and enhancing AI capabilities

• Production-Ready AI Infrastructure - Enterprise-grade reliability and security built specifically for AI workloads

• Advanced Anti-Bot Protection Bypass - Built-in stealth mode, ad blocking, automatic CAPTCHA solving, and rotating proxies for uninterrupted AI operation

• AI-First Design - Native support for multiple AI frameworks including LangChain, LlamaIndex, MCP, and more

Jump right in

npm install @hyperbrowser/sdk dotenv

yarn add @hyperbrowser/sdk dotenv

pip install hyperbrowser python-dotenv

uv add hyperbrowser python-dotenv

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const hbClient = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const result = await hbClient.agents.claudeComputerUse.startAndWait({
    task: "What is the top post on Hacker News?",
  });

  console.log(`Output:\n\n${result.data?.finalResult}`);
};

main().catch((err) => {
  console.error(`Error: ${err.message}`);
});

import os
from hyperbrowser import Hyperbrowser
from hyperbrowser.models import StartBrowserUseTaskParams
from dotenv import load_dotenv

load_dotenv()

hb_client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


def main():
    resp = hb_client.agents.claude_computer_use.start_and_wait(
        StartBrowserUseTaskParams(
            task="What is the top post on Hacker News?"
        )
    )

    print(f"Output:\n\n{resp.data.final_result}")


if __name__ == "__main__":
    try:
        main()
    except Exception as e:
        print(f"Error: {e}")

To view more details check out the Agents Claude Computer Use page.

Hyperbrowser gives you access to leading AI Agents that can execute tasks and use the browser autonomously. Get started easily with Browser Use, Claude Computer Use, and OpenAI CUA.

To see these agents in action without having to set anything up, checkout HyperPilot. You can try out all the latest and greatest AI browser agents.

npm install @hyperbrowser/sdk dotenv

yarn add @hyperbrowser/sdk dotenv

pip install hyperbrowser python-dotenv

uv add hyperbrowser python-dotenv

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const hbClient = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const result = await hbClient.agents.browserUse.startAndWait({
    task: "What is the top post on Hacker News?",
  });

  console.log(`Output:\n\n${result.data?.finalResult}`);
};

main().catch((err) => {
  console.error(`Error: ${err.message}`);
});

import os
from hyperbrowser import Hyperbrowser
from hyperbrowser.models import StartBrowserUseTaskParams
from dotenv import load_dotenv

load_dotenv()

hb_client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


def main():
    resp = hb_client.agents.browser_use.start_and_wait(
        StartBrowserUseTaskParams(
            task="What is the top post on Hacker News?"
        )
    )

    print(f"Output:\n\n{resp.data.final_result}")


if __name__ == "__main__":
    try:
        main()
    except Exception as e:
        print(f"Error: {e}")

To view more details check out the Agents Browser Use page.

npm install @hyperbrowser/sdk dotenv

yarn add @hyperbrowser/sdk dotenv

pip install hyperbrowser python-dotenv

uv add hyperbrowser python-dotenv

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const crawlResult = await client.crawl.startAndWait({
    url: "https://hyperbrowser.ai",
    maxPages: 5,
  });
  console.log("Crawl result:", crawlResult);
};

main();

import asyncio
import os
from dotenv import load_dotenv
from hyperbrowser import AsyncHyperbrowser as Hyperbrowser
from hyperbrowser.models import StartCrawlJobParams

# Load environment variables from .env file
load_dotenv()

# Initialize Hyperbrowser client
client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


async def main():
    # Start crawling and wait for completion
    crawl_result = await client.crawl.start_and_wait(
        StartCrawlJobParams(url="https://hyperbrowser.ai", max_pages=5)
    )
    print("Crawl result:")
    print(crawl_result.model_dump_json(indent=2))


if __name__ == "__main__":
    asyncio.run(main())

To view more details check out the Crawl page.

npm install @hyperbrowser/sdk dotenv zod

yarn add @hyperbrowser/sdk dotenv zod

pip install hyperbrowser python-dotenv pydantic

uv add hyperbrowser python-dotenv pydantic

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";
import { z } from "zod";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const schema = z.object({
    productName: z.string(),
    productOverview: z.string(),
    keyFeatures: z.array(z.string()),
    pricing: z.array(
      z.object({
        plan: z.string(),
        price: z.string(),
        features: z.array(z.string()),
      })
    ),
  });

  // Handles both starting and waiting for extract job response
  const result = await client.extract.startAndWait({
    urls: ["https://hyperbrowser.ai"],
    prompt:
      "Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.",
    schema: schema,
  });

  console.log("result", JSON.stringify(result, null, 2));
};

main();

import os
import json
from typing import List
from dotenv import load_dotenv
from hyperbrowser import Hyperbrowser
from hyperbrowser.models import StartExtractJobParams
from pydantic import BaseModel


# Load environment variables from .env file
load_dotenv()

# Initialize Hyperbrowser client
client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


class PricingSchema(BaseModel):
    plan: str
    price: str
    features: List[str]


class ExtractSchema(BaseModel):
    product_name: str
    product_overview: str
    key_features: List[str]
    pricing: List[PricingSchema]


def main():
    result = client.extract.start_and_wait(
        params=StartExtractJobParams(
            urls=["https://hyperbrowser.ai"],
            prompt="Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.",
            schema=ExtractSchema,
        )
    )
    print("result:", json.dumps(result.data, indent=2))


main()

To view more details check out the Extract page.

npm install @hyperbrowser/sdk dotenv

yarn add @hyperbrowser/sdk dotenv

pip install hyperbrowser python-dotenv

uv add hyperbrowser python-dotenv

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const hbClient = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const result = await hbClient.agents.cua.startAndWait({
    task: "What is the top post on Hacker News?",
  });

  console.log(`Output:\n\n${result.data?.finalResult}`);
};

main().catch((err) => {
  console.error(`Error: ${err.message}`);
});

import os
from hyperbrowser import Hyperbrowser
from hyperbrowser.models import StartBrowserUseTaskParams
from dotenv import load_dotenv

load_dotenv()

hb_client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


def main():
    resp = hb_client.agents.cua.start_and_wait(
        StartBrowserUseTaskParams(
            task="What is the top post on Hacker News?"
        )
    )

    print(f"Output:\n\n{resp.data.final_result}")


if __name__ == "__main__":
    try:
        main()
    except Exception as e:
        print(f"Error: {e}")

To view more details check out the Agents OpenAI CUA page.

npm install puppeteer-core @hyperbrowser/sdk dotenv

yarn add puppeteer-core @hyperbrowser/sdk dotenv

pip install pyppeteer hyperbrowser python-dotenv

uv add pyppeteer hyperbrowser python-dotenv

import { connect } from "puppeteer-core";
import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const session = await client.sessions.create();

  try {
    const browser = await connect({
      browserWSEndpoint: session.wsEndpoint,
      defaultViewport: null,
    });

    const [page] = await browser.pages();

    // Navigate to a website
    console.log("Navigating to Hacker News...");
    await page.goto("https://news.ycombinator.com/");
    const pageTitle = await page.title();
    console.log("Page 1:", pageTitle);
    await page.evaluate(() => {
      console.log("Page 1:", document.title);
    });

    await page.goto("https://example.com");
    console.log("Page 2:", await page.title());
    await page.evaluate(() => {
      console.log("Page 2:", document.title);
    });

    await page.goto("https://apple.com");
    console.log("Page 3:", await page.title());
    await page.evaluate(() => {
      console.log("Page 3:", document.title);
    });

    await page.goto("https://google.com");
    console.log("Page 4:", await page.title());
    await page.evaluate(() => {
      console.log("Page 4:", document.title);
    });
  } catch (err) {
    console.error(`Encountered error: ${err}`);
  } finally {
    await client.sessions.stop(session.id);
  }
};

main();

import asyncio
from pyppeteer import connect
import os
from dotenv import load_dotenv
from hyperbrowser import AsyncHyperbrowser

# Load environment variables from .env file
load_dotenv()

client = AsyncHyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


async def main():
    session = await client.sessions.create()

    try:
        browser = await connect(
            browserWSEndpoint=session.ws_endpoint, defaultViewport=None
        )

        pages = await browser.pages()
        page = pages[0]

        # Navigate to a website
        print("Navigating to Hacker News...")
        await page.goto("https://news.ycombinator.com/")
        page_title = await page.title()
        print("Page 1:", page_title)
        await page.evaluate("() => { console.log('Page 1:', document.title); }")

        await page.goto("https://example.com")
        page_title = await page.title()
        print("Page 2:", page_title)
        await page.evaluate("() => { console.log('Page 2:', document.title); }")

        await page.goto("https://apple.com")
        page_title = await page.title()
        print("Page 3:", page_title)
        await page.evaluate("() => { console.log('Page 3:', document.title); }")

        await page.goto("https://google.com")
        page_title = await page.title()
        print("Page 4:", page_title)
        await page.evaluate("() => { console.log('Page 4:', document.title); }")

    except Exception as e:
        print(f"Error: {e}")
    finally:
        await client.sessions.stop(session.id)


# Run the async main function
if __name__ == "__main__":
    asyncio.run(main())

To see how browser sessions work, check out the Sessions Overview.

Hyperbrowser provides powerful and intuitive endpoints to effortlessly handle your web scraping needs. Whether you're looking to scrape individual web pages, crawl entire sites, or extract structured data, Hyperbrowser simplifies the process with easy to use and robust functionality.

Unlock seamless browser automation with Hyperbrowser. Automate repetitive web tasks, testing, and interactions efficiently using industry-leading automation frameworks. Get started quickly with Puppeteer, Playwright, or Selenium.

Browser Use is an open-source tool designed to make websites accessible for AI agents by enabling them to interact with web pages as a human user would. It provides a framework that allows AI systems to navigate, interpret, and manipulate web content, facilitating tasks such as data extraction, web automation, and testing.

Hyperbrowser's browser-use agent allows you to easily execute agent tasks on the web utilizing browser-use with just a simple call. Hyperbrowser exposes endpoints for starting/stopping a browser-use task and for getting it's status and results.

By default, browser-use tasks are handled in an asynchronous manner of first starting the task and then checking it's status until it is completed. However, if you don't want to handle the monitoring yourself, our SDKs provide a simple function that handles the whole flow and returns the data once the task is completed.

Installation

Usage

Session Configurations

​OpenAI's Computer-Using Agent (CUA) is an advanced AI model designed to interact with computer interfaces much like a human user. It can navigate graphical user interfaces (GUIs), perform tasks such as clicking buttons, typing text, and managing multi-step processes. This capability allows CUA to automate complex workflows without relying on specialized APIs. Actually, this is the model that powers Operator, OpenAI's AI agent capable of executing various web-based tasks like filling out forms, ordering products, and scheduling appointments. Operator utilizes CUA to mimic human interactions within a browser, allowing it to handle repetitive or intricate tasks on behalf of users.

Hyperbrowser's CUA agent allows you to easily execute agent tasks on the web utilizing OpenAI's Computer-Using Agent with just a simple call. Hyperbrowser exposes endpoints for starting/stopping a CUA task and for getting it's status and results.

By default, CUA tasks are handled in an asynchronous manner of first starting the task and then checking it's status until it is completed. However, if you don't want to handle the monitoring yourself, our SDKs provide a simple function that handles the whole flow and returns the data once the task is completed.

Installation

Usage

Session Configurations

Claude's Computer Use allows Claude to directly interact with your computer to perform tasks much like a human. This capability allows Claude to move the cursor, click buttons, type text, and navigate the web, thereby automating complex, multi-step workflows.

Hyperbrowser's Claude Computer Use agent allows you to easily execute agent tasks on the web with just a simple call. Hyperbrowser exposes endpoints for starting/stopping a Claude Computer Use task and for getting it's status and results.

By default, these tasks are handled in an asynchronous manner of first starting the task and then checking it's status until it is completed. However, if you don't want to handle the monitoring yourself, our SDKs provide a simple function that handles the whole flow and returns the data once the task is completed.

Installation

Usage

Session Configurations

The Scrape API allows you to get the data you want from web pages using with a single call. You can scrape page content and capture it's data in various formats.

Hyperbrowser exposes endpoints for starting a scrape request and for getting it's status and results. By default, scraping is handled in an asynchronous manner of first starting the job and then checking it's status until it is completed. However, with our SDKs, we provide a simple function that handles the whole flow and returns the data once the job is completed.

Installation

Usage

Response

The Start Scrape Job POST /scrape endpoint will return a jobId in the response which can be used to get information about the job in subsequent requests.

The Get Scrape Job GET /scrape/{jobId} will return the following data:

The status of a scrape job can be one of pending, running, completed, failed . There can also be other optional fields like error with an error message if an error was encountered, and html and links in the data object depending on which formats are requested for the request.

Session Configurations

You can also provide configurations for the session that will be used to execute the scrape job just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs.

Scrape Configurations

You can also provide optional parameters for the scrape job itself such as the formats to return, only returning the main content of the page, setting the maximum timeout for navigating to a page, etc.

Batch Scrape

Batch Scrape works the same as regular scrape, except instead of a single URL, you can provide a list of up to 1,000 URLs to scrape at once.

Response

The Start Batch Scrape Job POST /scrape/batch endpoint will return a jobId in the response which can be used to get information about the job in subsequent requests.

The Get Batch Scrape Job GET /scrape/batch/{jobId} will return the following data:

The status of a batch scrape job can be one of pending, running, completed, failed . The results of all the scrapes will be an array in the data field of the response. Each scraped page will be returned in the order of the initial provided urls, and each one will have its own status and information.

As with the single scrape, by default, batch scraping is handled in an asynchronous manner of first starting the job and then checking it's status until it is completed. However, with our SDKs, we provide a simple function (client.scrape.batch.startAndWait) that handles the whole flow and returns the data once the job is completed.

The Crawl API allows you to crawl through an entire website and get all it's data with a single call.

Hyperbrowser exposes endpoints for starting a crawl request and for getting it's status and results. By default, crawling is handled in an asynchronous manner of first starting the job and then checking it's status until it is completed. However, with our SDKs, we provide a simple function that handles the whole flow and returns the data once the job is completed.

Installation

npm install @hyperbrowser/sdk

yarn add @hyperbrowser/sdk

pip install hyperbrowser

Usage

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  // Handles both starting and waiting for crawl job response
  const crawlResult = await client.crawl.startAndWait({
    url: "https://hyperbrowser.ai",
  });
  console.log("Crawl result:", crawlResult);
};

main();

import os
from dotenv import load_dotenv
from hyperbrowser import Hyperbrowser
from hyperbrowser.models.crawl import StartCrawlJobParams

# Load environment variables from .env file
load_dotenv()

# Initialize Hyperbrowser client
client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


# Start crawling and wait for completion
crawl_result = client.crawl.start_and_wait(
    StartCrawlJobParams(url="https://hyperbrowser.ai")
)
print("Crawl result:", crawl_result)

curl -X POST https://app.hyperbrowser.ai/api/crawl \
    -H 'Content-Type: application/json' \
    -H 'x-api-key: <YOUR_API_KEY>' \
    -d '{
        "url": "https://hyperbrowser.ai"
    }'

curl https://app.hyperbrowser.ai/api/crawl/{jobId} \
    -H 'x-api-key: <YOUR_API_KEY>'

Response

The Start Crawl Job POST /crawl endpoint will return a jobId in the response which can be used to get information about the job in subsequent requests.

{
    "jobId": "962372c4-a140-400b-8c26-4ffe21d9fb9c"
}

The Get Crawl Job GET /crawl/{jobId} will return the following data:

{
  "jobId": "962372c4-a140-400b-8c26-4ffe21d9fb9c",
  "status": "completed",
  "totalCrawledPages": 2,
  "totalPageBatches": 1,
  "currentPageBatch": 1,
  "batchSize": 20,
  "data": [
    {
      "url": "https://example.com",
      "status": "completed",
      "metadata": {
        "title": "Example Page",
        "description": "A sample webpage"
      },
      "markdown": "# Example Page\nThis is content...",
    },
    ...
  ]
}

The status of a crawl job can be one of pending, running, completed, failed . There can also be other optional fields like error with an error message if an error was encountered, and html and links in the data object depending on which formats are requested for the request.

Unlike the scrape endpoint, the crawl endpoint returns a list in the data field with the all the pages that were crawled in the current page batch. The SDKs also provide a function which will start the crawl job, wait until it's complete, and return all the crawled pages for the entire crawl.

To see the full schema, checkout the API Reference.

Additional Crawl Configurations

The crawl endpoint provides additional parameters you can provide to tailor the crawl to your needs. You can narrow down the pages crawled by setting a limit to the maximum number of pages visited, only including paths that match a certain pattern, excluding paths that match another pattern, etc.

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  // Handles both starting and waiting for crawl job response
  const crawlResult = await client.crawl.startAndWait({
    url: "https://hyperbrowser.ai",
    maxPages: 5,
    includePatterns: ["/blogs/*"],
  });
  console.log("Crawl result:", crawlResult);
};

main();

import os
from dotenv import load_dotenv
from hyperbrowser import Hyperbrowser
from hyperbrowser.models.crawl import StartCrawlJobParams

# Load environment variables from .env file
load_dotenv()

# Initialize Hyperbrowser client
client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


# Start crawling and wait for completion
crawl_result = client.crawl.start_and_wait(
    StartCrawlJobParams(
        url="https://hyperbrowser.ai",
        max_pages=5,
        include_patterns: ["/blogs/*"],
    )
)
print("Crawl result:", crawl_result)

Session Configurations

You can also provide configurations for the session that will be used to execute the crawl job just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs.

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const crawlResult = await client.crawl.startAndWait({
    url: "https://example.com",
    sessionOptions: {
      useProxy: true,
      solveCaptchas: true,
      proxyCountry: "US",
      locales: ["en"],
    },
  });
  console.log("Crawl result:", crawlResult);
};

main();

import os
from dotenv import load_dotenv
from hyperbrowser import Hyperbrowser
from hyperbrowser.models.crawl import StartCrawlJobParams
from hyperbrowser.models.session import CreateSessionParams

# Load environment variables from .env file
load_dotenv()

# Initialize Hyperbrowser client
client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


# Start crawling and wait for completion
crawl_result = client.crawl.start_and_wait(
    StartCrawlJobParams(
        url="https://example.com",
        session_options=CreateSessionParams(use_proxy=True, solve_captchas=True),
    )
)
print("Crawl result:", crawl_result)

Scrape Configurations

You can also provide optional scrape options for the crawl job such as the formats to return, only returning the main content of the page, setting the maximum timeout for navigating to a page, etc.

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const crawlResult = await client.crawl.startAndWait({
    url: "https://example.com",
    scrapeOptions: {
      formats: ["markdown", "html", "links"],
      onlyMainContent: false,
      timeout: 10000,
    },
  });
  console.log("Crawl result:", crawlResult);
};

main();

import os
from dotenv import load_dotenv
from hyperbrowser import Hyperbrowser
from hyperbrowser.models.scrape import ScrapeOptions
from hyperbrowser.models.crawl import StartCrawlJobParams

# Load environment variables from .env file
load_dotenv()

# Initialize Hyperbrowser client
client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


# Start crawling and wait for completion
crawl_result = client.crawl.start_and_wait(
    StartCrawlJobParams(
        url="https://example.com",
        scrape_options=ScrapeOptions(
            formats=["html", "links", "markdown"], only_main_content=False, timeout=10000
        ),
    )
)
print("Crawl result:", crawl_result)

For a full reference on the crawl endpoint, checkout the API Reference, or read the Advanced Scraping Guide to see more advanced options for scraping.

Hyperbrowser profiles allow you to reuse browser state like cookies, local storage, and network cache across multiple sessions. This can improve performance and enable seamless workflows requiring persistent data.

How Profiles Work

A profile is essentially a saved snapshot of a browser's user data directory. By default, each Hyperbrowser session uses a fresh user data directory to ensure isolation.

When you create a profile and then use and persist it in a new session, Hyperbrowser saves that session's user data directory. You can then attach the profile to future sessions to pick up where you left off.

Common use cases for profiles include:

• Reusing authenticated sessions

• Improving page load times with a primed cache

• Preserving application state across sessions

Using Profiles

1. Create a Profile

First, create a new profile using the Profiles API. Note the returned id - you'll need this to attach the profile to sessions.

const profile = await client.profiles.create();
console.log("profile", profile.id);

profile = client.profiles.create()
print("profile", profile.id)

curl -X POST 'https://app.hyperbrowser.ai/api/profile' \
-H 'Content-Type: application/json' \
-H 'x-api-key: YOUR_API_KEY' \
-d '{}'

2. Attach Profile to Session

When creating a new session with the Sessions API, include the id in the profile field:

const session = await client.sessions.create({
  profile: {
    id: "<PROFILE_ID>",
    persistChanges: true, // set to true for the first session of a new profile
  },
});

session = client.sessions.create(
    params=CreateSessionParams(
        profile=CreateSessionProfile(
            id="<PROFILE_ID>",
            # set to true for the first session of a new profile
            persist_changes=True,
        ),
    )
)

curl -X POST 'https://app.hyperbrowser.ai/api/session' \
-H 'Content-Type: application/json' \
-H 'x-api-key: YOUR_API_KEY' \
-d '{
    "profile": {
      "id": "<PROFILE_ID>"
    }
}'

Set "persistChanges": true in the profile object if you want the session to update the profile with any changes to cookies, cache, etc. You will need to do this for the first time you use a new profile in a session so it can be used in subsequent sessions.

3. Reuse Profile

Attach the same id to additional sessions to reuse browser state. Created profiles are reusable until you delete them.

Deleting Profiles

Delete unused profiles to free up resources. Once deleted, a profile is no longer attachable to sessions.

To delete a profile, send a DELETE request to the Profiles API with the target id:

const response = await client.profiles.delete("<PROFILE_ID>");
console.log("response", response);

response = client.profiles.delete("<PROFILE_ID>")
print("response", response)

curl -X DELETE 'https://app.hyperbrowser.ai/api/profile/<PROFILE_ID>' \
-H 'x-api-key: YOUR_API_KEY'

Deletion is irreversible, so be sure you no longer need a profile before deleting.

Session Parameters are shared between the:

• New Session

• Scrape

• Crawl

• Extract

• Agent Tasks

This is a comprehensive list and description of all parameters available when setting up a session.

Enabling Stealth Mode with useStealth

• Type: boolean

• Description: When set to true, the session will be launched in stealth mode, which employs various techniques to make the browser harder to detect as an automated tool.

• Default: false

Using a Proxy with useProxy

• Type: boolean

• Description: When set to true, the session will be launched with a proxy server.

• Default: false

Specifying a Custom Proxy Server with proxyServer

• Type: string

• Description: The hostname or IP address of the proxy server to use for the session. This option is only used when useProxy is set to true.

• Default: undefined

Providing Proxy Server Authentication with proxyServerUsername and proxyServerPassword

• Type: string

• Description: The username and password to use for authenticating with the proxy server, if required. These options are only used when useProxy is set to true and the proxy server requires authentication.

• Default: undefined

Selecting a Proxy Location Country with proxyCountry

• Type: string

• Enum: ["US", "GB", "CA", ...]

• Description: The country where the proxy server should be located.

• Default: "US"

Specifying Operating Systems with operatingSystems

• Type: array

• Items: string

• Enum: ["windows", "android", "macos", "linux", "ios"]

• Description: An array of operating systems to use for fingerprinting.

• Default: undefined

Choosing Device Types with device

• Type: array

• Items: string

• Enum: ["desktop", "mobile"]

• Description: An array of device types to use for fingerprinting.

• Default: undefined

Selecting Browser Platforms with platform

• Type: array

• Items: string

• Enum: ["chrome", "firefox", "safari", "edge"]

• Description: An array of browser platforms to use for fingerprinting.

• Default: undefined

Setting Browser Locales with locales

• Type: array

• Items: string

• Enum: ["en", "es", "fr", ...]

• Description: An array of browser locales to specify the language for the browser.

• Default: ["en"]

Customizing Screen Resolution with screen

• Type: object

• Properties:

  • width (number, default 1280): The screen width in pixels.

  • height (number, default 720): The screen height in pixels.

• Description: An object specifying the screen resolution to emulate in the session.

Solving CAPTCHAs Automatically with solveCaptchas

• Type: boolean

• Description: When set to true, the session will attempt to automatically solve any CAPTCHAs encountered during the session.

• Default: false

Blocking Ads with adblock

• Type: boolean

• Description: When set to true, the session will attempt to block ads and other unwanted content during the session.

• Default: false

Blocking Trackers with trackers

• Type: boolean

• Description: When set to true, the session will attempt to block web trackers and other privacy-invasive technologies during the session.

• Default: false

Blocking Annoyances with annoyances

• Type: boolean

• Description: When set to true, the session will attempt to block common annoyances like pop-ups, overlays, and other disruptive elements during the session.

• Default: false

Automatically accepting cookies with acceptCookies

• Type: boolean

• Description: When set to true, the session will attempt to accept all cookies on sites that are visited.

• Default: false

Enabling web recording with enableWebRecording

• Type: boolean

• Description: When set to true, the session will be recorded using rrweb.

• Default: true

• Learn more

Reuse browser state(cookies, local storage, etc.) across sessions with profile

• Type: object

• Properties:

  • id

    • Type: string

    • Description: ID of the profile to use for the browser session.

  • persistChanges

    • Type: boolean

    • Description: When set to true, changes will be persisted to this profile on session close.

• Description: An object specifying the profile configuration.

• Learn more

The Extract API allows you to get data in a structured format for any provided URLs with a single call.

Hyperbrowser exposes endpoints for starting an extract request and for getting it's status and results. By default, extracting is handled in an asynchronous manner of first starting the job and then checking it's status until it is completed. However, with our SDKs, we provide a simple function that handles the whole flow and returns the data once the job is completed.

Installation

npm install @hyperbrowser/sdk dotenv zod

yarn add @hyperbrowser/sdk dotenv zod

pip install hyperbrowser python-dotenv pydantic

Usage

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";
import { z } from "zod";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const schema = z.object({
    productName: z.string(),
    productOverview: z.string(),
    keyFeatures: z.array(z.string()),
    pricing: z.array(
      z.object({
        plan: z.string(),
        price: z.string(),
        features: z.array(z.string()),
      })
    ),
  });

  // Handles both starting and waiting for extract job response
  const result = await client.extract.startAndWait({
    urls: ["https://hyperbrowser.ai"],
    prompt:
      "Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.",
    schema: schema,
  });

  console.log("result", JSON.stringify(result, null, 2));
};

main();

import os
import json
from typing import List
from dotenv import load_dotenv
from hyperbrowser import Hyperbrowser
from hyperbrowser.models import StartExtractJobParams
from pydantic import BaseModel


# Load environment variables from .env file
load_dotenv()

# Initialize Hyperbrowser client
client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


class PricingSchema(BaseModel):
    plan: str
    price: str
    features: List[str]


class ExtractSchema(BaseModel):
    product_name: str
    product_overview: str
    key_features: List[str]
    pricing: List[PricingSchema]


def main():
    result = client.extract.start_and_wait(
        params=StartExtractJobParams(
            urls=["https://hyperbrowser.ai"],
            prompt="Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.",
            schema=ExtractSchema,
        )
    )
    print("result:", json.dumps(result.data, indent=2))


main()

curl -X POST https://app.hyperbrowser.ai/api/extract \
    -H 'Content-Type: application/json' \
    -H 'x-api-key: <YOUR_API_KEY>' \
    -d '{
        "urls": ["https://hyperbrowser.ai"],
        "prompt": "Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.",
        "schema": {
          "type": "object",
          "properties": {
            "productName": {
              "type": "string"
            },
            "productOverview": {
              "type": "string"
            },
            "keyFeatures": {
              "type": "array",
              "items": {
                "type": "string"
              }
            },
            "pricing": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "plan": {
                    "type": "string"
                  },
                  "price": {
                    "type": "string"
                  },
                  "features": {
                    "type": "array",
                    "items": {
                      "type": "string"
                    }
                  }
                },
                "required": [
                  "plan",
                  "price",
                  "features"
                ]
              }
            }
          },
          "required": [
            "productName",
            "productOverview",
            "keyFeatures",
            "pricing"
          ]
        }
    }'

curl https://app.hyperbrowser.ai/api/extract/{jobId} \
    -H 'x-api-key: <YOUR_API_KEY>'

You can configure the extract request with the following parameters:

• urls - A required list of urls you want to use to extract data from. To allow crawling for any of the urls provided in the list, simply add /* to the end of the url (https://hyperbrowser.ai/*). This will crawl other pages on the site with the same origin and find relevant pages to use for the extraction context.

• schema - A strict json schema you want the returned data to be structured as. Gives the best results.

• prompt - A prompt describing how you want the data structured. Useful if you don't have a specific schema in mind.

• maxLinks - The maximum number of links to look for if performing a crawl for any given url.

• waitFor - A delay in milliseconds to wait after the page loads before initiating the scrape to get data for extraction from page. This can be useful for allowing dynamic content to fully render. This is also useful for waiting to detect CAPTCHAs on the page if you have solveCaptchas set to true in the sessionOptions.

• sessionOptions - Options for the session.

You must provide either a schema or a prompt in your request, and if both are provided the schema takes precedence.

For the Node SDK, you can simply pass in a zod schema for ease of use or an actual json schema. For the Python SDK, you can pass in a pydantic model or an actual json schema.

Response

The Start Extract Job POST /extract endpoint will return a jobId in the response which can be used to get information about the job in subsequent requests.

{
    "jobId": "962372c4-a140-400b-8c26-4ffe21d9fb9c"
}

The Get Extract Job GET /extract/{jobId} will return the following data:

{
  "jobId": "962372c4-a140-400b-8c26-4ffe21d9fb9c",
  "status": "completed",
  "data": {
    "pricing": [
      {
        "plan": "Free",
        "price": "$0",
        "features": [
          "3,000 Credits Included",
          "5 Concurrent Browsers",
          "7 Days Data Retention",
          "Basic Stealth Mode"
        ]
      },
      {
        "plan": "Startup",
        "price": "$30 / Month",
        "features": [
          "18,000 Credits Included",
          "25 Concurrent Browsers",
          "30 Day Data Retention",
          "Auto Captcha Solving",
          "Basic Stealth Mode"
        ]
      },
      {
        "plan": "Scale",
        "price": "$100 / Month",
        "features": [
          "60,000 Credits Included",
          "100 Concurrent Browsers",
          "30 Day Data Retention",
          "Auto Captcha Solving",
          "Advanced Stealth Mode"
        ]
      },
      {
        "plan": "Enterprise",
        "price": "Custom",
        "features": [
          "Volume discounts available",
          "Premium Support",
          "HIPAA/SOC 2",
          "250+ Concurrent Browsers",
          "180+ Day Data Retention",
          "Auto Captcha Solving",
          "Advanced Stealth Mode"
        ]
      }
    ],
    "keyFeatures": [
      "Run headless browsers to automate tasks like web scraping, testing, and form filling.",
      "Use browsers to scrape and structure web data at scale for analysis and insights.",
      "Integrate with AI agents to enable browsing, data collection, and interaction with web apps.",
      "Automatically solve captchas to streamline automation workflows.",
      "Operate browsers in stealth mode to bypass bot detection and stay undetected.",
      "Manage browser sessions with logging, debugging, and secure resource isolation."
    ],
    "productName": "Hyperbrowser",
    "productOverview": "Hyperbrowser is a platform for running and scaling headless browsers in secure, isolated containers. Built for web automation and AI-driven use cases."
  }
}

The status of an extract job can be one of pending, running, completed, failed . There can also be an optional error field with an error message if an error was encountered.

To see the full schema, checkout the API Reference.

Session Configurations

You can also provide configurations for the session that will be used to execute the extract job just as you would when creating a new session itself. These could include using a proxy or solving CAPTCHAs. To see the full list of session configurations, checkout the Session API Reference.

import { config } from "dotenv";
import { z } from "zod";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const schema = z.object({
    productName: z.string(),
    productOverview: z.string(),
    keyFeatures: z.array(z.string()),
    pricing: z.array(
      z.object({
        plan: z.string(),
        price: z.string(),
        features: z.array(z.string()),
      })
    ),
  });

  const result = await client.extract.startAndWait({
    urls: ["https://hyperbrowser.ai"],
    prompt:
      "Extract the product name, an overview of the product, its key features, and its pricing plans from the page.",
    schema: schema,
    // include sessionOptions
    sessionOptions: {
      useProxy: true,
      solveCaptchas: true,
    },
  });

  console.log("result", JSON.stringify(result, null, 2));
};

main();

import os
from dotenv import load_dotenv
from hyperbrowser import Hyperbrowser
from hyperbrowser.models.extract import StartExtractJobParams
from pydantic import BaseModel


load_dotenv()


client = Hyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


class PricingSchema(BaseModel):
    plan: str
    price: str
    features: List[str]


class ExtractSchema(BaseModel):
    product_name: str
    product_overview: str
    key_features: List[str]
    pricing: List[PricingSchema]


def main():
    result = client.extract.start_and_wait(
        params=StartExtractJobParams(
            urls=["https://hyperbrowser.ai"],
            prompt="Extract the product name, an overview of the product, its key features, and a list of its pricing plans from the page.",
            schema=ExtractSchema,
            # include session_options
            session_options=CreateSessionParams(use_proxy=True, solve_captchas=True),
        )
    )
    print("result:", json.dumps(result.data, indent=2))


main()

For a full reference on the extract endpoint, checkout the API Reference.

Hyperbrowser's Live View feature allows you to observe and interact with your browser sessions in real-time. You can use Live View to:

• Debug and troubleshoot your scripts

• Monitor the progress of long-running automations

• Provide a way for end-users to interact with the browser

• Perform hybrid workflows, such as ones where human-in-loop interactions are required.

How it Works

Live View works by providing a secure, authenticated URL that embeds a live stream of the browser session. The URL can be accessed in any modern web browser.

Whenever you create a new session or get the details of an existing session, Hyperbrowser returns a liveUrl field with a unique URL tied to that specific session. The URL remains valid as long as the session is active and the token in the URL hasn't expired (Securing Live View).

const session = await hbClient.sessions.create();
const liveUrl = session.liveUrl;

session = hb_client.sessions.create()
live_url = session.live_url

The returned liveUrl will look something like:

https://app.hyperbrowser.ai/live?token=<TOKEN>&keepAlive=true

You can now open this URL in a web browser to view the live session.

Embedding Live View

You can embed a Live View into your own web application using an iframe:

<iframe src="<LIVE_URL>"></iframe>

This is useful for providing a seamless experience to your end-users. For example, you could use an embedded Live View to:

• Show the progress of a web scraping job

• Allow users to complete a complex workflow that requires manual intervention

• Provide a preview of an automated process before committing it

Securing Live View

Live View URLs are secured with authentication and encryption. Only users with the correct URL can access the Live View.

However, anyone with the URL can view (and potentially interact with) the session. Be sure to protect Live View URLs as sensitive secrets, especially if you're embedding them in a public web page.

The token in the liveUrl will expire after 12 hours. In this case, you can simply call the GET request for the Sessions API to get the details for the given session id which will also return a new liveUrl with a refreshed token.

Hyperbrowser allows you to record and replay your browser sessions. It uses rrweb, an open-source web session replay library. Session recordings let you:

• Visually debug test failures and errors

• Analyze user behavior and interactions

• Share reproducible bug reports

• Save and archive session data

Enabling Session Recording

To record a session, set the enableWebRecording option to true when creating a new Hyperbrowser session:

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const session = await client.sessions.create({
    enableWebRecording: true,
  });
};

main();

import asyncio
import os
from dotenv import load_dotenv
from hyperbrowser import AsyncHyperbrowser
from hyperbrowser.models.session import CreateSessionParams

load_dotenv()

client = AsyncHyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


async def main():
    session = await client.sessions.create(
        params=CreateSessionParams(
            enable_web_recording=True,
        )
    )


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

This will record all browser interactions, DOM changes, and network requests for the duration of the session.

Retrieving Recordings

1. Note the id of the session you want to replay

2. Use the Session Recordings API to download the recording, or if you are using the SDKs, you can just call the getRecording function:

import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";

config();

const client = new Hyperbrowser({
  apiKey: process.env.HYPERBROWSER_API_KEY,
});

const main = async () => {
  const recordingData = await client.sessions.getRecording(
    "91e96d43-0dd2-4882-8d3a-613b12583ba2"
  );
};

main();

import asyncio
import os
from dotenv import load_dotenv
from hyperbrowser import AsyncHyperbrowser

load_dotenv()

client = AsyncHyperbrowser(api_key=os.getenv("HYPERBROWSER_API_KEY"))


async def main():
    recording_data = await client.sessions.get_recording(
        "91e96d43-0dd2-4882-8d3a-613b12583ba2"
    )


if __name__ == "__main__":
    import asyncio

    asyncio.run(main())

curl -X GET 'https://app.hyperbrowser.ai/api/session/{sessionId}/recording' \
-H 'x-api-key: YOUR_API_KEY

The recording data will be returned in rrweb's JSON format.

Replaying Recordings

Using rrweb's Player

Here's an example of using rrweb's player to replay a recording:

1. Include the rrweb player script on your page:

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/rrweb@latest/dist/rrweb.min.css"/>
<script src="https://cdn.jsdelivr.net/npm/rrweb@latest/dist/rrweb.min.js"></script>

1. Add a container element for your player

<div id="player"></div>

1. Initialize the player with your recording data

// if using rrweb npm package
import rrwebPlayer from "rrweb-player";
import "rrweb-player/dist/style.css";

const recordingData = YOUR_RECORDING_DATA

const replayer = new rrwebPlayer({
  target: document.getElementById('player'),
  props: {
    events: recordingData,
    showController: true,
    autoPlay: true,
  },
});