Crawl
Crawl a site and all it's links.
2
Setup your Environment
To use Hyperbrowser with your code, you will need an API Key. You can get one easily from the dashboard. Once you have your API Key, add it to your .env
file as HYPERBROWSER_API_KEY
.
3
Crawl a Site
Next, you can crawl any site by simply setting up the Hyperbrowser client and providing the site's url.
import { Hyperbrowser } from "@hyperbrowser/sdk";
import { config } from "dotenv";
config();
const client = new Hyperbrowser({
apiKey: process.env.HYPERBROWSER_API_KEY,
});
const main = async () => {
const crawlResult = await client.crawl.startAndWait({
url: "https://hyperbrowser.ai",
maxPages: 5,
});
console.log("Crawl result:", crawlResult);
};
main();
4
View Crawl in Dashboard
You can view all your crawls in the dashboard and see all their related information.
To view more details check out the Crawl page.
Last updated