1 reply

How to persist the context with Crawlee?

Hey,
so I am fairly new to Crawlee and have spent the better part of two days trying to figure out how to have persistent context when crawling with Crawlee.

I am scraping a website that requires login, and I would like to avoid the overhead of always logging in when a new crawler is ran.

I am able to very easily create and persist the context and cookies in the project root directory without Crawlee, only using PlaywrightCrawler like so:

import { firefox } from 'playwright';
const browser = await firefox.launchPersistentContext('./persistent-context', {
  headless: false,
});

import { firefox } from 'playwright';
const browser = await firefox.launchPersistentContext('./persistent-context', {
  headless: false,
});

import { firefox } from 'playwright';
const browser = await firefox.launchPersistentContext('./persistent-context', {
  headless: false,
});

import { firefox } from 'playwright';
const browser = await firefox.launchPersistentContext('./persistent-context', {
  headless: false,
});

can anyone point me in the right direction on how to achieve the same with Crawlee?

How to persist the context with Crawlee?

Similar Threads

Similar Threads

Similar Threads