Apify

Apify Discord Mirror

Updated last year

New fingerprint per new request with PlaywrightCrawler/Firefox ?

At a glance

The community member wants to achieve a unique fingerprint for every request in a Playwright Crawler, where cookies and sessions are not shared between requests. They have tried the hack of setting retireBrowserAfterPageCount=2 in browserPoolOptions, which gives a unique fingerprint every two requests, but it's not a perfect solution.

In the comments, another community member suggests two options: 1) setting retireBrowserAfterPageCount=1, which will open a new browser window for every request but may have worse performance, and 2) trying the "hacky" experimental containers feature in Crawlee.

There is no explicitly marked answer in the comments.

Useful resources
nnew_in_town
·
Hi all,
what I want to achieve:

  • every request should have unique fingerprint - this is important!
  • cookies, etc. not shared between requests
  • PlaywrightCrawler
  • no sessions - every request is independent, (no login or similar)
  • Firefox
  • performance/throughput is not a number one prio
At the moment I almost have this with the hack retireBrowserAfterPageCount=2 in browserPoolOptions: this gives a unique fingerprint every two requests, which... isn't perfect (and starting a new browser instance so often looks strange)

In this thread: https://discord.com/channels/801163717915574323/1060467542616965150
a solution for browser pool (without crawler) was suggested.

I would like to have both: new fingerprint per request and PlaywrightCrawler.
Is it possible?
L
1 comment
LLukas Krivka
Well, you can do retireBrowserAfterPageCount=1, it just has a bit worse performance since it needs to open new browser window every time.

Second option is to try hacky https://crawlee.dev/api/puppeteer-crawler/interface/PuppeteerLaunchContext#experimentalContainers
Add a reply
Sign up and join the conversation on Discord