Make PlaywrightCrawler less unique and avoid blocking? (canvas/fonts/plugins/permissions...)

I checked my program (PlaywrightCrawler) against this thing: https://amiunique.org/fingerprint
Used US residential proxy, did 3 screenshots, see below
It seems - there are some areas where Crawlee could do better (be less unique, less detectable)!

Here the list (these things are red on the screenshots):
- User Agent (I used fingerprint generator for this!)
- Canvas
- Navigator properties
- List of fonts
- List of plugins
- Permissions

Some settings in my PlaywrightCrawler:
useFingerprints: trueuseFingerprints: true, useFingerprintCache: falseuseFingerprintCache: false, launcher: firefoxlauncher: firefox

Regarding list of plugins: I use some JS code (pluginContentpluginContent string) taken from here: crawlee-jsCrawlee vs bot detection systems - Plugins length is not OK
and inject it into page this way:

    preNavigationHooks: [
        async ({ page, request }) => {
            await page.addInitScript({ content: pluginContent });
        },

    preNavigationHooks: [
        async ({ page, request }) => {
            await page.addInitScript({ content: pluginContent });
        },

Well, this code/hack... it simulates presence of some PDF plugins... but I have an impression there are better solutions for plugins/fonts/permissions...

My Fingerprint- Am I Unique ?

Check if your browser has a unique fingerprint, how identifiable you are on the Internet

Apify & Crawlee•3y ago•

10 replies

sacred-emerald

Make PlaywrightCrawler less unique and avoid blocking? (canvas/fonts/plugins/permissions...)

    preNavigationHooks: [
        async ({ page, request }) => {
            await page.addInitScript({ content: pluginContent });
        },

    preNavigationHooks: [
        async ({ page, request }) => {
            await page.addInitScript({ content: pluginContent });
        },

Well, this code/hack... it simulates presence of some PDF plugins... but I have an impression there are better solutions for plugins/fonts/permissions...