optimistic-gold•2y ago

Crawle failed creepjs test

i use crawlee and here is my setup https://gist.github.com/lechuhuuha/79cca650becde698949877e7f93e584e here is the node modules version

  "dependencies": {
    "chalk": "^2.4.2",
    "crawlee": "^3.8.2",
    "dotenv": "^8.0.0",
    "ghost-cursor": "^1.2.0",
    "ghost-cursor-playwright": "^2.1.0",
    "mz": "^2.7.0",
    "playwright": "^1.42.1",
    "puppeteer": "19.8.0",
    "puppeteer-afp": "^1.0.5",
    "puppeteer-extra": "2.1.6",
    "puppeteer-extra-plugin-adblocker": "^2.13.6",
    "puppeteer-extra-plugin-anonymize-ua": "^2.4.6",
    "puppeteer-extra-plugin-stealth": "2.11.2",
    "puppeteer-extra-plugin-user-preferences": "^2.4.1",
    "request": "2.88.2",
    "request-promise": "0.0.1",
    "simple-node-logger": "^21.8.12",
    "simple-youtube-api": "5.1.1",
    "xvfb": "^0.4.0",
    "yt-search": "2.10.4"
  },

  "dependencies": {
    "chalk": "^2.4.2",
    "crawlee": "^3.8.2",
    "dotenv": "^8.0.0",
    "ghost-cursor": "^1.2.0",
    "ghost-cursor-playwright": "^2.1.0",
    "mz": "^2.7.0",
    "playwright": "^1.42.1",
    "puppeteer": "19.8.0",
    "puppeteer-afp": "^1.0.5",
    "puppeteer-extra": "2.1.6",
    "puppeteer-extra-plugin-adblocker": "^2.13.6",
    "puppeteer-extra-plugin-anonymize-ua": "^2.4.6",
    "puppeteer-extra-plugin-stealth": "2.11.2",
    "puppeteer-extra-plugin-user-preferences": "^2.4.1",
    "request": "2.88.2",
    "request-promise": "0.0.1",
    "simple-node-logger": "^21.8.12",
    "simple-youtube-api": "5.1.1",
    "xvfb": "^0.4.0",
    "yt-search": "2.10.4"
  },

whenever i go to https://abrahamjuliot.github.io/creepjs/ the website always show me 0 score Its say i lied about these thing in images how can i fix this with crawlee or playwright thanks

Gist

crawlee.js

GitHub Gist: instantly share code, notes, and snippets.

6 Replies

optimistic-goldOP•2y ago

bump

Saurav Jain•2y ago

Our team will reply here soon. Expect a little delay due to few holidays. 🙂

ondro_k•2y ago

Hi, it looks like fingerprints are the cause. I got 50%+ score with fingerprints disabled. You can disable it by adding

browserPoolOptions: {
    useFingerprints: false,
},

browserPoolOptions: {
    useFingerprints: false,
},

to PlaywrightCrawler's constructor

optimistic-goldOP•2y ago

oke thanks verymuch will try its out how can we add playwright steath to its and what its compare to solution provide by crawlee

ondro_k•2y ago

you can follow this tutorial to add playwright-stealth

Using puppeteer-extra and playwright-extra | Crawlee

puppeteer-extra and playwright-extra are community-built

ondro_k•2y ago

The comparisson is here

GitHub

GitHub - apify/fingerprint-suite: Browser fingerprinting tools for ...

Browser fingerprinting tools for anonymizing your scrapers. Developed by Apify. - apify/fingerprint-suite

Crawle failed creepjs test

Did you find this page helpful?