extended-salmon•12mo ago

How to launch a crawlee browser that I can manually pass the cloudfare anti-bot protection

This is my code to launch the browser with headless: false mode. I mannualy input the URL and try to pass the captcha challenges. But the challenges keep failing This is the code

const { launchPuppeteer } = require('crawlee');
const puppeteerExtra = require('puppeteer-extra');
const stealthPlugin = require('puppeteer-extra-plugin-stealth');

// Use the stealth plugin
puppeteerExtra.use(stealthPlugin());

const main = async () => {
    // Launch the browser without running any crawl
    const browser = await launchPuppeteer({
        // !!! You need to specify this option to tell Crawlee to use puppeteer-extra as the launcher !!!
        launcher: puppeteerExtra,
        launchOptions: {
            // Other puppeteer options work as usual
            headless: false,
        },
    });

    // Create and navigate new page 
    console.log('Open target page');
    const page = await browser.newPage();

    // Now you can play around with the browser
    console.log('Browser launched. You can now interact with it.');
    
    // Keep the script running
    await new Promise((resolve) => {
        console.log('Press Ctrl+C to exit.');
    });
};

main().catch(console.error);

const { launchPuppeteer } = require('crawlee');
const puppeteerExtra = require('puppeteer-extra');
const stealthPlugin = require('puppeteer-extra-plugin-stealth');

// Use the stealth plugin
puppeteerExtra.use(stealthPlugin());

const main = async () => {
    // Launch the browser without running any crawl
    const browser = await launchPuppeteer({
        // !!! You need to specify this option to tell Crawlee to use puppeteer-extra as the launcher !!!
        launcher: puppeteerExtra,
        launchOptions: {
            // Other puppeteer options work as usual
            headless: false,
        },
    });

    // Create and navigate new page 
    console.log('Open target page');
    const page = await browser.newPage();

    // Now you can play around with the browser
    console.log('Browser launched. You can now interact with it.');
    
    // Keep the script running
    await new Promise((resolve) => {
        console.log('Press Ctrl+C to exit.');
    });
};

main().catch(console.error);

This is the test URL https://www.nivod.cc/

4 Replies

Hall•12mo ago

View post on community site

This post has been pushed to the community knowledgebase. Any replies in this thread will be synced to the community site.

Apify Community

extended-salmonOP•12mo ago

This is what happened to me in video

Oleg V.•12mo ago

This section should help: https://docs.apify.com/academy/anti-scraping/mitigation Try to play around with proxies and fingerprints.

Mitigation | Academy | Apify Documentation

After learning about the various different anti-scraping techniques websites use, learn how to mitigate them with a few different techniques.

Oleg V.•12mo ago

Also, try to use Playwright + firefox . It's more undetectable than pptr

How to launch a crawlee browser that I can manually pass the cloudfare anti-bot protection

Did you find this page helpful?