How to prevent following redirects to other domains?

I see there is a way to prevent this once page loads with something like this:

    await page.setRequestInterception(true);

    page.on('request', async (request) => {
      if (ok) {
        await request.continue();
      } else {
        await request.abort();
      }
    }

    await page.setRequestInterception(true);

    page.on('request', async (request) => {
      if (ok) {
        await request.continue();
      } else {
        await request.abort();
      }
    }

But how about when I have a url in queue, is there a nice way to have the request interception setup in the crawler options instead and have it defined globally?

Apify & Crawlee•4y ago•

4 replies

primary-violet

How to prevent following redirects to other domains?

I see there is a way to prevent this once page loads with something like this:

    await page.setRequestInterception(true);

    page.on('request', async (request) => {
      if (ok) {
        await request.continue();
      } else {
        await request.abort();
      }
    }

    await page.setRequestInterception(true);

    page.on('request', async (request) => {
      if (ok) {
        await request.continue();
      } else {
        await request.abort();
      }
    }

But how about when I have a url in queue, is there a nice way to have the request interception setup in the crawler options instead and have it defined globally?

Continue the conversation

Join the Discord to ask follow-up questions and connect with the community

A&C

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

15,108 Members

Join

How to prevent following redirects to other domains?

How to prevent following redirects to other domains?

Continue the conversation

Apify & Crawlee

Continue the conversation

Apify & Crawlee

Similar Threads