Downloading an image using puppeteer example

Hi, I found this simple example using Puppeteer, that downloads images as you visit the page and I'm wonderign how I can encorporate it into my crawlee scraper.

    this.page.on('response', async (response) => {
      const matches = /.*\.(jpg|png|svg|gif)$/.exec(response.url());

      console.log(matches);

      if (matches && (matches.length === 2)) {
        const extension = matches[1];
        const buffer = await response.buffer();

        fs.writeFileSync(`downloads/${this.request.userData}.${extension}`, buffer, 'base64');
        counter += 1;
      }
    });

    this.page.on('response', async (response) => {
      const matches = /.*\.(jpg|png|svg|gif)$/.exec(response.url());

      console.log(matches);

      if (matches && (matches.length === 2)) {
        const extension = matches[1];
        const buffer = await response.buffer();

        fs.writeFileSync(`downloads/${this.request.userData}.${extension}`, buffer, 'base64');
        counter += 1;
      }
    });

Apify & Crawlee•4y ago•

2 replies

primary-violet

Downloading an image using puppeteer example

Hi, I found this simple example using Puppeteer, that downloads images as you visit the page and I'm wonderign how I can encorporate it into my crawlee scraper.

    this.page.on('response', async (response) => {
      const matches = /.*\.(jpg|png|svg|gif)$/.exec(response.url());

      console.log(matches);

      if (matches && (matches.length === 2)) {
        const extension = matches[1];
        const buffer = await response.buffer();

        fs.writeFileSync(`downloads/${this.request.userData}.${extension}`, buffer, 'base64');
        counter += 1;
      }
    });

    this.page.on('response', async (response) => {
      const matches = /.*\.(jpg|png|svg|gif)$/.exec(response.url());

      console.log(matches);

      if (matches && (matches.length === 2)) {
        const extension = matches[1];
        const buffer = await response.buffer();

        fs.writeFileSync(`downloads/${this.request.userData}.${extension}`, buffer, 'base64');
        counter += 1;
      }
    });

Continue the conversation

Join the Discord to ask follow-up questions and connect with the community

A&C

Apify & Crawlee

This is the official developer community of Apify and Crawlee.

14,851 Members

Join

Downloading an image using puppeteer example

Downloading an image using puppeteer example

Continue the conversation

Apify & Crawlee

Continue the conversation

Apify & Crawlee

Similar Threads