correct-apricot•13mo ago

why is crawlee running old deleted code for the first url?

main.ts

import { PlaywrightCrawler } from 'crawlee';
import { router } from './routes.js';

const startUrls = [
  11 links
  ...
];

const crawler = new PlaywrightCrawler({
    requestHandler: router,
    maxRequestsPerCrawl: 1,
    maxRequestRetries: 0,
    maxRequestsPerMinute: 16,
    maxConcurrency: 1,
    headless: false,
});

await crawler.run(startUrls);

import { PlaywrightCrawler } from 'crawlee';
import { router } from './routes.js';

const startUrls = [
  11 links
  ...
];

const crawler = new PlaywrightCrawler({
    requestHandler: router,
    maxRequestsPerCrawl: 1,
    maxRequestRetries: 0,
    maxRequestsPerMinute: 16,
    maxConcurrency: 1,
    headless: false,
});

await crawler.run(startUrls);

I've updated route.ts and after npm start everything works fine for the last 10 links but for the first link it is running old deleted code for some reason(confirmed by testing multiple times, editing different things). I tried deleting request queue but same thing happened. I'm at a loss for what is happening.

3 Replies

Marco•13mo ago

Hello! This is not an expected behavior: could you share more details? Why do you say it's running deleted code? Are you running it locally or on the platform?

Pepa J•13mo ago

Hi @Big Duck , It does sound like some leftovers in the RequestQueeue. How do you run the Actor?

Oleg V.•12mo ago

also, if You are using VS code, You possibly have "auto-save" option being disabled ?

why is crawlee running old deleted code for the first url?

Did you find this page helpful?