slow-yellowS
Apify & Crawleeβ€’3y agoβ€’
14 replies
slow-yellow

Could not find file at storage/key_value_stores/default/SDK_SESSION_POOL_STATE.json

Hi there! πŸ‘‹

I'm crawling some pages from different countries using proxy configurations.

Function running crawling:

async crawl(proxyUrl: string, sites: string[]) {
    const config = new Configuration({
      disableBrowserSandbox: true,
    });

    const proxyConfiguration = new ProxyConfiguration({
      proxyUrls: [proxyUrl],
    });

    const urlsHash: Record<string, string[]> = {};

    const crawler = new PuppeteerCrawler(
      {
        proxyConfiguration,
        maxRequestRetries: 0,
        requestHandler: async ({ page }) => {
          const pageUrl = page.url();

          page.on('request', (request) => {
            ...
          });

          await this.autoScroll(page);

          await page.waitForNetworkIdle({
            idleTime: 1000,
          });
        },
      },
      config,
    );

    await crawler.run(sites);

    return urlsHash;
  }


After the first run finished(and a promise is resolved), I call this function again with another params, but get the following error:
Could not find file at /usr/src/app/storage/key_value_stores/default/SDK_SESSION_POOL_STATE.json


Could someone help what is wrong?

I suppose that a session after the first run was cleared and the file from the error message was deleted, but it seems like it expects to have this file.
Was this page helpful?