7 replies

custom CheerioCrawler User-Agent?

hi, could anyone show me an example of CheerioCrawler User-Agent? I had tried

preNavigationHooks

preNavigationHooks

but got a stranger error

  preNavigationHooks: [
    function customUserAgent(_ctx, opts = {}) {
      opts.headers = opts.headers || {};
      opts.useHeaderGenerator = true;
      opts.headerGeneratorOptions = {
        browsers: [ 'chrome', 'safari' ],
        devices: [ 'mobile' ],
        operatingSystems: [ 'android', 'ios' ],
      };
    },
  ]

  preNavigationHooks: [
    function customUserAgent(_ctx, opts = {}) {
      opts.headers = opts.headers || {};
      opts.useHeaderGenerator = true;
      opts.headerGeneratorOptions = {
        browsers: [ 'chrome', 'safari' ],
        devices: [ 'mobile' ],
        operatingSystems: [ 'android', 'ios' ],
      };
    },
  ]

but got error:

RequestError: No headers based on this input can be generated.

RequestError: No headers based on this input can be generated.

I had tried use this options with

header-generator

header-generator

then it works, but don't work with CheerioCrawler hooks

custom CheerioCrawler User-Agent?

Similar Threads