fascinating-indigo•3y ago

How to pass UserData when executing crawler

When I do await crawler.run(['https://crawlee.dev'], { userData: { depth: 0 } }); I got this error: Uncaught ArgumentError ArgumentError: Did not expect property userData to exist, got [object Object] in object options How can I set userData in option? `

4 Replies

HonzaS•3y ago

await crawler.run([{url: 'https://crawlee.dev', userData: { depth: 0 } }]);

Alexey Udovydchenko•3y ago

userData is part of request, not part of crawler

afraid-scarlet•3y ago

You passed the userData as the second parameter of the run() method instead of within the request. Hence the ArgumentError. Try this instead:

await crawler.run([{
    url: 'https://crawlee.dev',
    userData: {
        depth: 0,
    },
}]);

await crawler.run([{
    url: 'https://crawlee.dev',
    userData: {
        depth: 0,
    },
}]);

rare-sapphire•3y ago

Also, something that was not obvious to me was how to get the user data on the route. you can do it like so

const routeHandler = async ({ request, crawler, page, log }) => {
    // get user data
    let { depth } = request.userData  
}

const routeHandler = async ({ request, crawler, page, log }) => {
    // get user data
    let { depth } = request.userData  
}

How to pass UserData when executing crawler

Did you find this page helpful?