quickest-silver•3y ago

Parse RSS XML

I'm trying to parse a RSS feed, I add it to additionalMimeTypes, but then it writes a ERROR CheerioCrawler: Request failed and reached maximum retries. ReferenceError: $ is not defined Any advice how to do it correctly, please? Thanks a lot!

5 Replies

HonzaS•3y ago

can you share the RSS feed url? you can try to change content-type header in response in postNavigationHooks

quickest-silverOP•3y ago

Thanks Honza, Its here: https://mladypodnikatel.cz/feed Now I'm stuck like mentioned https://github.com/apify/crawlee/issues/271, althought that the bug is solved.

GitHub

Issues · apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast. - Issues · apify/crawlee

HonzaS•3y ago

so add this to your crawler constructor

postNavigationHooks:[({response})=>{
        response.headers['content-type'] = 'application/xhtml+xml';
    }]

postNavigationHooks:[({response})=>{
        response.headers['content-type'] = 'application/xhtml+xml';
    }]

quickest-silverOP•3y ago

Perfect, it works 🥳 Thank you

genetic-orange•13mo ago

@HonzaS saving my ass again ❤️

Parse RSS XML

Did you find this page helpful?