recent-tealR
Apify & Crawlee12mo ago
2 replies
recent-teal

SAME_HOSTNAME not working on non www URLs

When using the EnqueueStrategy.SAME_HOSTNAME I noticed it does not work properly on non www urls.

In the debugger I noticed it passes origin to the _check_enqueue_strategy but it uses the context.request.loaded_url if available.
So every URL that is checked will mismatch because of the difference in hostname

I tested this with multiple urls with & without www prefix and got the same behaviour.
Screenshot_2025-02-03_at_13.10.28.png
Screenshot_2025-02-03_at_13.13.31.png
Was this page helpful?