xenial-black•13mo ago
How to set the delay between requests?
crawler = PlaywrightCrawler(
max_requests_per_crawl=10,
max_request_retries=0,
headless=True,
browser_type='chromium',
)
I want to set 3600ms delay between requests
1 Reply
Hi, please refer this code
import asyncio
from apify import PlaywrightCrawler
async def handle_page_function(page, request):
# Your page processing logic here
print(f'Processing {request.url}')
# Introduce a delay of 3600ms (3.6 seconds)
await asyncio.sleep(3.6)
crawler = PlaywrightCrawler(
max_requests_per_crawl=10,
max_request_retries=0,
headless=True,
browser_type='chromium',
request_handler=handle_page_function, # Attach the handler function
)
Start the crawler
crawler.run()