crawlee-js
apify-platform
crawlee-python
💻hire-freelancers
🚀actor-promotion
💫feature-request
💻creators-and-apify
🗣general-chat
🎁giveaways
programming-memes
🌐apify-announcements
🕷crawlee-announcements
👥community
Crawlee's Compatibility with Kivy
Hey all,
I was wondering if there are any specific compatibility issues with using crawlee within a kivy app, because whenever I run the script attached to grab the name of a product in amazon, it gives a recursion error that stems within the default_handler, specifically when the url is pulled from the context.
FWIW, the UI pipeline is App -> ScreenManager -> Main Screen & Second Screen, wherein the main screen transitions to the second screen via switch_to()
...
Solution:
Hey, @Glitchy_mess
Try adding
configure_logging=False when initializing the crawler.
```python...
Using browser_new_context_options with PlaywrightBrowserPlugin
Hello,
I'm very confused on how to use browser_new_context_options because the error code that i get implies that storage_state is not a variable that Playwright's new_context call supports. However on the playwright docs, storage_state seems to support uploading cookies as a dictionary, and when I try running playwright by itself and uploading cookies through this method, everything works fine.
A lot of the cookie related questions in this forum seems to be before the msot recent build, so I was wondering what the syntax should be to properly load cookies through the PlaywrightBrowserPlugin class, as everything else seems to work just fine once that's sorted.
...
Solution:
figured out the issue in dms, for anyone looking up this sort of issue in the future, you'd want to have something akin to this script
some lines of interest are:
```
"expires": int(parts[4]) if parts[4] else None, #type: ignore
"http_only": parts[3].lower() == 'true',...
Memory is critically overloaded
I have an AWS EC2 instance with 64GB memory.
My crawler is running in a docker container. The
CRAWLEE_MEMORY_MBYTES is set to 61440
My docker config
```...