Glitchy_mess•2w ago

Packaging crawlee with pyinstaller

More of an open question, but has anyone had luck with using pyinstaller to package crawlee into an executable? Currently getting a FileNotFoundError after packaging the script where crawlee can't find a file in Local\Temp\ [some random list of letters] \apify_fingerprint_datapoints\data\input-network-definition; pyinstaller has some stuff in its FAQ about import errors but this seems very specific?

Solution:

yep, it ended up being a mix of the two acting weird running something like ```powershell $env:PLAYWRIGHT_BROWSERS_PATH='0' #playwright config command to force it to acknowledge the browser you want to use because it breaks if you run pyinstaller by itself >> playwright install firefox #use whatever browser you're using crawlee with...

Jump to solution

6 Replies

Jan Buchar•2w ago

Hi there, I have no experience with pyinstaller, but what you describe here looks like there may be a file missing in the apify_fingerprint_datapoints package. Could you look into the path that in the error message and see what files are present there?

Glitchy_messOP•2w ago

so that's the weird thing, if i run the file off of vscode it a) runs just fine and b) doesn't generate any temp files in my Local\Temp; the closest thing i could see is maybe "apify-playwright-[web browser]-taac-[random letters]" but even that doesn't hold anything akin to apify_fingerprint_datapoints i'd be remiss to forget mentioning that, when cross referencing when files were modified, the only thing added to tmp is a mat-debug file, which is just empty, and a mozilla-temp-files folder, which is also empty so i'm not sure what crawlee's looking for when trying to ask for the fingerprint

Jan Buchar•2w ago

well, crawlee is looking for a data file with browser fingerprints that is contained in the apify_fingerprint_datapoints package... but I'm not sure why it thinks that it should be located in your temp directory - maybe that's something pyinstaller does?

Solution

Glitchy_mess•2w ago

yep, it ended up being a mix of the two acting weird running something like

$env:PLAYWRIGHT_BROWSERS_PATH='0' #playwright config command to force it to acknowledge the browser you want to use because it breaks if you run pyinstaller by itself
>> playwright install firefox #use whatever browser you're using crawlee with
>> pyinstaller -F --collect-data apify_fingerprint_datapoints --collect-data playwright pythonScript.py

$env:PLAYWRIGHT_BROWSERS_PATH='0' #playwright config command to force it to acknowledge the browser you want to use because it breaks if you run pyinstaller by itself
>> playwright install firefox #use whatever browser you're using crawlee with
>> pyinstaller -F --collect-data apify_fingerprint_datapoints --collect-data playwright pythonScript.py

lets you grab the fingerprints before building the actual app itself

Glitchy_messOP•2w ago

@Jan Buchar thanks for bumping me in that direction !!! looking at the pyinstaller arguments led to finding --collect-data

Jan Buchar•2w ago

glad to have helped 🙂

Packaging crawlee with pyinstaller

Did you find this page helpful?