Ppopular-magenta
image download/upload empty zip issues
2023-06-02T21:07:23.777Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202105/...
2023-06-02T21:07:23.783Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:24.367Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202102/...
2023-06-02T21:07:24.947Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:24.965Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202304/...
2023-06-02T21:07:25.971Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:25.980Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:26.407Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202205/...
2023-06-02T21:07:26.504Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:27.089Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202211/...
2023-06-02T21:07:27.354Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202009/...
2023-06-02T21:07:27.511Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:28.055Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:28.486Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202108/...
2023-06-02T21:07:29.165Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202303/...
2023-06-02T21:07:29.298Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202010/...
2023-06-02T21:07:29.558Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:30.253Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:31.284Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202302/...
2023-06-02T21:07:31.377Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202301/...
2023-06-02T21:07:32.058Z INFO BasicCrawler: All the requests from request list and/or request queue have been processed, the crawler will shut down.
2023-06-02T21:07:32.265Z INFO BasicCrawler: Crawl finished. Final request statistics: {"requestsFinished":20,"requestsFailed":0,"retryHistogram":[20],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":820,"requestsFinishedPerMinute":139,"requestsFailedPerMinute":0,"requestTotalDurationMillis":16405,"requestsTotal":20,"crawlerRuntimeMillis":8603}
2023-06-02T21:07:32.267Z INFO All images in iteration 0 were processed
2023-06-02T21:07:32.268Z INFO Archiving Images...
2023-06-02T21:07:32.318Z INFO Archive has been written
2023-06-02T21:07:32.539Z INFO Will save output data to: key-value-store
2023-06-02T21:07:32.541Z INFO Post-download processed data length:
2023-06-02T21:07:32.911Z INFO END OF ITERATION STATS:
2023-06-02T21:07:32.912Z INFO *** STATS ***
2023-06-02T21:07:32.914Z INFO Total: 20, Uploaded: 0, Failed: 20, Skipped: 0, Duplicates: 0
2023-06-02T21:07:33.074Z INFO Downloading finished
2023-06-02T21:07:33.076Z INFO Actor finished successfully (exit code 0)2023-06-02T21:07:23.777Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202105/...
2023-06-02T21:07:23.783Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:24.367Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202102/...
2023-06-02T21:07:24.947Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:24.965Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202304/...
2023-06-02T21:07:25.971Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:25.980Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:26.407Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202205/...
2023-06-02T21:07:26.504Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:27.089Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202211/...
2023-06-02T21:07:27.354Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202009/...
2023-06-02T21:07:27.511Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:28.055Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:28.486Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202108/...
2023-06-02T21:07:29.165Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202303/...
2023-06-02T21:07:29.298Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202010/...
2023-06-02T21:07:29.558Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:30.253Z INFO Downloading image https://image.api.playstation.com/gs2-sec/appkgo/prod/C...
2023-06-02T21:07:31.284Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202302/...
2023-06-02T21:07:31.377Z INFO Downloading image https://image.api.playstation.com/vulcan/ap/rnd/202301/...
2023-06-02T21:07:32.058Z INFO BasicCrawler: All the requests from request list and/or request queue have been processed, the crawler will shut down.
2023-06-02T21:07:32.265Z INFO BasicCrawler: Crawl finished. Final request statistics: {"requestsFinished":20,"requestsFailed":0,"retryHistogram":[20],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":820,"requestsFinishedPerMinute":139,"requestsFailedPerMinute":0,"requestTotalDurationMillis":16405,"requestsTotal":20,"crawlerRuntimeMillis":8603}
2023-06-02T21:07:32.267Z INFO All images in iteration 0 were processed
2023-06-02T21:07:32.268Z INFO Archiving Images...
2023-06-02T21:07:32.318Z INFO Archive has been written
2023-06-02T21:07:32.539Z INFO Will save output data to: key-value-store
2023-06-02T21:07:32.541Z INFO Post-download processed data length:
2023-06-02T21:07:32.911Z INFO END OF ITERATION STATS:
2023-06-02T21:07:32.912Z INFO *** STATS ***
2023-06-02T21:07:32.914Z INFO Total: 20, Uploaded: 0, Failed: 20, Skipped: 0, Duplicates: 0
2023-06-02T21:07:33.074Z INFO Downloading finished
2023-06-02T21:07:33.076Z INFO Actor finished successfully (exit code 0)I'm using
lukaskrivka/images-download-uploadlukaskrivka/images-download-upload to grab all image urls from a dataset from one of my actors. You can see above that it's retrieving the urls successfully and appears to process/download them but the resulting zip file is empty every time. Anyone have any insight or suggestions for alternatives that work?
