make request for cookies inside createSessionFunction

At a glance

The community member is trying to use the createSessionFunction to create a session and set some basic cookies from a response. The main issue is how to make a request to an endpoint to get the cookies inside the createSessionFunction without breaking the flow of the crawler. One community member suggests using a library like "got-scraping" to make the request and then set the cookies. However, another community member is having trouble setting the cookies using session.setCookies() as the cookies are not being set in the key-value pair. There is no explicitly marked answer in the comments.

Useful resources

ccurioussoul

Dear all, I am trying to use createSessionFunction to create a session and set some basic cookies from a response. The problem is how can I make a request to an endpoint to get a cookies inside createSessionFunction ?

My basic code is this and am wondering what is the best way to get cookies without breaking the flow of the crawler.

 createSessionFunction: async(sessionPool,options) => {

        const session1 = await sessionPool.getSession();
        const proxyurl = await proxyConfigurationEU.newUrl(session1.id);
        // Get cookies 


       session1.setCookiesFromResponse(response);
       
    
       return session1

    }

2 comments

RRado Ch.

Hi, you can make the request here (for example by using got-scraping: https://github.com/apify/got-scraping) and then set the cookies. Check out also: https://docs.apify.com/academy/api-scraping/general-api-scraping/cookies-headers-tokens

ccurioussoul

Hey . Yes I am doing the same. But when I try to set the cookies to a sesssion via session.setCookies(). It doesnt set. I dont see the cookies being set in my key value pair.

Add a reply

Apify Discord Mirror

make request for cookies inside createSessionFunction