Bbroad-salmon
How to parse out results in Langchain ApifyDatasetLoader
I'm using the Google Search Results Scraper which provides a single JSON with Paid, Organic and a few other keys. I'd like to parse out the Organic title and urls into a Langchain agent, but it's clear how to iterate over them. Any suggestions?
loader = apify.call_actor(
actor_id="apify/google-search-scraper",
# Prepare the Actor input
run_input={"queries": query,
"maxPagesPerQuery": 1,
"resultsPerPage": 100,
"customDataFunction": """async ({ input, $, request, response, html }) => {return {pageTitle: $('title').text(),};}""",},
dataset_mapping_function=lambda item: Document(
page_content=item["url"] or "", metadata={"source": item["url"]}
),
)
loader = apify.call_actor(
actor_id="apify/google-search-scraper",
# Prepare the Actor input
run_input={"queries": query,
"maxPagesPerQuery": 1,
"resultsPerPage": 100,
"customDataFunction": """async ({ input, $, request, response, html }) => {return {pageTitle: $('title').text(),};}""",},
dataset_mapping_function=lambda item: Document(
page_content=item["url"] or "", metadata={"source": item["url"]}
),
)
This is the official developer community of Apify and Crawlee.
13,923Members
Resources
Recent Announcements
Similar Threads
Was this page helpful?
