Apify Discord Mirror

Updated 2 months ago

user_data is not working as expected

At a glance
The community member is experiencing an issue where the tab_number values pushed to user_data are different from the values retrieved in another handler. For example, when pushing values from 1 to 19, the community member gets tab_number as 1 instead of the correct value of 19. The community member has provided the relevant code and is unable to find the error. A comment suggests that the issue might be related to the target site or selectors, and the community member is advised to debug the values at each step to pinpoint the problem.
For some reason, when I push data to user_data (str type in this case), when I get user_data variables in another handler I get different values.
In this case the error is on tab_number. When I push tab_number to user_data the values seems to be good (values ranged from 1 to 100). But when I get tab_number through tab_handler I get a different value.
For example, for values from 1 to 19, I get tab_number 1, instead of the correct tab_number: tab_number pushed to user_data: "19", tab_number requested from user_data: "1".
I cannot find the error. Here is the code:

@router.handler('tabs')
async def tabs_handler(context: PlaywrightCrawlingContext) -> None:

tab_id = context.request.user_data["tab_id"]

await context.page.wait_for_selector('#tabsver > ul > li > a')

tabs = await context.page.locator('#tabsver > ul > li > a').all()

for tab in tabs:
tab_name = await tab.text_content()
tab_number = tab_name.replace("Tab number ", "").strip()
if tab_name:
await context.enqueue_links(
selector = f'#tabsver > ul > li > a:has-text("{tab_name}")',
label = "tab",
user_data = {"tab_id": tab_id,
"tab_number": tab_number
},
)

@router.handler('tab')
async def tab_handler(context: PlaywrightCrawlingContext) -> None:

tab_id = context.request.user_data["tab_id"]
tab_number = context.request.user_data["tab_number"]
O
1 comment
I think the issue might be related to the target site or selectors. Try debugging the value at each step to pinpoint the problem.
Add a reply
Sign up and join the conversation on Discord