Dataset Info

Hi ! I want to show/log dataset statistic right after adding values using code below

            ## data => 1000-ish items

            # 100 records each to default dataset
            for i in range(0, len(data), 100): await Actor.push_data(data[i:i+100])
            
            # open default dataset
            ds = await Actor.open_dataset()
            
            # get info
            info = await ds.get_info()
            for k,v in info.items():
                print(k, v)

            ## data => 1000-ish items

            # 100 records each to default dataset
            for i in range(0, len(data), 100): await Actor.push_data(data[i:i+100])
            
            # open default dataset
            ds = await Actor.open_dataset()
            
            # get info
            info = await ds.get_info()
            for k,v in info.items():
                print(k, v)

the records added successfully, however the stats seems not updated immediately, shown in logs itemCount = 0 :

2023-04-07T13:44:07.862Z id 8iukTelIPvlR7grEh
2023-04-07T13:44:07.864Z name None
2023-04-07T13:44:07.866Z userId ****
2023-04-07T13:44:07.868Z createdAt 2023-04-07 13:44:00.911000+00:00
2023-04-07T13:44:07.870Z modifiedAt 2023-04-07 13:44:00.911000+00:00
2023-04-07T13:44:07.871Z accessedAt 2023-04-07 13:44:02.891000+00:00
2023-04-07T13:44:07.873Z itemCount 0
2023-04-07T13:44:07.875Z cleanItemCount 0
2023-04-07T13:44:07.876Z actId O6knBihxFig55ve53
2023-04-07T13:44:07.878Z actRunId XrCVnyE7a6EXEbSv5
...
2023-04-07T13:44:07.881Z stats {'readCount': 0, 'writeCount': 0, 'storageBytes': 0}
2023-04-07T13:44:07.883Z fields []

2023-04-07T13:44:07.862Z id 8iukTelIPvlR7grEh
2023-04-07T13:44:07.864Z name None
2023-04-07T13:44:07.866Z userId ****
2023-04-07T13:44:07.868Z createdAt 2023-04-07 13:44:00.911000+00:00
2023-04-07T13:44:07.870Z modifiedAt 2023-04-07 13:44:00.911000+00:00
2023-04-07T13:44:07.871Z accessedAt 2023-04-07 13:44:02.891000+00:00
2023-04-07T13:44:07.873Z itemCount 0
2023-04-07T13:44:07.875Z cleanItemCount 0
2023-04-07T13:44:07.876Z actId O6knBihxFig55ve53
2023-04-07T13:44:07.878Z actRunId XrCVnyE7a6EXEbSv5
...
2023-04-07T13:44:07.881Z stats {'readCount': 0, 'writeCount': 0, 'storageBytes': 0}
2023-04-07T13:44:07.883Z fields []

do I missing something? any help greatly appreciated Thanks.

6 Replies

like-gold•3y ago

Passed the question to the team. I have a feeling that something like that has happened before, and it might be that somehow stats is updated once in a while, but I might as well remember some completely different issues, so will wait for some certain response. Will write back with more info! Got the response: The updates to stats are throttled - so it won't show immediately. If you add some sleep between push_data and get_info it should show proper numbers.

!!!Joefree!!! 👑OP•3y ago

Thanks for the info 👍 . so how many seconds do I need to sleep. Or is there a status / event that I can watch to indicate that the stats is up to date. 🧐

like-gold•3y ago

I think it should work even with like 1 second waiting. I could not check it now, so if you could try it out - please let me know if that would not work

!!!Joefree!!! 👑OP•3y ago

I've tried several time using sleep(1) also asyncio.sleep(1). the result vary, sometimes updated, sometimes zero. I guess it need longer sleeps.

like-gold•3y ago

Can you try like 1 second, not 1 millisecond?

!!!Joefree!!! 👑OP•3y ago

Hey Andrey, I believe sleep(1) is meant 1 second. also asyncio.sleep(1). however I am not sure using sleep is reliable option. still searching...

Dataset Info

Did you find this page helpful?