sensitive-blue•16mo ago

Baby's first pull request (# 2521), fix: utils.parseOpenGraph() can parse arrays of images w/ props

Long time open-source user, first time contributor. I noticed some issues using the utils.parseOpenGraph() function. For me, honestly it wouldn't work at all as a consumer. It could be my dev env because it's Windows 11 and Node.js doesn't always work right on Windows. It kept returning an empty object no matter whether I gave it a $ CheerioAPI context or a string of HTML. After forking the repo, I was able to run the tests just fine (not everything passed, but that module's tests did). When I started to dive deeper into it, I noticed it returned back the oddest thing. For structured Open Graph objects, think og:image, for instance, which may also have associated og:image:width, etc., it would return arrays for all the individual values. And those values weren't in certain indices where you could rebuild an object if you knew them all because while they were in the correct order they didn't skip over undefined og:image:* values from the previous og:image Open Graph properties where image was an array of objects. In other words, the first width og:image:width value might not be able to be reassociated with the correct og:image value upon consumption of this API call. So I went a different direction with this API in my fork, setting up TS interfaces and making an effort to fill them in with valid objects. I do wonder if you guys like what I have done with it. It breaks the previous API signatures, because it doesn't return these arrays of values but instead returns structured objects. I would be okay with splitting this parseOpenGraph() function I have written off to its own npm package if that's better for everybody. There's still more to improve about it, because right now it's a mix of the previous way of parsing and the way I decided to parse it. I came here because I wonder about your thoughts on it. Here's my PR: https://github.com/apify/crawlee/pull/2521. I have no idea how to do a successful pull request. I'm such a newb.

GitHub

Build software better, together

GitHub is where people build software. More than 100 million people use GitHub to discover, fork, and contribute to over 420 million projects.