Jerome•17mo ago

Output schema

Hi, I'm having 2 issues with output schema definition: - I can't make transformations.unwind work, the table only shows undefined values -> Is there a working example I can check somewhere? - When instead using transformations.flatten, it seems the table can't show fields with array or object format, it shows undefined instead. Am I missing something here? For reference, my result is formatted like this, and i want to unwind/flatten the data part:

[
  {
    "metadata": {
      "timestamp": "2024-07-08T09:51:31.942Z",
      "run_id": "lWmckTaBAlbeKTM33"
    },
    "data": {
      "url": "some_url.com",
      "title": "some title",
      "attributes": ["a", "b"]
    }
  },
  {...}
]

[
  {
    "metadata": {
      "timestamp": "2024-07-08T09:51:31.942Z",
      "run_id": "lWmckTaBAlbeKTM33"
    },
    "data": {
      "url": "some_url.com",
      "title": "some title",
      "attributes": ["a", "b"]
    }
  },
  {...}
]

And I'd like the table to show the columns: - "metadata" as object (this works) - "url" as link (works only when using flatten, not unwind) - "title" as text (same) - "attributes" as array (shows undefined) Here is the docs I've been using https://docs.apify.com/platform/actors/development/actor-definition/output-schema

7 Replies

Oleg V.•17mo ago

Please provide your exact reproduction (JSON with output_schema). It should be straightforward and shouldn't cause any issues. The documentation is clear and simple, so there might be a mistake in your JSON structure.

JeromeOP•17mo ago

Hi @Oleg V. , thanks for your answer. Here is my output definition:

{
    "actorSpecification": 1,
    "fields": {},
    "views": {
        "price_monitor": {
            "title": "Price Monitor",
            "description": "`PriceMonitor` output schema.",
            "transformation": {
                "fields": [
                    "metadata",
                    "data.url",
                    "data.id",
                    "data.title",
                    "data.brand",
                    "data.price"
                ],
                "flatten": [
                    "data"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "metadata": {
                        "format": "object"
                    },
                    "data.url": {
                        "label": "URL",
                        "format": "link"
                    },
                    "data.id": {
                        "label": "ID",
                        "format": "text"
                    },
                    "data.title": {
                        "label": "Title",
                        "format": "text"
                    },
                    "data.brand": {
                        "label": "Brand",
                        "format": "text"
                    },
                    "data.price": {
                        "label": "Price",
                        "format": "object"
                    }
                }
            }
        }
    }
}

{
    "actorSpecification": 1,
    "fields": {},
    "views": {
        "price_monitor": {
            "title": "Price Monitor",
            "description": "`PriceMonitor` output schema.",
            "transformation": {
                "fields": [
                    "metadata",
                    "data.url",
                    "data.id",
                    "data.title",
                    "data.brand",
                    "data.price"
                ],
                "flatten": [
                    "data"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "metadata": {
                        "format": "object"
                    },
                    "data.url": {
                        "label": "URL",
                        "format": "link"
                    },
                    "data.id": {
                        "label": "ID",
                        "format": "text"
                    },
                    "data.title": {
                        "label": "Title",
                        "format": "text"
                    },
                    "data.brand": {
                        "label": "Brand",
                        "format": "text"
                    },
                    "data.price": {
                        "label": "Price",
                        "format": "object"
                    }
                }
            }
        }
    }
}

As you see (first image) the price should be an object but is instead undefined, even though the price json is present in the full data (second image).

JeromeOP•17mo ago

If i try using unwind with this config:

{
    "actorSpecification": 1,
    "fields": {},
    "views": {
        "price_monitor": {
            "title": "Price Monitor",
            "description": "`PriceMonitor` output schema.",
            "transformation": {
                "fields": [
                    "metadata",
                    "url",
                    "id",
                    "title",
                    "brand",
                    "price"
                ],
                "flatten": [
                    "unwind"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "metadata": {
                        "format": "object"
                    },
                    "url": {
                        "label": "URL",
                        "format": "link"
                    },
                    "id": {
                        "label": "ID",
                        "format": "text"
                    },
                    "title": {
                        "label": "Title",
                        "format": "text"
                    },
                    "brand": {
                        "label": "Brand",
                        "format": "text"
                    },
                    "price": {
                        "label": "Price",
                        "format": "object"
                    }
                }
            }
        }
    }
}

{
    "actorSpecification": 1,
    "fields": {},
    "views": {
        "price_monitor": {
            "title": "Price Monitor",
            "description": "`PriceMonitor` output schema.",
            "transformation": {
                "fields": [
                    "metadata",
                    "url",
                    "id",
                    "title",
                    "brand",
                    "price"
                ],
                "flatten": [
                    "unwind"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "metadata": {
                        "format": "object"
                    },
                    "url": {
                        "label": "URL",
                        "format": "link"
                    },
                    "id": {
                        "label": "ID",
                        "format": "text"
                    },
                    "title": {
                        "label": "Title",
                        "format": "text"
                    },
                    "brand": {
                        "label": "Brand",
                        "format": "text"
                    },
                    "price": {
                        "label": "Price",
                        "format": "object"
                    }
                }
            }
        }
    }
}

then it's all undefined

Oleg V.•17mo ago

unwind usage should look like this:

 
                 "transformation": {
                      "fields": [
                          "searchQuery",
                          "resultsTotal",
                          "organicResults"
                      ],
                      "unwind": [
                          "organicResults"
                      ]
                  },

 
                 "transformation": {
                      "fields": [
                          "searchQuery",
                          "resultsTotal",
                          "organicResults"
                      ],
                      "unwind": [
                          "organicResults"
                      ]
                  },

And not

                "flatten": [
                    "unwind"
                ]

                "flatten": [
                    "unwind"
                ]

try to debug it with smaller / simpler output object. and then increase complexity step by step (I mean add nested obj, and so on...). check this working example for inspiration:

        "views": {
          "overview": {
            "title": "Overview",
            "description": "",
            "transformation": {
              "fields": [
                "thumbnailImage",
                "asin",
                "title",
                "description",
                "brand",
                "stars",
                "reviewsCount",
                "price.value",
                "breadCrumbs",
                "url"
              ],
              "flatten": [
                "price"
              ]
            },
            "display": {
              "component": "table",
              "properties": {
                "thumbnailImage": {
                  "label": "Image",
                  "format": "image"
                },
                "asin": {
                  "label": "ASIN"
                },
                "description": {
                  "format": "text"
                },
                "stars": {
                  "format": "number"
                },
                "reviewsCount": {
                  "format": "number"
                },
                "price.value": {
                  "label": "Price",
                  "format": "number"
                },
                "breadCrumbs": {
                  "label": "Categories"
                },
                "url": {
                  "label": "URL",
                  "format": "link"
                }
              }
            }
          }
        }

        "views": {
          "overview": {
            "title": "Overview",
            "description": "",
            "transformation": {
              "fields": [
                "thumbnailImage",
                "asin",
                "title",
                "description",
                "brand",
                "stars",
                "reviewsCount",
                "price.value",
                "breadCrumbs",
                "url"
              ],
              "flatten": [
                "price"
              ]
            },
            "display": {
              "component": "table",
              "properties": {
                "thumbnailImage": {
                  "label": "Image",
                  "format": "image"
                },
                "asin": {
                  "label": "ASIN"
                },
                "description": {
                  "format": "text"
                },
                "stars": {
                  "format": "number"
                },
                "reviewsCount": {
                  "format": "number"
                },
                "price.value": {
                  "label": "Price",
                  "format": "number"
                },
                "breadCrumbs": {
                  "label": "Categories"
                },
                "url": {
                  "label": "URL",
                  "format": "link"
                }
              }
            }
          }
        }

JeromeOP•17mo ago

Ofc my bad for the typo, I did use "unwind" as a key, but still could not make it work. If i do "unwing": ["data"], and data contains a url field, what should i list in the fields? I've tried url and data.url adn both end up undefined. In your example with flatten none of the fields are of type array or object (they are image, text or number only), which are the types I'm having trouble with...

JeromeOP•17mo ago

So with unwind I'm able to move children into a parent object as expected, but in the table view it always appears as undefined. What should be in the field list when using unwind?

JeromeOP•17mo ago

Any insight on this @Oleg V.?

Output schema

Did you find this page helpful?