-
Notifications
You must be signed in to change notification settings - Fork 49
Feature Request: Automatically create parent ImageCollection during EE ingestion #532
Description
Description
Currently, when ingesting assets into Google Earth Engine via weather-mv, the process fails if the destination ImageCollection does not already exist. This requires manual intervention to pre-create the collection before running an ingestion pipeline.
In automated streaming or batch workflows, the parent collection path may be dynamic (e.g., based on a new model version or experiment name). If the collection isn't manually initialized, the ingestion job throws an error, leading to pipeline failures.
Proposed Solution
Update the Earth Engine ingestion logic to check for the existence of the parent ImageCollection before attempting to write the asset.
- If the collection exists, proceed as usual.
- If the collection does not exist, the tool should automatically create it (equivalent to
ee.data.createAssetwith typeIMAGE_COLLECTION).
Alternative Considered
Manually creating collections via the EE Code Editor or earthengine CLI, which is not scalable for automated, multi-tenant, or dynamic pipelines.