Skip to content

Feature Request: Automatically create parent ImageCollection during EE ingestion #532

@j9sh264

Description

@j9sh264

Description

Currently, when ingesting assets into Google Earth Engine via weather-mv, the process fails if the destination ImageCollection does not already exist. This requires manual intervention to pre-create the collection before running an ingestion pipeline.

In automated streaming or batch workflows, the parent collection path may be dynamic (e.g., based on a new model version or experiment name). If the collection isn't manually initialized, the ingestion job throws an error, leading to pipeline failures.

Proposed Solution

Update the Earth Engine ingestion logic to check for the existence of the parent ImageCollection before attempting to write the asset.

  • If the collection exists, proceed as usual.
  • If the collection does not exist, the tool should automatically create it (equivalent to ee.data.createAsset with type IMAGE_COLLECTION).

Alternative Considered

Manually creating collections via the EE Code Editor or earthengine CLI, which is not scalable for automated, multi-tenant, or dynamic pipelines.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions