Implement API client and add Pydantic classes

[shreddd]

Summary of changes:

• preserve original coordinates from ingested objects. make a separate mongo only (not in schema) field for geojson values
• use pydantic model for data
• add additional API calls
• create a python client to access API.
• introduce --clean flag for full DB ingest

[shreddd]

Note - this uses a local copy of the pydantic schema file bertron_schema_pydantic.py - eventually this should be installed as a dependency once the PR to make bertron-schema a package is approved.

ber-data/bertron-schema#41

[eecavanna]

I am surprised to see indexes being created each time an entity gets inserted (as opposed to after all entities have been inserted).

Edit: Maybe these are effectively "no op"s when the index already exists.

[eecavanna]

LGTM! I left a few comments/suggestions.

(Side note: I am surprised a review comment from Copilot hasn't appeared on the "Conversation" tab yet.)

[Copilot]

Pull Request Overview

This PR integrates Pydantic models into the server and client code, refactors geospatial query handling, and adds a cleanup option to the ingestion workflow.

• Replace raw MongoDB documents with bertron_schema_pydantic.Entity instances in server endpoints
• Centralize document-to-entity conversion and add logging for validation failures
• Enhance ingestion script with clean flag and switch to GeoJSON field

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/server.py	Use Pydantic Entity, refactor geospatial endpoints, add convert_document_to_entity and logging
src/bertron_client.py	Update client methods to consume Entity model and reconstruct metadata
mongodb/ingest_data.py	Add clean_collections method, switch to geojson field, update indexes
mongodb/gold-example.json	Remove outdated example data
docker-compose.yml	Add --clean flag to ingestion command

Comments suppressed due to low confidence (3)

src/server.py:165

• The nearby-entities endpoint no longer returns metadata (query_type, center, radius_meters) that clients may rely on. Consider adding those fields or a generic metadata object to maintain API compatibility.

return {"documents": entities, "count": len(entities)}

src/server.py:238

• The bounding-box endpoint no longer includes query_type or the bounding-box coordinates in its response. Clients reconstructing request context may break; consider returning those values or a metadata object.

return {"documents": entities, "count": len(entities)}

src/server.py:42

• [nitpick] The new conversion logic in get_all_entities and the centralized convert_document_to_entity are not covered by existing tests. Add unit tests for these to prevent regressions.

def get_all_entities():

[Copilot]

The return type is annotated as Optional[Entity] but the function always returns an Entity. Updating the signature to return Entity improves type accuracy.

Suggested change

	) -> Optional[bertron_schema_pydantic.Entity]:
	) -> bertron_schema_pydantic.Entity:

[eecavanna]

Edit: This comment is obsolete.

---

Show/hide comment

I renamed the PR (its name will appear in the release notes):

- Pydantic models
+ Implement API client and add local copy of Pydantic classes from `bertron-schema` repo

I recommend renaming it again once the local copy of the Pydantic class definitions has been removed from this branch (assuming that happens before this branch is merged into main).

[eecavanna]

Edit: This comment is obsolete.

---

Show/hide comment

Hi @shreddd, PR ber-data/bertron-schema#41 has been merged into bertron-schema. I recommend removing the local copy of the Pydantic class definition file and importing it from the bertron-schema package instead.

Here's a command you can use to pip install the bertron-schema package:

pip install git+https://github.com/ber-data/bertron-schema.git

[shreddd]

Note - I updated the code to use the pip installable version instead.