Building and deploying a productionized GPT-2-based code autocompletion tool, from model training to a working VS Code extension with ML lifecycle infrastructure.

In this project, we will attempt to recreate GitHub Copilot. Since properly recreating the service requires an immense amount of compute for model training, we will seek to build a Mini Copilot.

This repository contains both educational materials (starter notebooks for basic training) and our actual implementation of recreating GitHub Copilot. Education materials can be found under education/, and our implementation can be found under src/.

Accompanying educational slide decks can be found here (requires UMich login).

Recreating GitHub Copilot involves various parts -- the underlying code completion model, the endpoint serving the model, and the actual application (VSCode extension) using the model.

• Code completion model: To train the underlying code completion model, we train gpt-2 on a code completion task (code found under src/model). The model training pipeline includes shared feature logic between training and inference for consistency.
• Serving the model: To serve the model, we deploy a versioned, API-based inference service on GCP Cloud Run (container image found under src/backend). The service includes observability for latency and errors, and supports model versioning and rollback capabilities.
• ML Lifecycle Infrastructure: We implement model versioning, rollback, and inference logging to support safe iteration, reproducibility, and monitoring of production ML workloads (see src/model/registry.py and src/model/model_registry.json).
• VSCode extension: To build the VSCode extension, we use the registerInlineCompletionItemProvider VSCode API (full implementation found under src/extension). This extension calls our Cloud Run endpoint to supply real-time code completion.