Skip to content

A NestJS service that mirrors your GitHub starred repositories into Postgres, including README snapshots, cron-based scheduling, and Prometheus metrics

License

Notifications You must be signed in to change notification settings

x0rium/gh-star-sync

Repository files navigation

GitHub Star Sync

This project provides a robust backend solution for creating a persistent, local mirror of a user's starred repositories on GitHub. It automatically fetches repository data via the GitHub API and stores it in a relational database, making it easy to query, analyze, or display your starred projects.

Key Features:

  • Periodic Synchronization: Uses a scheduler (cron job) to automatically keep the local database in sync with your GitHub stars. The schedule is configurable via environment variables.
  • Rich Data Storage: Saves essential repository metadata, including description, language, star count, and key dates (created, pushed, starred).
  • README Fetching: Includes functionality to download and store the README.md content for each repository.
  • Scalable Architecture: Built on the powerful and modular NestJS framework.
  • Type-Safe Database Access: Leverages Prisma for a modern, type-safe database workflow.
  • Observability: Exposes key application metrics in Prometheus format.
  • Containerized: Comes with a multi-stage Dockerfile for easy deployment.
  • CI/CD: Includes a GitHub Actions workflow for continuous integration.

Built With

Getting Started

To get a local copy up and running, follow these simple steps.

Prerequisites

  • Node.js (v22 or newer)
  • pnpm
  • A running PostgreSQL instance (or another database of your choice)
  • Docker (optional, for containerized setup)

Installation

  1. Clone the repository

    git clone https://github.com/x0rium/gh-star-sync.git
    cd gh-star-sync
  2. Install dependencies

    pnpm install
  3. Set up environment variables

    Create a .env file in the root of the project by copying the example file:

    cp .env.example .env

    Now, open the .env file and fill in the required values. See the Environment Variables section for a full list.

  4. Run database migrations

    This will set up the database schema based on your prisma/schema.prisma file.

    pnpm prisma migrate dev

Running the Application

# Development mode with hot-reloading
$ pnpm run start:dev

# Production mode
$ pnpm run build
$ pnpm run start:prod

Once running, the application will automatically trigger the synchronization job based on the schedule and feature flags defined in your .env file.

Running with Docker

You can also run the application using Docker.

  1. Build the Docker image:

    docker build -t gh-star-sync .
  2. Run the Docker container: Make sure to provide the necessary environment variables.

    docker run -p 3000:3000 --env-file ./.env gh-star-sync

API Endpoints

  • GET /health: Returns the health status of the application.
    {
      "status": "ok"
    }
  • GET /metrics: Exposes application metrics in Prometheus format.

Database Schema

This service persists starred repositories into a single table named Repository.

erDiagram
  Repository {
    Int id PK "autoincrement"
    BigInt githubId UK "GitHub repository ID"
    String name
    String fullName
    String description "nullable"
    String url
    String language "nullable"
    Int stars
    String readmeContent "nullable"
    DateTime readmeFetchedAt "nullable"
    DateTime repoCreatedAt
    DateTime repoPushedAt
    DateTime repoStarredAt
    DateTime createdAt "default now()"
    DateTime updatedAt "auto updated"
  }
Loading

Prisma model source: see prisma/schema.prisma

model Repository {
  id               Int       @id @default(autoincrement())
  githubId         BigInt    @unique
  name             String
  fullName         String
  description      String?
  url              String
  language         String?
  stars            Int
  readmeContent    String?
  readmeFetchedAt  DateTime?
  repoCreatedAt    DateTime
  repoPushedAt     DateTime
  repoStarredAt    DateTime
  createdAt        DateTime  @default(now())
  updatedAt        DateTime  @updatedAt
}

Notes

  • Primary key: id (autoincrement).
  • Unique key: githubId (ensures one row per GitHub repository).
  • Optional fields: description, language, readmeContent, readmeFetchedAt.
  • Temporal metadata:
    • repoCreatedAt — repository creation time on GitHub.
    • repoPushedAt — last push time on GitHub.
    • repoStarredAt — when this user starred the repository.
    • createdAt / updatedAt — row timestamps (managed by Prisma).

Environment Variables

Variable Description Default
DATABASE_URL Your database connection string.
GITHUB_TOKEN Your GitHub Personal Access Token (PAT) with 'repo' scope.
GITHUB_USERNAME The GitHub username whose stars you want to sync.
PORT The port the application will listen on. 3000
ENABLE_SYNC_ON_BOOT Whether to run the synchronization job when the application starts. (true or false) true
ENABLE_SYNC_CRON Whether to enable the scheduled synchronization job. (true or false) true
CRON_SCHEDULE The cron schedule for the synchronization job. */5 * * * *
METRICS_ENABLED Whether to expose the /metrics endpoint. (true or false) true

Running Tests

# Unit and e2e tests
$ pnpm run test

# Test coverage report
$ pnpm run test:cov

Continuous Integration

This project uses GitHub Actions for CI. The workflow is defined in .github/workflows/ci.yml. It automatically runs on every push and pull request to the main branch, performing the following checks:

  • Linting
  • Building
  • Running tests

License

Distributed under the MIT License. See LICENSE for more information.

About

A NestJS service that mirrors your GitHub starred repositories into Postgres, including README snapshots, cron-based scheduling, and Prometheus metrics

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors