Mindtrace Documentation Pipeline [1/2]

.github/workflows/docs.yml

@@ -0,0 +1,69 @@
+name: Build and Publish Docs
+
+on:
+  push:
+    branches:
+      - dev
+      - main
+  workflow_dispatch:
+  release:
+    types: [published]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event_name == 'push' && github.ref}}
+  cancel-in-progress: true
+
+permissions:
+  contents: write
+  pages: write
+  pull-requests: write
+
+jobs:
+  deploy:
+    name: Publish Docs
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    strategy:
+      matrix:
+        python-version: ["3.12"]
+    steps:
+      - name: Checkout the repository
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          fetch-depth: 0
+
+      - name: Install uv and set Python ${{ matrix.python-version }}
+        uses: astral-sh/setup-uv@e92bafb6253dcd438e0484186d7669ea7a8ca1cc # v6.4.3
+        with:
+          python-version: ${{ matrix.python-version }}
+          activate-environment: true
+
+      - name: Install dependencies
+        run: |
+          uv sync --group docs
+
+      - name: Configure git for github-actions
+        run: |
+          git config --global user.name "github-actions[bot]"
+          git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com"
+
+      - name: Deploy Development Docs
+        if: (github.event_name == 'push' && github.ref == 'refs/heads/dev') || github.event_name == 'workflow_dispatch'
+        run: |
+          uv run mike deploy --push dev
+
+      - name: Set Default to Dev
+        if: (github.event_name == 'push' && github.ref == 'refs/heads/dev') || github.event_name == 'workflow_dispatch'
+        run: |
+          uv run mike set-default dev --push
+
+      - name: Deploy Release Docs
+        if: github.event_name == 'release' && github.event.action == 'published'
+        run: |
+          latest_tag=$(git describe --tags `git rev-list --tags --max-count=1`)
+          uv run mike deploy --push --update-aliases $latest_tag latest
+
+      - name: Deploy Main Docs
+        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
+        run: |
+          uv run mike deploy --push main

docs/cluster/README.md

@@ -0,0 +1 @@
+../../mindtrace/cluster/README.md

docs/cluster/api.md

@@ -0,0 +1,6 @@
+# Cluster Package API Reference
+::: mindtrace.cluster
+    options:
+      show_root_heading: false
+      show_source: false
+      heading_level: 4

docs/core/README.md

@@ -0,0 +1 @@
+../../mindtrace/core/README.md

docs/core/api.md

@@ -0,0 +1,6 @@
+# Core Package API Reference
+::: mindtrace.core
+    options:
+      show_root_heading: false
+      show_source: false
+      heading_level: 4

docs/database/README.md

@@ -0,0 +1 @@
+../../mindtrace/database/README.md

docs/database/api.md

@@ -0,0 +1,6 @@
+# Database Package API Reference
+::: mindtrace.database
+    options:
+      show_root_heading: false
+      show_source: false
+      heading_level: 4

docs/hardware/README.md

@@ -0,0 +1 @@
+../../mindtrace/hardware/README.md

docs/hardware/api.md

@@ -0,0 +1,6 @@
+# Hardware Package API Reference
+::: mindtrace.hardware
+    options:
+      show_root_heading: false
+      show_source: false
+      heading_level: 4

docs/index.md

@@ -0,0 +1,120 @@
+# :material-rocket-launch: Welcome to Mindtrace
+
+Unifying asset inspection under a single package, Mindtrace brings together industrial hardware, machine learning, and automation, bridging the gap between edge devices and scalable intelligence.
+
+Monitor, automate, and scale your on-prem AI solutions effortlessly with **Mindtrace** package.  
+From training robust AI models, hardware integration to deploying and scaling edge intelligence for **Asset inspection**, Mindtrace enables full-cycle orchestration.  
+
+It seamlessly connects **hardware, data, and machine learning**, empowering teams to deploy context-aware decision systems, derive real-time insights, and visualize results through interactive dashboards
+
+---
+
+## :material-star: Features
+
+
+-   :fontawesome-solid-network-wired: **Unified Architecture**  
+    Integrates data ingestion, model training, deployment, and system coordination under one modular ecosystem that scales seamlessly across services and clusters.
+
+-   :material-cpu-64-bit: **Hardware-Aware Intelligence**  
+    Connect directly to PLCs, cameras, and sensors for real-time inference, control, and closed-loop feedback across industrial environments.
+
+-   :material-brain: **ML-Native Design**  
+    Provides end-to-end pipelines for dataset management, model registry, and distributed training workflows.
+
+-   :material-database: **Datalake Integration**  
+    Built-in connectors and APIs for storing, indexing, and retrieving structured or unstructured data — powering analytics, retraining, and traceability.
+
+-   :material-cloud-sync: **Cluster-Aware Orchestration**  
+    Enables coordinated operation across multiple nodes or services, supporting distributed execution and horizontal scaling.
+
+-   :material-account-group: **Service Collaboration Layer**  
+    Seamlessly launch, register, and interconnect FastAPI or MCP-based microservices through a unified control plane and shared state system.
+
+
+---
+
+## :material-layers-triple: Layered Architecture
+
+Mindtrace is organized into a layered workspace to support ML components as Python modules with clearly defined boundaries and dependencies. We use a level-based system for organizing modules based on dependency direction and build order.
+
+### **Level 1: Core**
+- `core`: Foundational utilities and base classes used across all other modules.
+
+### **Level 2: Core Consumers**
+- `jobs`: Job execution and backend interfaces.
+- `registry`: Artifact and metadata management.
+- `database`: Redis, Mongo, and DB access layers.
+- `services`: Service base classes, authentication, and gateways.
+- `storage`: Storage functionality for cloud storage integration.
+- `ui`: Optional UI libraries and components.
+
+### **Level 3: Infrastructure Modules**
+- `hardware`: Interfaces for cameras, PLCs, scanners, etc.
+- `cluster`: Runtime cluster management, nodes, and workers.
+- `datalake`: Dataset interfaces for HuggingFace and Mindtrace datasets.
+- `models`: Core model definitions and leaderboard utilities.
+
+### **Level 4: Automation**
+- `automation`: Integration of pipelines and orchestration using level 2–3 modules.
+
+### **Level 5: Applications**
+- `apps`: End-user applications composed of all previous levels.
+  - E.g., Demo pipelines
+
+---
+
+## :material-arrow-down-bold: Dependency Flow
+
+Each layer only depends on modules in lower levels.
+
+| Module     | Depends On                                           |
+|------------|------------------------------------------------------|
+| `core`     | –                                                    |
+| `jobs`     | `core`                                               |
+| `registry` | `core`                                               |
+| `database` | `core`, `registry`                                   |
+| `services` | `core`                                               |
+| `storage`  | –                                                    |
+| `ui`       | `core`                                               |
+| `cluster`  | `core`, `jobs`, `registry`, `database`, `services`   |
+| `datalake` | `core`, `registry`, `database`, `services`           |
+| `models`   | `core`, `registry`, `services`                       |
+| `hardware` | `core`, `services`, `storage`                        | 
+| `automation` | `core`, `registry`, `database`, `services`, `datalake`, `models`, `cluster` |
+| `apps`     | `core`, `registry`, `database`, `services`, `datalake`, `models`, `cluster`, `jobs`, `hardware`, `ui`, `automation` |
+
+---
+
+## :material-link: Useful Links
+
+- :material-github: [GitHub Repository](https://github.com/Mindtrace/mindtrace)
+- :material-package: [PyPI Package](https://pypi.org/project/mindtrace/)
+
+---
+## :material-console: Quick Start
+
+### :material-download: Installation
+
+```bash
+# Install the full Mindtrace package
+uv add mindtrace
+
+# Or install a minimal dependency chain
+uv add mindtrace-datalake
+```
+
+### :material-code-tags: Basic Usage
+
+```python
+from mindtrace import core, registry, database, services
+```
+
+---
+
+## :material-rocket: Contribute
+
+We welcome contributions! Whether you're fixing bugs, adding features, or improving documentation, your help makes Mindtrace better.
+
+- :material-book-open: [Contributing Guide](https://github.com/Mindtrace/mindtrace/blob/dev/CONTRIBUTING.md) - Learn how to get started
+- :material-github: [GitHub Issues](https://github.com/Mindtrace/mindtrace/issues) - Report bugs or suggest features
+- :material-source-branch: [Pull Requests](https://github.com/Mindtrace/mindtrace/pulls) - Submit your contributions

docs/jobs/README.md

@@ -0,0 +1 @@
+../../mindtrace/jobs/README.md

docs/jobs/api.md

@@ -0,0 +1,6 @@
+# Jobs Package API Reference
+::: mindtrace.jobs
+    options:
+      show_root_heading: false
+      show_source: false
+      heading_level: 4

docs/services/README.md

@@ -0,0 +1 @@
+../../mindtrace/services/README.md

docs/services/api.md

@@ -0,0 +1,6 @@
+# Services Package API Reference
+::: mindtrace.services
+    options:
+      show_root_heading: false
+      show_source: false
+      heading_level: 4

docs/storage/README.md

@@ -0,0 +1 @@
+../../mindtrace/storage/README.md

docs/storage/api.md

@@ -0,0 +1,6 @@
+# Storage Package API Reference
+::: mindtrace.storage
+    options:
+      show_root_heading: false
+      show_source: false
+      heading_level: 4

mindtrace/core/README.md

@@ -50,7 +50,7 @@ The core module is organized into several submodules:
 ### Logging (`logging/`)
 - Structured logging configuration and utilities
 
-## API Reference
+
 
 ### Core Classes
 
@@ -78,7 +78,7 @@ print(config["MINDTRACE_DIR_PATHS"]["TEMP"])
 print(config.MINDTRACE_DIR_PATHS.TEMP)
 ```
 
-For detailed usage of the Config class—including how it’s used within the Mindtrace class—refer to the [Usage documentation](../../samples/core/config/README.md)
+For detailed usage of the Config class—including how it's used within the Mindtrace class—refer to the [Usage documentation](https://github.com/Mindtrace/mindtrace/tree/dev/samples/core/config)
 
 #### `Logging`
 The `get_logger` function provides a unified way to configure logging across your application. It can return either a standard Python logger or a structlog
@@ -285,4 +285,3 @@ Dynamically instantiate a class from string reference.
 from mindtrace.core import instantiate_target
 instance = instantiate_target("my.module.MyClass", param="value")
 ```
-

mindtrace/services/README.md

@@ -9,7 +9,7 @@
 [Architecture](#architecture)<br>
 [Auto-generation for Connection Managers](#auto-generation-for-connection-managers)<br>
 [Usage Example](#usage-example)<br>
-[Testing & Coverage](#testing--coverage)<br>
+[Testing & Coverage](#testing-and-coverage)<br>
 [API Reference](#api-reference)<br>
 [MCP Integration: Exposing Service Endpoints as Tools](#mcp-integration-exposing-service-endpoints-as-tools)<br>
 [Remote MCP Server Usage with Cursor](#remote-mcp-server-usage-with-cursor)
@@ -69,7 +69,7 @@ cm.status()  # now generated as a method
 
 ## Usage Example
 
-See [`mindtrace/services/sample/echo_service.py`](./mindtrace/services/sample/echo_service.py) for a full example. Basic usage:
+See [`mindtrace/services/sample/echo_service.py`](https://github.com/Mindtrace/mindtrace/blob/dev/mindtrace/services/sample/echo_service.py) for a full example. Basic usage:
 
 ```python
 from mindtrace.services import Service
@@ -115,7 +115,7 @@ class EchoService(Service):
         return EchoOutput(echoed=payload.message)
 ```
 
-## Testing & Coverage
+## Testing and Coverage
 
 The test runner supports unit, integration, and stress tests:
 
@@ -181,7 +181,7 @@ The Model Context Protocol (MCP) is a protocol for exposing service functionalit
   This makes the `echo` function available both as a REST endpoint and as an MCP tool.
 
 ### Example: EchoService with MCP
-See [`mindtrace/services/sample/echo_mcp.py`](./mindtrace/services/sample/echo_mcp.py):
+See [`mindtrace/services/sample/echo_mcp.py`](https://github.com/Mindtrace/mindtrace/blob/dev/mindtrace/services/sample/echo_mcp.py):
 ```python
 from mindtrace.services.samples.echo_mcp import EchoService
 
@@ -286,9 +286,9 @@ asyncio.run(main())
 - The MCP client allows you to list and call tools programmatically.
 
 For trial purposes, see the sample files:
-- [`mindtrace/services/sample/echo_mcp.py`](./mindtrace/services/sample/echo_mcp.py)
-- [`samples/services/echo_mcp_service.py`](../../samples/services/echo_mcp_service.py)
-- [`samples/services/mcp/mcp_client.py`](../../samples/services/mcp/mcp_client.py)
+- [`mindtrace/services/sample/echo_mcp.py`](https://github.com/Mindtrace/mindtrace/blob/dev/mindtrace/services/sample/echo_mcp.py)
+- [`samples/services/echo_mcp_service.py`](https://github.com/Mindtrace/mindtrace/blob/dev/samples/services/echo_mcp_service.py)
+- [`samples/services/mcp/mcp_client.py`](https://github.com/Mindtrace/mindtrace/blob/dev/samples/services/mcp/mcp_client.py)
 
 
 ## Remote MCP Server Usage with Cursor