Skip to content

questinrest/tracelite

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 

Repository files navigation

tracelite

A lightweight tracing Proof-of-Concept, tracelite, has been implemented in the tracelite/ directory. It relies exclusively on the Python Standard Library (zero external dependencies) to demonstrate the most powerful architectural patterns found in enterprise LLM observability frameworks (LangSmith, DeepEval, Opik).

What It Can Do (Capabilities)

  • Seamless Context Propagation: Automatically tracks hierarchical parent -> child span lineage gracefully, even across asynchronous function execution.
  • Lineage Flattening (Dotted Order): Implements LangSmith's genius dotted_order design, enabling instant reconstruction of trace waterfalls without running expensive recursive database queries on parent_id.
  • Zero-Blocking Performance: Integrates a queue.Queue background daemon worker (Inspired by DeepEval/Opik), ensuring the instrumentation adds virtually < 1ms of overhead to your application's main runtime loop.
  • LLM-First Strongly Typed Spans: Offers native classes (LlmSpan, RetrieverSpan, ToolSpan) for explicitly tracking token counts, hyper-parameters, and tools (Inspired by DeepEval).
  • Time-Sorted Stable IDs: Uses UUID4 prefixes infused with hex UNIX timestamps to mimic UUID7, meaning span IDs can be chronologically sorted just by reading the ID string.
  • Decorator & Context Manager APIs: Exposes a frictionless developer experience matching LangSmith/Opik via @observe() or with span():.

Architecture Overview & Implementation Details

Tracelite's architecture is a hybridized model specifically designed to capture the best attributes of DeepEval, LangSmith, and Opik while discarding their heavy infrastructure.

graph TD
    A[User_Application_Code] -->|observe decorator| B[decorators.py]
    A -->|span context| B

    subgraph Context_Propagation
        B -->|push span| C[ContextVar_Stack]
        C -->|read lineage| D[models.py_initialization]
    end

    subgraph Core_Primitives
        D --> E[LlmSpan]
        D --> F[RetrieverSpan]
        D --> G[BaseSpan]
    end

    subgraph Background_Exporter
        B -->|enqueue on exit| H[Queue]
        H -->|daemon thread reads| I[exporter.py]
        I -->|write JSONL| J[traces.jsonl]
    end

    E -.-> H
    F -.-> H
    G -.-> H
Loading

Core Components (tracelite/)

  1. models.py: Lean standard dataclasses defining the core payloads (Trace, BaseSpan, LlmSpan, RetrieverSpan). Features smart constructor logic that fetches the current span_lineage to build the dotted_order string instantly upon span initialization.
  2. context.py: A hyper-safe propagation utility utilizing purely immutable tuple state stored inside a ContextVar. This mitigates all cross-task threading contamination risks while maintaining lightning-quick performance.
  3. exporter.py: Automatically spins up a background thread singleton which reads completed spans from an internal queue.Queue, serializing and batching data asynchronously to an output traces.jsonl file.
  4. decorators.py: Simplistic function wrappers utilizing inspect to auto-capture arguments/returns, bridging standard and async Python code elegantly to trace lifecycles.
  5. utils.py: Provides the microsecond-accurate time-sorted ID generator logic to mirror UUID7 logic purely via standard library.

What Is Taken From Where (And Why It's Better)

Concept / Logic Block Taken From Why It's Better in Tracelite What Was Omitted
Context ContextVar Usage Opik We use Opik's immutable tuple stack. When an async child span pushes itself, it creates a new tuple. If tasks context-switch, they cannot mistakenly mutate another task's lineage list. This provides LangSmith's thread-safety without the heavy CPU overhead of LangSmith's copy_context(). DeepEval's mutable Singleton logic which is prone to thread-leakage and required complex WeakKeyDictionary hacks.
Span Storage/Lineage LangSmith We adopted the Dotted Order string paradigm (uuid1.uuid2.uuid3). A span encodes its entire lineage path. A UI or backend can render the parent/child trace tree waterfall instantly just by looking at an individual span payload, zero database JOINs required. Opik's relational parent_span_id concept, which requires $O(n)$ recursive database graph traversals just to draw one UI trace.
Span Domain Typing DeepEval The decorator supports explicit GenAI types (@observe(span_type="llm") yields an LlmSpan). This matches GenAI developer mental maps natively (e.g. explicitly tracking prompt_tokens rather than stuffing them in a messy untyped generic tags dict). Generic opentelemetry span types found in boilerplate SDKs.
Exporter Architecture DeepEval / Opik A native Python queue.Queue managed by a single daemon thread running a batching loop. LangSmith's heavy multi-region client and fallback logic.
ID Generation Strategy LangSmith We construct pseudo-UUID7 strings (timestamp_hex + uuid4). The span ID fundamentally contains the execution time. UUID4 usage across Opik/DeepEval which provides zero temporal ordering context.

Usage Example

import tracelite
from tracelite import observe, span

@observe(span_type="retriever", name="vector_search")
def fetch_documents(query):
    # Active span metadata
    tracelite.current_span().set_metadata({"top_k": 3})
    return ["doc_1", "doc_2"]

@observe(span_type="chain", name="rag_pipeline")
def rag_answer(query):
    docs = fetch_documents(query)
    
    with span("llm_generation", span_type="llm") as llm_span:
        llm_span.set_inputs({"prompt": f"Docs: {docs}, Query: {query}"})
        response = "Tracelite is extremely lightweight."
        llm_span.set_metrics({"prompt_tokens": 10, "completion_tokens": 5})
        llm_span.set_outputs({"completion": response})
        
    return response

# Traces log invisibly to traces.jsonl
rag_answer("What is the easiest way to observe LLMs?")
tracelite.exporter.shutdown() # Force flush queue at program exit

About

tracelite: A custom-built local only LLM tracing tooling

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages