Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: runpod-workers/worker-basic
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: Jakelolipopp/llama-cpp-python-worker
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 10 commits
  • 3 files changed
  • 1 contributor

Commits on Oct 15, 2025

  1. First test commit

    Vincent committed Oct 15, 2025
    Configuration menu
    Copy the full SHA
    0471d6c View commit details
    Browse the repository at this point in the history

  2. Auto-detect .gguf model on /runpod-volume and initialize Llama; add s… 

    …tartup checks in handler, and install llama-cpp-python in Dockerfile
    Vincent committed Oct 15, 2025
    Configuration menu
    Copy the full SHA
    0f91b23 View commit details
    Browse the repository at this point in the history

  3. Fix handler return statement (remove stray '+')

    Vincent committed Oct 15, 2025
    Configuration menu
    Copy the full SHA
    dcbf59f View commit details
    Browse the repository at this point in the history

  4. change base

    Vincent committed Oct 15, 2025
    Configuration menu
    Copy the full SHA
    1101e29 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2025

  1. Hoping to get cuda working

    Vincent committed Oct 17, 2025
    Configuration menu
    Copy the full SHA
    f3a9bb0 View commit details
    Browse the repository at this point in the history

  2. first real attempt at getting the llm stuff working

    Vincent committed Oct 17, 2025
    Configuration menu
    Copy the full SHA
    2a6750c View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2025

  1. Add clean_stream parser for channel-marked streaming output and use i… 

    …t in handler; increase Llama context (n_ctx=16384*4), disable verbose logging, add re import, and remove sleep/seconds handling
    Vincent committed Oct 22, 2025
    Configuration menu
    Copy the full SHA
    e65e166 View commit details
    Browse the repository at this point in the history

  2. Restore Llama context size to 16384 (remove *4 multiplier)

    Vincent committed Oct 22, 2025
    Configuration menu
    Copy the full SHA
    01e32b4 View commit details
    Browse the repository at this point in the history

  3. try to fix streaming

    Vincent committed Oct 22, 2025
    Configuration menu
    Copy the full SHA
    6232c70 View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2025

  1. Make handler async and return early error messages instead of yieldin… 

    …g (fix streaming)
    Vincent committed Oct 29, 2025
    Configuration menu
    Copy the full SHA
    ad1c173 View commit details
    Browse the repository at this point in the history
Loading