Kwami is an independent, reusable 3D interactive AI companion library (npm: kwami) for building companions with a real-time 3D “blob” body, voice/AI capabilities, and a personality system.

This repository is the KWAMI ecosystem monorepo: the core library plus playground + apps (Candy Machine, Marketplace, DAO, Website) and Solana programs that use it.

Version 1.5.11 - See what's new

• kwami/: the core npm library
• pg/: interactive playground for exploring + tuning features
• app/, candy/, market/, dao/, web/: ecosystem apps built on top of the core library
• solana/: on-chain programs (token/NFT infrastructure)
• docs/: documentation (EN/ES) for the full ecosystem

Kwami is built around three core components:

• 🎨 Body - 3D audio-reactive blob with real-time animations and touch interaction
• 🧠 Mind - AI capabilities (TTS, STT, Conversations) with multi-provider support
• 🎭 Soul - Personality system with 10-dimensional emotional intelligence

• ✅ Audio-Reactive 3D Blob - Real-time visualization with WebGL
• ✅ Multiple Shader Skins - 3Colors collection (Poles, Donut, Vintage)
• ✅ Interactive Touch Effects - Liquid-like ripples on click
• ✅ Animation States - Idle, listening, thinking, speaking
• ✅ AI Voice Integration - ElevenLabs & OpenAI TTS support
• ✅ ElevenLabs Agents - Full agent management with Tools & Knowledge Base APIs
• ✅ Emotional Personalities - Rich personality system with emotional traits
• ✅ Background System - Gradients, images, videos with glass effects
• ✅ TypeScript First - Fully typed for excellent DX
• ✅ Framework Agnostic - Use with React, Vue, Svelte, or vanilla JS

# npm
npm install kwami

# bun (recommended)
bun add kwami

# deno
deno add npm:kwami

All dependencies included: three, simplex-noise, @elevenlabs/elevenlabs-js

All ecosystem projects now include Docker support with 3 runtime options:

# Node.js (default)
docker build -f docker/Dockerfile -t kwami-app .

# Bun (fast builds)
docker build -f docker/Dockerfile.bun -t kwami-app:bun .

# Deno (secure runtime)
docker build -f docker/Dockerfile.deno -t kwami-app:deno .

👉 Complete Docker Deployment Guide (EN) / ES

import { Kwami } from "kwami";

const canvas = document.querySelector("canvas") as HTMLCanvasElement;

const kwami = new Kwami(canvas, {
  body: {
    audioFiles: ["/audio/track.mp3"],
    initialSkin: "Poles",
    blob: {
      resolution: 180,
      colors: { x: "#ff0066", y: "#00ff66", z: "#6600ff" },
    },
  },
  mind: {
    provider: "elevenlabs",
    apiKey: "your-api-key",
    voiceId: "your-voice-id",
  },
  soul: {
    name: "Kaya",
    emotionalTraits: {
      happiness: 75,
      empathy: 95,
      energy: 60,
    },
  },
});

// Play audio and animate
await kwami.body.audio.play();

// Speak with AI voice
await kwami.speak("Hello! I am Kwami.");

// Randomize appearance
kwami.body.blob.setRandomBlob();

Docs live under docs/ (English + Spanish):

• Docs Home (EN): docs/en/index.md
• Docs Home (ES): docs/es/index.md

• Quick Start - Get running in minutes
• Installation - Detailed setup for all environments
• Core Concepts - Mind/Body/Soul architecture explained

• Body - 3D blob and scene management
• Mind - AI capabilities + provider system
• Soul - Personality + emotional traits

• Configuration - Configuration reference
• Animations - Animation states and effects

• Architecture Overview - System overview
• Mind Architecture - Provider internals
• Body Architecture - Rendering pipeline
• Soul Architecture - Personality system

• Advanced Topics - Performance, custom development, testing

• Kwami API - Complete API documentation

Try Kwami live at kwami.io

Run locally:

npm run playground

Features:

• 🎨 Rotating Sidebar System - Mind, Body, Soul configuration
• 🎙️ Voice Conversations (Beta) - WebSocket-based interactions
• 🤖 Complete Mind Menu - 50+ ElevenLabs configuration options
• ✨ Personality Editor - Real-time personality customization
• 🌈 Background Manager - Gradients, images, videos (15+ included)
• 🎛️ Audio Effects - Fine-tune frequency response
• 📥 GLB Export - Download blob as 3D model

See Playground README for details.

The core npm package (kwami) - a standalone, reusable 3D interactive AI companion library featuring a Mind-Body-Soul architecture. Published to npm for easy integration into any JavaScript/TypeScript project. This is the heart of the KWAMI ecosystem, providing the foundational 3D blob visualization, AI voice capabilities, and personality system that powers all other projects.

• npm Package: @kwami
• Source Code: kwami/
• Documentation: docs/en/kwami/README.md

Interactive demo with full configuration UI for testing all Kwami features.

• Documentation: docs/en/pg/README.md

Full-featured Nuxt 4 web application providing a complete Kwami experience. Built with glassmorphic UI, multi-language support (en, fr, es), Supabase authentication, and ElevenLabs voice integration. A production-ready implementation of the Kwami 3D AI companion.

• Documentation: docs/en/app/README.md

Smart contracts for QWAMI token and KWAMI NFT system with 10B supply by 2100.

• Documentation: docs/en/solana/README.md

Mint your own KWAMI NFTs on Solana with unique DNA-based validation.

• Documentation: docs/en/candy/README.md

Buy, sell, and trade KWAMI NFTs on the Solana blockchain.

• Documentation: docs/en/market/README.md

Decentralized governance platform for KWAMI NFT holders to participate in community decisions.

• Documentation: docs/en/dao/README.md

Public-facing website showcasing the KWAMI ecosystem.

• Documentation: docs/en/web/README.md

Complete documentation for all components and APIs organized by project. Start here: docs/en/index.md (English) / docs/es/index.md (Spanish)

// 3D blob with audio reactivity
kwami.body.blob.setSkin("Poles");
kwami.body.blob.setColors("#ff0000", "#00ff00", "#0000ff");
kwami.body.setBackgroundGradient("#1a1a2e", "#16213e");

// Interactive animations
kwami.body.enableBlobInteraction();
kwami.body.startListening(); // Microphone mode
kwami.body.startThinking(); // Contemplative animation

// Multi-provider AI system
await kwami.mind.initialize({
  provider: "elevenlabs", // or "openai"
  apiKey: "your-key",
  voiceId: "voice-id",
});

// Text-to-Speech
await kwami.mind.speak("Hello world!");

// Voice fine-tuning
kwami.mind.setStability(0.5);
kwami.mind.setSimilarityBoost(0.75);

// Advanced: Create agents with Tools & Knowledge Base
import { AgentConfigBuilder } from "kwami";

const config = new AgentConfigBuilder()
  .withName("Support Agent")
  .withVoice("voice_id")
  .withLLM("gpt-4o")
  .withTools([
    { name: "create_ticket", url: "https://api.example.com/tickets" },
  ])
  .withKnowledgeBase([{ knowledge_base_id: "kb_123" }])
  .build();

const agent = await kwami.mind.createAgent(config);

// Preset personalities
kwami.soul.loadPresetPersonality("friendly"); // Kaya
kwami.soul.loadPresetPersonality("professional"); // Nexus
kwami.soul.loadPresetPersonality("playful"); // Spark

// Custom emotional traits
kwami.soul.setEmotionalTrait("happiness", 80);
kwami.soul.setEmotionalTrait("energy", 90);

// Generate AI context
const systemPrompt = kwami.soul.getSystemPrompt();

const kwami = new Kwami(canvas, {
  body: {
    audioFiles: ["/audio/music.mp3"],
    blob: { spikes: { x: 0.5, y: 0.5, z: 0.5 } },
  },
});
await kwami.body.audio.play();

const kwami = new Kwami(canvas, {
  mind: { provider: "elevenlabs", apiKey: "key" },
  soul: { name: "Assistant" },
});

await kwami.listen();
kwami.think();
await kwami.speak("How can I help you?");

const kwami = new Kwami(canvas, {
  body: { initialSkin: "Poles" },
  mind: { provider: "elevenlabs" },
  soul: {
    name: "Kaya",
    emotionalTraits: {
      happiness: 75,
      empathy: 95,
      socialness: 90,
    },
  },
});

kwami.soul.loadPresetPersonality("friendly");
await kwami.mind.startConversation({
  systemPrompt: kwami.soul.getSystemPrompt(),
  onUserSpeaking: () => kwami.setState("listening"),
  onAISpeaking: () => kwami.setState("speaking"),
});

🏗️ Major Project Restructuring:

• Core Library: Moved to kwami/ folder for npm publishing
• Documentation: Reorganized into docs/ with project-specific folders (1_kwami through 8_web)
• Workspace Setup: Configured npm workspaces for all projects
• Playground: Renamed from playground/ to pg/
• Clean Separation: Each project now has its own isolated structure
• CI/CD: Updated GitHub Actions to build and publish from kwami/

📦 Improved Organization:

• Root now contains only monorepo-level files
• Each project has its own README, CHANGELOG, and documentation
• Clear separation between core library and applications
• Proper workspace dependency management

Complete ElevenLabs Conversational AI Agents Integration

Professional-grade agent management with Tools and Knowledge Base:

import { AgentConfigBuilder } from "kwami";

// Create an advanced agent with tools and knowledge
const config = new AgentConfigBuilder()
  .withName("Support Agent")
  .withVoice("voice_id")
  .withLLM("gpt-4o")
  .withPrompt("You are a helpful support agent")
  .withTools([
    {
      name: "create_ticket",
      description: "Create support ticket",
      url: "https://api.example.com/tickets",
    },
  ])
  .withKnowledgeBase([{ knowledge_base_id: "kb_123" }])
  .withMaxDuration(1800)
  .build();

const agent = await kwami.mind.createAgent(config);

Key Features:

• ✨ AgentConfigBuilder - Fluent API for agent configuration with validation
• 🔧 Tools API - Create custom tools with webhooks and parameters
• 📚 Knowledge Base API - RAG support with document management (URL, text, file)
• 🌊 Multi-Agent Workflows - Orchestrate complex conversations with conditional routing
• 🎯 Complete Type Coverage - Full TypeScript types for all ElevenLabs APIs
• ✅ Validation System - Built-in configuration validation with detailed errors
• 📖 Comprehensive Examples - Complete documentation and test suite

APIs Implemented:

• Agent CRUD operations (create, read, update, delete, duplicate, list)
• Tools management (create tools, manage webhooks, parameter schemas)
• Knowledge Base (create KBs, add documents, RAG indexing, semantic search)
• Workflow system (multi-agent orchestration, conditional routing, node types)
• Conversation management (list, get details, download audio, send feedback)

See Mind Examples for comprehensive usage guides.

• Unique DNA-based NFTs with 1 trillion limit
• Solana blockchain integration with Metaplex standard
• Candy machine for minting KWAMI NFTs
• DNA registry with on-chain validation
• Real-time WebSocket updates (Socket.IO 4.8.1)

NEW: Governance platform for KWAMI NFT holders built with Nuxt 4 and Solana.

cd dao
npm install
npm run dev

Features:

• 🔌 Multi-wallet support (Phantom, Solflare, Backpack, etc.)
• ✅ Automatic KWAMI NFT holder verification
• 💰 QWAMI token balance tracking for governance
• 📝 Create and vote on proposals
• 🗳️ Token-weighted voting system
• 📊 Real-time voting statistics

DAO Membership:

• Own at least 1 KWAMI NFT
• Hold 100+ QWAMI tokens to participate in governance
• One QWAMI = One vote

See dao/README.md and docs/en/dao/README.md for complete documentation.

• Multi-vendor AI provider support (ElevenLabs, OpenAI)
• Hot-swappable providers at runtime
• Unified provider interface for extensibility
• Enhanced background system with Three.js gradient overlays
• 10-dimensional emotional personality system
• Complete documentation

• 🎭 1.3.1 – Expanded Soul personality templates to 20+ presets
• 🧠 1.3.2 – Introduced Kwami Skills System (JSON/YAML-driven behaviors)
• 🎨 1.3.3 – Mind UI polish with provider logos and refined tabs
• 🧪 1.3.4 – Full Vitest suite (238 tests), hardened KwamiAudio and Soul defaults

See CHANGELOG for complete history.

┌──────────────────────────────────────┐
│             KWAMI                    │
│         (Orchestrator)               │
├──────────┬──────────────┬────────────┤
│   BODY   │     MIND     │    SOUL    │
│  Visual  │   AI/Voice   │ Personality│
│          │              │            │
│  Scene   │  Providers   │  Emotional │
│  Blob    │  - 11Labs    │  - Traits  │
│  Audio   │  - OpenAI    │  - Prompts │
└──────────┴──────────────┴────────────┘

Data Flow:

Audio → Web Audio API → FFT → Blob Vertices → Render
User → Mind Provider → Soul Context → LLM → TTS → Animation

See Architecture Overview for diagrams and details.

• Production: kwami.io
• Branch: main (protected)
• Auto-deploy: On merge to main

feature/* → [PR] → dev → [Review] → main → Deploy

See Contributing Guide for details.

We welcome contributions! Please:

1. Read CONTRIBUTING.md
2. Fork and create branch from dev
3. Make changes with tests
4. Submit PR to dev (not main)

• Automated Publishing: Package auto-publishes to npm when version changes on main
• Setup Guide: .github/NPM_SETUP.md
• Release Process: See CONTRIBUTING.md

Dual License:

• Personal/Non-Commercial: Apache License 2.0
• Commercial: Requires separate license

Contact: Alex Colls

See LICENSE for full terms.

Built with:

• THREE.js - 3D graphics
• simplex-noise - Smooth noise
• ElevenLabs - AI voice synthesis
• TypeScript - Type safety

• Documentation (EN): docs/en/index.md
• Documentation (ES): docs/es/index.md
• Issues: GitHub Issues
• Discussions: GitHub Discussions

• 🌐 Live Demo: kwami.io
• 📦 npm Package: npmjs.com/package/kwami
• 💻 GitHub: github.com/alexcolls/kwami
• 📚 Docs (EN): docs/en/index.md
• 📚 Docs (ES): docs/es/index.md
• 📝 Changelog: CHANGELOG.md

Made with ❤️ by the KWAMI team