Skip to content

feat: Matrix transport support#29

Merged
PleasePrompto merged 28 commits intoPleasePrompto:mainfrom
n-haminger:feature/matrix-transport
Mar 11, 2026
Merged

feat: Matrix transport support#29
PleasePrompto merged 28 commits intoPleasePrompto:mainfrom
n-haminger:feature/matrix-transport

Conversation

@n-haminger
Copy link
Copy Markdown
Contributor

@n-haminger n-haminger commented Mar 7, 2026

Summary

Add Matrix as a first-class messaging transport alongside Telegram — laying the foundation for making ductor a truly transport-agnostic agent framework.

What is Matrix?

Matrix is an open, decentralized communication protocol for real-time messaging. Unlike Telegram, Matrix is:

  • Self-hostable — run your own homeserver with full data sovereignty
  • Federated — servers talk to each other, no single point of failure
  • End-to-end encrypted — via the Olm/Megolm protocol
  • Open standard — no vendor lock-in, multiple client implementations (Element, FluffyChat, etc.)

Why Matrix?

  • Users running ductor on private infrastructure often prefer self-hosted communication
  • E2EE support means agent conversations stay private even on shared servers
  • The decentralized architecture aligns with ductor's "runs on your machine" philosophy
  • Matrix rooms can be used for multi-user agent access without Telegram group limitations

Foundation for more transports

This PR introduces a transport registry pattern (transport_registry.py + BotProtocol interface) that makes adding new transports straightforward. Adding Discord, Slack, or any other messaging platform requires only:

  1. A bot implementation conforming to BotProtocol
  2. A transport adapter for MessageBus delivery
  3. A one-line entry in _TRANSPORT_FACTORIES

All orchestrator logic, session management, CLI execution, and automation systems remain transport-agnostic.

Changes

Core Transport Layer

  • ductor_bot/matrix/ — Full Matrix transport: bot, sender, transport, buttons, media, credentials, typing, formatting, startup
  • ductor_bot/transport_registry.py — Transport factory dispatch
  • ductor_bot/bot/protocol.py — Shared BotProtocol interface
  • ductor_bot/config.pyMatrixConfig, transport field, media_files_days rename

Features

  • Segment-based streaming (Matrix doesn't support message editing)
  • Clean chat: tool/system markers suppressed, only reasoning + final summary visible
  • Reaction-based buttons (emoji digits, since Matrix has no inline keyboards)
  • E2EE support via matrix-nio[e2e]
  • Incoming media handling (images, audio, video, files)
  • Full command parity (!// prefix)

Transport-Neutral Refactoring

  • telegram_tools/media_tools/ (scans both telegram_files/ and matrix_files/)
  • telegram_files_daysmedia_files_days (backwards-compatible)
  • Setup wizard supports transport selection during onboarding
  • create_agent.py supports --transport matrix for sub-agents
  • Transport-neutral identity text and documentation

Documentation (14 files updated)

  • README.md, docs/config.md, docs/architecture.md, docs/system_overview.md
  • Module docs: bot.md, matrix.md, bus.md, orchestrator.md, cron.md, logging.md, multiagent.md
  • Guides: installation.md, developer_quickstart.md, automation.md

Test plan

  • All existing tests pass (891+ tests, pre-existing failures excluded)
  • Matrix bot starts and connects to homeserver
  • Text messages, commands, streaming work via Matrix
  • File send/receive works (images, audio, documents)
  • Tool/system markers suppressed in Matrix chat
  • Telegram transport unaffected (default behavior preserved)
  • media_files_days backwards-compatible with telegram_files_days
  • Sub-agent creation with --transport matrix works
  • E2EE room communication (requires matrix-nio[e2e])

🤖 Generated with Claude Code

@PleasePrompto
Copy link
Copy Markdown
Owner

Hey n-haminger, thanks for the support! 😄
I’ll take a closer look at Matrix first, and then I’m planning to take over the integration from you.

Quick note:
If you’d like to appear as a contributor, you’ll need to change your email address when creating the PR. Otherwise, you won’t be listed as one. Github will then assign it automatically!

n-haminger and others added 26 commits March 8, 2026 05:13
…cations

When update_check is false in config.json, the UpdateObserver background
task is not started. This is useful when updates are managed via git-based
cron sync instead of the built-in PyPI upgrade flow.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…dling

Previously, when the CLI timed out mid-execution, the session ID was lost
(especially for new sessions where the ResultEvent never arrived), the user
received a generic error message, and the next message started a fresh
session without context.

Changes:
- Increase default cli_timeout from 600s to 1800s (30 min)
- Capture session_id from SystemInitEvent during streaming so it survives
  a timeout kill before the final ResultEvent
- Add dedicated _handle_timeout() flow that persists the session for
  auto-resume on the next user message
- Add clear timeout_error_text() user-facing message explaining what
  happened and that the session is preserved

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Introduce an alternative Matrix transport alongside Telegram via matrix-nio.
Key changes:
- BotProtocol abstraction for transport-agnostic bot interface
- NotificationService protocol replacing hardcoded aiogram dependency
- MatrixBot with full message handling, streaming, buttons, and formatting
- MatrixConfig model with homeserver/credentials/room settings
- AgentStack/Supervisor support for mixed transport agents
- Autouse test fixture preventing pytest from killing real systemd service
- 35 new tests for matrix utility modules (id_map, buttons, formatting)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix observer wiring: use orchestrator.wire_observers_to_bus() instead
  of manually calling non-existent methods
- Fix restart marker: correct filename (restart-requested) and API
  (write_restart_marker instead of mark_restart_requested)
- Fix message dispatch: use handle_message/handle_message_streaming
  instead of non-existent run_message
- Fix auto-join: accept invites when allowed_rooms is empty (= all allowed)
- Add ! command prefix for Matrix (Element intercepts / commands)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add all Telegram commands to Matrix (!help, !status, !model, !memory,
  !cron, !sessions, !tasks, !diagnose, !upgrade, !info, !session,
  !agent_commands, !showfiles, !agents, !agent_start/stop/restart)
- Route orchestrator commands via orch.handle_message() with ButtonGrid
  rendering as numbered text lists
- Fix !new: use orch.handle_message → cmd_reset() instead of end_session()
- Fix !stop/!stop_all: use orch.abort(chat_id) instead of kill_all_active()
- Fix line break rendering: convert \n to <br> in Matrix HTML output
- Build help text dynamically from BOT_COMMANDS + MULTIAGENT_SUB_COMMANDS

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matrix bots can now operate in group rooms with mention-only mode,
matching the existing Telegram group_mention_only behaviour. Bot
responds only to @mentions (body/HTML pill) or replies to its own
messages. Named rooms are always treated as groups regardless of
member_count (fixes false DM detection after fresh join). Auto-leave
for unauthorized room invites. SubAgentConfig now accepts
group_mention_only from agents.json.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace m.replace edit-in-place streaming with a buffer+flush approach:
- Text between tool calls is sent as separate messages
- Tool activity markers ([TOOL: ...]) are suppressed
- Final segment gets button extraction
- Removed unused MatrixStreamEditor import

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove hardcoded Telegram rules from workspace RULES.md template
- Add transport-specific rule constants (_TRANSPORT_TELEGRAM, _TRANSPORT_MATRIX)
  injected at runtime based on config.transport
- Matrix rules: HTML formatting, !-prefix commands, text-based buttons
- Telegram rules: 4096-char limit, mobile-friendly, inline keyboard buttons
- Make identity strings transport-aware (sub-agents show their transport type)
- Pass config.transport through lifecycle → inject_runtime_environment()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Address all SHOULD-FIX items from PR review:
- Credentials file permissions (0o600 via os.open)
- room_send error logging for failed sends and uploads
- Deduplicate _BUTTON_RE regex (single source in formatting.py)
- Extract shared cron sanitization to bus/cron_sanitize.py
- Fix _split_text to split raw text before HTML conversion
- Add sync_forever exception logging
- Fix invite race condition with _leaving_rooms guard set
- Remove unused field import from sender.py
- Fix empty input handling in _convert_markdown

Fix async inter-agent communication for Matrix:
- Add initial sync() + _populate_rooms_from_sync() at startup
- Track _last_active_room as fallback delivery target
- MatrixNotificationService.notify() falls back to notify_all()
- notify_all() uses _last_active_room when allowed_rooms is empty
- on_async_interagent_result() warns and falls back on chat_id=0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add JOIN_NOTIFICATION.md support: agents send and pin a welcome
  message on first contact (/start, group add, room invite)
- Fix matrix broadcast bug: _broadcast() and broadcast() now fall back
  to _last_active_room when allowed_rooms is empty, matching
  notify_all() behavior. Fixes silent cron result loss.
- Add --description flag to create_agent.py for auto-populating
  JOIN_NOTIFICATION.md on sub-agent creation
- Update agent_tools RULES.md with join notification docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace fragile text-number button matching with reaction-based
interaction. Bot sends emoji reactions (1️⃣ 2️⃣ 3️⃣...) on selector
messages, user clicks a reaction to select. Properly routes
callback_data through selector handlers (model, cron, session, task)
instead of feeding labels as chat messages.

- ButtonTracker stores callback_data alongside labels, supports
  reaction matching via event_id + emoji key
- MatrixBot registers m.reaction event handler
- New _handle_button_callback routes callbacks through the same
  selector handlers as Telegram
- Text-number input preserved as fallback
- 17 button tests (10 new for reactions)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…d all status tags

- Typing indicator was cleared by Matrix after each sent message during
  streaming, making it look like the bot stopped working. Now re-set
  after every flushed segment and status tag.
- Send [TOOL: name] tags as bold messages (matching Telegram behavior).
- Add on_system_status callback for THINKING, COMPACTING, RECOVERING,
  and TIMEOUT tags (previously Matrix-only gap).
- Add matrix.md module documentation.
- Add Matrix cross-reference in bot.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a background keep-alive task to MatrixTypingContext that re-sends
the typing notification every 5 seconds. Matrix clients (Element) clear
the indicator when the bot sends a message, and a single re-set right
after sending was not reliable. The periodic keep-alive ensures the
indicator stays visible throughout the entire streaming response.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add --transport, --homeserver, --user-id, --password, --allowed-users,
  --allowed-rooms flags for Matrix agent creation
- Make --password optional (can be added to agents.json later)
- Auto-detect transport from flags (--homeserver → matrix, --token → telegram)
- Validate Matrix user IDs and room IDs
- Reject conflicting Telegram/Matrix flags
- Update RULES.md with Matrix agent setup documentation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add transport selection step to the onboarding wizard. Users now choose
between Telegram and Matrix before entering transport-specific credentials.

Matrix flow prompts for:
- Homeserver URL (HTTPS validation)
- Bot user ID (@localpart:domain validation)
- Password (for initial login)
- Allowed users (Matrix user ID format)

_write_config() extended with transport parameter and Matrix config
writing. Telegram flow unchanged. Matrix validation messages in
__main__.py now reference `ductor onboarding`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- setup_wizard.md: document transport selection step, Matrix credential
  prompts, and transport-aware configuration gate
- matrix.md: document typing keep-alive mechanism, credential flow,
  and setup wizard integration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…eo, files)

Add Matrix media reception pipeline analogous to the existing Telegram
media handler. Agents can now receive and process files sent via Matrix.

- matrix/media.py: download media from homeserver, build agent prompts
- matrix/bot.py: register callbacks for RoomMessageImage/Audio/Video/File
- workspace/paths.py: add matrix_files_dir property
- cleanup/observer.py: include matrix_files in periodic cleanup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…try + tests

Review-driven cleanup before merge:
- K1: Remove dead streaming.py (never imported)
- K2: Add error_callback to resolve_matrix_media for user feedback
- K3: Fix sync_forever full_state=False (avoid full state on reconnect)
- W1: Add type annotations (MatrixRoom, RoomMessageText/Media)
- W2: Fix redundant except clause in media.py
- W3/W4: Extract resolve_broadcast_rooms() shared helper (DRY)
- W5: Move update_index to files/storage.py (transport-agnostic)
- W6: Refactor command handler from if-chain to dispatch table

Transport extensibility:
- New transport_registry.py with create_bot() factory + dispatch dicts
- __main__.py: _IS_CONFIGURED_CHECKS, _TRANSPORT_VALIDATORS dicts
- supervisor.py: _TRANSPORT_IDENTITY_CHANGED dict

Tests (132 new Matrix tests):
- test_transport.py: background/heartbeat/interagent/task/cron delivery
- test_media.py: download, resolve, MIME detection, prompt building
- test_credentials.py: all 3 login modes + error cases
- test_sender.py: send_rich, file upload, text splitting
- test_transport_registry.py: factory dispatch

712 tests pass (1 pre-existing failure in test_auth.py).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ming

Add clean_intermediate option to StreamingConfig (default: false).
When enabled, tool markers ([TOOL: ...]) and system markers ([THINKING])
are redacted when the next reasoning text arrives, and all intermediate
messages are redacted after the final summary is sent — leaving only
the final response visible in the chat.

- sender.py: add redact_message() and redact_messages() helpers
- bot.py: track per-request marker/intermediate event IDs in
  _run_streaming(), redact via asyncio.ensure_future (non-blocking)
- config.py: add clean_intermediate field to StreamingConfig

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When clean_intermediate is enabled, tool/system markers are simply
not sent rather than sent-then-redacted. This avoids the "message
deleted" indicators that Element shows for redacted events.

Reasoning text and the final summary are still sent normally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tool and system markers ([TOOL: ...], [THINKING], etc.) are no longer
sent in Matrix chats. Only reasoning text and the final summary are
visible. This is now hardcoded behavior for Matrix — no config flag
needed. The tag parameter in _flush_and_tag() is kept for logging but
not sent to the room.

Remove clean_intermediate from StreamingConfig (no longer needed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename telegram-centric names to transport-neutral equivalents:

- telegram_tools/ → media_tools/ (scripts work with all transports)
- telegram_files_days → media_files_days (backwards-compat alias kept)
- Remove hardcoded telegram_files/ from required dirs (created on-demand)
- Add media_files_dir(transport) helper to DuctorPaths

Update scripts to be transport-aware:
- list_files.py: scans both telegram_files/ and matrix_files/
- file_info.py: accepts paths from any media directory

Update documentation:
- README.md: mention Matrix alongside Telegram
- installation.md: list both transport options as requirements
- cleanup.md, config.md: update field names and descriptions
- RULES.md: route to media_tools/ instead of telegram_tools/

Fix identity text:
- Remove hardcoded "Telegram" from inter-agent messages
- create_agent.py: conditional output based on transport

Fix pre-existing test gaps:
- test_init_wizard.py: mock _ask_transport + pass transport param

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update all documentation to reflect dual transport support (Telegram
and Matrix). Add transport field, MatrixConfig schema, and Matrix auth
sections. Generalize Telegram-specific language across README,
architecture, config, automation, and module docs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add `interagent_port` to AgentConfig so multiple ductor instances on
the same host can each bind their internal API to a different port.
Default remains 8799 for backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@n-haminger n-haminger force-pushed the feature/matrix-transport branch from d21dd5d to b902cc3 Compare March 8, 2026 05:14
@n-haminger
Copy link
Copy Markdown
Contributor Author

Hey n-haminger, thanks for the support! 😄
I’ll take a closer look at Matrix first, and then I’m planning to take over the integration from you.

Quick note:
If you’d like to appear as a contributor, you’ll need to change your email address when creating the PR. Otherwise, you won’t be listed as one. Github will then assign it automatically!

Thank you for the note, I think now the right email address should be assigned.

n-haminger and others added 2 commits March 9, 2026 13:14
nio 0.25.2 expects a file-like object, not raw bytes. Passing bytes
directly to client.upload() crashes the agent sync loop.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…diately

Matrix !stop fix:
- Message processing now runs as asyncio.create_task() instead of blocking
  the nio sync callback, so !stop/!interrupt can be processed immediately
  while a long-running CLI call is in flight
- Added _dispatch_with_lock(), _run_handler_with_lock() for per-chat
  serialization via LockPool
- Split commands into _IMMEDIATE_COMMANDS (stop, interrupt, help, etc.)
  that bypass the lock and deferred commands that run as background tasks

New !interrupt command (Matrix + Telegram):
- Sends SIGINT to the CLI process instead of killing it — equivalent to
  pressing ESC in the terminal, cancels the current tool without exiting
- interrupt_process() in process_tree.py (SIGINT on POSIX, CTRL_C on Win)
- ProcessRegistry.interrupt_all() with _interrupted flag so orchestrator
  flows can suppress the "Session Error" message after a user-initiated
  interrupt
- Telegram: handled pre-lock in SequentialMiddleware (before abort check)
- Matrix: handled as immediate command in _COMMAND_DISPATCH
- "esc" and "interrupt" bare words moved from ABORT_WORDS to
  INTERRUPT_WORDS so they trigger soft interrupt instead of full kill

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@PleasePrompto PleasePrompto merged commit 208ac1c into PleasePrompto:main Mar 11, 2026
PleasePrompto added a commit that referenced this pull request Mar 11, 2026
Merge updated main (including PR #29 Matrix transport by n-haminger)
into feature branch. Conflicts resolved with our restructured codebase.
Removed duplicate interrupt checks introduced by auto-merge.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PleasePrompto added a commit that referenced this pull request Mar 11, 2026
Restructure the messenger layer into a transport-agnostic plugin
architecture. New transports (Discord, Slack, etc.) can be added by
implementing BotProtocol + registering a factory — no changes needed
in orchestrator, session, cron, or any other core module.

Key additions:
- BotProtocol, NotificationService, TransportRegistry
- Shared command classification, callback routing, BaseSendOpts
- SessionKey factory methods with transport-prefixed storage
- MultiBotAdapter for parallel transport operation
- MessengerCapabilities feature matrix
- /interrupt command (soft SIGINT) for all transports
- Matrix concurrency fix (nio sync loop no longer blocked)
- Dead code removal, deduplication, zero lint/typecheck errors

Built on Matrix transport support by n-haminger (PR #29).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@PleasePrompto
Copy link
Copy Markdown
Owner

Great work on the Matrix transport! 🎉 The implementation is solid — streaming, reactions, media, group mentions, typing indicators — all working well.

I've merged this and then restructured the entire messenger layer on top of your work to create a transport plugin architecture that makes it easy to add more messengers in the future (Discord, Slack, etc.).

What changed

Architecture (messenger/ module)

Your code lived in bot/ (Telegram) and matrix/ (Matrix) as separate top-level modules. I moved everything into a unified messenger/ package:

messenger/
├── protocol.py          # BotProtocol — 7 methods any transport must implement
├── notifications.py     # NotificationService — 2-method protocol (notify, notify_all)
├── registry.py          # Transport factory registry (no if/else, just a dict)
├── commands.py          # Shared command classification (direct/orchestrator/multiagent)
├── callback_router.py   # Shared button/selector callback routing
├── send_opts.py         # BaseSendOpts base class
├── capabilities.py      # Feature matrix per transport (buttons, reactions, editing...)
├── multi.py             # MultiBotAdapter for parallel transport operation
├── telegram/            # All Telegram code (was bot/)
└── matrix/              # All Matrix code (was matrix/)

Adding a new transport now requires

  1. Implement BotProtocol in messenger/discord/bot.py
  2. Implement TransportAdapter for envelope delivery
  3. One line in registry.py: _TRANSPORT_FACTORIES["discord"] = _create_discord
  4. SessionKey.for_transport("discord", chat_id) works automatically

Zero changes needed in orchestrator, session, cron, webhook, or heartbeat code.

Session isolation

SessionKey now uses transport-prefixed storage keys (tg:123, mx:456) with factory methods:

SessionKey.telegram(chat_id, topic_id)
SessionKey.matrix(chat_id)
SessionKey.for_transport("discord", chat_id)

New feature: /interrupt

Built on top of your !stop concurrency fix (the create_task + LockPool pattern). /interrupt sends SIGINT instead of SIGTERM — cancels the current tool execution without killing the agent session. Works on both Telegram and Matrix. Also added bare-word triggers: esc, interrupt, skip, überspringen.

Code quality

  • Removed dead code across 11 files (381 deletions)
  • Deduplicated shared utilities (file browser, command sets)
  • Fixed all mypy errors (18 → 0) without type: ignore workarounds
  • Fixed all ruff lint errors without noqa comments
  • Fixed Matrix bugs: message replay, stop_all across transports, Docker file paths, missing awaits
  • Extracted MatrixStreamEditor class from inline streaming logic
  • 3284 tests passing

Thanks for the contribution — the Matrix support is a great addition to ductor!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants