[Feature] QBIT LLM API Integration & Autonomous Avatar

[SeanChangX]

Summary

Enable QBIT to integrate with user-provided LLM APIs and act as an autonomous digital avatar when the owner is offline.

Problem / Motivation

Currently, QBIT does not support:

• User-provided LLM integration
• Autonomous responses when the owner is offline
• Persistent conversational memory

This limits QBIT’s ability to function as a continuous digital presence.

Proposed Solution

Introduce native LLM API integration and a memory-driven response system that allows claimed QBITs to:

• Use a user-configured LLM provider
• Maintain conversational context (“brain”)
• Automatically reply to pokes when the owner is offline

Key Requirements

1. LLM Configuration (Dashboard)

• Users can:

  • Select provider/model
  • Input and manage API keys
  • Configure usage limits (token / request / budget)
  • Define QBIT personality/avatar profile (bounded length)

• Constraint:

  • QBIT must be claimed before configuration

2. Autonomous Runtime Behavior

• QBIT can:
  • Detect owner offline status
  • Auto-reply to incoming poke events
  • Generate context-aware responses
  • Apply personality/avatar profile consistently

3. Memory System

• Maintain conversational context across interactions
• Separate memory:
  • owner ↔ QBIT
  • other users ↔ QBIT
• Support:
  • incremental updates per interaction
  • automatic compression/summarization

4. System Integration

• Extend existing QBIT messaging/reply pipeline to support:
  • QBIT → user responses (not only user → QBIT)
• Define:
  • memory storage layer
  • retrieval flow during inference
  • summarization pipeline

Acceptance Criteria

• Claimed QBIT can connect to a valid LLM configuration
• Usage limits are enforced correctly
• Personality/avatar profile affects generated responses
• QBIT auto-replies when owner is offline
• Memory updates correctly per interaction
• Owner vs other users memory is properly separated
• Memory size remains bounded via compression
• System handles API/key/quota failures gracefully

Non-Goals

• Building a proprietary LLM
• Supporting multi-agent QBIT collaboration (future work)
• Real-time streaming responses (optional later)

Open Questions

• How to define "owner offline"? (heartbeat? last active timestamp?)
• Should memory be:
  • per-user isolated?
  • partially shared/global?
• What compression strategy is preferred? (LLM summarize vs heuristic)
• Do we need fallback models/providers by default?

Risks / Considerations

• Cost explosion due to uncontrolled usage
• API key security and leakage risk
• Unintended or unsafe autonomous replies
• Memory drift / hallucination from aggressive summarization

Dependencies

• QBIT dashboard (settings UI)
• Messaging/reply pipeline
• Storage layer (for memory)
• Background job system (for summarization/compression)

Sub-Issues

• Dashboard: LLM configuration UI
• Backend: API key management & encryption
• Runtime: Offline detection + auto-reply flow
• Memory: schema + storage design
• Memory: compression/summarization pipeline
• Quota: usage tracking & enforcement
• Guardrails: moderation + rate limiting
• Observability: metrics + logging
• Fallback handling (provider / quota failure)