Bot Overlay API - Display bot messages in call view

[LexioJ]

Bot Overlay API - Feature Request for Nextcloud Talk

Related Issue

This proposal builds on #15879 (Chat overlay in call view) by extending the concept to bot APIs.

Problem Statement

Current situation:

• Bots send messages via ChatManager->sendMessage() that appear in the chat timeline
• During video calls, these messages are only visible if the chat sidebar is open
• Important bot notifications (time warnings, status updates, reminders) are easily missed
• Opening the sidebar reduces video space and interrupts call flow

Real-world use case:
The Agenda Bot tracks meeting agenda items and sends time warnings (80%, 100%, 120% of planned duration). During active calls, participants often miss these critical timing alerts because they don't have the chat sidebar open.

Proposed Solution

Add a new Bot API endpoint that allows bots to display temporary overlay messages in the call view, similar to the live transcription interface.

API Design

Endpoint:

POST /ocs/v2.php/apps/spreed/api/v1/bot/{token}/overlay

Headers:

X-Nextcloud-Talk-Bot-Random: <random-seed>
X-Nextcloud-Talk-Bot-Signature: <signature>

Request Body:

{
  "message": "🎯 Current agenda: Budget Discussion (8/10 min)",
  "type": "info|warning|error",
  "duration": 5000,
  "priority": "normal|high"
}

Response:

• 201: Overlay message queued successfully
• 400: Invalid parameters or call not active
• 401: Bot authentication failed
• 429: Rate limit exceeded

Technical Implementation

Backend (lib/Controller/BotController.php):

/**
 * Display overlay message in call view
 *
 * @param string $token Conversation token
 * @param string $message The message to display
 * @param string $type Message type (info|warning|error)
 * @param int $duration Display duration in milliseconds (1000-10000)
 * @param string $priority Message priority (normal|high)
 */
#[BruteForceProtection(action: 'bot')]
#[OpenAPI(scope: 'bots')]
#[PublicPage]
public function sendOverlay(
    string $token, 
    string $message, 
    string $type = 'info',
    int $duration = 5000,
    string $priority = 'normal'
): DataResponse

Frontend (new component or extend LiveTranscriptionRenderer.vue):

• Reuse existing bubble/layered UI pattern from live transcriptions
• Display bot icon/avatar instead of participant avatar
• Color-code by message type (info=blue, warning=orange, error=red)
• Auto-dismiss after specified duration
• Position: bottom center (same as transcriptions)
• Show only during active calls

Event Flow:

Bot → POST /api/v1/bot/{token}/overlay
  → BotController validates & authenticates
    → Signaling server broadcasts overlay event
      → Frontend receives & displays in CallView
        → Auto-dismisses after duration

Rate Limiting & Abuse Prevention

• Rate limit: Max 1 overlay per bot per 30 seconds
• Max queue: 3 pending overlays per conversation
• Duration limits: 1-10 seconds only
• Message length: Max 200 characters
• Call requirement: Only works during active calls
• Bot permissions: Uses existing bot authentication

UI Specifications

Overlay appearance:

┌─────────────────────────────────────────┐
│ [Bot Icon] Bot Name                     │
│ 🎯 Current agenda: Budget (8/10 min)    │
└─────────────────────────────────────────┘

Styling:

• Reuse .transcript CSS from LiveTranscriptionRenderer
• Bot-specific background color (e.g., --color-primary-element-light)
• Icon: Use bot avatar or generic bot icon
• Font: Same as live transcriptions
• Animation: Fade in/out transitions

Positioning:

• Bottom center of call view
• 20% inset from sides (same as transcriptions)
• Stack multiple messages if needed
• Z-index: Above video, below call controls

Benefits

1. For bot developers:

   • Simple REST API, no WebRTC knowledge needed
   • Consistent with existing bot authentication
   • Clear visibility for important bot messages

2. For users:

   • See bot notifications without opening sidebar
   • Maintain full video view during calls
   • Better awareness of time-sensitive information

3. For Nextcloud Talk:

   • Reuses existing UI components (low development cost)
   • Natural extension of issue Chat overlay in call view (reuse live-transcription layered bubbles for regular chat) #15879
   • Enables richer bot interactions

Example Use Cases

Agenda Bot:

{
  "message": "⚠️ Time Alert: Budget discussion has reached planned time (10/10 min)",
  "type": "warning",
  "duration": 7000,
  "priority": "high"
}

Reminder Bot:

{
  "message": "🔔 Reminder: Meeting ends in 5 minutes",
  "type": "info",
  "duration": 5000,
  "priority": "normal"
}

Voting Bot:

{
  "message": "📊 Poll closed: 5 votes received",
  "type": "info",
  "duration": 4000,
  "priority": "normal"
}

Alternatives Considered

1. Toast notifications - Too transient, no backend API
2. Room description updates - Not visible during calls
3. Enhanced chat messages - Still requires sidebar open
4. Custom UI extension - Requires separate app, maintenance burden

Implementation Phases

Phase 1: Proof of Concept

• Basic endpoint with minimal validation
• Simple overlay display (reuse transcription component)
• Test with Agenda Bot

Phase 2: Polish

• Rate limiting & abuse prevention
• UI refinements (colors, animations)
• Documentation

Phase 3: Enhancement (future)

• Interactive overlays (buttons, actions)
• Persistent overlays (manual dismiss)
• Positioning options

Questions for Discussion

1. Should overlays be conversation-level (visible to all) or per-participant?
2. Should moderators be able to disable bot overlays?
3. Should there be a user preference to hide/show bot overlays?
4. Should overlays be logged to chat history?
5. Alternative names: "Bot Notifications", "Bot Alerts", "Bot Banners"?

Related Work

• Chat overlay in call view (reuse live-transcription layered bubbles for regular chat) #15879 - Chat overlay in call view
• Live transcription feature (Talk 20+)
• Bot API framework (Talk 16+)
• Reaction toaster system

Proof of Concept

I'm willing to develop a proof-of-concept implementation to demonstrate feasibility and gather feedback through screenshots.

---

Maintainer feedback welcome! Is this direction aligned with Talk's bot API roadmap?

[nickvergessen]

Definetily a cool idea 👍