Skip to content
BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
static
static
 
 
templates
templates
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
app.py
app.py
 
 
requirements.txt
requirements.txt
 
 

Repository files navigation

Voice Assistant AI 🎙️

An advanced, full-stack Voice Assistant application powered by the sophisticated natural language capabilities of OpenAI's GPT-3.5-Turbo and the robust voice transcription/synthesis engines of IBM Watson.

This application provides a highly polished, modern web interface allowing users to communicate with an AI assistant through both voice and text inputs. It supports dynamic client-side API key configuration, reducing server-side dependencies and enhancing security by allowing individual users to supply their own credentials.

📑 Table of Contents

  1. Features
  2. Architecture Overview
  3. Technology Stack
  4. Prerequisites
  5. Installation & Setup
  6. Configuration & Usage
  7. API Endpoints
  8. Error Handling Strategy

✨ Features

  • Voice & Text Input: Engage with the AI using your microphone or by typing messages directly.
  • Dynamic API Key Management: An intuitive "Settings" modal allows users to save OpenAI and IBM keys locally in the browser (localStorage), removing the strict requirement of hard-coded server .env files.
  • Premium UI/UX Design: Features a dark-mode, glassmorphism aesthetic with animated gradient backgrounds, pulsing microphone animations, and clean message bubbles.
  • Real-Time Audio Playback: The assistant speaks back its responses using IBM Watson's Text-to-Speech synthesizer.
  • Robust Error Handling: Standardized backend and frontend error pipelines ensure graceful fallbacks when microphones fail, APIs are misconfigured, or network drops occur.

🏗️ Architecture Overview

The application follows a standard Client-Server architecture:

  1. Frontend (Client): A vanilla JavaScript application that manages UI state, records audio using the MediaRecorder API, manages localStorage for API keys, and orchestrates sequential fetch requests to the backend.
  2. Backend (Server): A Python Flask RESTful API that acts as a proxy bridge. It receives audio/text from the client, dynamically applies the API keys passed in the HTTP Headers, and routes the requests to the respective external AI providers (OpenAI or IBM).

🛠️ Technology Stack

  • Frontend: HTML5, CSS3 (Vanilla), JavaScript (ES6+). No heavy frameworks.
  • Backend Web Server: Python 3.x, Flask (werkzeug).
  • AI Models:
    • OpenAI: gpt-3.5-turbo via the official openai Python SDK.
    • IBM Watson: SpeechToTextV1 and TextToSpeechV1 via the ibm-watson Python SDK.
  • Environment Management: python-dotenv for optional server-wide fallbacks.

📋 Prerequisites

Before you begin, ensure you have the following accounts and credentials ready:

  1. OpenAI Account: An active API key (sk-...).
  2. IBM Cloud Account:
    • A Speech to Text (STT) service instance (API Key and URL).
    • A Text to Speech (TTS) service instance (API Key and URL).
  3. Python Environment: Python 3.8+ installed on your system.

🚀 Installation & Setup

  1. Navigate to the Project Directory Ensure you are in the Voice_Assistant root directory where app.py is located.

  2. Install Python Dependencies Run the following command to install Flask, OpenAI, and IBM Watson SDKs:

    pip install -r requirements.txt

  3. (Optional) Configure Server-Wide Environment Variables If you want to set global default keys, you can edit the generated .env file in the root directory:

    OPENAI_API_KEY=your_openai_api_key_here
IBM_STT_API_KEY=your_ibm_stt_api_key_here
IBM_STT_URL=your_ibm_stt_url_here
IBM_TTS_API_KEY=your_ibm_tts_api_key_here
IBM_TTS_URL=your_ibm_tts_url_here

    Note: Keys set in the UI will override these environment variables.

  4. Start the Flask Server

    python app.py

    The server will start on http://127.0.0.1:5000 with debug mode enabled.

⚙️ Configuration & Usage

  1. Open your web browser and navigate to http://127.0.0.1:5000.
  2. Configure UI Keys:
    • Click the Gear Icon in the top right corner.
    • Enter your OpenAI API Key, IBM STT Key & URL, and IBM TTS Key & URL.
    • Click Save Keys. These are now securely stored in your browser's local storage.
  3. Voice Interaction:
    • Click the circular microphone button.
    • Allow browser microphone permissions when prompted.
    • Speak your prompt, then click the microphone button again to stop.
    • The app will transcribe your audio, consult the AI, and play back the spoken answer!
  4. Text Interaction:
    • Type your prompt in the text bar and hit Enter or click the purple Send icon.

🌐 API Endpoints

The Flask backend exposes the following internal endpoints. All endpoints support dynamic key injection via HTTP Headers.

POST /transcribe

Converts uploaded WAV audio into text.

  • Headers Needed: X-IBM-STT-API-KEY, X-IBM-STT-URL
  • Payload: multipart/form-data containing an audio file.
  • Response: {"transcript": "Hello world"}

POST /ask

Sends a text prompt to OpenAI and returns the AI's response.

  • Headers Needed: X-OPENAI-API-KEY
  • Payload: application/json {"prompt": "Hello world"}
  • Response: {"answer": "Hi there! How can I help you?"}

POST /synthesize

Converts text into spoken audio.

  • Headers Needed: X-IBM-TTS-API-KEY, X-IBM-TTS-URL
  • Payload: application/json {"text": "Hi there! How can I help you?"}
  • Response: Binary audio stream (audio/wav).

🛡️ Error Handling Strategy

This application implements comprehensive error handling at every layer:

  • Missing Configurations: If keys are missing (both in the UI and .env), the frontend intercepts the action and forces the Settings modal open. If forced via direct API calls, the backend returns a clean HTTP 400 JSON response indicating the missing configuration.
  • External API Failures: If OpenAI or IBM servers are unreachable, or if an invalid API key is provided, the backend catches the Exception, logs it to the server console to protect stack traces, and returns a sanitized HTTP 500 JSON payload to the client.
  • Global Flask Handlers: Unmatched routes (404) and overarching internal errors (500) are captured by @app.errorhandler wrappers to prevent standard HTML error pages from breaking the JSON fetch pipelines.

🔮 Future Integrations & Improvements

Looking to contribute? Here are several ways to upgrade this project for market readiness:

  • Conversation Memory: Implement a session database (e.g., SQLite, PostgreSQL) to pass past chat context to OpenAI for continuous flowing conversations.
  • Real-Time Audio WebSockets: Replace standard HTTP POST with WebSockets/SSE to stream audio and minimize response latency.
  • Multilingual Support: Auto-detect language and configure IBM STT/TTS and OpenAI to respond in the user's native tongue.
  • User Authentication: Add OAuth or Firebase Auth to allow users to log in securely and sync their API keys/history across devices.
  • Voice Customization & Emotion: Integrate ElevenLabs API for realistic voices, or apply IBM Emotion Detection to adjust the AI's vocal tone dynamically.
  • Agentic Function Calling: Implement OpenAI's "Function Calling" to allow the assistant to trigger APIs (e.g., check weather, set alarms).
  • Scalable Deployment: Dockerize the application and migrate the backend to FastAPI for better async audio handling on cloud platforms like AWS.

🤝 Hire Me

Looking for a custom AI Voice Assistant, chatbot, or full-stack integration for your business?

I am currently open to job offers and freelance projects! If you'd like to hire me to build out customizations of this architecture or completely new AI-driven applications, please feel free to reach out via my GitHub profile or connect with me directly. Let's build something amazing together!

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages