DoYouTrustMe – 3D AI Avatar Chatbot

Research prototype from a Bachelor’s thesis on trust in 3D AI avatars. The project is about a 3D avatar interface with a classical text interface for information retrieval in a university context. It consists of a SvelteKit client (3D rendering + evaluation UI) and a Python Flask server (STT, TTS, LLM proxy).

⚠️ Research prototype – no guarantees for correctness, stability, or production use. ⚠️ Backend server (local) available here: 👉 https://github.com/lechiffrebeats/3d-ai-avatars-server

🏗 Architecture (Short Overview)

• Web Client (SvelteKit + Three.js)

  • Renders the 3D avatar, plays audio, handles keyboard & microphone input.

• SvelteKit App Server

  • If you have a GPU or commercial api like Azure, Gcloud etc you can of course directly use the server.ts
  • Else if CPU & local it might be easier to go Python and use the server.ts as proxy for the 3d-ai-avatars-server

• Python Server (In this thesis was a Flaskapp on a Ubuntu VM)

  • Endpoints for /gwdg/chat, /stt, /tts, etc.
  • Whisper (STT), Piper (TTS), phoneme/viseme extraction.
  • Acts as a proxy to the GWDG LLM (meta-llama-3.1-8b-rag) or if you have cuda/gpu on do it locally https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct.

👀 Avatars (Licensing Notice)

The primary avatar models used in this project are paid assets purchased from CGTrader. Due to licensing restrictions, the original .glb/.fbx files cannot be included in this public repository as they are not allowed to be redistributed. Either use own models or contact 📩

🔧 Tech Stack

Frontend

• SvelteKit – https://kit.svelte.dev/
• Vercel (Client Hosting) – https://vercel.com/
• Cloudflare (Protection Layer) – https://www.cloudflare.com/
• Supabase (Evaluation Storage, DE Servers) – https://supabase.com/
• Three.js (3D Rendering) – https://threejs.org/

Backend

• Python / Flask – https://flask.palletsprojects.com/
• Nginx – https://nginx.org/
• Gunicorn – https://gunicorn.org/

AI / Speech / Processing

• ARCANA (HPC GWDG) – https://docs.hpc.gwdg.de/services/arcana/index.html
• Bournemouth Forced Aligner (BFA) – https://github.com/tabahi/bournemouth-forced-aligner
• Whisper (OpenAI) – https://platform.openai.com/research/whisper
• PiperTTS – https://github.com/OHF-Voice/piper1-gpl

For the full list of software usage and licensing, see:
/about?p=oss

⚙️ Environment Variables (Excerpt)

# VM / Python Server
SERVER_API_URL=       # where ever 3d-ai-avatars-server runs (for this work it was some vm of the university)
SERVER_API_KEY=       # create via RSA for example, this is to acces your 3d-ai-avatars-server

2️⃣ Start the SvelteKit App

npm install
npm run dev -- --open

Frontend runs at http://localhost:5173/, communicating with SERVER_API_URL.

📜 Licensing

• npm licenses: npx license-checker --json --out static/licenses/npm-licenses.json
• Python licenses: pip-licenses --format=json --output-file=pip-licenses.json