A conversational language learning app powered by Inworld AI Runtime. Practice speaking with an AI tutor, get real-time feedback on your responses, and build vocabulary with auto-generated flashcards.

• Node.js (v20 or higher)
• npm
• An Inworld AI account and API key
• An AssemblyAI account and API key (for speech-to-text)

git clone https://github.com/inworld-ai/language-learning-node
cd language-learning-node

npm install

This installs dependencies for the root, backend, and frontend automatically.

Create a backend/.env file:

INWORLD_API_KEY=your_inworld_base64_key
ASSEMBLY_AI_API_KEY=your_assemblyai_key

For development (with auto-reload on file changes):

npm run dev

This starts both the backend (port 3000) and frontend dev server (port 5173) concurrently.

Open http://localhost:5173

For production:

npm run build
npm start

Open http://localhost:3000

Without Supabase, the app works in anonymous mode using localStorage.

a) Create a Supabase project at supabase.com

b) Push the database schema:

npx supabase login
npx supabase link --project-ref YOUR_PROJECT_REF
npx supabase db push

This creates all tables, indexes, RLS policies, and the match_memories function for semantic search.

Find your project ref in the Supabase dashboard URL: supabase.com/dashboard/project/YOUR_PROJECT_REF

c) Add Supabase variables to backend/.env:

SUPABASE_URL=https://YOUR_PROJECT.supabase.co
SUPABASE_SECRET_KEY=your_secret_key

d) Create frontend/.env.local:

VITE_SUPABASE_URL=https://YOUR_PROJECT.supabase.co
VITE_SUPABASE_PUBLISHABLE_KEY=your_anon_key

Find these in: Supabase Dashboard > Settings > API

language-learning-node/
├── backend/
│   ├── src/
│   │   ├── __tests__/        # Backend unit tests
│   │   ├── config/           # Language, LLM & server configuration
│   │   ├── graphs/           # Inworld Runtime conversation graphs
│   │   │   ├── configs/      # Graph JSON configurations
│   │   │   └── nodes/        # Custom graph nodes
│   │   ├── helpers/          # Audio utils, connection management
│   │   ├── prompts/          # Nunjucks prompt templates
│   │   ├── services/         # Server components (WS handler, API routes)
│   │   ├── types/            # TypeScript types
│   │   ├── utils/            # Logger
│   │   └── server.ts         # Entry point
│   └── vitest.config.ts      # Backend test config
├── frontend/
│   ├── src/
│   │   ├── __tests__/        # Frontend unit tests
│   │   ├── components/       # React components
│   │   ├── config/           # Language configuration
│   │   ├── context/          # App state & auth
│   │   ├── hooks/            # Custom React hooks
│   │   ├── services/         # WebSocket client, audio, storage
│   │   ├── styles/           # CSS
│   │   └── types/            # TypeScript types
│   └── vitest.config.ts      # Frontend test config
├── supabase/
│   └── migrations/           # Database schema
├── render.yaml               # Render deployment config
└── package.json              # Monorepo scripts

The app uses a real-time audio streaming architecture:

1. Frontend captures microphone audio and streams it via WebSocket
2. Backend processes audio through an Inworld Runtime graph:
   • AssemblyAI handles speech-to-text with voice activity detection
   • LLM generates contextual responses in the target language
   • TTS converts responses back to audio
3. Flashcards are auto-generated from conversation vocabulary
4. Response feedback provides grammar and usage corrections

When Supabase is configured, the app stores and retrieves user memories using semantic search:

• Automatic memory creation: Every few conversation turns, the system extracts memorable facts
• Semantic retrieval: Relevant memories are retrieved using vector similarity search (pgvector)
• Personalized responses: The AI uses retrieved memories to personalize conversations

Memory types:

• learning_progress: Vocabulary struggles, grammar patterns, learning achievements
• personal_context: Interests, goals, preferences shared by the user

Without Supabase, the app works in anonymous mode using localStorage (no memory persistence).

# Run all tests (backend + frontend)
npm test --prefix backend

# Backend tests only
npm run test:backend --prefix backend

# Frontend tests only
npm test --prefix frontend

# Watch mode (backend)
npm run test:watch --prefix backend

Tests cover critical paths: audio conversion, language configuration, storage persistence, and flashcard deduplication.

Bug Reports: GitHub Issues

General Questions: For general inquiries and support, please email us at [email protected]

We welcome contributions! Please see CONTRIBUTING.md for guidelines on how to contribute to this project.

This project is licensed under the MIT License - see the LICENSE file for details.