AI-Powered Home Assistant

Problem

Smart home systems require complex UIs or YAML automations. Non-technical users cannot easily control their homes with natural language, and unvalidated LLM output poses security risks when connected to physical systems.

Solution

Built a modular async pipeline: natural language input → GPT-4o with structured output constraints → JSON Schema validation → allow-list check (services + entities) → Home Assistant REST API execution. The LLM never directly controls hardware.

Architecture

• User input → Prompt engineering layer (system prompt + few-shot examples) → GPT-4o structured output
• JSON Schema validation → Allow-list check (services + entities) → Home Assistant REST API
• AsyncIO handles multiple concurrent commands without blocking
• Modular design separates LLM interaction, validation, execution, and error handling into independent components

Security & Reliability

• Allow-listed services — only pre-approved Home Assistant services can be invoked
• Entity validation — LLM output checked against known entity IDs before execution
• API key protection — secrets managed via environment variables, never in code or prompts
• JSON Schema validation — structured output validated before reaching execution layer
• Input sanitization — user input cleaned before prompt construction (prompt injection defense)
• Error isolation — failures in one command don't cascade to other operations
• Rate limiting — prevents runaway LLM calls from overwhelming the system

Results

• Converting natural language to validated smart-home actions with reliable execution
• Modular architecture enables easy addition of new services and entities
• Security-first design prevents unauthorized device control

Tech Stack

Artifacts