armistace/blog
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

designing_and_building_an_ai_enhanced_cctv_system_at_home #22

Merged
armistace merged 14 commits from designing_and_building_an_ai_enhanced_cctv_system_at_home into master 2026-02-03 13:24:16 +10:00
Conversation 0 Commits 14 Files Changed 1 +35 -33
1 changed files with 35 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit ae661993c1 - Show all commits

76
src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md
View File

@ -1,55 +1,57 @@
Okay, here's the draft blog post, aiming for a professional tone, minimising code examples, and incorporating the provided information. I've tried to smooth out some of the more casual phrasing while retaining the enthusiasm. I've also included the Mermaid diagram as requested.
## Designing and Building an AI Enhanced CCTV System at Home
---
Over the past six months, I’ve been immersed in a project that blends home automation with cutting-edge technology: building an AI-enhanced CCTV system for my homelab. The goal was to achieve locally controlled camera data, coupled with the benefits of AI-powered semantic search and intelligent notifications. This isn't about replacing existing security measures; it's about augmenting them with a layer of intelligent automation.
## Designing and Building an AI-Enhanced CCTV System at Home
The core concept revolves around leveraging artificial intelligence to provide context and actionable insights from surveillance footage, all while maintaining complete control over your data. The possibilities are genuinely impressive, and the potential for future development is quite remarkable.
For the past six months, I've been immersed in designing a home CCTV system for my homelab. The goal? Locally controlled, owned camera data, combined with the power of AI-enhanced semantic search and intelligent notifications. It's a project that started as a bit of a tinkering exercise, but the potential it unlocks is genuinely impressive.
### The Foundation: Frigate and Containerization
The core of this system revolves around Frigate ([https://frigate.video/](https://frigate.video/)), a remarkably intuitive NVR (Network Video Recorder) software. Frigate’s strength lies in its ease of use and its container-native architecture, allowing for repeatable builds managed through a simple YAML configuration file ([https://docs.frigate.video/configuration/reference](https://docs.frigate.video/configuration/reference)). This has been invaluable for testing different configurations and camera setups. While I haven't yet fully publicised the complete system configuration (still working on securing some credentials!), my intention is to share it openly so others can replicate the setup.
At the heart of this system lies Frigate, an open-source Network Video Recorder (NVR) software. What sets Frigate apart is its intuitive interface and container-native design. This allows for easy configuration and repeatable builds, a significant advantage when experimenting with different camera setups and settings. The configuration is managed through a straightforward YAML file, simplifying the process and promoting consistency.
Exploring Frigate’s extensive options has been a fascinating learning experience. I’ve been able to fine-tune camera connections, experiment with FFmpeg configurations, and even implement looping of internal and external streams to reduce the load on the cameras themselves. It’s provided a real insight into how commercial NVR and cloud-based systems operate.
While the complete configuration details remain private due to ongoing development and security considerations, my intention is to eventually share the setup publicly, allowing others to learn from and adapt the design.
My current setup utilises three (with plans to expand to four) TP-Link C540 cameras, acquired during a Black Friday sale for a steal at AUD$50 each ([https://www.amazon.com.au/gp/product/B0BJP6JJB3?smid=ANEGB3WVEVKZB&th=1](https://www.amazon.com.au/gp/product/B0BJP6JJB3?smid=ANEGB3WVEVKZB&th=1)). These cameras offer pan and tilt functionality, and even include onboard human detection and tracking, which is working exceptionally well. While they lack zoom capabilities, they perfectly suit my surveillance needs.
Exploring the extensive options within Frigate has provided a deeper understanding of how commercial NVR systems operate, offering valuable insights into their underlying architecture.
**The System Architecture**
### Hardware Selection: Affordable and Effective
```mermaid
graph LR
The system currently utilizes three TP-Link 'vigi' C540 cameras, acquired during a Black Friday sale for a reasonable AUD$50 each. These cameras offer a compelling combination of features, including pan and tilt functionality and onboard human detection. While they lack zoom capabilities, they more than adequately meet the surveillance requirements for this project.
### AI Integration: Object Detection and Semantic Search
Frigate seamlessly integrates with the TP-Link cameras, providing the foundational NVR capabilities. It also offers APIs for integrating machine learning, enabling object detection for cars, motorbikes, and animals. This functionality is particularly useful for identifying potential hazards or unusual activity on your property.
The system’s intelligence is further enhanced through integration with Ollama, an AI service. This allows for the transmission of snapshots and detection data to a vision-capable machine learning model, which provides contextual information. For example, a simple text query like "Red SUV" or "White Ute" can quickly narrow down search results, providing specific details about detected vehicles. This capability significantly enhances the system's usefulness and efficiency.
### Storage and Scalability
To accommodate the continuous stream of video data, the system utilizes a dedicated server with an SSD for running Frigate and a 2TB hard drive for video storage. This configuration provides sufficient capacity for seven days of full history, 14 days of detections, and 30 days of alerts across the current four-camera setup.
### Home Assistant: Orchestrating Automation
The integration with Home Assistant elevates the system's capabilities to a new level. This allows object detections and other camera data to be leveraged for automated actions. For instance, Home Assistant, connected to Ollama and Matrix, can trigger notifications based on object detections and even request AI analysis to provide additional context before sending an alert. This results in a powerful, locally controlled, and AI-enhanced CCTV system.
### System Architecture
[mermaid]
graph TD
Camera --> Frigate Object Detections
Camera --> FrigateNVR
FrigateObjectDetections --> OllamaAI[Ollama (qwen3-vl-4b) for semantic search AI enhancement]
Frigate Object Detections --> Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement
Frigate NVR --> HomeAssistant
HomeAssistant --> OllamaAI2[Ollama (qwen3-vl-2b) for context enhancement]
HomeAssistant --> MatrixNotifications[Send response via Matrix]
style OllamaAI fill:#f9f,stroke:#333,stroke-width:2px
style OllamaAI2 fill:#f9f,stroke:#333,stroke-width:2px
```
HomeAssistant --(Object Detection from Frigate)--> Copy Image
Copy Image --> Send image to Ollama (qwen3-vl-2b) for context enhancement
Send image to Ollama (qwen3-vl-2b) --> send response back via Matrix
Camera --> Future Camera
[/mermaid]
**AI-Powered Intelligence**
The entire system operates locally within a dedicated node on my Proxmox cluster, leveraging Docker containers for efficient resource management.
Frigate’s integration with object detectors ([https://docs.frigate.video/configuration/object_detectors](https://docs.frigate.video/configuration/object_detectors)) is a game-changer. It provides basic NVR and vision capabilities, and can detect cars, motorbikes, and animals – a significant upgrade from standard CCTV. If your hardware includes a relatively recent GPU, this detection can be performed locally. Frigate relies on models from Hugging Face ([https://huggingface.co/](https://huggingface.co/)), requiring internet access for initial deployment and occasional model updates.
### Future Enhancements and Ongoing Development
A particularly exciting development is Frigate’s integration with Ollama. This allows me to send detections and alerts to a generative AI service, enabling powerful semantic search capabilities. For example, I can now search for "Red SUV" or "White Ute" within my camera footage and quickly narrow down results based on AI-provided context. This significantly enhances the utility of the system.
While the current system provides a solid foundation, several enhancements are planned. A key area of focus is refining the prompts used for semantic context enhancement and notifications. Currently, the system generates alerts for every detected person or vehicle, which can be overwhelming. Future development will incorporate AI-driven decision-making to filter out irrelevant alerts and prioritize those requiring immediate attention.
**Storage and Integration**
### Acknowledgements
For storage, I’ve dedicated an SSD for the Frigate software and a 2TB HDD to store video data. This should provide approximately 7 days of full history, 14 days of detections, and 30 days of alerts across the four cameras. Initial results suggest this capacity is sufficient.
The system seamlessly integrates with Home Assistant ([https://docs.frigate.video/integrations/home-assistant/](https://docs.frigate.video/integrations/home-assistant/)). This allows object detections and other camera metadata to be leveraged within Home Assistant’s automation platform. Because I already use Home Assistant to connect to Ollama and Matrix, I can trigger notifications based on object detection, provide AI with context about the detected object type, and then request further analysis and context before sending the notification via Matrix to my phone. The result is a truly powerful, locally-owned, and AI-enhanced CCTV system.
**The Hardware Foundation**
All of this runs locally on a dedicated node within my Proxmox cluster ([https://blog.aridgwayweb.com/proxmox-cluster-1.html](https://blog.aridgwayweb.com/proxmox-cluster-1.html)), deployed using Docker LXC containers. This provides a robust and isolated environment for the system.
**Looking Ahead**
While the system is already delivering impressive results, there’s still room for improvement. The prompts used for both semantic context enhancement and notifications are currently a little rudimentary and require refinement. My long-term goal is to leverage AI to make decisions about *whether* to send a notification, as I currently receive alerts for every person or vehicle that passes by. However, even in its current state, the system provides valuable metadata enhancement for targeted semantic searches and AI-powered context for notifications.
Finally, a huge thank you to my Dad. As someone who isn't particularly adept at running ethernet cables through roofs, I relied heavily on his expertise to get the wired cameras installed. It was a significant undertaking, and his help was invaluable. I couldn't have built this system without his guidance and assistance.
Finally, I want to extend a sincere thank you to my father. His assistance with the physical installation of the network infrastructure was invaluable. His expertise and willingness to lend a hand were essential to the project's success.
---
Let me know if you'd like any adjustments or further refinements!