armistace/blog
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

AI CCTV system design and build

Browse Source
This commit is contained in:
Blog Creator 2026-02-03 01:07:48 +00:00
parent ae661993c1
commit 1ce3984028
1 changed files with 23 additions and 44 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

63
src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md
View File

@ -1,57 +1,36 @@
## Designing and Building an AI Enhanced CCTV System at Home
Designing and Building an AI Enhanced CCTV System at Home
Over the past six months, I’ve been immersed in a project that blends home automation with cutting-edge technology: building an AI-enhanced CCTV system for my homelab. The goal was to achieve locally controlled camera data, coupled with the benefits of AI-powered semantic search and intelligent notifications. This isn't about replacing existing security measures; it's about augmenting them with a layer of intelligent automation.
Over the past six months, I’ve been immersed in a project to build a sophisticated CCTV system for my homelab. The goal? Locally controlled camera data combined with the benefits of AI-powered semantic search and intelligent notifications. This isn't about creating a futuristic surveillance hub, but about building a genuinely useful, locally managed system capable of some impressive feats.
The core concept revolves around leveraging artificial intelligence to provide context and actionable insights from surveillance footage, all while maintaining complete control over your data. The possibilities are genuinely impressive, and the potential for future development is quite remarkable.
The system revolves around Frigate, an open-source Network Video Recorder (NVR) software. It offers an intuitive interface and is designed to be container-native, making deployment and management straightforward. Configuration is handled through YAML files, providing a structured approach to camera setup. This has allowed for rapid testing and refinement of different configurations, saving considerable time and frustration. While the complete configuration remains private due to ongoing refinements, the intention is to eventually share it publicly.
### The Foundation: Frigate and Containerization
The foundation of the system consists of three (with plans to expand to four) TP-Link ‘vigi’ C540 cameras, acquired during a Black Friday sale for a remarkably affordable AUD$50 each. These cameras offer pan and tilt functionality, onboard human detection, and follow mode, providing a solid base for surveillance.
At the heart of this system lies Frigate, an open-source Network Video Recorder (NVR) software. What sets Frigate apart is its intuitive interface and container-native design. This allows for easy configuration and repeatable builds, a significant advantage when experimenting with different camera setups and settings. The configuration is managed through a straightforward YAML file, simplifying the process and promoting consistency.
The true innovation lies in the integration of AI. Frigate connects to machine learning models, enabling object detection – identifying cars, motorbikes, and animals. This goes beyond simple detection; it leverages external AI models to analyze images and provide context. For example, a detected vehicle can be identified as a "silver sedan," and a description like "silver sedan approaching driveway" can be generated. This contextual information is then relayed to Home Assistant, a powerful open-source home automation platform.
While the complete configuration details remain private due to ongoing development and security considerations, my intention is to eventually share the setup publicly, allowing others to learn from and adapt the design.
Home Assistant acts as the central hub, integrating the CCTV system with other smart home devices. It receives object detections from Frigate, triggers notifications based on those detections, and even uses the information to control other devices. This integration is further enhanced by connecting Home Assistant to Ollama, allowing for the generation of metadata and the delivery of context-rich notifications via Matrix.
Exploring the extensive options within Frigate has provided a deeper understanding of how commercial NVR systems operate, offering valuable insights into their underlying architecture.
The architecture of the system is as follows:
### Hardware Selection: Affordable and Effective
The system currently utilizes three TP-Link 'vigi' C540 cameras, acquired during a Black Friday sale for a reasonable AUD$50 each. These cameras offer a compelling combination of features, including pan and tilt functionality and onboard human detection. While they lack zoom capabilities, they more than adequately meet the surveillance requirements for this project.
### AI Integration: Object Detection and Semantic Search
Frigate seamlessly integrates with the TP-Link cameras, providing the foundational NVR capabilities. It also offers APIs for integrating machine learning, enabling object detection for cars, motorbikes, and animals. This functionality is particularly useful for identifying potential hazards or unusual activity on your property.
The system’s intelligence is further enhanced through integration with Ollama, an AI service. This allows for the transmission of snapshots and detection data to a vision-capable machine learning model, which provides contextual information. For example, a simple text query like "Red SUV" or "White Ute" can quickly narrow down search results, providing specific details about detected vehicles. This capability significantly enhances the system's usefulness and efficiency.
### Storage and Scalability
To accommodate the continuous stream of video data, the system utilizes a dedicated server with an SSD for running Frigate and a 2TB hard drive for video storage. This configuration provides sufficient capacity for seven days of full history, 14 days of detections, and 30 days of alerts across the current four-camera setup.
### Home Assistant: Orchestrating Automation
The integration with Home Assistant elevates the system's capabilities to a new level. This allows object detections and other camera data to be leveraged for automated actions. For instance, Home Assistant, connected to Ollama and Matrix, can trigger notifications based on object detections and even request AI analysis to provide additional context before sending an alert. This results in a powerful, locally controlled, and AI-enhanced CCTV system.
### System Architecture
[mermaid]
```mermaid
graph TD
Camera --> FrigateObjectDetections
Frigate Object Detections --> Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement
Frigate NVR --> HomeAssistant
HomeAssistant --(Object Detection from Frigate)--> Copy Image
Copy Image --> Send image to Ollama (qwen3-vl-2b) for context enhancement
Send image to Ollama (qwen3-vl-2b) --> send response back via Matrix
FrigateObjectDetections --> SendSnapshot
SendSnapshot --> Ollama[qwen3-vl-4b for Semantic Search AI Enhancement]
FrigateObjectDetections --> HomeAssistant
HomeAssistant --> ObjectDetection
ObjectDetection --> CopyImage
CopyImage --> Ollama[qwen3-vl-2b for Context Enhancement]
Ollama --> Matrix
Camera --> FutureCamera
[/mermaid]
```
The entire system operates locally within a dedicated node on my Proxmox cluster, leveraging Docker containers for efficient resource management.
The system operates within a dedicated node on my Proxmox cluster, utilizing Docker LXC for efficient resource management.
### Future Enhancements and Ongoing Development
While the current system provides a solid foundation, several enhancements are planned. A key area of focus is refining the prompts used for semantic context enhancement and notifications. Currently, the system generates alerts for every detected person or vehicle, which can be overwhelming. Future development will incorporate AI-driven decision-making to filter out irrelevant alerts and prioritize those requiring immediate attention.
### Acknowledgements
Finally, I want to extend a sincere thank you to my father. His assistance with the physical installation of the network infrastructure was invaluable. His expertise and willingness to lend a hand were essential to the project's success.
Storage is handled with an SSD for Frigate’s operation and a 2TB HDD for video history and metadata. This setup provides sufficient capacity for seven days of full history, fourteen days of detections, and thirty days of alerts.
While the system is functional and delivers impressive results, ongoing refinements are planned. The current notification prompts are being refined to improve accuracy and reduce unnecessary alerts. The ultimate goal is to leverage AI to intelligently determine when notifications are warranted, rather than generating alerts for every detected object.
Finally, a sincere thank you is extended to my father, whose assistance was invaluable in running Ethernet cables through the roof for the wired camera setup. His contribution was essential to the project's success.
This project represents a blend of cutting-edge AI and practical hardware, resulting in a robust and personalized smart home security solution.