diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 7c83972..40e03fc 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -1,55 +1,57 @@ -Okay, here's the draft blog post, aiming for a professional tone, minimising code examples, and incorporating the provided information. I've tried to smooth out some of the more casual phrasing while retaining the enthusiasm. I've also included the Mermaid diagram as requested. +## Designing and Building an AI Enhanced CCTV System at Home ---- +Over the past six months, I’ve been immersed in a project that blends home automation with cutting-edge technology: building an AI-enhanced CCTV system for my homelab. The goal was to achieve locally controlled camera data, coupled with the benefits of AI-powered semantic search and intelligent notifications. This isn't about replacing existing security measures; it's about augmenting them with a layer of intelligent automation. -## Designing and Building an AI-Enhanced CCTV System at Home +The core concept revolves around leveraging artificial intelligence to provide context and actionable insights from surveillance footage, all while maintaining complete control over your data. The possibilities are genuinely impressive, and the potential for future development is quite remarkable. -For the past six months, I've been immersed in designing a home CCTV system for my homelab. The goal? Locally controlled, owned camera data, combined with the power of AI-enhanced semantic search and intelligent notifications. It's a project that started as a bit of a tinkering exercise, but the potential it unlocks is genuinely impressive. +### The Foundation: Frigate and Containerization -The core of this system revolves around Frigate ([https://frigate.video/](https://frigate.video/)), a remarkably intuitive NVR (Network Video Recorder) software. Frigate’s strength lies in its ease of use and its container-native architecture, allowing for repeatable builds managed through a simple YAML configuration file ([https://docs.frigate.video/configuration/reference](https://docs.frigate.video/configuration/reference)). This has been invaluable for testing different configurations and camera setups. While I haven't yet fully publicised the complete system configuration (still working on securing some credentials!), my intention is to share it openly so others can replicate the setup. +At the heart of this system lies Frigate, an open-source Network Video Recorder (NVR) software. What sets Frigate apart is its intuitive interface and container-native design. This allows for easy configuration and repeatable builds, a significant advantage when experimenting with different camera setups and settings. The configuration is managed through a straightforward YAML file, simplifying the process and promoting consistency. -Exploring Frigate’s extensive options has been a fascinating learning experience. I’ve been able to fine-tune camera connections, experiment with FFmpeg configurations, and even implement looping of internal and external streams to reduce the load on the cameras themselves. It’s provided a real insight into how commercial NVR and cloud-based systems operate. +While the complete configuration details remain private due to ongoing development and security considerations, my intention is to eventually share the setup publicly, allowing others to learn from and adapt the design. -My current setup utilises three (with plans to expand to four) TP-Link C540 cameras, acquired during a Black Friday sale for a steal at AUD$50 each ([https://www.amazon.com.au/gp/product/B0BJP6JJB3?smid=ANEGB3WVEVKZB&th=1](https://www.amazon.com.au/gp/product/B0BJP6JJB3?smid=ANEGB3WVEVKZB&th=1)). These cameras offer pan and tilt functionality, and even include onboard human detection and tracking, which is working exceptionally well. While they lack zoom capabilities, they perfectly suit my surveillance needs. +Exploring the extensive options within Frigate has provided a deeper understanding of how commercial NVR systems operate, offering valuable insights into their underlying architecture. -**The System Architecture** +### Hardware Selection: Affordable and Effective -```mermaid -graph LR - Camera --> FrigateObjectDetections - Camera --> FrigateNVR - FrigateObjectDetections --> OllamaAI[Ollama (qwen3-vl-4b) for semantic search AI enhancement] - FrigateNVR --> HomeAssistant - HomeAssistant --> OllamaAI2[Ollama (qwen3-vl-2b) for context enhancement] - HomeAssistant --> MatrixNotifications[Send response via Matrix] - style OllamaAI fill:#f9f,stroke:#333,stroke-width:2px - style OllamaAI2 fill:#f9f,stroke:#333,stroke-width:2px -``` +The system currently utilizes three TP-Link 'vigi' C540 cameras, acquired during a Black Friday sale for a reasonable AUD$50 each. These cameras offer a compelling combination of features, including pan and tilt functionality and onboard human detection. While they lack zoom capabilities, they more than adequately meet the surveillance requirements for this project. -**AI-Powered Intelligence** +### AI Integration: Object Detection and Semantic Search -Frigate’s integration with object detectors ([https://docs.frigate.video/configuration/object_detectors](https://docs.frigate.video/configuration/object_detectors)) is a game-changer. It provides basic NVR and vision capabilities, and can detect cars, motorbikes, and animals – a significant upgrade from standard CCTV. If your hardware includes a relatively recent GPU, this detection can be performed locally. Frigate relies on models from Hugging Face ([https://huggingface.co/](https://huggingface.co/)), requiring internet access for initial deployment and occasional model updates. +Frigate seamlessly integrates with the TP-Link cameras, providing the foundational NVR capabilities. It also offers APIs for integrating machine learning, enabling object detection for cars, motorbikes, and animals. This functionality is particularly useful for identifying potential hazards or unusual activity on your property. -A particularly exciting development is Frigate’s integration with Ollama. This allows me to send detections and alerts to a generative AI service, enabling powerful semantic search capabilities. For example, I can now search for "Red SUV" or "White Ute" within my camera footage and quickly narrow down results based on AI-provided context. This significantly enhances the utility of the system. +The system’s intelligence is further enhanced through integration with Ollama, an AI service. This allows for the transmission of snapshots and detection data to a vision-capable machine learning model, which provides contextual information. For example, a simple text query like "Red SUV" or "White Ute" can quickly narrow down search results, providing specific details about detected vehicles. This capability significantly enhances the system's usefulness and efficiency. -**Storage and Integration** +### Storage and Scalability -For storage, I’ve dedicated an SSD for the Frigate software and a 2TB HDD to store video data. This should provide approximately 7 days of full history, 14 days of detections, and 30 days of alerts across the four cameras. Initial results suggest this capacity is sufficient. +To accommodate the continuous stream of video data, the system utilizes a dedicated server with an SSD for running Frigate and a 2TB hard drive for video storage. This configuration provides sufficient capacity for seven days of full history, 14 days of detections, and 30 days of alerts across the current four-camera setup. -The system seamlessly integrates with Home Assistant ([https://docs.frigate.video/integrations/home-assistant/](https://docs.frigate.video/integrations/home-assistant/)). This allows object detections and other camera metadata to be leveraged within Home Assistant’s automation platform. Because I already use Home Assistant to connect to Ollama and Matrix, I can trigger notifications based on object detection, provide AI with context about the detected object type, and then request further analysis and context before sending the notification via Matrix to my phone. The result is a truly powerful, locally-owned, and AI-enhanced CCTV system. +### Home Assistant: Orchestrating Automation -**The Hardware Foundation** +The integration with Home Assistant elevates the system's capabilities to a new level. This allows object detections and other camera data to be leveraged for automated actions. For instance, Home Assistant, connected to Ollama and Matrix, can trigger notifications based on object detections and even request AI analysis to provide additional context before sending an alert. This results in a powerful, locally controlled, and AI-enhanced CCTV system. -All of this runs locally on a dedicated node within my Proxmox cluster ([https://blog.aridgwayweb.com/proxmox-cluster-1.html](https://blog.aridgwayweb.com/proxmox-cluster-1.html)), deployed using Docker LXC containers. This provides a robust and isolated environment for the system. +### System Architecture -**Looking Ahead** +[mermaid] +graph TD + Camera --> Frigate Object Detections + Frigate Object Detections --> Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement + Frigate NVR --> HomeAssistant + HomeAssistant --(Object Detection from Frigate)--> Copy Image + Copy Image --> Send image to Ollama (qwen3-vl-2b) for context enhancement + Send image to Ollama (qwen3-vl-2b) --> send response back via Matrix + Camera --> Future Camera +[/mermaid] -While the system is already delivering impressive results, there’s still room for improvement. The prompts used for both semantic context enhancement and notifications are currently a little rudimentary and require refinement. My long-term goal is to leverage AI to make decisions about *whether* to send a notification, as I currently receive alerts for every person or vehicle that passes by. However, even in its current state, the system provides valuable metadata enhancement for targeted semantic searches and AI-powered context for notifications. +The entire system operates locally within a dedicated node on my Proxmox cluster, leveraging Docker containers for efficient resource management. -Finally, a huge thank you to my Dad. As someone who isn't particularly adept at running ethernet cables through roofs, I relied heavily on his expertise to get the wired cameras installed. It was a significant undertaking, and his help was invaluable. I couldn't have built this system without his guidance and assistance. +### Future Enhancements and Ongoing Development + +While the current system provides a solid foundation, several enhancements are planned. A key area of focus is refining the prompts used for semantic context enhancement and notifications. Currently, the system generates alerts for every detected person or vehicle, which can be overwhelming. Future development will incorporate AI-driven decision-making to filter out irrelevant alerts and prioritize those requiring immediate attention. + +### Acknowledgements + +Finally, I want to extend a sincere thank you to my father. His assistance with the physical installation of the network infrastructure was invaluable. His expertise and willingness to lend a hand were essential to the project's success. ---- - -Let me know if you'd like any adjustments or further refinements! \ No newline at end of file