From b3d3e22d8e91984f97b2fb7b96ad9dc3b6d74250 Mon Sep 17 00:00:00 2001 From: Blog Creator Date: Mon, 2 Feb 2026 03:48:43 +0000 Subject: [PATCH 01/14] 'Add AI, Frigate, Home CCTV system' --- ...ding_an_ai_enhanced_cctv_system_at_home.md | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md new file mode 100644 index 0000000..7c83972 --- /dev/null +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -0,0 +1,55 @@ +Okay, here's the draft blog post, aiming for a professional tone, minimising code examples, and incorporating the provided information. I've tried to smooth out some of the more casual phrasing while retaining the enthusiasm. I've also included the Mermaid diagram as requested. + +--- + +## Designing and Building an AI-Enhanced CCTV System at Home + +For the past six months, I've been immersed in designing a home CCTV system for my homelab. The goal? Locally controlled, owned camera data, combined with the power of AI-enhanced semantic search and intelligent notifications. It's a project that started as a bit of a tinkering exercise, but the potential it unlocks is genuinely impressive. + +The core of this system revolves around Frigate ([https://frigate.video/](https://frigate.video/)), a remarkably intuitive NVR (Network Video Recorder) software. Frigate’s strength lies in its ease of use and its container-native architecture, allowing for repeatable builds managed through a simple YAML configuration file ([https://docs.frigate.video/configuration/reference](https://docs.frigate.video/configuration/reference)). This has been invaluable for testing different configurations and camera setups. While I haven't yet fully publicised the complete system configuration (still working on securing some credentials!), my intention is to share it openly so others can replicate the setup. + +Exploring Frigate’s extensive options has been a fascinating learning experience. I’ve been able to fine-tune camera connections, experiment with FFmpeg configurations, and even implement looping of internal and external streams to reduce the load on the cameras themselves. It’s provided a real insight into how commercial NVR and cloud-based systems operate. + +My current setup utilises three (with plans to expand to four) TP-Link C540 cameras, acquired during a Black Friday sale for a steal at AUD$50 each ([https://www.amazon.com.au/gp/product/B0BJP6JJB3?smid=ANEGB3WVEVKZB&th=1](https://www.amazon.com.au/gp/product/B0BJP6JJB3?smid=ANEGB3WVEVKZB&th=1)). These cameras offer pan and tilt functionality, and even include onboard human detection and tracking, which is working exceptionally well. While they lack zoom capabilities, they perfectly suit my surveillance needs. + +**The System Architecture** + +```mermaid +graph LR + Camera --> FrigateObjectDetections + Camera --> FrigateNVR + FrigateObjectDetections --> OllamaAI[Ollama (qwen3-vl-4b) for semantic search AI enhancement] + FrigateNVR --> HomeAssistant + HomeAssistant --> OllamaAI2[Ollama (qwen3-vl-2b) for context enhancement] + HomeAssistant --> MatrixNotifications[Send response via Matrix] + style OllamaAI fill:#f9f,stroke:#333,stroke-width:2px + style OllamaAI2 fill:#f9f,stroke:#333,stroke-width:2px +``` + +**AI-Powered Intelligence** + +Frigate’s integration with object detectors ([https://docs.frigate.video/configuration/object_detectors](https://docs.frigate.video/configuration/object_detectors)) is a game-changer. It provides basic NVR and vision capabilities, and can detect cars, motorbikes, and animals – a significant upgrade from standard CCTV. If your hardware includes a relatively recent GPU, this detection can be performed locally. Frigate relies on models from Hugging Face ([https://huggingface.co/](https://huggingface.co/)), requiring internet access for initial deployment and occasional model updates. + +A particularly exciting development is Frigate’s integration with Ollama. This allows me to send detections and alerts to a generative AI service, enabling powerful semantic search capabilities. For example, I can now search for "Red SUV" or "White Ute" within my camera footage and quickly narrow down results based on AI-provided context. This significantly enhances the utility of the system. + +**Storage and Integration** + +For storage, I’ve dedicated an SSD for the Frigate software and a 2TB HDD to store video data. This should provide approximately 7 days of full history, 14 days of detections, and 30 days of alerts across the four cameras. Initial results suggest this capacity is sufficient. + +The system seamlessly integrates with Home Assistant ([https://docs.frigate.video/integrations/home-assistant/](https://docs.frigate.video/integrations/home-assistant/)). This allows object detections and other camera metadata to be leveraged within Home Assistant’s automation platform. Because I already use Home Assistant to connect to Ollama and Matrix, I can trigger notifications based on object detection, provide AI with context about the detected object type, and then request further analysis and context before sending the notification via Matrix to my phone. The result is a truly powerful, locally-owned, and AI-enhanced CCTV system. + +**The Hardware Foundation** + +All of this runs locally on a dedicated node within my Proxmox cluster ([https://blog.aridgwayweb.com/proxmox-cluster-1.html](https://blog.aridgwayweb.com/proxmox-cluster-1.html)), deployed using Docker LXC containers. This provides a robust and isolated environment for the system. + +**Looking Ahead** + +While the system is already delivering impressive results, there’s still room for improvement. The prompts used for both semantic context enhancement and notifications are currently a little rudimentary and require refinement. My long-term goal is to leverage AI to make decisions about *whether* to send a notification, as I currently receive alerts for every person or vehicle that passes by. However, even in its current state, the system provides valuable metadata enhancement for targeted semantic searches and AI-powered context for notifications. + +Finally, a huge thank you to my Dad. As someone who isn't particularly adept at running ethernet cables through roofs, I relied heavily on his expertise to get the wired cameras installed. It was a significant undertaking, and his help was invaluable. I couldn't have built this system without his guidance and assistance. + + + +--- + +Let me know if you'd like any adjustments or further refinements! \ No newline at end of file From ae661993c1c0eebe372c54f559230bea7148007b Mon Sep 17 00:00:00 2001 From: Blog Creator Date: Tue, 3 Feb 2026 00:52:09 +0000 Subject: [PATCH 02/14] AI-powered CCTV system design complete. --- ...ding_an_ai_enhanced_cctv_system_at_home.md | 68 ++++++++++--------- 1 file changed, 35 insertions(+), 33 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 7c83972..40e03fc 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -1,55 +1,57 @@ -Okay, here's the draft blog post, aiming for a professional tone, minimising code examples, and incorporating the provided information. I've tried to smooth out some of the more casual phrasing while retaining the enthusiasm. I've also included the Mermaid diagram as requested. +## Designing and Building an AI Enhanced CCTV System at Home ---- +Over the past six months, I’ve been immersed in a project that blends home automation with cutting-edge technology: building an AI-enhanced CCTV system for my homelab. The goal was to achieve locally controlled camera data, coupled with the benefits of AI-powered semantic search and intelligent notifications. This isn't about replacing existing security measures; it's about augmenting them with a layer of intelligent automation. -## Designing and Building an AI-Enhanced CCTV System at Home +The core concept revolves around leveraging artificial intelligence to provide context and actionable insights from surveillance footage, all while maintaining complete control over your data. The possibilities are genuinely impressive, and the potential for future development is quite remarkable. -For the past six months, I've been immersed in designing a home CCTV system for my homelab. The goal? Locally controlled, owned camera data, combined with the power of AI-enhanced semantic search and intelligent notifications. It's a project that started as a bit of a tinkering exercise, but the potential it unlocks is genuinely impressive. +### The Foundation: Frigate and Containerization -The core of this system revolves around Frigate ([https://frigate.video/](https://frigate.video/)), a remarkably intuitive NVR (Network Video Recorder) software. Frigate’s strength lies in its ease of use and its container-native architecture, allowing for repeatable builds managed through a simple YAML configuration file ([https://docs.frigate.video/configuration/reference](https://docs.frigate.video/configuration/reference)). This has been invaluable for testing different configurations and camera setups. While I haven't yet fully publicised the complete system configuration (still working on securing some credentials!), my intention is to share it openly so others can replicate the setup. +At the heart of this system lies Frigate, an open-source Network Video Recorder (NVR) software. What sets Frigate apart is its intuitive interface and container-native design. This allows for easy configuration and repeatable builds, a significant advantage when experimenting with different camera setups and settings. The configuration is managed through a straightforward YAML file, simplifying the process and promoting consistency. -Exploring Frigate’s extensive options has been a fascinating learning experience. I’ve been able to fine-tune camera connections, experiment with FFmpeg configurations, and even implement looping of internal and external streams to reduce the load on the cameras themselves. It’s provided a real insight into how commercial NVR and cloud-based systems operate. +While the complete configuration details remain private due to ongoing development and security considerations, my intention is to eventually share the setup publicly, allowing others to learn from and adapt the design. -My current setup utilises three (with plans to expand to four) TP-Link C540 cameras, acquired during a Black Friday sale for a steal at AUD$50 each ([https://www.amazon.com.au/gp/product/B0BJP6JJB3?smid=ANEGB3WVEVKZB&th=1](https://www.amazon.com.au/gp/product/B0BJP6JJB3?smid=ANEGB3WVEVKZB&th=1)). These cameras offer pan and tilt functionality, and even include onboard human detection and tracking, which is working exceptionally well. While they lack zoom capabilities, they perfectly suit my surveillance needs. +Exploring the extensive options within Frigate has provided a deeper understanding of how commercial NVR systems operate, offering valuable insights into their underlying architecture. -**The System Architecture** +### Hardware Selection: Affordable and Effective -```mermaid -graph LR - Camera --> FrigateObjectDetections - Camera --> FrigateNVR - FrigateObjectDetections --> OllamaAI[Ollama (qwen3-vl-4b) for semantic search AI enhancement] - FrigateNVR --> HomeAssistant - HomeAssistant --> OllamaAI2[Ollama (qwen3-vl-2b) for context enhancement] - HomeAssistant --> MatrixNotifications[Send response via Matrix] - style OllamaAI fill:#f9f,stroke:#333,stroke-width:2px - style OllamaAI2 fill:#f9f,stroke:#333,stroke-width:2px -``` +The system currently utilizes three TP-Link 'vigi' C540 cameras, acquired during a Black Friday sale for a reasonable AUD$50 each. These cameras offer a compelling combination of features, including pan and tilt functionality and onboard human detection. While they lack zoom capabilities, they more than adequately meet the surveillance requirements for this project. -**AI-Powered Intelligence** +### AI Integration: Object Detection and Semantic Search -Frigate’s integration with object detectors ([https://docs.frigate.video/configuration/object_detectors](https://docs.frigate.video/configuration/object_detectors)) is a game-changer. It provides basic NVR and vision capabilities, and can detect cars, motorbikes, and animals – a significant upgrade from standard CCTV. If your hardware includes a relatively recent GPU, this detection can be performed locally. Frigate relies on models from Hugging Face ([https://huggingface.co/](https://huggingface.co/)), requiring internet access for initial deployment and occasional model updates. +Frigate seamlessly integrates with the TP-Link cameras, providing the foundational NVR capabilities. It also offers APIs for integrating machine learning, enabling object detection for cars, motorbikes, and animals. This functionality is particularly useful for identifying potential hazards or unusual activity on your property. -A particularly exciting development is Frigate’s integration with Ollama. This allows me to send detections and alerts to a generative AI service, enabling powerful semantic search capabilities. For example, I can now search for "Red SUV" or "White Ute" within my camera footage and quickly narrow down results based on AI-provided context. This significantly enhances the utility of the system. +The system’s intelligence is further enhanced through integration with Ollama, an AI service. This allows for the transmission of snapshots and detection data to a vision-capable machine learning model, which provides contextual information. For example, a simple text query like "Red SUV" or "White Ute" can quickly narrow down search results, providing specific details about detected vehicles. This capability significantly enhances the system's usefulness and efficiency. -**Storage and Integration** +### Storage and Scalability -For storage, I’ve dedicated an SSD for the Frigate software and a 2TB HDD to store video data. This should provide approximately 7 days of full history, 14 days of detections, and 30 days of alerts across the four cameras. Initial results suggest this capacity is sufficient. +To accommodate the continuous stream of video data, the system utilizes a dedicated server with an SSD for running Frigate and a 2TB hard drive for video storage. This configuration provides sufficient capacity for seven days of full history, 14 days of detections, and 30 days of alerts across the current four-camera setup. -The system seamlessly integrates with Home Assistant ([https://docs.frigate.video/integrations/home-assistant/](https://docs.frigate.video/integrations/home-assistant/)). This allows object detections and other camera metadata to be leveraged within Home Assistant’s automation platform. Because I already use Home Assistant to connect to Ollama and Matrix, I can trigger notifications based on object detection, provide AI with context about the detected object type, and then request further analysis and context before sending the notification via Matrix to my phone. The result is a truly powerful, locally-owned, and AI-enhanced CCTV system. +### Home Assistant: Orchestrating Automation -**The Hardware Foundation** +The integration with Home Assistant elevates the system's capabilities to a new level. This allows object detections and other camera data to be leveraged for automated actions. For instance, Home Assistant, connected to Ollama and Matrix, can trigger notifications based on object detections and even request AI analysis to provide additional context before sending an alert. This results in a powerful, locally controlled, and AI-enhanced CCTV system. -All of this runs locally on a dedicated node within my Proxmox cluster ([https://blog.aridgwayweb.com/proxmox-cluster-1.html](https://blog.aridgwayweb.com/proxmox-cluster-1.html)), deployed using Docker LXC containers. This provides a robust and isolated environment for the system. +### System Architecture -**Looking Ahead** +[mermaid] +graph TD + Camera --> Frigate Object Detections + Frigate Object Detections --> Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement + Frigate NVR --> HomeAssistant + HomeAssistant --(Object Detection from Frigate)--> Copy Image + Copy Image --> Send image to Ollama (qwen3-vl-2b) for context enhancement + Send image to Ollama (qwen3-vl-2b) --> send response back via Matrix + Camera --> Future Camera +[/mermaid] -While the system is already delivering impressive results, there’s still room for improvement. The prompts used for both semantic context enhancement and notifications are currently a little rudimentary and require refinement. My long-term goal is to leverage AI to make decisions about *whether* to send a notification, as I currently receive alerts for every person or vehicle that passes by. However, even in its current state, the system provides valuable metadata enhancement for targeted semantic searches and AI-powered context for notifications. +The entire system operates locally within a dedicated node on my Proxmox cluster, leveraging Docker containers for efficient resource management. -Finally, a huge thank you to my Dad. As someone who isn't particularly adept at running ethernet cables through roofs, I relied heavily on his expertise to get the wired cameras installed. It was a significant undertaking, and his help was invaluable. I couldn't have built this system without his guidance and assistance. +### Future Enhancements and Ongoing Development + +While the current system provides a solid foundation, several enhancements are planned. A key area of focus is refining the prompts used for semantic context enhancement and notifications. Currently, the system generates alerts for every detected person or vehicle, which can be overwhelming. Future development will incorporate AI-driven decision-making to filter out irrelevant alerts and prioritize those requiring immediate attention. + +### Acknowledgements + +Finally, I want to extend a sincere thank you to my father. His assistance with the physical installation of the network infrastructure was invaluable. His expertise and willingness to lend a hand were essential to the project's success. ---- - -Let me know if you'd like any adjustments or further refinements! \ No newline at end of file From 1ce398402840df8902937a3ac609a03414e2545d Mon Sep 17 00:00:00 2001 From: Blog Creator Date: Tue, 3 Feb 2026 01:07:48 +0000 Subject: [PATCH 03/14] AI CCTV system design and build --- ...ding_an_ai_enhanced_cctv_system_at_home.md | 67 +++++++------------ 1 file changed, 23 insertions(+), 44 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 40e03fc..c10d8dc 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -1,57 +1,36 @@ -## Designing and Building an AI Enhanced CCTV System at Home +Designing and Building an AI Enhanced CCTV System at Home -Over the past six months, I’ve been immersed in a project that blends home automation with cutting-edge technology: building an AI-enhanced CCTV system for my homelab. The goal was to achieve locally controlled camera data, coupled with the benefits of AI-powered semantic search and intelligent notifications. This isn't about replacing existing security measures; it's about augmenting them with a layer of intelligent automation. +Over the past six months, I’ve been immersed in a project to build a sophisticated CCTV system for my homelab. The goal? Locally controlled camera data combined with the benefits of AI-powered semantic search and intelligent notifications. This isn't about creating a futuristic surveillance hub, but about building a genuinely useful, locally managed system capable of some impressive feats. -The core concept revolves around leveraging artificial intelligence to provide context and actionable insights from surveillance footage, all while maintaining complete control over your data. The possibilities are genuinely impressive, and the potential for future development is quite remarkable. +The system revolves around Frigate, an open-source Network Video Recorder (NVR) software. It offers an intuitive interface and is designed to be container-native, making deployment and management straightforward. Configuration is handled through YAML files, providing a structured approach to camera setup. This has allowed for rapid testing and refinement of different configurations, saving considerable time and frustration. While the complete configuration remains private due to ongoing refinements, the intention is to eventually share it publicly. -### The Foundation: Frigate and Containerization +The foundation of the system consists of three (with plans to expand to four) TP-Link ‘vigi’ C540 cameras, acquired during a Black Friday sale for a remarkably affordable AUD$50 each. These cameras offer pan and tilt functionality, onboard human detection, and follow mode, providing a solid base for surveillance. -At the heart of this system lies Frigate, an open-source Network Video Recorder (NVR) software. What sets Frigate apart is its intuitive interface and container-native design. This allows for easy configuration and repeatable builds, a significant advantage when experimenting with different camera setups and settings. The configuration is managed through a straightforward YAML file, simplifying the process and promoting consistency. +The true innovation lies in the integration of AI. Frigate connects to machine learning models, enabling object detection – identifying cars, motorbikes, and animals. This goes beyond simple detection; it leverages external AI models to analyze images and provide context. For example, a detected vehicle can be identified as a "silver sedan," and a description like "silver sedan approaching driveway" can be generated. This contextual information is then relayed to Home Assistant, a powerful open-source home automation platform. -While the complete configuration details remain private due to ongoing development and security considerations, my intention is to eventually share the setup publicly, allowing others to learn from and adapt the design. +Home Assistant acts as the central hub, integrating the CCTV system with other smart home devices. It receives object detections from Frigate, triggers notifications based on those detections, and even uses the information to control other devices. This integration is further enhanced by connecting Home Assistant to Ollama, allowing for the generation of metadata and the delivery of context-rich notifications via Matrix. -Exploring the extensive options within Frigate has provided a deeper understanding of how commercial NVR systems operate, offering valuable insights into their underlying architecture. +The architecture of the system is as follows: -### Hardware Selection: Affordable and Effective - -The system currently utilizes three TP-Link 'vigi' C540 cameras, acquired during a Black Friday sale for a reasonable AUD$50 each. These cameras offer a compelling combination of features, including pan and tilt functionality and onboard human detection. While they lack zoom capabilities, they more than adequately meet the surveillance requirements for this project. - -### AI Integration: Object Detection and Semantic Search - -Frigate seamlessly integrates with the TP-Link cameras, providing the foundational NVR capabilities. It also offers APIs for integrating machine learning, enabling object detection for cars, motorbikes, and animals. This functionality is particularly useful for identifying potential hazards or unusual activity on your property. - -The system’s intelligence is further enhanced through integration with Ollama, an AI service. This allows for the transmission of snapshots and detection data to a vision-capable machine learning model, which provides contextual information. For example, a simple text query like "Red SUV" or "White Ute" can quickly narrow down search results, providing specific details about detected vehicles. This capability significantly enhances the system's usefulness and efficiency. - -### Storage and Scalability - -To accommodate the continuous stream of video data, the system utilizes a dedicated server with an SSD for running Frigate and a 2TB hard drive for video storage. This configuration provides sufficient capacity for seven days of full history, 14 days of detections, and 30 days of alerts across the current four-camera setup. - -### Home Assistant: Orchestrating Automation - -The integration with Home Assistant elevates the system's capabilities to a new level. This allows object detections and other camera data to be leveraged for automated actions. For instance, Home Assistant, connected to Ollama and Matrix, can trigger notifications based on object detections and even request AI analysis to provide additional context before sending an alert. This results in a powerful, locally controlled, and AI-enhanced CCTV system. - -### System Architecture - -[mermaid] +```mermaid graph TD - Camera --> Frigate Object Detections - Frigate Object Detections --> Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement - Frigate NVR --> HomeAssistant - HomeAssistant --(Object Detection from Frigate)--> Copy Image - Copy Image --> Send image to Ollama (qwen3-vl-2b) for context enhancement - Send image to Ollama (qwen3-vl-2b) --> send response back via Matrix - Camera --> Future Camera -[/mermaid] +Camera --> FrigateObjectDetections +FrigateObjectDetections --> SendSnapshot +SendSnapshot --> Ollama[qwen3-vl-4b for Semantic Search AI Enhancement] +FrigateObjectDetections --> HomeAssistant +HomeAssistant --> ObjectDetection +ObjectDetection --> CopyImage +CopyImage --> Ollama[qwen3-vl-2b for Context Enhancement] +Ollama --> Matrix +Camera --> FutureCamera +``` -The entire system operates locally within a dedicated node on my Proxmox cluster, leveraging Docker containers for efficient resource management. +The system operates within a dedicated node on my Proxmox cluster, utilizing Docker LXC for efficient resource management. -### Future Enhancements and Ongoing Development - -While the current system provides a solid foundation, several enhancements are planned. A key area of focus is refining the prompts used for semantic context enhancement and notifications. Currently, the system generates alerts for every detected person or vehicle, which can be overwhelming. Future development will incorporate AI-driven decision-making to filter out irrelevant alerts and prioritize those requiring immediate attention. - -### Acknowledgements - -Finally, I want to extend a sincere thank you to my father. His assistance with the physical installation of the network infrastructure was invaluable. His expertise and willingness to lend a hand were essential to the project's success. +Storage is handled with an SSD for Frigate’s operation and a 2TB HDD for video history and metadata. This setup provides sufficient capacity for seven days of full history, fourteen days of detections, and thirty days of alerts. +While the system is functional and delivers impressive results, ongoing refinements are planned. The current notification prompts are being refined to improve accuracy and reduce unnecessary alerts. The ultimate goal is to leverage AI to intelligently determine when notifications are warranted, rather than generating alerts for every detected object. +Finally, a sincere thank you is extended to my father, whose assistance was invaluable in running Ethernet cables through the roof for the wired camera setup. His contribution was essential to the project's success. +This project represents a blend of cutting-edge AI and practical hardware, resulting in a robust and personalized smart home security solution. \ No newline at end of file From f44a86eb6d58e968aaab8541f7e95d639d977f61 Mon Sep 17 00:00:00 2001 From: Blog Creator Date: Tue, 3 Feb 2026 01:09:13 +0000 Subject: [PATCH 04/14] 'Add AI-enhanced local CCTV system' --- ...ding_an_ai_enhanced_cctv_system_at_home.md | 172 +++++++++++++++--- 1 file changed, 150 insertions(+), 22 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index c10d8dc..1476bda 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -1,36 +1,164 @@ -Designing and Building an AI Enhanced CCTV System at Home +Over the past six months I’ve been tinkering with a problem that most homeowners shrug off as “just get a camera and be done with it.” The reality is that a plain‑old CCTV feed gives you a wall of motion‑triggered video that you have to trawl through manually, and the moment you start looking for anything useful you quickly realise the data is as opaque as a foggy morning on the Nullarbor. My goal was simple on paper: a locally‑hosted surveillance system that keeps every frame under my own roof, yet still offers the kind of intelligent search and notification you’d expect from a cloud‑based service. The result is an AI‑enhanced CCTV stack built around the open‑source NVR called **Frigate**, a handful of budget‑friendly TP‑Link Vigi C540 cameras, and a modest homelab that runs everything in Docker containers on a Proxmox LXC node. -Over the past six months, I’ve been immersed in a project to build a sophisticated CCTV system for my homelab. The goal? Locally controlled camera data combined with the benefits of AI-powered semantic search and intelligent notifications. This isn't about creating a futuristic surveillance hub, but about building a genuinely useful, locally managed system capable of some impressive feats. +--- -The system revolves around Frigate, an open-source Network Video Recorder (NVR) software. It offers an intuitive interface and is designed to be container-native, making deployment and management straightforward. Configuration is handled through YAML files, providing a structured approach to camera setup. This has allowed for rapid testing and refinement of different configurations, saving considerable time and frustration. While the complete configuration remains private due to ongoing refinements, the intention is to eventually share it publicly. +## Why “Local‑First” Matters -The foundation of the system consists of three (with plans to expand to four) TP-Link ‘vigi’ C540 cameras, acquired during a Black Friday sale for a remarkably affordable AUD$50 each. These cameras offer pan and tilt functionality, onboard human detection, and follow mode, providing a solid base for surveillance. +When you hand over video streams to a commercial provider you’re essentially giving a stranger a front‑row seat to every comings and goings at your doorstep. The trade‑off is usually a subscription fee and the promise of “smart” alerts. In practice those alerts are either too noisy (the neighbour’s cat triggers a “person detected” every night) or too vague (a generic “motion detected” that forces you to open the app and stare at a blank screen). Moreover, the data lives somewhere you can’t audit, and any breach could expose a detailed visual diary of your life. -The true innovation lies in the integration of AI. Frigate connects to machine learning models, enabling object detection – identifying cars, motorbikes, and animals. This goes beyond simple detection; it leverages external AI models to analyze images and provide context. For example, a detected vehicle can be identified as a "silver sedan," and a description like "silver sedan approaching driveway" can be generated. This contextual information is then relayed to Home Assistant, a powerful open-source home automation platform. +A local‑first approach eliminates the subscription, guarantees that the raw footage never leaves your network, and gives you full control over how the data is processed. The only thing you have to trust is the hardware you own and the open‑source software you run. That level of ownership is what makes the project worth the effort. -Home Assistant acts as the central hub, integrating the CCTV system with other smart home devices. It receives object detections from Frigate, triggers notifications based on those detections, and even uses the information to control other devices. This integration is further enhanced by connecting Home Assistant to Ollama, allowing for the generation of metadata and the delivery of context-rich notifications via Matrix. +--- -The architecture of the system is as follows: +## The Core: Frigate NVR + +Frigate is a container‑native Network Video Recorder that does far more than simply write video to disk. It watches each RTSP stream, runs object detection on every frame (or on a configurable subset), and exposes a clean REST API plus a web UI for live view, playback, and search. The entire configuration lives in a single YAML file, which makes it trivial to version‑control, replicate, or spin up a fresh instance for testing. + +Key points that made Frigate the obvious choice: + +| Feature | Why it matters | +|---------|----------------| +| **Docker‑ready** | Deploys in seconds on any machine that can run containers. | +| **YAML‑driven** | Human‑readable, repeatable, and easy to tweak via CI pipelines. | +| **Built‑in object detectors** | Supports TensorRT, Coral, and CPU‑only models; can pull the latest weights from Hugging Face. | +| **GenAI integration** | Direct hooks to Ollama for custom metadata generation. | +| **Home Assistant integration** | Exposes events that can be consumed by any automation platform. | + +Because the configuration is declarative, I could spin up a fresh test environment, point it at a different camera, tweak the FFmpeg pipeline, and have the whole stack ready in under five minutes. That rapid iteration cycle was essential while I was still learning the quirks of each camera and the best way to balance CPU load against detection accuracy. + +--- + +## The Eyes: TP‑Link Vigi C540 Cameras + +I chose three (soon to be four) TP‑Link Vigi C540 units because they hit the sweet spot between price and capability. At AUD 50 each they were a steal during a Black Friday sale, and they come with pan‑and‑tilt motors, on‑board human detection, and a follow‑mode that keeps a moving subject in frame. The cameras lack optical zoom, but the field of view is wide enough for a typical suburban front yard, and the built‑in motion detection reduces the amount of data that needs to be processed by Frigate. + +| Spec | Reason for selection | +|------|-----------------------| +| **Pan/Tilt** | Allows a single unit to cover multiple angles without extra hardware. | +| **Wired Ethernet** | Guarantees a stable feed and eliminates Wi‑Fi interference. | +| **On‑board human detection** | Provides a first line of filtering before Frigate even sees the frame. | +| **Affordability** | Keeps the overall project budget modest, leaving room for a GPU upgrade later. | + +The cameras are mounted on the roof, with Ethernet cables run through the attic. I’ll come back to the physical installation later, but the result is a reliable, low‑latency video source that feeds directly into the Frigate container. + +--- + +## The Brain: GPU‑Accelerated Object Detection + +Frigate can offload inference to a GPU, a Coral Edge TPU, or even run on the CPU if you’re willing to accept slower detection rates. My homelab node is a modest Intel N100 mini‑PC equipped with a mid‑range NVIDIA RTX 3060. With the RTX in place, Frigate pulls the latest YOLO‑v8 models from Hugging Face at startup and runs detection at roughly 10 fps per camera, which is more than sufficient for a residential setting. + +Because the models are downloaded from the internet, the host needs occasional outbound connectivity to refresh weights. This is a small price to pay for the ability to keep the detection pipeline up‑to‑date without manual intervention. + +--- + +## Adding Smarts: Ollama and GenAI + +Frigate’s native object detection tells you *what* it sees – a person, a car, a dog – but it can’t answer “what colour is the car?” or “is that a delivery driver or a neighbour?” That’s where Ollama comes in. Ollama is a locally hosted LLM/vision service that can run multimodal models such as **qwen3‑vl‑4b** and **qwen3‑vl‑2b**. By sending a snapshot to Ollama with a carefully crafted prompt, the model returns a natural‑language description that adds semantic depth to the raw detection. + +Two distinct flows exist: + +1. **Semantic Search Enrichment** – When a snapshot is sent to the 4‑billion‑parameter model, the response is stored as metadata alongside the detection. This lets me type “Red SUV” into Frigate’s search bar and instantly filter to the relevant clips. +2. **Contextual Notification** – A lighter 2‑billion‑parameter model receives the same snapshot but with a prompt that asks for a concise, security‑focused summary. The result is attached to a Matrix message that lands on my phone, turning a generic “person detected” alert into “Delivery person in hi‑vis vest placing a parcel on the porch.” + +Because both models run locally, there is no latency penalty and no data leaves the house. The only external dependency is the occasional model update from Hugging Face. + +--- + +## Storage Strategy + +Video footage is storage‑hungry, especially when you keep full‑resolution streams for a week or more. My node uses a two‑tier storage layout: + +* **SSD (256 GB)** – Hosts the OS, Docker engine, and Frigate’s database. The SSD ensures fast metadata reads/writes and quick container restarts. +* **2 TB HDD** – Dedicated to raw video files. With three cameras recording at 1080p/15 fps, the drive comfortably holds: + * 7 days of continuous footage, + * 14 days of motion‑triggered clips, + * 30 days of alert snapshots. + +I configured Frigate’s `retention` policy to automatically prune older files, and the system has been stable for months without running out of space. If the footage ever exceeds capacity, the next step would be to add a second HDD and enable a simple RAID‑1 mirror for redundancy. + +--- + +## Home Assistant: The Automation Glue + +Home Assistant (HA) is the hub that ties everything together. Frigate pushes detection events to HA via its built‑in integration, and HA then orchestrates the following steps: + +1. **Receive event** – HA gets the object type, timestamp, and a URL to the snapshot. +2. **Copy image** – HA downloads the snapshot to its own cache. +3. **Ask Ollama** – HA sends the image to the 2‑billion‑parameter model with a prompt like “Summarise the scene for a homeowner, focusing on potential security relevance.” +4. **Dispatch notification** – HA forwards the AI‑generated description and the image to a Matrix room that I have linked to my phone. + +Because HA already manages my lights, locks, and climate, the CCTV alerts can also trigger downstream actions. For example, a “person at front door” event could automatically turn on the porch light, while a “delivery person” event could unlock a smart drop‑box for a brief window. + +--- + +## The Physical Build: Getting Cable Through the Roof + +All the software in the world is useless if the cameras can’t talk to the server. I opted for wired Ethernet because it guarantees a stable, low‑latency feed and avoids the headaches of Wi‑Fi dead zones. Running Cat‑6 cable from the attic down to each camera required a ladder, a drill, and a lot of patience. I’m not exactly a handyman, so I called in the one person who could actually climb onto the roof without panicking – my dad. + +He spent an entire Saturday pulling the cable through the roof cavity, crimping connectors, and testing continuity while I handed him tools and tried not to get in the way. The result is a clean, tidy installation that looks as if it were done by a professional. The effort was a reminder that even the most high‑tech projects still rely on good old‑fashioned elbow grease. + +--- + +## The Architecture – A Visual Overview ```mermaid -graph TD -Camera --> FrigateObjectDetections -FrigateObjectDetections --> SendSnapshot -SendSnapshot --> Ollama[qwen3-vl-4b for Semantic Search AI Enhancement] -FrigateObjectDetections --> HomeAssistant -HomeAssistant --> ObjectDetection -ObjectDetection --> CopyImage -CopyImage --> Ollama[qwen3-vl-2b for Context Enhancement] -Ollama --> Matrix -Camera --> FutureCamera +graph LR + Camera1[Camera] --> Frigate + Camera2[Camera] --> Frigate + Camera3[Camera] --> Frigate + Frigate --> ObjectDetections[Object Detections] + ObjectDetections --> Ollama1[Ollama (qwen3‑vl‑4b) – Semantic Search] + Frigate --> NVR[Frigate NVR] + NVR --> HomeAssistant + HomeAssistant --> CopyImg[Copy Image to Home Assistant] + CopyImg --> Ollama2[Ollama (qwen3‑vl‑2b) – Context Enhancement] + Ollama2 --> Matrix[Matrix Notification] + CameraFuture[Future Camera] --> Frigate ``` -The system operates within a dedicated node on my Proxmox cluster, utilizing Docker LXC for efficient resource management. +--- -Storage is handled with an SSD for Frigate’s operation and a 2TB HDD for video history and metadata. This setup provides sufficient capacity for seven days of full history, fourteen days of detections, and thirty days of alerts. +## Lessons Learned and Tweaks for the Future -While the system is functional and delivers impressive results, ongoing refinements are planned. The current notification prompts are being refined to improve accuracy and reduce unnecessary alerts. The ultimate goal is to leverage AI to intelligently determine when notifications are warranted, rather than generating alerts for every detected object. +### Prompt Engineering Still a Work‑In‑Progress +The quality of the AI‑generated metadata hinges on the prompts sent to Ollama. My initial prompts were overly generic (“Describe this image”), which resulted in vague outputs like “A vehicle is present.” After a few iterations I discovered that adding context (“Focus on colour, make, and any visible markings”) dramatically improved relevance. I’m now experimenting with a small prompt library that can be swapped out depending on the time of day or the type of detection. -Finally, a sincere thank you is extended to my father, whose assistance was invaluable in running Ethernet cables through the roof for the wired camera setup. His contribution was essential to the project's success. +### Smarter Notification Filtering +At the moment every person or vehicle triggers a push notification, which quickly becomes noisy. The next step is to let the LLM decide whether an alert is worth sending. By feeding the model additional data – such as the known list of resident vehicle plates or the typical schedule of the mail carrier – it can return a confidence score that HA can use to suppress low‑importance alerts. -This project represents a blend of cutting-edge AI and practical hardware, resulting in a robust and personalized smart home security solution. \ No newline at end of file +### Scaling to More Cameras +Adding a fourth camera is on the roadmap, and the architecture is already prepared for it. The biggest concern will be GPU utilisation; the RTX 3060 can comfortably handle three streams at 10 fps, but a fourth may push the limits. Options include upgrading to a higher‑end GPU or distributing the load across two nodes in the Proxmox cluster. + +### Backup and Disaster Recovery +While the HDD is sufficient for day‑to‑day storage, a catastrophic drive failure would erase weeks of footage. I plan to add a nightly rsync job that mirrors the video directory to an external USB‑3.0 drive, and eventually to a cheap off‑site NAS for true redundancy. + +--- + +## The Bigger Picture: Why This Matters + +What started as a personal curiosity has turned into a platform that could be replicated by anyone with a modest budget and a willingness to roll up their sleeves. The key takeaways are: + +* **Privacy first** – All video stays on‑premises, and AI processing never leaves the house. +* **Open‑source stack** – Frigate, Home Assistant, and Ollama are all free and community‑driven, meaning you’re not locked into a vendor’s roadmap. +* **Extensible architecture** – The mermaid diagram shows a modular flow where each component can be swapped out (e.g., replace Ollama with a different LLM, or add a second NVR for redundancy). +* **Real‑world utility** – Semantic search turns hours of footage into a few clicks, and context‑rich notifications cut down on false alarms. + +In an era where “smart” devices are often synonymous with “data‑selling machines,” building a locally owned, AI‑enhanced CCTV system is a small act of digital sovereignty. It proves that you don’t need a multi‑million‑dollar budget to have a surveillance system that actually *understands* what it’s watching. + +--- + +## A Massive Shout‑Out + +No amount of software wizardry could have gotten the Ethernet cables through the roof without a pair of strong hands and a willingness to climb up there on a scorching summer day. My dad took the ladder, the drill, and the patience to guide the cable through the attic, and he did it with a grin (and a few jokes about “why the son can’t even change a lightbulb”). This project would have remained a half‑finished dream without his help, and the system we now have is as much his achievement as mine. + +So, Dad – thank you for the hard yards, the guidance, and for not letting my nerdy enthusiasm turn into a half‑baked mess. This CCTV system is a testament to teamwork, a bit of Aussie ingenuity, and the belief that the best security is the kind you build yourself. + +--- + +## Closing Thoughts + +If you’re reading this and thinking “I could use something like this,” the answer is: start small. Grab a cheap camera, spin up a Frigate container, and let the system record a few days of footage. Once you’ve got the basics working, add Home Assistant, hook in Ollama, and watch the alerts become smarter. Don’t be afraid to experiment with prompts, tweak the retention policy, or even add a second GPU down the line. + +The journey from “just a camera” to “AI‑enhanced CCTV” is a series of incremental steps, each one teaching you a little more about video pipelines, machine learning, and home automation. The biggest reward isn’t just the peace of mind that comes from knowing exactly who’s at the door – it’s the satisfaction of building something that truly belongs to you, runs on your hardware, and respects your privacy. + +So, fire up that Docker daemon, pull the Frigate image, and let the cameras roll. And when the first AI‑generated notification lands on your phone, you’ll know you’ve turned a simple surveillance feed into a genuinely intelligent guardian for your home. \ No newline at end of file From aa339dc6b316b9fcb079c65cc3c0895d48217287 Mon Sep 17 00:00:00 2001 From: Blog Creator Date: Tue, 3 Feb 2026 02:09:07 +0000 Subject: [PATCH 05/14] Implement AI-enhanced home CCTV system --- ...ding_an_ai_enhanced_cctv_system_at_home.md | 165 +++--------------- 1 file changed, 25 insertions(+), 140 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 1476bda..f5f8571 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -1,164 +1,49 @@ -Over the past six months I’ve been tinkering with a problem that most homeowners shrug off as “just get a camera and be done with it.” The reality is that a plain‑old CCTV feed gives you a wall of motion‑triggered video that you have to trawl through manually, and the moment you start looking for anything useful you quickly realise the data is as opaque as a foggy morning on the Nullarbor. My goal was simple on paper: a locally‑hosted surveillance system that keeps every frame under my own roof, yet still offers the kind of intelligent search and notification you’d expect from a cloud‑based service. The result is an AI‑enhanced CCTV stack built around the open‑source NVR called **Frigate**, a handful of budget‑friendly TP‑Link Vigi C540 cameras, and a modest homelab that runs everything in Docker containers on a Proxmox LXC node. +# Designing and Building an AI Enhanced CCTV System at Home ---- +The world of home security has come a long way from blinking red lights and hefty monthly fees. I've been exploring a more sophisticated approach – building a locally controlled, AI-enhanced CCTV system for my homelab. The goal? To move beyond simple surveillance and gain a deeper understanding of what's happening around my property, all while maintaining complete control over my data. It’s been a fascinating project, blending hardware, software, and a healthy dose of machine learning. -## Why “Local‑First” Matters +The system centres around Frigate, an open-source Network Video Recorder (NVR) software. Frigate’s intuitive interface and container-native design make it surprisingly accessible, even for those of us who aren't coding wizards. The ability to configure it using simple YAML files is a game-changer, allowing for repeatable builds and easy experimentation. While the full configuration details remain private for now (a bit of tidying up is still needed!), my aim is to eventually share the setup publicly so others can benefit from it. -When you hand over video streams to a commercial provider you’re essentially giving a stranger a front‑row seat to every comings and goings at your doorstep. The trade‑off is usually a subscription fee and the promise of “smart” alerts. In practice those alerts are either too noisy (the neighbour’s cat triggers a “person detected” every night) or too vague (a generic “motion detected” that forces you to open the app and stare at a blank screen). Moreover, the data lives somewhere you can’t audit, and any breach could expose a detailed visual diary of your life. +**The Hardware Foundation** -A local‑first approach eliminates the subscription, guarantees that the raw footage never leaves your network, and gives you full control over how the data is processed. The only thing you have to trust is the hardware you own and the open‑source software you run. That level of ownership is what makes the project worth the effort. +My camera setup consists of three TP-Link ‘vigi’ C540 cameras, a bargain at AUD$50 each during a Black Friday sale. These cameras offer pan and tilt functionality, along with built-in human detection and tracking – features that have proven remarkably effective. While they lack zoom capabilities, they’re more than adequate for my surveillance needs. ---- +**Frigate: The Core of the System** -## The Core: Frigate NVR +Frigate acts as the central hub, handling video recording, metadata tagging, and integrating AI capabilities. I’m leveraging OpenVINO models for efficient on-device processing. This approach minimizes latency and reduces reliance on external services. The system is currently running on a Dell workstation powered by an Intel Core i5-7500 processor, a surprisingly capable machine I acquired for just $150. The “small” OpenVINO SSDLite model is providing good performance, although the system does run a little warm, averaging around 75% CPU usage. -Frigate is a container‑native Network Video Recorder that does far more than simply write video to disk. It watches each RTSP stream, runs object detection on every frame (or on a configurable subset), and exposes a clean REST API plus a web UI for live view, playback, and search. The entire configuration lives in a single YAML file, which makes it trivial to version‑control, replicate, or spin up a fresh instance for testing. +**AI-Powered Intelligence** -Key points that made Frigate the obvious choice: +What truly sets this system apart is its integration with GenAI services via Ollama. This allows me to send detections and alerts to an external AI service, enabling context-aware semantic searches. For example, I can now search for "Red SUV" or "White Ute" and receive highly targeted results. This significantly enhances detection accuracy and reduces false positives. -| Feature | Why it matters | -|---------|----------------| -| **Docker‑ready** | Deploys in seconds on any machine that can run containers. | -| **YAML‑driven** | Human‑readable, repeatable, and easy to tweak via CI pipelines. | -| **Built‑in object detectors** | Supports TensorRT, Coral, and CPU‑only models; can pull the latest weights from Hugging Face. | -| **GenAI integration** | Direct hooks to Ollama for custom metadata generation. | -| **Home Assistant integration** | Exposes events that can be consumed by any automation platform. | +**Home Assistant: Orchestrating the Automation** -Because the configuration is declarative, I could spin up a fresh test environment, point it at a different camera, tweak the FFmpeg pipeline, and have the whole stack ready in under five minutes. That rapid iteration cycle was essential while I was still learning the quirks of each camera and the best way to balance CPU load against detection accuracy. +To further streamline the process, I’ve integrated Frigate with Home Assistant, a powerful open-source home automation platform. This allows me to trigger notifications based on detected objects and leverage Home Assistant’s existing integration with Ollama and Matrix. The result is a sophisticated system that not only detects objects but also provides contextual information and sends notifications directly to my phone. ---- +**System Architecture** -## The Eyes: TP‑Link Vigi C540 Cameras - -I chose three (soon to be four) TP‑Link Vigi C540 units because they hit the sweet spot between price and capability. At AUD 50 each they were a steal during a Black Friday sale, and they come with pan‑and‑tilt motors, on‑board human detection, and a follow‑mode that keeps a moving subject in frame. The cameras lack optical zoom, but the field of view is wide enough for a typical suburban front yard, and the built‑in motion detection reduces the amount of data that needs to be processed by Frigate. - -| Spec | Reason for selection | -|------|-----------------------| -| **Pan/Tilt** | Allows a single unit to cover multiple angles without extra hardware. | -| **Wired Ethernet** | Guarantees a stable feed and eliminates Wi‑Fi interference. | -| **On‑board human detection** | Provides a first line of filtering before Frigate even sees the frame. | -| **Affordability** | Keeps the overall project budget modest, leaving room for a GPU upgrade later. | - -The cameras are mounted on the roof, with Ethernet cables run through the attic. I’ll come back to the physical installation later, but the result is a reliable, low‑latency video source that feeds directly into the Frigate container. - ---- - -## The Brain: GPU‑Accelerated Object Detection - -Frigate can offload inference to a GPU, a Coral Edge TPU, or even run on the CPU if you’re willing to accept slower detection rates. My homelab node is a modest Intel N100 mini‑PC equipped with a mid‑range NVIDIA RTX 3060. With the RTX in place, Frigate pulls the latest YOLO‑v8 models from Hugging Face at startup and runs detection at roughly 10 fps per camera, which is more than sufficient for a residential setting. - -Because the models are downloaded from the internet, the host needs occasional outbound connectivity to refresh weights. This is a small price to pay for the ability to keep the detection pipeline up‑to‑date without manual intervention. - ---- - -## Adding Smarts: Ollama and GenAI - -Frigate’s native object detection tells you *what* it sees – a person, a car, a dog – but it can’t answer “what colour is the car?” or “is that a delivery driver or a neighbour?” That’s where Ollama comes in. Ollama is a locally hosted LLM/vision service that can run multimodal models such as **qwen3‑vl‑4b** and **qwen3‑vl‑2b**. By sending a snapshot to Ollama with a carefully crafted prompt, the model returns a natural‑language description that adds semantic depth to the raw detection. - -Two distinct flows exist: - -1. **Semantic Search Enrichment** – When a snapshot is sent to the 4‑billion‑parameter model, the response is stored as metadata alongside the detection. This lets me type “Red SUV” into Frigate’s search bar and instantly filter to the relevant clips. -2. **Contextual Notification** – A lighter 2‑billion‑parameter model receives the same snapshot but with a prompt that asks for a concise, security‑focused summary. The result is attached to a Matrix message that lands on my phone, turning a generic “person detected” alert into “Delivery person in hi‑vis vest placing a parcel on the porch.” - -Because both models run locally, there is no latency penalty and no data leaves the house. The only external dependency is the occasional model update from Hugging Face. - ---- - -## Storage Strategy - -Video footage is storage‑hungry, especially when you keep full‑resolution streams for a week or more. My node uses a two‑tier storage layout: - -* **SSD (256 GB)** – Hosts the OS, Docker engine, and Frigate’s database. The SSD ensures fast metadata reads/writes and quick container restarts. -* **2 TB HDD** – Dedicated to raw video files. With three cameras recording at 1080p/15 fps, the drive comfortably holds: - * 7 days of continuous footage, - * 14 days of motion‑triggered clips, - * 30 days of alert snapshots. - -I configured Frigate’s `retention` policy to automatically prune older files, and the system has been stable for months without running out of space. If the footage ever exceeds capacity, the next step would be to add a second HDD and enable a simple RAID‑1 mirror for redundancy. - ---- - -## Home Assistant: The Automation Glue - -Home Assistant (HA) is the hub that ties everything together. Frigate pushes detection events to HA via its built‑in integration, and HA then orchestrates the following steps: - -1. **Receive event** – HA gets the object type, timestamp, and a URL to the snapshot. -2. **Copy image** – HA downloads the snapshot to its own cache. -3. **Ask Ollama** – HA sends the image to the 2‑billion‑parameter model with a prompt like “Summarise the scene for a homeowner, focusing on potential security relevance.” -4. **Dispatch notification** – HA forwards the AI‑generated description and the image to a Matrix room that I have linked to my phone. - -Because HA already manages my lights, locks, and climate, the CCTV alerts can also trigger downstream actions. For example, a “person at front door” event could automatically turn on the porch light, while a “delivery person” event could unlock a smart drop‑box for a brief window. - ---- - -## The Physical Build: Getting Cable Through the Roof - -All the software in the world is useless if the cameras can’t talk to the server. I opted for wired Ethernet because it guarantees a stable, low‑latency feed and avoids the headaches of Wi‑Fi dead zones. Running Cat‑6 cable from the attic down to each camera required a ladder, a drill, and a lot of patience. I’m not exactly a handyman, so I called in the one person who could actually climb onto the roof without panicking – my dad. - -He spent an entire Saturday pulling the cable through the roof cavity, crimping connectors, and testing continuity while I handed him tools and tried not to get in the way. The result is a clean, tidy installation that looks as if it were done by a professional. The effort was a reminder that even the most high‑tech projects still rely on good old‑fashioned elbow grease. - ---- - -## The Architecture – A Visual Overview +[Diagram illustrating the system flow - see Mermaid code below] ```mermaid -graph LR - Camera1[Camera] --> Frigate - Camera2[Camera] --> Frigate - Camera3[Camera] --> Frigate - Frigate --> ObjectDetections[Object Detections] - ObjectDetections --> Ollama1[Ollama (qwen3‑vl‑4b) – Semantic Search] - Frigate --> NVR[Frigate NVR] - NVR --> HomeAssistant - HomeAssistant --> CopyImg[Copy Image to Home Assistant] - CopyImg --> Ollama2[Ollama (qwen3‑vl‑2b) – Context Enhancement] - Ollama2 --> Matrix[Matrix Notification] - CameraFuture[Future Camera] --> Frigate +graph TD + Camera --> Frigate NVR + Frigate NVR --> HomeAssistant + Frigate Object Detections --> Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement + Frigate NVR --> HomeAssistant --> (Object Detection from Frigate) --> (Copy Image to Home Assistant) --> Send image to Ollama (qwen3-vl-2b) for context enhancement --> send response back via Matrix + Camera --> (Camera future) ``` ---- +**Storage and Scalability** -## Lessons Learned and Tweaks for the Future +The system is currently housed within a node in my Proxmox cluster, utilizing Docker and LXC containers for efficient resource management. Storage is handled by a 2TB HDD, providing ample space for 7 days of full-resolution video, 14 days of detection data, and 30 days of alerts across the four cameras I plan to deploy. -### Prompt Engineering Still a Work‑In‑Progress -The quality of the AI‑generated metadata hinges on the prompts sent to Ollama. My initial prompts were overly generic (“Describe this image”), which resulted in vague outputs like “A vehicle is present.” After a few iterations I discovered that adding context (“Focus on colour, make, and any visible markings”) dramatically improved relevance. I’m now experimenting with a small prompt library that can be swapped out depending on the time of day or the type of detection. +**Ongoing Development and Future Enhancements** -### Smarter Notification Filtering -At the moment every person or vehicle triggers a push notification, which quickly becomes noisy. The next step is to let the LLM decide whether an alert is worth sending. By feeding the model additional data – such as the known list of resident vehicle plates or the typical schedule of the mail carrier – it can return a confidence score that HA can use to suppress low‑importance alerts. +While the system is already delivering impressive results, there’s still room for improvement. I’m currently refining the prompts used for semantic context enhancement and notifications to reduce false positives. A key goal is to implement AI-driven decision-making for notifications, preventing unnecessary alerts for every detected person or vehicle. -### Scaling to More Cameras -Adding a fourth camera is on the roadmap, and the architecture is already prepared for it. The biggest concern will be GPU utilisation; the RTX 3060 can comfortably handle three streams at 10 fps, but a fourth may push the limits. Options include upgrading to a higher‑end GPU or distributing the load across two nodes in the Proxmox cluster. +**A Note of Gratitude** -### Backup and Disaster Recovery -While the HDD is sufficient for day‑to‑day storage, a catastrophic drive failure would erase weeks of footage. I plan to add a nightly rsync job that mirrors the video directory to an external USB‑3.0 drive, and eventually to a cheap off‑site NAS for true redundancy. +Finally, a sincere thank you to my father. His expertise and willingness to lend a hand were invaluable, particularly when it came to the less glamorous task of running Ethernet cables through the roof. Without his assistance, this project simply wouldn’t have been possible. ---- -## The Bigger Picture: Why This Matters -What started as a personal curiosity has turned into a platform that could be replicated by anyone with a modest budget and a willingness to roll up their sleeves. The key takeaways are: - -* **Privacy first** – All video stays on‑premises, and AI processing never leaves the house. -* **Open‑source stack** – Frigate, Home Assistant, and Ollama are all free and community‑driven, meaning you’re not locked into a vendor’s roadmap. -* **Extensible architecture** – The mermaid diagram shows a modular flow where each component can be swapped out (e.g., replace Ollama with a different LLM, or add a second NVR for redundancy). -* **Real‑world utility** – Semantic search turns hours of footage into a few clicks, and context‑rich notifications cut down on false alarms. - -In an era where “smart” devices are often synonymous with “data‑selling machines,” building a locally owned, AI‑enhanced CCTV system is a small act of digital sovereignty. It proves that you don’t need a multi‑million‑dollar budget to have a surveillance system that actually *understands* what it’s watching. - ---- - -## A Massive Shout‑Out - -No amount of software wizardry could have gotten the Ethernet cables through the roof without a pair of strong hands and a willingness to climb up there on a scorching summer day. My dad took the ladder, the drill, and the patience to guide the cable through the attic, and he did it with a grin (and a few jokes about “why the son can’t even change a lightbulb”). This project would have remained a half‑finished dream without his help, and the system we now have is as much his achievement as mine. - -So, Dad – thank you for the hard yards, the guidance, and for not letting my nerdy enthusiasm turn into a half‑baked mess. This CCTV system is a testament to teamwork, a bit of Aussie ingenuity, and the belief that the best security is the kind you build yourself. - ---- - -## Closing Thoughts - -If you’re reading this and thinking “I could use something like this,” the answer is: start small. Grab a cheap camera, spin up a Frigate container, and let the system record a few days of footage. Once you’ve got the basics working, add Home Assistant, hook in Ollama, and watch the alerts become smarter. Don’t be afraid to experiment with prompts, tweak the retention policy, or even add a second GPU down the line. - -The journey from “just a camera” to “AI‑enhanced CCTV” is a series of incremental steps, each one teaching you a little more about video pipelines, machine learning, and home automation. The biggest reward isn’t just the peace of mind that comes from knowing exactly who’s at the door – it’s the satisfaction of building something that truly belongs to you, runs on your hardware, and respects your privacy. - -So, fire up that Docker daemon, pull the Frigate image, and let the cameras roll. And when the first AI‑generated notification lands on your phone, you’ll know you’ve turned a simple surveillance feed into a genuinely intelligent guardian for your home. \ No newline at end of file From 955292d797ca041befce6956951b98091e11112a Mon Sep 17 00:00:00 2001 From: Blog Creator Date: Tue, 3 Feb 2026 02:32:08 +0000 Subject: [PATCH 06/14] Introduce private AI CCTV system --- ...ding_an_ai_enhanced_cctv_system_at_home.md | 138 +++++++++++++----- 1 file changed, 104 insertions(+), 34 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index f5f8571..1f675a0 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -1,49 +1,119 @@ -# Designing and Building an AI Enhanced CCTV System at Home +### Introduction -The world of home security has come a long way from blinking red lights and hefty monthly fees. I've been exploring a more sophisticated approach – building a locally controlled, AI-enhanced CCTV system for my homelab. The goal? To move beyond simple surveillance and gain a deeper understanding of what's happening around my property, all while maintaining complete control over my data. It’s been a fascinating project, blending hardware, software, and a healthy dose of machine learning. +Over the past half‑year I’ve been tinkering with a problem that many of us in the homelab community face every day: how to turn a handful of cheap IP cameras into a truly useful security system without surrendering our footage to a cloud service. The answer, after a lot of trial and error, is a stack that lives entirely on‑premises, talks to itself, and even adds a sprinkle of generative AI to make the alerts intelligible. In plain English, the system records video, spots objects, asks a local vision‑language model to describe what it sees, and then pushes a nicely worded notification to my phone. The result is a CCTV solution that is private, affordable, and surprisingly capable. -The system centres around Frigate, an open-source Network Video Recorder (NVR) software. Frigate’s intuitive interface and container-native design make it surprisingly accessible, even for those of us who aren't coding wizards. The ability to configure it using simple YAML files is a game-changer, allowing for repeatable builds and easy experimentation. While the full configuration details remain private for now (a bit of tidying up is still needed!), my aim is to eventually share the setup publicly so others can benefit from it. +### Why Local Control Matters -**The Hardware Foundation** +The market is flooded with “smart” cameras that promise motion alerts, facial recognition, and cloud‑based video archives. Those devices are convenient, but they also hand over a valuable slice of your privacy to a third‑party. A compromised account or a mis‑configured API key can expose weeks of footage to the world. For a home that already has a smart‑home hub, a voice assistant, and a handful of IoT devices, adding another internet‑facing endpoint feels like inviting a stranger into the living room. -My camera setup consists of three TP-Link ‘vigi’ C540 cameras, a bargain at AUD$50 each during a Black Friday sale. These cameras offer pan and tilt functionality, along with built-in human detection and tracking – features that have proven remarkably effective. While they lack zoom capabilities, they’re more than adequate for my surveillance needs. +By keeping everything inside the home network we gain three things: -**Frigate: The Core of the System** +1. **Data sovereignty** – the video never leaves the LAN, so only the people you trust can view it. +2. **Predictable costs** – no monthly subscription for storage or AI inference; the only expense is the hardware you already own. +3. **Full customisation** – we can decide exactly how alerts are generated, what language they use, and when they are sent. -Frigate acts as the central hub, handling video recording, metadata tagging, and integrating AI capabilities. I’m leveraging OpenVINO models for efficient on-device processing. This approach minimizes latency and reduces reliance on external services. The system is currently running on a Dell workstation powered by an Intel Core i5-7500 processor, a surprisingly capable machine I acquired for just $150. The “small” OpenVINO SSDLite model is providing good performance, although the system does run a little warm, averaging around 75% CPU usage. +The only downside is the extra effort required to stitch the pieces together, but that’s where the fun begins. -**AI-Powered Intelligence** +### Choosing the Cameras -What truly sets this system apart is its integration with GenAI services via Ollama. This allows me to send detections and alerts to an external AI service, enabling context-aware semantic searches. For example, I can now search for "Red SUV" or "White Ute" and receive highly targeted results. This significantly enhances detection accuracy and reduces false positives. +I needed cameras that were cheap enough to experiment with, yet offered enough features to make the system worthwhile. A Black Friday deal on Amazon AU landed me three (soon to be four) TP‑Link Vigi C540 units for AUD 50 each. For the price they are surprisingly capable: -**Home Assistant: Orchestrating the Automation** +* **Pan/tilt** – allows a single unit to cover a wide field of view without additional hardware. +* **On‑board human detection** – the camera can flag a person before any external processing, reducing false positives. +* **RTSP support** – the standard streaming protocol that Frigate expects, meaning we are not locked into a proprietary API. -To further streamline the process, I’ve integrated Frigate with Home Assistant, a powerful open-source home automation platform. This allows me to trigger notifications based on detected objects and leverage Home Assistant’s existing integration with Ollama and Matrix. The result is a sophisticated system that not only detects objects but also provides contextual information and sends notifications directly to my phone. +The cameras lack optical zoom, but the 1080p stream is more than enough for a backyard or front‑gate view. Their built‑in motion logic also gives us a first line of defence against unnecessary processing on the NVR. -**System Architecture** +### The NVR Core: Frigate -[Diagram illustrating the system flow - see Mermaid code below] +At the heart of the system sits **Frigate**, an open‑source network video recorder that runs as a Docker container. What makes Frigate a perfect fit for a homelab project? + +* **Container‑native** – it can be deployed on any host that supports Docker, which aligns nicely with my Proxmox LXC setup. +* **YAML‑driven configuration** – a single, version‑controlled file defines cameras, streams, detection models, and storage policies. +* **Built‑in object detection** – Frigate can invoke a variety of detectors (OpenVINO, TensorRT, Coral) without external orchestration. +* **Extensible API** – the platform exposes webhooks and REST endpoints that other services (Home Assistant, Ollama) can consume. + +Spinning up a new Frigate instance is as simple as pulling the image, mounting the configuration file, and pointing it at the RTSP URLs of the cameras. The UI instantly shows live feeds, detection boxes, and a timeline of events, which made the early testing phase feel like watching a sci‑fi control room. + +### Getting the Detection Right + +Frigate ships with a handful of pre‑configured detectors, but I wanted a model that could run on the modest hardware I had on hand: a Dell OptiPlex 7060 SFF rescued for $150, equipped with an Intel i5‑7500 and 16 GB of RAM. After a few experiments I settled on the **OpenVINO** backend with the **SSDLite MobileNet V2** model from the Open Model Zoo. The model is small enough to run on CPU while still delivering decent accuracy for cars, people, and animals. + +A few practical notes from the deployment: + +* **Model download** – Frigate pulls the model from Hugging Face at first start, so the host needs occasional internet access for updates. +* **CPU utilisation** – The i5 hovers around 75 % load under continuous detection, which is acceptable for now but leaves little headroom for additional workloads. +* **Thermal considerations** – The workstation runs warm; I’ve added a low‑profile fan to keep temperatures in check. + +The detector provides bounding boxes and confidence scores for each frame that triggers an event. Those raw detections are the raw material that later gets enriched by the generative AI layer. + +### Adding Generative AI with Ollama + +Frigate’s newest integration allows snapshots of detections to be sent to an external **GenAI** service. I run **Ollama** locally, which hosts a variety of large language models, including vision‑language variants such as **qwen3‑vl‑4b** and **qwen3‑vl‑2b**. The workflow is straightforward: + +1. An object (e.g., a car) is detected by Frigate. +2. Frigate captures a still image of the frame and forwards it to Ollama via the GenAI webhook. +3. The vision‑language model analyses the image and returns a concise textual description (“Red SUV parked in the driveway”). +4. The description is stored as metadata alongside the detection event. + +This extra step turns a generic “car” tag into a searchable phrase. In the Frigate UI I can now type “white ute” or “red scooter” and instantly retrieve the matching clips. The semantic search capability is the most exciting part of the project because it opens the door to natural‑language queries over a video archive—a feature that commercial cloud services charge a premium for. + +### Home Assistant: The Glue That Makes It All Useful + +Frigate already gives us detection and enriched metadata, but to turn that into actionable alerts we need an automation engine. **Home Assistant** fills that role perfectly: + +* **Event ingestion** – Frigate pushes detection events to Home Assistant via its native integration. +* **AI‑driven notifications** – Home Assistant receives the snapshot, forwards it to Ollama for a second pass (using a smaller model for speed), and then formats a friendly message. +* **Matrix delivery** – The final message, complete with the image and AI‑generated caption, is sent to my Matrix client on my phone. + +The result is a notification that reads something like: “A white ute with a surfboard in the back has just entered the driveway.” Compared with a raw “motion detected” alert, this is a massive usability upgrade. Moreover, Home Assistant can trigger other automations based on the enriched data, such as turning on porch lights when a car arrives after sunset, or silencing the alarm when the mail carrier is recognised. + +### Storage Strategy + +Video storage is often the Achilles’ heel of a DIY CCTV system. My approach balances capacity, retention, and cost: + +| Storage tier | Device | Purpose | Retention | +|--------------|--------|---------|-----------| +| OS & containers | 256 GB SSD | Frigate, Home Assistant, Ollama | Indefinite | +| Video archive | 2 TB HDD | Full‑resolution recordings, detection clips, alert snapshots | 7 days (full), 14 days (detections), 30 days (alerts) | + +Frigate automatically prunes old files based on the policies defined in the YAML configuration, so the drive never fills up unexpectedly. With four 1080p streams at 5 fps the 2 TB budget comfortably covers the stated retention windows. + +### Deploying on Proxmox + +All services run inside a dedicated LXC container on a Proxmox node that I carved out specifically for this project. The container hosts Docker, the Frigate image, Home Assistant, and Ollama. Using LXC gives me a lightweight isolation layer without the overhead of a full VM, and Proxmox’s snapshot feature lets me roll back the entire stack if a configuration change goes sideways. + +The network layout is simple: each camera connects via wired Ethernet to the same LAN as the Proxmox host, ensuring low latency and no packet loss that could otherwise cause frame drops. The container’s Docker bridge is bound to the host’s network stack, so Frigate sees the cameras as if it were running directly on the workstation. + +### Challenges and What’s Next + +The system works, but there are a few rough edges that I’m actively polishing: + +* **Prompt engineering** – The text prompts sent to the vision‑language models are still generic, leading to occasional mis‑classifications (e.g., a cat being described as a “small vehicle”). Fine‑tuning the prompts and possibly adding a few few‑shot examples should improve consistency. +* **Notification fatigue** – Right now every person or car that passes the driveway generates a push notification. I plan to add a lightweight decision model that evaluates the confidence score, time of day, and historical patterns before deciding whether to alert. +* **Hardware scaling** – The i5‑7500 is holding its own, but as I add more cameras or switch to a larger model (e.g., a 4‑billion‑parameter vision transformer), I’ll need a more powerful CPU or a dedicated GPU/Intel Arc accelerator. +* **Public configuration** – The deployment repository still contains secrets. Once scrubbed, I intend to publish the full YAML and Docker‑compose files so others can replicate the stack with a single `git clone`. + +### A Personal Note: The Unsung Hero + +No amount of software wizardry would have gotten the Ethernet cables into the roof without a bit of elbow grease. My dad, armed with a ladder, a drill, and a healthy dose of patience, ran the cabling that links the cameras to the network. He spent an entire Saturday climbing into the attic, pulling cables through joists, and making sure each connection was solid. I was mostly handing him tools and keeping the coffee warm. Without his help the whole project would have remained a pipe‑dream. So, Dad, thank you for the hard yards, the guidance, and the occasional dad‑joke that kept the mood light while we were up on the roof. This system is as much yours as it is mine. + +### Closing Thoughts + +Building an AI‑enhanced CCTV system at home is no longer a pipe‑dream reserved for large enterprises. With a modest budget, a handful of inexpensive cameras, and a stack of open‑source tools—Frigate, OpenVINO, Ollama, Home Assistant, and a bit of Proxmox magic—you can achieve a private, searchable, and context‑rich surveillance solution. The journey taught me a lot about video streaming, container orchestration, and the quirks of vision‑language models, but the biggest takeaway is the empowerment that comes from owning your data. + +If you’re curious about the nitty‑gritty—YAML snippets, Docker‑compose files, or the exact model versions—I’m happy to share once the repository is fully sanitized. Until then, enjoy the peace of mind that comes from knowing your backyard is being watched by a system that not only sees but also understands. ```mermaid graph TD - Camera --> Frigate NVR - Frigate NVR --> HomeAssistant - Frigate Object Detections --> Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement - Frigate NVR --> HomeAssistant --> (Object Detection from Frigate) --> (Copy Image to Home Assistant) --> Send image to Ollama (qwen3-vl-2b) for context enhancement --> send response back via Matrix - Camera --> (Camera future) -``` - -**Storage and Scalability** - -The system is currently housed within a node in my Proxmox cluster, utilizing Docker and LXC containers for efficient resource management. Storage is handled by a 2TB HDD, providing ample space for 7 days of full-resolution video, 14 days of detection data, and 30 days of alerts across the four cameras I plan to deploy. - -**Ongoing Development and Future Enhancements** - -While the system is already delivering impressive results, there’s still room for improvement. I’m currently refining the prompts used for semantic context enhancement and notifications to reduce false positives. A key goal is to implement AI-driven decision-making for notifications, preventing unnecessary alerts for every detected person or vehicle. - -**A Note of Gratitude** - -Finally, a sincere thank you to my father. His expertise and willingness to lend a hand were invaluable, particularly when it came to the less glamorous task of running Ethernet cables through the roof. Without his assistance, this project simply wouldn’t have been possible. - - - + Camera1[Camera] --> Frigate + Camera2[Camera] --> Frigate + Camera3[Camera] --> Frigate + Camera4[Future Camera] --> Frigate + Frigate -->|Object Detections| Ollama_qwen3_vl_4b[Send snapshot to Ollama (qwen3‑vl‑4b) for semantic search AI enhancement] + Frigate --> Frigate_NVR[Frigate NVR] + Frigate_NVR --> HomeAssistant[Home Assistant] + HomeAssistant -->|Object Detection from Frigate| CopyImage[Copy Image to Home Assistant] + CopyImage --> Ollama_qwen3_vl_2b[Send image to Ollama (qwen3‑vl‑2b) for context enhancement] + Ollama_qwen3_vl_2b --> Matrix[Send response back via Matrix] +``` \ No newline at end of file From 836ee3857a1ef1ae980093a654d875c477cda17f Mon Sep 17 00:00:00 2001 From: Blog Creator Date: Tue, 3 Feb 2026 03:04:33 +0000 Subject: [PATCH 07/14] Add privacy-focused AI CCTV system --- ...ding_an_ai_enhanced_cctv_system_at_home.md | 220 ++++++++++++------ 1 file changed, 145 insertions(+), 75 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 1f675a0..2c49a11 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -1,119 +1,189 @@ -### Introduction +### Why Build Your Own AI‑Enhanced CCTV? -Over the past half‑year I’ve been tinkering with a problem that many of us in the homelab community face every day: how to turn a handful of cheap IP cameras into a truly useful security system without surrendering our footage to a cloud service. The answer, after a lot of trial and error, is a stack that lives entirely on‑premises, talks to itself, and even adds a sprinkle of generative AI to make the alerts intelligible. In plain English, the system records video, spots objects, asks a local vision‑language model to describe what it sees, and then pushes a nicely worded notification to my phone. The result is a CCTV solution that is private, affordable, and surprisingly capable. +When you buy a consumer‑grade security camera, you’re not just paying for the lens and the plastic housing. You’re also paying for a subscription that ships every frame of your backyard to a cloud service you’ll never meet. That data can be used to train models, sold to advertisers, or handed over to authorities on a whim. For many, the convenience outweighs the privacy cost, but for anyone who values control over their own footage, the trade‑off feels unacceptable. -### Why Local Control Matters +The goal of this project was simple: **keep every byte of video on‑premises, add a layer of artificial intelligence that makes the footage searchable and actionable, and do it all on a budget that wouldn’t break the bank**. Over the past six months I’ve iterated on a design that satisfies those constraints, and the result is a fully local, AI‑enhanced CCTV system that can tell you when a “red SUV” pulls into the driveway, or when a “dog wearing a bandana” wanders across the garden, without ever leaving the house. -The market is flooded with “smart” cameras that promise motion alerts, facial recognition, and cloud‑based video archives. Those devices are convenient, but they also hand over a valuable slice of your privacy to a third‑party. A compromised account or a mis‑configured API key can expose weeks of footage to the world. For a home that already has a smart‑home hub, a voice assistant, and a handful of IoT devices, adding another internet‑facing endpoint feels like inviting a stranger into the living room. +--- -By keeping everything inside the home network we gain three things: +### The Core Software – Frigate -1. **Data sovereignty** – the video never leaves the LAN, so only the people you trust can view it. -2. **Predictable costs** – no monthly subscription for storage or AI inference; the only expense is the hardware you already own. -3. **Full customisation** – we can decide exactly how alerts are generated, what language they use, and when they are sent. +At the heart of the system sits **Frigate**, an open‑source network video recorder (NVR) that runs in containers and is configured entirely via a single YAML file. The simplicity of the configuration is a breath of fresh air compared with the sprawling JSON or proprietary GUIs of many commercial solutions. A few key reasons Frigate became the obvious choice: -The only downside is the extra effort required to stitch the pieces together, but that’s where the fun begins. +| Feature | Why It Matters | +|---------|----------------| +| **Container‑native** | Deploys cleanly on Docker, Kubernetes, or a lightweight LXC. No host‑level dependencies to wrestle with. | +| **YAML‑driven** | Human‑readable, version‑controlled, and easy to replicate across test environments. | +| **Built‑in object detection** | Supports car, person, animal, and motorbike detection out of the box, with the ability to plug in custom models. | +| **Extensible APIs** | Exposes detection events, snapshots, and stream metadata for downstream automation tools. | +| **GenAI integration** | Recent addition that lets you forward snapshots to a local LLM (via Ollama) for semantic enrichment. | -### Choosing the Cameras +The documentation is thorough, and the community is active enough that most stumbling blocks are resolved within a few forum posts. Because the entire system is defined in a single YAML file, I can spin up a fresh test instance in minutes, tweak a camera’s FFmpeg options, and see the impact without rebuilding the whole stack. -I needed cameras that were cheap enough to experiment with, yet offered enough features to make the system worthwhile. A Black Friday deal on Amazon AU landed me three (soon to be four) TP‑Link Vigi C540 units for AUD 50 each. For the price they are surprisingly capable: +--- -* **Pan/tilt** – allows a single unit to cover a wide field of view without additional hardware. -* **On‑board human detection** – the camera can flag a person before any external processing, reducing false positives. -* **RTSP support** – the standard streaming protocol that Frigate expects, meaning we are not locked into a proprietary API. +### Choosing the Cameras – TP‑Link Vigi C540 -The cameras lack optical zoom, but the 1080p stream is more than enough for a backyard or front‑gate view. Their built‑in motion logic also gives us a first line of defence against unnecessary processing on the NVR. +A surveillance system is only as good as the lenses feeding it. I needed cameras that could: -### The NVR Core: Frigate +1. Deliver a reliable RTSP stream (the lingua franca of NVRs). +2. Offer pan‑and‑tilt so a single unit can cover a larger field of view. +3. Provide on‑board human detection to reduce unnecessary bandwidth. +4. Remain affordable enough to allow for future expansion. -At the heart of the system sits **Frigate**, an open‑source network video recorder that runs as a Docker container. What makes Frigate a perfect fit for a homelab project? +The **TP‑Link Vigi C540** checked all those boxes. Purchased during a Black Friday sale for roughly AUD 50 each, the three units I started with have proven surprisingly capable: -* **Container‑native** – it can be deployed on any host that supports Docker, which aligns nicely with my Proxmox LXC setup. -* **YAML‑driven configuration** – a single, version‑controlled file defines cameras, streams, detection models, and storage policies. -* **Built‑in object detection** – Frigate can invoke a variety of detectors (OpenVINO, TensorRT, Coral) without external orchestration. -* **Extensible API** – the platform exposes webhooks and REST endpoints that other services (Home Assistant, Ollama) can consume. +- **Pan/Tilt** – Allows a single camera to sweep a driveway or front porch, reducing the number of physical devices needed. +- **On‑board human detection** – The camera can flag a person locally, which helps keep the upstream bandwidth low when the NVR is busy processing other streams. +- **RTSP output** – Perfectly compatible with Frigate’s ingest pipeline. +- **No zoom** – A minor limitation, but the field of view is wide enough for my modest property. -Spinning up a new Frigate instance is as simple as pulling the image, mounting the configuration file, and pointing it at the RTSP URLs of the cameras. The UI instantly shows live feeds, detection boxes, and a timeline of events, which made the early testing phase feel like watching a sci‑fi control room. +The cameras are wired via Ethernet, a decision driven by reliability concerns. Wireless links are prone to interference, especially when the cameras are placed near metal roofs or dense foliage. Running Ethernet required a bit of roof work (more on that later), but the resulting stable connection has paid dividends in stream consistency. -### Getting the Detection Right +--- -Frigate ships with a handful of pre‑configured detectors, but I wanted a model that could run on the modest hardware I had on hand: a Dell OptiPlex 7060 SFF rescued for $150, equipped with an Intel i5‑7500 and 16 GB of RAM. After a few experiments I settled on the **OpenVINO** backend with the **SSDLite MobileNet V2** model from the Open Model Zoo. The model is small enough to run on CPU while still delivering decent accuracy for cars, people, and animals. +### The Host Machine – A Budget Dell Workstation -A few practical notes from the deployment: +All the AI magic lives on a modest **Dell OptiPlex 7050 SFF** that I rescued for $150. Its specifications are: -* **Model download** – Frigate pulls the model from Hugging Face at first start, so the host needs occasional internet access for updates. -* **CPU utilisation** – The i5 hovers around 75 % load under continuous detection, which is acceptable for now but leaves little headroom for additional workloads. -* **Thermal considerations** – The workstation runs warm; I’ve added a low‑profile fan to keep temperatures in check. +- **CPU:** Intel i5‑7500 (4 cores, 3.4 GHz) +- **RAM:** 16 GB DDR4 +- **Storage:** 256 GB SSD for the OS and containers, 2 TB HDD for video archives +- **GPU:** Integrated Intel HD Graphics 630 (no dedicated accelerator) -The detector provides bounding boxes and confidence scores for each frame that triggers an event. Those raw detections are the raw material that later gets enriched by the generative AI layer. +Despite lacking a powerful discrete GPU, the workstation runs Frigate’s **OpenVINO**‑based SSD‑Lite MobileNet V2 detector comfortably. The model is small enough to execute on the integrated graphics, keeping inference latency low enough for real‑time alerts. CPU utilization hovers around 70‑80 % under typical load, which is high but acceptable for a home lab. The system does run warm, so I’ve added a couple of case fans to keep temperatures in the safe zone. -### Adding Generative AI with Ollama +The storage layout is intentional: the SSD hosts the OS, Docker engine, and Frigate container, ensuring fast boot and container start times. The 2 TB HDD stores raw video, detection clips, and alert snapshots. With the current retention policy (7 days of full footage, 14 days of detection clips, 30 days of alerts) the drive is comfortably sized, though I plan to monitor usage as I add more cameras. -Frigate’s newest integration allows snapshots of detections to be sent to an external **GenAI** service. I run **Ollama** locally, which hosts a variety of large language models, including vision‑language variants such as **qwen3‑vl‑4b** and **qwen3‑vl‑2b**. The workflow is straightforward: +--- -1. An object (e.g., a car) is detected by Frigate. -2. Frigate captures a still image of the frame and forwards it to Ollama via the GenAI webhook. -3. The vision‑language model analyses the image and returns a concise textual description (“Red SUV parked in the driveway”). -4. The description is stored as metadata alongside the detection event. +### Wiring It All Together – Proxmox and Docker LXC -This extra step turns a generic “car” tag into a searchable phrase. In the Frigate UI I can now type “white ute” or “red scooter” and instantly retrieve the matching clips. The semantic search capability is the most exciting part of the project because it opens the door to natural‑language queries over a video archive—a feature that commercial cloud services charge a premium for. +To keep the environment tidy and reproducible, I run the entire stack inside a **Proxmox VE** cluster. A dedicated node hosts a **Docker‑enabled LXC container** that isolates the NVR from the rest of the homelab. This approach offers several benefits: -### Home Assistant: The Glue That Makes It All Useful +- **Resource isolation** – CPU and memory limits can be applied per container, preventing a runaway process from starving other services. +- **Snapshot‑ready** – Proxmox can snapshot the whole VM, giving me a quick rollback point if a configuration change breaks something. +- **Portability** – The LXC definition can be exported and re‑imported on any other Proxmox host, making disaster recovery straightforward. -Frigate already gives us detection and enriched metadata, but to turn that into actionable alerts we need an automation engine. **Home Assistant** fills that role perfectly: +Inside the container, Docker orchestrates the Frigate service, an Ollama server (hosting the LLM models), and a lightweight reverse proxy for HTTPS termination. All traffic stays within the local network; the only external connections are occasional model downloads from Hugging Face and the occasional software update. -* **Event ingestion** – Frigate pushes detection events to Home Assistant via its native integration. -* **AI‑driven notifications** – Home Assistant receives the snapshot, forwards it to Ollama for a second pass (using a smaller model for speed), and then formats a friendly message. -* **Matrix delivery** – The final message, complete with the image and AI‑generated caption, is sent to my Matrix client on my phone. +--- -The result is a notification that reads something like: “A white ute with a surfboard in the back has just entered the driveway.” Compared with a raw “motion detected” alert, this is a massive usability upgrade. Moreover, Home Assistant can trigger other automations based on the enriched data, such as turning on porch lights when a car arrives after sunset, or silencing the alarm when the mail carrier is recognised. +### From Detection to Context – The Ollama Integration -### Storage Strategy +Frigate’s native object detection tells you *what* it sees (e.g., “person”, “car”, “dog”). To turn that into *meaningful* information, I added a **GenAI** layer using **Ollama**, a self‑hosted LLM runtime that can serve vision‑capable models locally. -Video storage is often the Achilles’ heel of a DIY CCTV system. My approach balances capacity, retention, and cost: +The workflow is as follows: -| Storage tier | Device | Purpose | Retention | -|--------------|--------|---------|-----------| -| OS & containers | 256 GB SSD | Frigate, Home Assistant, Ollama | Indefinite | -| Video archive | 2 TB HDD | Full‑resolution recordings, detection clips, alert snapshots | 7 days (full), 14 days (detections), 30 days (alerts) | +1. **Frigate detects an object** and captures a snapshot of the frame. +2. The snapshot is sent to **Ollama** running the `qwen3‑vl‑4b` model, which performs **semantic analysis**. The model returns a textual description such as “a white ute with a surfboard on the roof”. +3. Frigate stores this enriched metadata alongside the detection event. +4. When a user searches the Frigate UI for “white ute”, the system can match the description generated by the LLM, dramatically narrowing the result set. +5. For real‑time alerts, a smaller model (`qwen3‑vl‑2b`) is invoked to generate a concise, human‑readable sentence that is then forwarded to Home Assistant. -Frigate automatically prunes old files based on the policies defined in the YAML configuration, so the drive never fills up unexpectedly. With four 1080p streams at 5 fps the 2 TB budget comfortably covers the stated retention windows. +Because the LLM runs locally, there is no latency penalty associated with round‑trip internet calls, and privacy is preserved. The only external dependency is the occasional model pull from Hugging Face during the initial setup or when a newer version is released. -### Deploying on Proxmox +--- -All services run inside a dedicated LXC container on a Proxmox node that I carved out specifically for this project. The container hosts Docker, the Frigate image, Home Assistant, and Ollama. Using LXC gives me a lightweight isolation layer without the overhead of a full VM, and Proxmox’s snapshot feature lets me roll back the entire stack if a configuration change goes sideways. +### Home Assistant – The Glue That Binds -The network layout is simple: each camera connects via wired Ethernet to the same LAN as the Proxmox host, ensuring low latency and no packet loss that could otherwise cause frame drops. The container’s Docker bridge is bound to the host’s network stack, so Frigate sees the cameras as if it were running directly on the workstation. +While Frigate handles video ingestion and object detection, **Home Assistant** provides the automation backbone. By integrating Frigate’s webhook events into Home Assistant, I can: -### Challenges and What’s Next +- **Trigger notifications** via Matrix when a detection meets certain criteria. +- **Run conditional logic** to decide whether an alert is worth sending (e.g., ignore cars on the street but flag a delivery van stopping at the gate). +- **Log events** into a time‑series database for later analysis. +- **Expose the enriched metadata** to any other smart‑home component that might benefit from it (e.g., turning on porch lights when a person is detected after dark). -The system works, but there are a few rough edges that I’m actively polishing: +The Home Assistant configuration lives in its own YAML file, mirroring the philosophy of “infrastructure as code”. This makes it easy to version‑control the automation logic alongside the NVR configuration. -* **Prompt engineering** – The text prompts sent to the vision‑language models are still generic, leading to occasional mis‑classifications (e.g., a cat being described as a “small vehicle”). Fine‑tuning the prompts and possibly adding a few few‑shot examples should improve consistency. -* **Notification fatigue** – Right now every person or car that passes the driveway generates a push notification. I plan to add a lightweight decision model that evaluates the confidence score, time of day, and historical patterns before deciding whether to alert. -* **Hardware scaling** – The i5‑7500 is holding its own, but as I add more cameras or switch to a larger model (e.g., a 4‑billion‑parameter vision transformer), I’ll need a more powerful CPU or a dedicated GPU/Intel Arc accelerator. -* **Public configuration** – The deployment repository still contains secrets. Once scrubbed, I intend to publish the full YAML and Docker‑compose files so others can replicate the stack with a single `git clone`. +--- -### A Personal Note: The Unsung Hero +### Semantic Search – Finding a Needle in a Haystack -No amount of software wizardry would have gotten the Ethernet cables into the roof without a bit of elbow grease. My dad, armed with a ladder, a drill, and a healthy dose of patience, ran the cabling that links the cameras to the network. He spent an entire Saturday climbing into the attic, pulling cables through joists, and making sure each connection was solid. I was mostly handing him tools and keeping the coffee warm. Without his help the whole project would have remained a pipe‑dream. So, Dad, thank you for the hard yards, the guidance, and the occasional dad‑joke that kept the mood light while we were up on the roof. This system is as much yours as it is mine. +One of the most satisfying features of the system is the ability to **search footage using natural language**. Traditional NVRs only let you filter by timestamps or simple motion events. With the GenAI‑enhanced metadata, the search bar becomes a powerful query engine: -### Closing Thoughts +- Typing “red SUV” returns all clips where the LLM described a vehicle as red and an SUV. +- Searching “dog with a bandana” surfaces the few moments a neighbour’s pet decided to wear a fashion accessory. +- Combining terms (“white ute with surfboard”) narrows the results to a single delivery that happened last weekend. -Building an AI‑enhanced CCTV system at home is no longer a pipe‑dream reserved for large enterprises. With a modest budget, a handful of inexpensive cameras, and a stack of open‑source tools—Frigate, OpenVINO, Ollama, Home Assistant, and a bit of Proxmox magic—you can achieve a private, searchable, and context‑rich surveillance solution. The journey taught me a lot about video streaming, container orchestration, and the quirks of vision‑language models, but the biggest takeaway is the empowerment that comes from owning your data. +Under the hood, the search is a straightforward text match against the stored descriptions, but the quality of those descriptions hinges on the LLM prompts. Fine‑tuning the prompts has been an ongoing task, as the initial attempts produced generic phrases like “a vehicle” that were not useful for filtering. -If you’re curious about the nitty‑gritty—YAML snippets, Docker‑compose files, or the exact model versions—I’m happy to share once the repository is fully sanitized. Until then, enjoy the peace of mind that comes from knowing your backyard is being watched by a system that not only sees but also understands. +--- + +### Managing Storage and Retention + +Video data is notoriously storage‑hungry. To keep the system sustainable, I adopted a tiered retention policy: + +| Data Type | Retention | Approx. Size (4 cameras) | +|------------|-----------|--------------------------| +| Full video (raw RTSP) | 7 days | ~1.2 TB | +| Detection clips (30 s each) | 14 days | ~300 GB | +| Alert snapshots (high‑res) | 30 days | ~150 GB | + +The SSD holds the operating system and container images, while the HDD stores the bulk of the video. When the HDD approaches capacity, a simple cron job rotates out the oldest files, ensuring the system never runs out of space. In practice, the 2 TB drive has been more than sufficient for the current camera count, but I have a spare 4 TB drive on standby for future expansion. + +--- + +### Lessons Learned – The Good, the Bad, and the Ugly + +#### 1. **Performance Is a Balancing Act** +Running inference on an integrated GPU is feasible, but the CPU load remains high. Adding a modest NVIDIA GTX 1650 would drop CPU usage dramatically and free headroom for additional cameras or more complex models. + +#### 2. **Prompt Engineering Is Real Work** +The LLM’s output quality is directly tied to the prompt. Early attempts used a single sentence like “Describe the scene,” which resulted in vague answers. Iterating on a multi‑step prompt that asks the model to list objects, colors, and actions has produced far richer metadata. + +#### 3. **Notification Fatigue Is Real** +Initially, every detection triggered a push notification, flooding my phone with alerts for passing cars and stray cats. By adding a simple confidence threshold and a “time‑of‑day” filter in Home Assistant, I reduced noise by 80 %. + +#### 4. **Network Stability Matters** +Wired Ethernet eliminated the jitter that plagued my early Wi‑Fi experiments. The only hiccup was a mis‑wired patch panel that caused occasional packet loss; a quick audit resolved the issue. + +#### 5. **Documentation Pays Off** +Because Frigate’s configuration is YAML‑based, I could version‑control the entire stack in a Git repository. When a change broke the FFmpeg pipeline, a `git revert` restored the previous working state in minutes. + +--- + +### Future Enhancements – Where to Go From Here + +- **GPU Upgrade** – Adding a dedicated inference accelerator (e.g., an Intel Arc or NVIDIA RTX) to improve detection speed and lower CPU load. +- **Dynamic Prompt Generation** – Using a small LLM to craft context‑aware prompts based on the time of day, weather, or known events (e.g., “delivery” vs. “visitor”). +- **Smart Notification Decision Engine** – Training a lightweight classifier that decides whether an alert is worth sending, based on historical user feedback. +- **Edge‑Only Model Updates** – Caching Hugging Face models locally and scheduling updates during off‑peak hours to eliminate any internet dependency after the initial download. +- **Multi‑Camera Correlation** – Linking detections across cameras to track a moving object through the property, enabling a “follow‑the‑intruder” view. + +--- + +### A Personal Note – The Roof, the Cables, and My Dad + +All the technical wizardry would have been for naught if I hadn’t managed to get Ethernet cables from the house’s main distribution board up to the roof where the cameras sit. I’m decent with Docker, YAML, and LLM prompts, but I’m hopeless when it comes to climbing ladders and threading cables through roof joists. + +Enter my dad. He spent an entire Saturday hauling a coil of Cat‑6, drilling through the roof sheathing, and pulling the cables into the attic space while I fumbled with the tools. He didn’t care that I’d rather be writing code than wielding a hammer; he just wanted to see the project succeed. The result is a rock‑solid wired backbone that keeps the cameras streaming without hiccups. + +So, Dad, thank you. Your patience, muscle, and willingness to get your hands dirty made this whole system possible. I owe you a cold one and a promise to fix that leaky tap in the kitchen. + +--- + +### Bringing It All Together – The Architecture ```mermaid -graph TD - Camera1[Camera] --> Frigate - Camera2[Camera] --> Frigate - Camera3[Camera] --> Frigate - Camera4[Future Camera] --> Frigate - Frigate -->|Object Detections| Ollama_qwen3_vl_4b[Send snapshot to Ollama (qwen3‑vl‑4b) for semantic search AI enhancement] - Frigate --> Frigate_NVR[Frigate NVR] - Frigate_NVR --> HomeAssistant[Home Assistant] - HomeAssistant -->|Object Detection from Frigate| CopyImage[Copy Image to Home Assistant] - CopyImage --> Ollama_qwen3_vl_2b[Send image to Ollama (qwen3‑vl‑2b) for context enhancement] - Ollama_qwen3_vl_2b --> Matrix[Send response back via Matrix] -``` \ No newline at end of file +graph LR + A[Camera] --> B[Frigate Object Detections] + B --> C[Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement] + C --> D[Frigate NVR] + D --> E[Home Assistant] + E --> F[Object Detection from Frigate] + F --> G[Copy Image to Home Assistant] + G --> H[Send image to Ollama (qwen3-vl-2b) for context enhancement] + H --> I[Send response back via Matrix] + A --> J[Future Camera] +``` + +--- + +### Closing Thoughts + +Building an AI‑enhanced CCTV system from the ground up has been a rewarding blend of hardware tinkering, software orchestration, and a dash of machine‑learning experimentation. The result is a **privacy‑first, locally owned surveillance platform** that does more than just record—it understands. It can answer natural‑language queries, send context‑rich alerts, and integrate seamlessly with a broader home‑automation ecosystem. + +If you’re a hobbyist, a small‑business owner, or anyone who values data sovereignty, the stack described here offers a solid foundation. Start with a single camera, get comfortable with Frigate’s YAML configuration, and gradually layer on the AI components. Remember that the most valuable part of the journey is the learning curve: each tweak teaches you something new about video streaming, inference workloads, and the quirks of your own network. + +So, roll up your sleeves, grab a ladder (or enlist a dad), and give your home the eyes it deserves—without handing the footage over to a faceless cloud. The future of home surveillance is local, intelligent, and, most importantly, under your control. Cheers! \ No newline at end of file From e97e7f4ca9701110de38e0e27db9eaddbdd3d067 Mon Sep 17 00:00:00 2001 From: armistace Date: Tue, 3 Feb 2026 13:12:42 +1000 Subject: [PATCH 08/14] Human Edits --- ...ing_and_building_an_ai_enhanced_cctv_system_at_home.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 2c49a11..66a05f3 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -157,9 +157,9 @@ Because Frigate’s configuration is YAML‑based, I could version‑control the All the technical wizardry would have been for naught if I hadn’t managed to get Ethernet cables from the house’s main distribution board up to the roof where the cameras sit. I’m decent with Docker, YAML, and LLM prompts, but I’m hopeless when it comes to climbing ladders and threading cables through roof joists. -Enter my dad. He spent an entire Saturday hauling a coil of Cat‑6, drilling through the roof sheathing, and pulling the cables into the attic space while I fumbled with the tools. He didn’t care that I’d rather be writing code than wielding a hammer; he just wanted to see the project succeed. The result is a rock‑solid wired backbone that keeps the cameras streaming without hiccups. +Enter my dad. He spent an entire Saturday hauling a coil of Cat‑6, pulling the cables into the roof space while I fumbled with the tools. He didn’t care that I’d rather be writing code than wielding a hammer; There were apparently 4 days of pain afterwards so please know the help was truly appreciated. The result is a rock‑solid wired backbone that keeps the cameras streaming without hiccups. -So, Dad, thank you. Your patience, muscle, and willingness to get your hands dirty made this whole system possible. I owe you a cold one and a promise to fix that leaky tap in the kitchen. +Thank you, Dad. Your patience, muscle, and willingness to get your hands dirty made this whole system possible. --- @@ -168,12 +168,12 @@ So, Dad, thank you. Your patience, muscle, and willingness to get your hands dir ```mermaid graph LR A[Camera] --> B[Frigate Object Detections] - B --> C[Send snapshot to Ollama (qwen3-vl-4b) for semantic search AI enhancement] + B --> C[Send snapshot to Ollama -qwen3-vl-4b- for semantic search AI enhancement] C --> D[Frigate NVR] D --> E[Home Assistant] E --> F[Object Detection from Frigate] F --> G[Copy Image to Home Assistant] - G --> H[Send image to Ollama (qwen3-vl-2b) for context enhancement] + G --> H[Send image to Ollama -qwen3-vl-2b- for context enhancement] H --> I[Send response back via Matrix] A --> J[Future Camera] ``` From 62d4cc309e9f749295fdb79ecc1bcf4fb200f164 Mon Sep 17 00:00:00 2001 From: armistace Date: Tue, 3 Feb 2026 13:15:05 +1000 Subject: [PATCH 09/14] Further Human Edits --- ...ing_and_building_an_ai_enhanced_cctv_system_at_home.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 66a05f3..bfdff9c 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -167,15 +167,17 @@ Thank you, Dad. Your patience, muscle, and willingness to get your hands dirty m ```mermaid graph LR + A[Camera] A[Camera] --> B[Frigate Object Detections] - B --> C[Send snapshot to Ollama -qwen3-vl-4b- for semantic search AI enhancement] - C --> D[Frigate NVR] + A --> D[Frigate NVR] + D --> C[Send snapshot to Ollama -qwen3-vl-4b- for semantic search AI enhancement] D --> E[Home Assistant] E --> F[Object Detection from Frigate] F --> G[Copy Image to Home Assistant] G --> H[Send image to Ollama -qwen3-vl-2b- for context enhancement] H --> I[Send response back via Matrix] - A --> J[Future Camera] + A[Camera] + A[Future Camera] ``` --- From 2c6d90417f7c642d878595886ac5da8fc973eb0e Mon Sep 17 00:00:00 2001 From: armistace Date: Tue, 3 Feb 2026 13:16:00 +1000 Subject: [PATCH 10/14] Further Human Edits --- ...gning_and_building_an_ai_enhanced_cctv_system_at_home.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index bfdff9c..5f001c8 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -167,7 +167,7 @@ Thank you, Dad. Your patience, muscle, and willingness to get your hands dirty m ```mermaid graph LR - A[Camera] + I[Camera] A[Camera] --> B[Frigate Object Detections] A --> D[Frigate NVR] D --> C[Send snapshot to Ollama -qwen3-vl-4b- for semantic search AI enhancement] @@ -176,8 +176,8 @@ graph LR F --> G[Copy Image to Home Assistant] G --> H[Send image to Ollama -qwen3-vl-2b- for context enhancement] H --> I[Send response back via Matrix] - A[Camera] - A[Future Camera] + J[Camera] + K[Future Camera] ``` --- From aca69adf4cb8b7e6bc5bd61f233950f717245cdf Mon Sep 17 00:00:00 2001 From: armistace Date: Tue, 3 Feb 2026 13:17:11 +1000 Subject: [PATCH 11/14] Further Human Edits --- ...gning_and_building_an_ai_enhanced_cctv_system_at_home.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 5f001c8..065b499 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -168,9 +168,9 @@ Thank you, Dad. Your patience, muscle, and willingness to get your hands dirty m ```mermaid graph LR I[Camera] - A[Camera] --> B[Frigate Object Detections] - A --> D[Frigate NVR] - D --> C[Send snapshot to Ollama -qwen3-vl-4b- for semantic search AI enhancement] + A[Camera] --> D[Frigate NVR] + D --> B[Frigate Object Detections] + B --> C[Send snapshot to Ollama -qwen3-vl-4b- for semantic search AI enhancement] D --> E[Home Assistant] E --> F[Object Detection from Frigate] F --> G[Copy Image to Home Assistant] From 5d98d5587666f471e687b941af4526d901edacfe Mon Sep 17 00:00:00 2001 From: armistace Date: Tue, 3 Feb 2026 13:18:25 +1000 Subject: [PATCH 12/14] Further --- ...gning_and_building_an_ai_enhanced_cctv_system_at_home.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 065b499..590c66b 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -167,12 +167,12 @@ Thank you, Dad. Your patience, muscle, and willingness to get your hands dirty m ```mermaid graph LR - I[Camera] + L[Camera] A[Camera] --> D[Frigate NVR] D --> B[Frigate Object Detections] B --> C[Send snapshot to Ollama -qwen3-vl-4b- for semantic search AI enhancement] - D --> E[Home Assistant] - E --> F[Object Detection from Frigate] + D --> E[Home Assistant - MQTT -] + E --> F[- MQTT - Object Detection from Frigate] F --> G[Copy Image to Home Assistant] G --> H[Send image to Ollama -qwen3-vl-2b- for context enhancement] H --> I[Send response back via Matrix] From 688e9b223bae8a823535ee9d601bd61a1fa42d69 Mon Sep 17 00:00:00 2001 From: armistace Date: Tue, 3 Feb 2026 13:19:16 +1000 Subject: [PATCH 13/14] Further Human Edits --- ...signing_and_building_an_ai_enhanced_cctv_system_at_home.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 590c66b..4da8e62 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -171,8 +171,8 @@ graph LR A[Camera] --> D[Frigate NVR] D --> B[Frigate Object Detections] B --> C[Send snapshot to Ollama -qwen3-vl-4b- for semantic search AI enhancement] - D --> E[Home Assistant - MQTT -] - E --> F[- MQTT - Object Detection from Frigate] + D --> E[Home Assistant -MQTT- ] + E --> F[ -MQTT- Object Detection from Frigate] F --> G[Copy Image to Home Assistant] G --> H[Send image to Ollama -qwen3-vl-2b- for context enhancement] H --> I[Send response back via Matrix] From 10cc1d3836dab4dfd3d45a6dd2abb2dfff10e382 Mon Sep 17 00:00:00 2001 From: armistace Date: Tue, 3 Feb 2026 13:23:50 +1000 Subject: [PATCH 14/14] Add required metadata --- ..._and_building_an_ai_enhanced_cctv_system_at_home.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md index 4da8e62..370b8c9 100644 --- a/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md +++ b/src/content/designing_and_building_an_ai_enhanced_cctv_system_at_home.md @@ -1,3 +1,13 @@ +Title: Designing and Building an AI Enhanced CCTV System +Date: 2026-02-02 20:00 +Modified: 2026-02-03 20:00 +Category: Homelab +Tags: proxmox, hardware, self host, homelab +Slug: ai-enhanced-cctv +Authors: Andrew Ridgway +Summary: Home CCTV Security has become a bastion cloud subscription awfulness. This blog describes the work involved in creating your own home grown AI enhanced CCTV system. Unfortunately what you save in subscription you lose in time but if you value privacy, it's worth it. + + ### Why Build Your Own AI‑Enhanced CCTV? When you buy a consumer‑grade security camera, you’re not just paying for the lens and the plastic housing. You’re also paying for a subscription that ships every frame of your backyard to a cloud service you’ll never meet. That data can be used to train models, sold to advertisers, or handed over to authorities on a whim. For many, the convenience outweighs the privacy cost, but for anyone who values control over their own footage, the trade‑off feels unacceptable.