This commit is contained in:
@@ -7,15 +7,19 @@ Slug: proxmox-cluster-1
|
||||
Authors: Andrew Ridgway
|
||||
Summary: Upgrade from a small docker-compose style server to full proxmox server with kubernetes, LXC, and a hypervisor
|
||||
|
||||
#### A quick summary of this post by
|
||||
#### A quick summary of this post by AI
|
||||
|
||||
I'm going to use AI to summarise this post here because it ended up quite long
|
||||
I'm going to use AI to summarise this post here because it ended up quite long I've edited it ;)
|
||||
|
||||
**Summary:**
|
||||
|
||||
* You've set up LXC containers for various services like Plex, databases (PostgreSQL, MySQL, MongoDB), Nginx, and file serving, taking advantage of Proxmox's ease of use and integration with standard Linux tools.
|
||||
* You're using QEMU-based virtual machines (VMs) sparingly due to resource concerns, but have set up a simple Kubernetes cluster across three nodes (Intel NUCs) using VMs. Additionally, you have a development VM for remote coding environments.
|
||||
* Your current plans include writing about your Kubernetes setup, Gitea CI/CD pipelines, and other tools like n8n, Grafana, and Matrix.
|
||||
Quick look at some of the things I've used Proxmox fr
|
||||
|
||||
* I've set up LXC containers for various services like Plex, databases (PostgreSQL, MySQL, MongoDB), Nginx, and file serving, taking advantage of Proxmox's ease of use and integration with standard Linux tools.
|
||||
* I'm using QEMU-based virtual machines (VMs) sparingly due to resource concerns, but have set up a simple Kubernetes cluster across three nodes (Intel NUCs) using VMs. Additionally, you have a development VM for remote coding environments.
|
||||
* My current plans include writing about your Kubernetes setup, Gitea CI/CD pipelines, and other tools like n8n, Grafana, and Matrix.
|
||||
|
||||
As part of the summary it came up with this interesting idea of "follow up" I'm leaving it here as I thought it was an interesting take on what I can write about in the future
|
||||
|
||||
**Follow-up Questions:**
|
||||
|
||||
@@ -37,7 +41,11 @@ I'm going to use AI to summarise this post here because it ended up quite long
|
||||
|
||||
|
||||
## A Picture is worth a thousand words
|
||||
<INSERT PICTURE HERE OF FINAL PRODUCT>
|
||||
<img alt="Proxmox Image" height="auto" width="100%" src="{attach}/images/proxmox.jpg">
|
||||
|
||||
_Yes I know the setup is a bit hacky but it works. Below is an image of the original architecture its changed a bit but you sort of get what's going on_
|
||||
|
||||
<img alt="Proxmox Architecture" height="auto" width="100%" src="{attach}/images/Server_Initial_Architecture.png">
|
||||
|
||||
## The idea
|
||||
For some time now I have been toying with the idea of a hypervisor. Initially my thoughts were to get some old blade servers and use those. That was until someone pointed out there power requirements. Looking at specs for some of these machines the power supplies would be 600 to 800 watts, which is fine until you realise that these have redundant powersupplies and are now potentially pulling up 1.5kW of energy... I'm not made of money!
|
||||
@@ -51,7 +59,7 @@ After doing some amazing reddit research and looking at various homelab ideas fo
|
||||
|
||||
First lets define what on earth Proxmox is
|
||||
|
||||
##### Proxmox
|
||||
#### Proxmox
|
||||
Proxmox VE (Virtual Environment) is an open-source server virtualization platform that has gained significant popularity among home lab enthusiasts due to its robustness, ease of use, and impressive feature set. Here's why Proxmox stands out as a fantastic choice for homelab clusters:
|
||||
|
||||
1. **Simultaneous Management of LXC Containers and VMs:**
|
||||
@@ -74,8 +82,11 @@ Proxmox VE (Virtual Environment) is an open-source server virtualization platfor
|
||||
Proxmox VE is an incredibly useful platform for homelab clusters due to its ability to manage both LXC containers and VMs efficiently, along with its advanced features like live migration and high availability. Whether you're looking to run diverse workloads or experiment with virtualization technologies, Proxmox VE is definitely worth considering.
|
||||
|
||||
**Relevant Links:**
|
||||
|
||||
- Official Proxmox VE website: <https://www.proxmox.com/>
|
||||
|
||||
- Proxmox VE documentation: <https://pve-proxmox-community.org/>
|
||||
|
||||
- Proxmox VE forums: <https://forum.proxmox.com/>
|
||||
|
||||
I'd like to thank the mistral-nemo LLM for writing that ;)
|
||||
@@ -84,21 +95,29 @@ I'd like to thank the mistral-nemo LLM for writing that ;)
|
||||
|
||||
To start to understand proxmox we do need to focus in on one important piece, LXC's these are containers but not docker container, below I've had mistral summarise some of the differences.
|
||||
|
||||
1. **Isolation Level**:
|
||||
- LXC uses Linux's built-in features like cgroups and namespaces for containerization. This provides a high degree of isolation between containers.
|
||||
- Docker also uses these features but it adds an additional layer called the "Docker Engine" which manages many aspects of the containers, including networking, storage, etc.
|
||||
**Isolation Level**:
|
||||
|
||||
2. **System Call Filtering**:
|
||||
- LXC does not have system call filtering by default. This means that processes inside LXC containers can make any syscall available on the host.
|
||||
- Docker provides system call filtering with its "rootless" mode or using a tool like AppArmor, which restricts the capabilities of processes running in containers.
|
||||
- LXC uses Linux's built-in features like cgroups and namespaces for containerization. This provides a high degree of isolation between containers.
|
||||
|
||||
- Docker also uses these features but it adds an additional layer called the "Docker Engine" which manages many aspects of the containers, including networking, storage, etc.
|
||||
|
||||
3. **Resource Management**:
|
||||
- LXC has built-in support for cgroup hierarchy management and does not enforce strict limits by default.
|
||||
- Docker enforces strict resource limits on every container by default.
|
||||
**System Call Filtering**:
|
||||
|
||||
4. **Networking**:
|
||||
- In LXC, each container gets its own network namespace but IP addresses are shared by default. Networking is managed using traditional Linux tools like `ip` or `bridge-utils`.
|
||||
- Docker provides a custom networking model with features like user-defined networks, service discovery, and automatic swarm mode integration.
|
||||
- LXC does not have system call filtering by default. This means that processes inside LXC containers can make any syscall available on the host.
|
||||
|
||||
- Docker provides system call filtering with its "rootless" mode or using a tool like AppArmor, which restricts the capabilities of processes running in containers.
|
||||
|
||||
**Resource Management**
|
||||
|
||||
- LXC has built-in support for cgroup hierarchy management and does not enforce strict limits by default.
|
||||
|
||||
- Docker enforces strict resource limits on every container by default.
|
||||
|
||||
**Networking**:
|
||||
|
||||
- In LXC, each container gets its own network namespace but IP addresses are shared by default. Networking is managed using traditional Linux tools like `ip` or `bridge-utils`.
|
||||
|
||||
- Docker provides a custom networking model with features like user-defined networks, service discovery, and automatic swarm mode integration.
|
||||
|
||||
What LXC is Focused On:
|
||||
|
||||
@@ -114,7 +133,7 @@ Given these differences, here's what LXC primarily focuses on:
|
||||
|
||||
So, while Docker provides a more polished and feature-rich container ecosystem, LXC offers a simple, lightweight, and flexible alternative for those who prefer to have more direct control over their containers and prefer using standard Linux tools.
|
||||
|
||||
Ever since I discovered Proxmox LXC containers, my server management has been a breeze. For my Plex setup, it's perfect - isolating each instance and keeping resourceo in check but by using device loading I can get a graphics card there for some sweet sweet hardware decoding. Same goes for my databases; PostgreSQL, MySQL, and MongoDB all run smoothly as individual LXCs. Nginx, too, has found its home here, handling reverse proxy duties without breaking a sweat. And for fileservering, what could be better than having a dedicated LXC for that? It's like having my own little server farm right at my fingertips!
|
||||
Ever since I discovered Proxmox LXC containers, my server management has been a breeze. For my Plex setup, it's perfect - isolating each instance and keeping resources in check but by using device loading I can get a graphics card there for some sweet sweet hardware decoding. Same goes for my databases; PostgreSQL, MySQL, and MongoDB all run smoothly as individual LXCs. Nginx, too, has found its home here, handling reverse proxy duties without breaking a sweat. And for fileservering, what could be better than having a dedicated LXC for that? It's like having my own little server farm right at my fingertips!
|
||||
|
||||
The LXC's have also been super easy to set up with the help of ttecks helper scripts [Proxmox Helper Scripts](https://community-scripts.github.io/Proxmox/) It was very sad to hear he had gotten [sick](https://www.reddit.com/r/Proxmox/comments/1gk19gm/ttecks_proxmoxve_helper_scripts_changes/) and I realy hope he gets well soon!
|
||||
|
||||
|
||||
Reference in New Issue
Block a user