Homelab

In the IT industry, it is not uncommon to operate a home lab. The motives may differ here: some like to host their content themselves, some others strive for certifications and would like to provide the necessary infrastructure. I myself have been running homelabs for over 10 years – an overview:

Jahre Function Type Operating system CPU RAM
2008-2010 Hypervisor HP ProLiant DL380 G3 ESX 3.5 2x Intel Xeon @ 3.2 Ghz 6 GB
Gateway Pentium Pro CRUX Linux Intel Pentium Pro @ 200 Mhz 96 MB
Web server IBM ThinkCentre S50 Debian Lenny Intel Pentium 4 @ 2.8 Ghz 512 MB
NAS DIY Debian Sarge AMD Athlon 64 X2 4200+ @ 2.2 Ghz 2 GB
Firewall IBM NetVista 8364-EXX IPCop 1.4, 1.9 Intel Pentium MMX @ 266 Mhz 256 MB
Firewall IBM ThinkCentre S50 Intel Celeron D @ 2.4 Ghz 512 MB
2010-2011 Hypervisor HP ProLiant DL140 G1 ESXi 3.5 2x Intel Xeon @ 2.4 Ghz 4 GB
NAS QNAP TS-509 Pro Linux Intel Celeron @ 1.6 Ghz 1 GB
Gateway ALIX.3C3 Debian Squeeze AMD Geode LX800 @ 500 Mhz 256 MB
Firewall ALIX.2D13 IPCop 2.x AMD Geode LX800 @ 500 Mhz 256 MB
2011-2012 Hypervisor DIY ESXi 4.1 Phenom X6 1090T @ 3.2 Ghz 16 GB
NAS DIY Debian Lenny AMD Athlon 64 X2 4200+ @ 2.2 Ghz 2 GB
2012 Hypervisor HP ProLiant MicroServer G7 N36L ESXi 5.0 AMD N36L @ 1.3 Ghz 8 GB
NAS HP ProLiant MicroServer G7 N36L CentOS 6 AMD N36L @ 1.3 Ghz 2 GB
2013-2014 Hypervisor HP ProLiant MicroServer G7 N40L ESXi 5.1/5.5 AMD N40L @ 1.5 Ghz 16 GB
Webserver Raspberry Pi B Debian Wheezy BCM2835 @ 700 Mhz 512 MB
2014-2016 Hypervisor / Nested NAS DIY ESXi 6.0, CentOS Intel Xeon E3-1230 @ 3.2 Ghz 32 GB
Web server Raspberry Pi 2 B CentOS 7 BCM2836 @ 900 Mhz 1 GB
2016-2017 Hypervisor DIY vSAN cluster ESXi 6.x 2x Intel i3-4360T @ 3.2 Ghz 64 GB
Firewall APU.1D4 IPFire 2.x AMD G T40E @ 1 Ghz 4 GB
2017-2019 Hypervisor DIY vSAN all-flash cluster ESXi 6.5 2x Intel Xeon D-1518 @ 2.2 Ghz 196 GB
NAS HP ProLiant MicroServer G8 CentOS 7.x Intel Xeon E3-1220Lv2 16 GB
More information and pictures can be found in the hardware history

In 2020, I radically reduced my Homelab – it now consists only of the following components:

Switch cabinet with Raspberry Pi and switch
  • a vServer at Hetzner
  • an HP ProLiant Microserver G8-NAS with 12 TB
  • a backup DAS with 12 TB
  • a FRITZ!Box
  • numerous Raspberry Pis for experiments
  • several ESP32 microcontrollers
  • an APC Smart-UPS 750 UPS to prevent data loss due to power failures
  • various Shelly Plug S to control and measure power consumption

Less is more

Compared to the last homelab- update in 2017, my current setup is significantly more minimalistic – some might argue that it doesn’t even live up to the term “homelab” anymore. The reason for this is that my professional background has changed. Whereas I used to be heavily involved in hardware and VMware-related topics, I now work in the Infrastructure as Code environment, where hardware and hypervisors are of secondary interest.

Beyond that, I also clearly ran out of steam when it came to troubleshooting hardware problems. Checking RAM modules, replacing SSDs worn out faster by VSAN, and restoring orphaned VMs – all of this is not much fun when you have already worked at least 8 hours. The administrative burden was not commensurate with the cost and benefit; and since I no longer serve such issues full-time anyway – why should I do it privately?

Instead of multiple hypervisors and two dozen VMs, I now simply run containers per application. Since the number of hand-picked and required services is manageable, this also reduces the administrative overhead. Hypervisors and dedicated Linux patch management are no longer needed. Required applications are run on the 24/7 running NAS and a Raspberry Pi. Configuration of hosts and applications is done with Ansible.

Raspberry Pi clusters

Various Raspberry Pis serve as a playground for tinkering projects – for example, I built two small Kubernetes clusters to familiarize myself with the topic. A discarded Pi 2 serves as a playback device on the TV. A Pi 4 with 8 GB serves as an ESXi hypervisor for further testing purposes.

After my DIY IPFire router died after not even 4 years – probably due to continuous operation – I decided not to buy a successor. This eliminates another level of complexity in the network – but also a DMZ and VPN access. Without further ado, I configured the latter on the FRITZ!Box. A DMZ is no longer needed, as I no longer host any productive services myself. Things that need to be available are run on a vServer at Hetzner. Past experience has shown that hardware and Internet always fail when you are currently relying on a self-hosted service and are not at home. The switch previously purchased for VSAN with 10 Gbit uplinks was replaced with a simple (unmanaged) silent Gbit switch.

Homelab setup sketch

NAS & DAS

HP ProLiant MicroServer Gen8
HP ProLiant MicroServer Gen8

I use an HP ProLiant MicroServer Gen8 with the following key data as my central data grave:

  • Intel Xeon E3-1220Lv2 processor (4×2.3 GHz, 3M cache, 17W TDP)
  • 16 GB ECC DDR3 memory
  • 250 GB Samsung EVO 850-SSD for applications and container
  • 3×6 TB SATA hard drives (Seagate Ironwolf) in RAID-5 mode

Data exchange between the clients is done via Samba, transfers of encrypted data are possible with the maximum of the 1G network of about 125 MB/s thanks to the AES-NI capable CPU.

For containers and some applications, an SSD was retrofitted via the DVD SATA port. Since HP unnecessarily limits the MicroServer’s firmware, the boot loader has to be swapped out on an SD card, since booting from the SSD is not possible. CentOS 7 is used as the operating system. The data is backed up weekly via a DAS (Lian-Li EX-503) with 12 TB in a RAID-5 array. An offsite backup via an external hard drive is done sporadically.

Container

One container is run per application (the only exception is Icinga2 monitoring) – on the NAS under Docker CE, on a Raspberry Pi Podman is running:

  • Pi-hole – Suppression of advertising and tracking
  • Eclipse Mosquitto – Microcontroller communication via MQTT
  • joke_api – Flat jokes and movie quotes as a Service for a DIY voice-assistant
  • Node-RED – Home automation
  • InfluxDB – Time-series database
  • Grafana – Visualizing InfluxDB data

Arduino, ESP32, etc.

Some time ago I started to work with Arduino and ESP32 microcontrollers. One of the first exercises was to read out various sensors, such as temperature or humidity sensors. In combination with MQTT and Node-RED, this allows for a quick thermal overview of the apartment. In combination with a time series database, such as InfluxDB, developments and trends can be observed.

Node-RED Intent-Handler

With Rhasspy, I have found a software that can be used to create an extensible offline voice-assistant. This can be extended independently with functions and commands – without the need for a cloud service. This is where Node-RED comes into play again – it serves as a central point of contact and executes recognized commands.

I also created a post series and a talk (video) about building the voice-assistant.

Conclusion

There is always potential for optimization. For example, the NAS is starting to age on the hardware side – but as long as there is no defect, I don’t want to replace it. As a successor device, I could imagine an Intel Xeon D homebrew with a more modern operating system. The individual containers could be managed with Kubernetes instead of docker-compose, even though this adds another layer of complexity.

Sharing is caring