Questions on Homelab setup

Hi!

So, the beauty of a home lab is, you can adjust to exactly what you need.

I had a 24x7 media server at home that I actually P2V’d yesterday with Veeam Agent, and installed onto ESXi, I’ve had no increase in power consumption, because the server was on all the time. But the benefit? Rather than it running VMware Workstarion and having 1st class & 2nd class workloads (AKA, poor performance on workstation relative to anything running native on Windows), now it’s all equally distributed. I’m using this host for my 24x7 needs: firewall & VPN, Wifi controller, and media server, gonna move my VB365 VM onto ESXi too next weekend.

So that’s my 24x7 host, spec’d as a quad core with 16GB RAM and SSD storage.

Then I have my extra 2x hosts that I power on when I need them for my lab. These are quad core with 32GB RAM but only traditional HDDs as they’re HPE and if you use aftermarket disks with HPE the fans go constant 100% spin speed.

This way I can leverage iLO to power on my extra hosts if/when I need them, even if I’m working away from home. But this way I don’t have a dramatic power bill.

I’ve seen some great options with labs recently and there’s a great 2nd hand market to get recent generation tech for £1-2k that is powerful enough you only need the one host. But I would suggest if you consolidate and are thinking of nested ESXi/Hyper-V etc then you need to consider the power draw of that server, will it be running 24x7 and wasting a ton of power & cooling idly.

Also worth checking out are the workstation class machines from Dell/HPE etc. These are missing some features such as iDRAC normally depending on vendor, but they’re far more compact normally and can still get dual socket with hundreds of GBs of RAM and more desktop class SSD/NVMe (a lot cheaper!).

Another approach can be with lower power/compact devices such as Intel NUC(s), it’s quite common to have a 3-4 node cluster of NUCs, their main limitation being direct attached storage options and add-in card support, but I see a lot of people use a reputable brand NAS such as Synology or QNAP and present some shared storage to the nodes that way.

Another consideration will be licensing. You’re not gonna want to purchase full retail VMware licensing for a lab. But you’d either be stuck reinstalling ESXi every 60 days for a trial license, or using the free version that doesn’t support any API access for backups, cluster support etc. I’d suggest either the VMware vExpert program if you’re accepted to get NFR Licenses, alternatively there’s the VMUG Advantage program. Both scenarios give you licenses valid for a year, VMUG Advantage costs money, normally $200 a year, but there are times you can get a license at a discount. Well known VMware employee William Lam is currently organising a group purchase which should be approx $170 instead, otherwise VMware have typically done discounts around VMware World events etc. This program also includes discounts on training & exams amongst the perks. Link here: https://williamlam.com/2022/06/2022-vmug-advantage-community-group-buy.html

Hope this helps! Happy to answer any Q’s 🙂

Hello @dloseke, thank you very much for sharing your thoughts. New server are really expensive except for republished servers. 

> My work lab consists of 6 PowerEdge R610’s running ESXI 6.7 (one died, I have three more in another room not in use). 

This is a great deal for me right now. How do you deal with the Power consumption? And from experience, I will have to agree with you and I quote “However, they often come at the cost of inefficiency as well”. This is why I am sceptical…

Another comment to make, be sure that your CPU generation is supported on ESXi 7.0, the older versions of ESXi are all EoL in October. If you’re investing in a lab, you want to know that you can keep up with the latest software that your customers should be running 🙂

#Humblebrag

The current work lab…..also looks nicer than a lot of productions environments I’ve seen.  But alas...it needs some reworking as it gets updated to version 2.  The pics are a bit Veeam off topic, but that HP behind the monitor is the box running the VBR/VDRO VM’s, and there’s proxies in the ESXI cluster to keep it Veeam related.

 

That is one sweet setup!

Hey! Absolutely you can! Here’s how I did it:

You’ll want to use a ‘daily driver’ OS for the laptop I imagine, as you won’t get any guarantees of supported hardware if you ran ESXi or Hyper-V natively on it. Windows or Linux doesn’t really matter.

Depending on Windows or Linux you’ve got some options, these will be ‘type-2’ Hypervisors in that, they’ll run on top of your OS, performance won’t be as good, but I doubt you’ll see any major bottlenecks here.

If you’re using Windows, you can install VMware Workstation, OR, Hyper-V, or even something like VirtualBox. On Linux you won’t have the Hyper-V option but the other options remain.

Now, you won’t be using Veeam to interact with these, instead you’ll be using nested virtualisation to run a supported Hypervisor within Workstation/Windows 10 Hyper-V/VirtualBox. Why you might ask? Because Veeam needs to leverage APIs to interact with a host for VM processing. The two main hypervisors that Veeam supports are ESXi and Microsoft Hyper-V. Now the complicated part: You’ll need to enable exposing virtualisation capabilities to your VM within whichever type-2 hypervisor you chose (Workstation/Windows 10 Hyper-V/VirtualBox), there are different ways of doing this depending on the application you choose to use, but they’re only a google search away, or state here what you’re planning to use and we can hopefully point you in the right direction.

Once you’ve deployed a virtual hypervisor such as ESXi or Hyper-V into your type-2 hypervisor, you can then either deploy a Veeam VM within your virtual hypervisor, or within your type-2. Your only networking requirements are that you create a bridged network between your host (laptop), your other VMs, and your virtual hypervisor.

Now onto some specifics:

ESXi free edition doesn’t provide the APIs required to backup data, either use a trial, or NFR key if possible to license this, you don’t actually need vCenter, just a licensed ESXi.

Fun fact: Your laptop is more powerful than the PC I used for my lab when I was studying my VMCE, it’s a quad core with 32GB RAM and I still achieved what you’re trying to do.

So, hopefully this helps you get started, and I’ll do my best to answer any questions I can to get you going!

Yes, im going ahead with VMWare Workstation 16. :) Thank you for the link! I was able to install and set it up. Does that mean i can skip this step?

Nice.   Previously I had a set of FC switches and a small SAN.  The opportunity I have now is infract an SSD SAN. I think I might actualy jump on it the more I think about it. I don’t know that many people that have 200TB at home worth of SSD.  I’ll connect it to my home assistant to monitor power maybe and do a blog post about power consumption and why owning too much storage is less efficient than the cloud or something haha. 

My “home lab” consists currently of a standalone PowerEdge R520 hand-me-down running ESXI 7.0 U3.  It’s running a few VM’s, and it’s somewhat used as a second site.  Previously it was an R610 that recently kicked the can.  Prior to that it was an IBM x3550 M2 and a x3650.  I have a Synology DS218+ as a backup repo and am running an Extreme Networks Summit 48 Port POE switch with a Ubiquiti USG Pro for the firewall, and AC Pro AP (need to expand my wireless). VM’s include a management/utility server (runs the Ubiquiti management app), VBR 11a, VBO, Pi-Hole, and a NVR (Milestone) for my older POE camera’s (Hikvision among others).  It’s all small and really more than I need for home use.  My time an energy, when I can find it, is dedicated to my “work lab”.

My work lab consists of 6 PowerEdge R610’s running ESXI 6.7 (one died, I have three more in another room not in use).  They are using an Equallogic PS6000 for shared storage through two Dell PowerConnect 6248 switches in a stack.  I’m using an old ASA5505 for the firewall.  My lab network extends to another rack in a separate room where I perform staging of client equipment and is interconnected with an Adtran NetVanta 1534 switch. There is a HP Z520 workstation with an added RAID card and drives that is running ESXI as well.  It has 2 VM’s, one running VBR 11a to backup the lab VM’s, and the other is slated for VDRO but I haven’t had a chance to set it up.  There is an APC 3000 VA UPS running things (currently I get about 7 minutes of runtime, but that’s good enough for power blinks).  VMware and Veeam are both NFR licenses for the lab environment.  I also have a couple R410’s laying around but they’re quite light on power so they’re not used.  All VM’s are Server 2016 or Server 2019 with a full domain.  I have more infrastructure here than most of my clients, but it has been proven to be handy multiple times.

Future plans for the work lab are to upgrade the SAN to a PS6210e that is already racked, and replacing the hosts with two R720’s, and an R820 that I have (once one of the R720’s returns from a client site).  I’m also in the process of replacing two R720’s in production with R730’s which should free up two more R720’s for lab use.  The lab will be upgraded to the latest build of ESXI 7.0 U3 as well although I might setup two of the R720’s into a Hyper-V failover cluster.  I was going to do AHV (Nutanix) at one point, but it’s less common for me and I’m starting to see more customers with Hyper-V, so labbing up that would be a better use of resources for me.

Your lab is what you make it, and that is often based on what you need and what you can get or afford.  I’m a bit of a scavenger as you can probably see.  Older servers tend to still have some life in them, parts may be easier to find if you have extra cold spares, etc.  However, they often come at the cost of inefficiency as well.  Some older servers are quite power hungry and can generate significant heat.  For me, the most important items are determining what the goals of the lab environment are and building around that.  Also need to have flexibility and versatility in the lab so that you can spin up whatever it is that you want or need to run without rebuilding everything.  Many folks get away with running on older workstations, NOC’s, Raspberry Pi’s, etc.  Seems like a lot of folks like to run Proxmox.  I’ve never dealt with it, but the little bit I’ve heard, I’m not sure why I at least would when I have perfectly good VMware.  I’ve never seen Proxmox running in a production environment, so it makes no sense to me to run it.  In a true home lab though, if you just need something to run your infrastructure, it might make sense, but if I need to emulate an actual production environment, it’s best to run a similar to production as possible.

Hi Everyone,

It's me again. i am just wondering why everytime i change the IP Address I am not able to access the web Url using that IP address for IPv4 and when i set it to static.

I used Bridge as connection. However, after resetting it back to original, i am able to access it.

Holy Guacomole! you have a huge basement and great lab. My basement is all done and my wife said no way to anything technical so I have to hide laptops in different places. I had one successfully acting as a server on the floor until…. my cat Carlos decided to chase after a spider down there… :(