My VMware ESXi Home Lab Upgrade

My VMware ESXi Home Lab Upgrade

Although the focus in my career right now is certainly more cloud focused in Amazon Web Services and Azure, I still use my home lab a lot.

For the last 5+ years my home lab had consisted of using 3x Intel NUC’s (i5 DC53427HYE), a Synology NAS for shared storage and an HP ProCurve switch. This setup served me well for most of those years. It has allowed me to get many of the certifications I have, progress in my career and have fun as well.

At the start of this year I decided it was time to give the home lab an overhaul. At first I looked at the newest generation of Intel NUC’s but really wasn’t looking forward to dropping over $1,300 on just partial compute (I’d still need to be RAM for each of the 3 NUC’s). I also wanted something that just worked, no more fooling around with network adapter drivers or doing this tweak or that tweak.

I also no longer needed to be concerned about something that had a tiny footprint. I also questioned if I really needed multiple physical ESXi hosts. My home lab isn’t running anything mission critical and if I really wanted I could always build additional nested VMware ESXi hosts on one powerful machine if I needed.

So in the end, the below is what I settled on. Replacing all of my compute, most of my networking and adding more storage!

Compute

Dell R720 2u server
2x Xeon E5-2650 v2 @ 2.60Ghz (16 cores / 32 threads total)
8 x 16GB PC3-12800R Samsung RAM (128GB total)
Samsung 970 EVO 1TB M.2 SSD (primary lab storage)
Samsung 860 EVO 500GB SATA SSD (secondary lab storage)
Generic PCIe to M.2 NVMe adapter for 970 EVO
PERC H710P RAID adapter with 1GB cache and BBU
iDRAC Enterprise

vmware vcenter home lab

So while the old Intel NUC machines served me well for the last several years. They certainly don’t compare to the power of this new server. A single Xeon E5-2650 v2 CPU a far better benchmark compared to all three of the NUC’s combined and this server has dual CPU’s. CPU is certainly no a concern.

Going from having 48GB memory total to now 128GB of RAM I am no longer powering off some machines to power up others – at least not at the moment. 😉

Thanks to the new local storage, virtual machines boot up nearly instantly. The Samsung 970 EVO is INSANE!

samsung 970 evo benchmark

samsung 970 evo 1tb

Dell R720 Server

dell r720 insides

dell r720 ram covered

Storage

Synology DS1815+ with 16GB RAM
4x Seagate ST3000VN000-1 3TB HDD (RAID-10)
2x SanDisk Ultra II 240GB SSD (RAID-1 Read/Write Cache)

Synology NAS storage manager

The Synology NAS is now used to store personal documents and photo’s that are sync’ed to my multiple devices using Synology Cloud Station Drive or DS Photo – I really like that pictures from my phone are automatically sent to my Synology.

It also stores my movies for my Plex media server and serves as a local backup repository.

Synology volume

Synology SSD cache

Network

Ubiquiti EdgeRouter X
Ubiquiti UniFi 24 port switch
Ubiquiti UniFi 8 port POE switch
Ubiquiti UniFi UAP-AC-LR

All of the Ubiquiti UniFi network gear is all managed by an Ubiquiti Controller hosted on a Linode VPS.

The network has been broken up into the following VLANs:

  • VLAN 1 – Main network where my PC, vCenter, PLEX, and so on reside.
  • VLAN 10 – Guest Wifi Network.
  • VLAN 20 – Internet of Things for my thermostat and Amazon Dot (which is never use).
  • VLAN 30 – Lab network for machines I don’t want necessarily on the main network.
  • VLAN 40 – Pentest network for the penetration testing VM’s.

Unifi home lab

Yeah I know, I still have some “N/A” icons. Maybe one of these days I’ll upgrade to a Ubiquiti Security Gateway. Right now the EdgeRouter X is working just fine.

Backups

To backup the virtual machines in the lab I’m using Altaro VM Backup, which backs up the VM’s stored on the Samsung SSD’s to a folder on the Synology NAS. Altaro offers a free NFR license to vExperts as well as a free (2VM per host) license to everyone. You can download it here.

Altaro VM Backup

Synology Cloud Sync is used to backup various folders on the NAS to Backblaze B2 which is super inexpensive. Just over 400GB is being backed up to Backblaze for less than $2.00 a month.

Synology Cloud Sync

The End

I believe that pretty well sums up the new ESXi home lab.

If you have any questions or want more details around something please leave a comment below or Tweet me at @tabor_mike and I’ll be happy to talk.

This Post Has 18 Comments

  1. I have a very similar setup. I use a full Ubiquiti setup as well as a 3 node Proxmox cluster ( 3 8 core xenon desktops with 32GB each) with a Synology DS1518+ with two expansion bays as the datastore. It works out surprisingly well!

    I currently use crashplan to backup my Synology but will have to look into Backblaze since it runs natively on the Synology.

    1. Ron I think you’ll like Backblaze. It’s been very reliable for me and for my amount of storage needed has saved me money. It’s also very easy to schedule when the sync happens within the Synology UI.

  2. Van i have more Details? How do you Store it in 19“?

    1. I’ll be happy to share more details! I thought about putting this in a small rack or even building my own rack. I even went so far as bought the rails to do so but decided not to do either. Instead the server is sitting on a shelf out in my garage.

      Dell R720 on a shelf

      The garage is insulated and even has a small air conditioner. Obviously not as nice or steady as an actual server room, but should do just fine for me home lab needs.

  3. Looks like a small production setup to me except for those samsung ssd’s. Am I wrong?

  4. Nice information. Thanks for sharing

  5. Why two switches if everything is internal on a single host? Keep all of the routing internal and use NSX?

    1. Kasey, I am using two switches due to the location of the server. My main switch and all the hardwired runs go to one part of the house and the server and a couple additional IP cameras run into the garage. Thus the smaller switch out there. The two switches are connected via a Cat6 run.

  6. On the VMware compatibility site I’m seeing the R720 only supports up to ESX 6.5U2. Are you running 6.7U2 (latest at this time) without issues on the R720? I know servers typically work for quite a while as I just now got stopped out with my DL360 G5 on 6.5 and couldn’t go to 6.7. In looking at my own lab upgraded I wanted to make sure I hopefully had some longevity to continue to run the latest ESX versions.

    1. Brian, the VMware compatibility is showing what has been certified compatible. The R720 is an older unit and Dell likely has no intentions in getting it certified.

      That said, I had zero issues with installing and using ESXi 6.7 U1 on my Dell R720. One thing to keep in mind is the CPU requirements now in ESXi 6.7. In fact I went with an R720 for several reasons, one of which because most of them come with CPU’s that are allowed per the 6.7 release notes. See here for a list of no longer supported CPUs – https://docs.vmware.com/en/VMware-vSphere/6.7/rn/vsphere-esxi-vcenter-server-67-release-notes.html#compatibility

      Hope this helps!

      -Michael

  7. Mike,

    I have been looking for R720. My only concern is the power consumption and noise. How much would it cost per month if R720 is on 24/7 ?
    Appreciate if you could provide the feedback.
    Thanks,

    1. Prab, cost is going to depend on how much you pay per kilowatt in your area. When the R720 was connected to my kill-o-watt meter it was averaging around 130 watts. For my area it’s around $100 a year. This is about a $75 increase, per year, over what I was paying with my 3x Intel NUC’s.

  8. Mike, this is an awesome post. I currently have R710 & I think I need to get rid of this & getting R720 for 6.7 upgrade. however, if I want to have share storage for vmotion purposes, I have to setup a NAS right?

    I currently have R710 with freenas running for my share storage but the speed is slow & something it freezes up my vcenter. It bugs the heck out of me. What are your thoughts on share storage?

    1. Kevin,

      If you’re running multiple ESXi hosts you’ll want some kind of shared storage for sure. I have a Synology NAS that I could use but for running my home lab VM’s I really don’t use it that much anymore. All my home lab VM’s are all ran locally using the SSD drives in my single R720 server.

      -Michael

      1. You’re absolutely correct. At home, I really don’t need multiple hosts. 1 host is enough. I saw that you also have VM for Pentest. What OS or what software do you use for Pentest? I would like to learn that also. Thanks

      2. Kevin,

        I run Metasploit, Kali, Parrot, Buscador, and various Windows 7 & 10 VM’s on that network.

        -Michael

Leave a Reply