Why? or The Buildout Phase 1

Over the past year working with Hyper Convergence has captured all my attention and imagination. My experience with hardware at a large level started about 18 years ago. The job was simple build all new computers for customers, repair and support all of them, recondition used computers, provide support on those, lead sales person, and finally train all employees. After a year, at the age of 16, I must have worked on, built, supported, and sold well over a couple hundred systems. I was making $6.50 hr, and was literately responsible for everything. The owner just sat in his office watching TV, to his credit he was an extremely nice person.

But i digress… The point of this is Hyper Convergence and after what you just read, you can easily assume there hasn’t been much real interest hardware projects for a decade now. Yet, Hyper Convergence makes it fun again. The only problem is cost of such toys, heck the cost of fully setup 100+ VM infrastructure for a Small business can well exceed 50k. The benefits to having such an infrastructure is a no-brainer, but we live in a time where SB owners still treat Tech as an investment, and not a utility. So the very notion of lets say 30k in hardware, 20k+ in man hours, and ongoing support costs is a non-starter.

I asked myself how do i fix problems otherwise? I take it to the Lab. This sparked the idea to create the cheapest viable, feature rich Vsphere Hyper Converged lab deployment.

With that said I will be documenting as much of the process as humanly possible. Providing images of build to screen shots and quasi how-to documentation. I know blogs prove very useful for Vsphere information and setups. Hopefully i get most of it right, and people can use this blog as a resource. My writing ability will easily bring home the “shows improvement” award.

Build Parameters:

  • Must be reasonably cheap
  • Must be full featured
  • Four ESXi hosts
  • Supports VSAN
  • Uses SAS HBA on HCL
  • VSAN support
  • Independent OOB network
  • fully self-contained (one power intput and internet link)
  • IPMI
  • Full featured Firewall/Router many GIGe ports + extensive vlan / VPN support
  • Layer 2+ Switching
  • Two magnetic disks and one SSD (Consumer)
  • All items available by normal online purchases
  • Four GIGe Ports per host
  • Over 16GB per host max memory capacity
  • Self-Contained independent rack/enclosure
  • Battery Backup with Network support
  • Power Distribution Unit with SSH+Network support
  • Secondary management LAN/Switching
  • Console Server for Terminal/Console to device serial connection
  • Be power friendly
  • Office friendly noise level (kill noise wherever whenever possible using creative methods)
  • No hanging chads!

Build out (1/2/14) Amazon+Newegg+Ebay

::: Server Hardware :::

  • 4x 1U Supermicro CSE-512L-260B Chassis (Black)

  • 4x Supermicro Ultra ATX DDR3 1600 LGA 1150 Motherboards X10SL7-F-O
    LSI 2308 HBA SAS
    8x SAS 6GB Ports

  • 4x Intel Xeon E3-1230V3 Haswell, 3.3GHz, 8MB L3 Cache LGA 1150, 80W Quad-Core Server Processor BX80646E31230V3

  • 4x Dynatron Copper Heatsink K129 For Intel Xeon3400

  • 8x Enzotech BGA Copper heatsinks
    Note: North bridge and LSI2308 chip heatsink replacement, two per host

  • 4x Crucial 16GB Kit (8GBx2) DDR3L 1600MT/s (PC3-12800) DR x8 ECC UDIMM 240-Pin Server Memory CT2KIT102472BD160B
    Note: Max is 32GB per host. Gradual upgrade to 32GB per host

  • 4x Intel DC S3700 Series Taylorsville 100GB SSD (SSDSC2BA100G301)
    Note: VSAN HCL listed, vsan SSD medium

  • 8x HGST Travelstar 7K1000 2.5-Inch 1TB 7200 RPM SATA III 32MB Cache Internal Hard Drive 0J22423
    Note: Two per host vsan magnetic storage

  • 4x Intel 320 Series 120GB SSD
    Note: ESXi installation, backed by iSCSI rsynced lun target

  • 4x IBM PRO/1000 PT DP Server Adpater (39Y6126)

  • 4x Supermicro Riser Card RSC-RR1U-E16

  • 4x HDPLEX 160W DC-ATX Power Supply
    Note: Silent power supply for each host

  • 4x HDPLEX Internal 120W AC-DC adapter
    Note: Internal AC adapter for each host

  • 8x EZ-Fit Lite MB290SP-B Drive Bay Adapter

::: Networking :::

  • SonicWall NSA 240

  • ZyXel 24-Port GbE L2 Switch (GS221024)

  • HP 1810-8G 8-Port Managed GIGe Layer 2 switch (J9802A)

  • 1400-8G 8-Port Unmanged GIGe Switch (J9077A#ABA)

  • Lantronix EDS2100 Hybrid Ethernet Terminal and Multiport Device Server

  • Lantronix UDS2100 Ethernet Serial Device Server

  • Lantronix Spider KVM-over-IP

  • 100+ Coboc Colored Cat 6 550MHz UTP Flat Ethernet

::: Power :::

  • 1U CyberPower OR1500LCDRM1U Smart App LCD UPS 1500VA 900W

  • CyberPower RMCARD202 Remote Management Card – SNMP/HTTP

  • 1U APC AP7900 Rack PDU/Switched/1U/15A/100/120V Surge Protector

  • 1U CyberPower PDU20BT6F8R Basic PDU 14-Outlets

  • Power Adapter Cord 20A, 12AWG (NEMA-L5-20R to NEMA-5-15P) 6-in

  • 10x 13A 16AWG Outlet Extension Power Cable

  • 4x Universal AC Adapter 70W w/tips

::: Rack Parts :::

  • 12U StarTech Office Server Cabinet with Wood Finish RKWOODCAB12

  • 4x Supermicro Kit (CSE-PT8L) Rails for 1U servers

  • 2x 1U Server Shelf Vented Shelves Rack Mount

  • 4x Dell 1U Cable Management Arm Kit NN006
    Note: Requires modification (drilled holes in wood rack for rear connection, zip ties for chassis connect)

  • 1U Universal Rack Shelf Rails

  • 1U Rack Mount Cable Manager Intellinet 169950

  • PUREX Technology 12 Ports Patch Panel Face Cover for keystone jack PXP-12KB
    shelf + keystone panel + keystones + zip ties = Makeshift rack enclosure for Sonicwall.

  • 10x Cable Matters Cat6 RJ45 Keystone Jack

  • Shaxon Keystone Blank Insert White 10pc

  • Menotek F-Pin (Coax) Connector Keystone Module

  • Quad Shield RG-6 Coax Cable 1ft

  • Server Rack Cabinet M6 Cage Nuts & Mounting Screws 50pc

::: Misc :::

  • 4x Noctua NF-B9 PWM Cooling Fan
    Note: < 20db via low noise adapters, rack exhaust

  • Noctua NA-SRC7 Accessory 4-pin Low-Noise Adaptors

  • 2x CargoLoc 32326 Nylon Cable Ties 650pc
    Note: No man should have such power!

  • VELCRO ONE-WRAP Thin Self-Gripping Cable Ties

  • Plastic Wire Cord Cable Clip Organizer 16pc