Today Dennis and I presented our Shoebox sized datacenter at the Italian VMUG Conference. You can find the slides of our presentation today in Italy here. We previously presented this during the Dutch VMUG Conference. You can find a post about the hardware here. I also wrote about the software we used and the performance results we achieved.
In the previous presentation we were using an old NAS devices. That box was too heavy to take on a plane, didn’t perform very well and, most importantly, was very unstable. So we decided to replace the Thecus NAS box with a self build NAS device for our presentation at VMUGIT. Since we wanted it to be in the same form factor as the other machines we decided to use another NUC board. This time with an i3 processor. We put 16GB of RAM in the board so we have a nice big RAM cache in the NAS. Then we had to solve the problem of how we were going to connect at least 3 drives to this board.
Luckily I found an ideal solution. Or so I thought. I found a mini PCIe SATA card with 2 ports. The board also has one onboard SATA port so this would enable us to connect 3 drives. But unfortunately the card seemed to be broken. After contacting the supplier they were quick to send a new one. But it still didn’t work. Somehow the card is incompatible with the Intel NUC board. I don’t know why. So my next move was to order a cheap SATA port multiplier. Which turned out to be incompatible with the onboad SATA port…..
Finally I decided to order two USB3-SATA adapters. I was afraid of high CPU load and other problems but in reality it all works surprisingly well. So currently the board has one disk connected to the local SATA port and two disks connected over USB3. The board runs CentOS 6.5 which is booted from USB pen drive. The three disks are in software RAID 5. The linux IET (iSCSI enterprise target) presents this RAID group to the ESXi servers over iSCSI. So no fancy NAS software, just plain linux + mdadm + tgtd.
The whole package is “wrapped” in the same 3d printed rack we used for the compute nodes and this is what it looks like:
The whole “datacenter” is now a lot more quiet, stable as a rock, and light as a feather :). Also the worst case performance of the NAS device is almost 5 times better then the clunky old NAS box. We also upgraded the PernixData FVP software one the ESXi nodes from 1.5 beta to 1.5 GA. This improved the accelerated performance even further. I’ll write another blog post about the actual performance figures. Here’s a picture of the whole Shoebox DC v2.0 to enjoy in the meantime.
If you want to build your own then consider using a board with more onboard SATA ports. But if you really want to give it a try, here are the 3D models I created. Warning: Some dremel action might be needed to make everything fit.
- NAS Base plate (With slots for max 5 disks)
- NAS Support (Goes on top of the sides linked below. It’s the top of the disk enclosure)
- Disk enclosure Sides (Same as compute nodes. Use support as top for whole rack)
- NAS NUC sides (holds the NUC board, sits on top of disk enclosure)
Once again, no warranty, this was designed in a hurry. Make sure you have a dremel handy to make it all fit :).