Building Another NAS.
A lot has changed in my life over the last 6 months or so, moving out of my flat and quitting my job to travel the world only to have 2020 suck all meaning out of every fibre of my being making me yearn for the eventuality of Earth being obliterated into inexistence, but until that time my storage as of late left much to be desired.
I have been reminded, daily, by my Macbook that my last backup was almost 1 year ago and on top of this the storage I use at home is nearing it's capacity as well as being degraded. I've been overlooking these things for quite some time but I have finally decided to remedy the situation.
Join me on my newest endeavour to build a new NAS, or don't of course.
Didn't You Already Build That Chonky Storage Server That One Time?
The Best NAS (Network Attached Storage) Devices for 2019/ 2020 'The Asustor AS5304T is a powerful four-bay NAS device equipped with top-notch hardware and a generous catalog of apps. 5.0 out of 5 stars The best prosumer router now available, but overkill for less active networks. Reviewed in the United States on February 9, 2016. A LAG-ethernet connected NAS unit for all local backups, six Squeezeboxes (wired and wireless) served by LMS locally, five Apple TVs, two network-managed cable boxes, several other smart TVs. The best ah-ha moment for me was when my 12 yo called me last year saying that his school bus arrived early (which does sometimes happen) and he missed it but had left the house at his normal time. I quickly looked up the time his phone left our house network and he had left the house 15 minutes late.
It is a recommended NAS and best budget NAS for programmer and developers because it provides number of such applications. Plus it supports third-party apps like WordPress, joomla etc. Some people need a lot of storage, Netgear ReadyNas RN214 is a high storage device that can support up to 32 TB of storage. Nasdaq 100 is a growth index - you can participate in the best success stories Nasdaq 100 offers good liquidity - some of the NDX components were the most valuable in the world at one time or another (Apple, Microsoft, and Amazon) Nasdaq 100 offers better relative performance than.
Yes. I actually had over 200TB of usable storage in that server after a while, however, one month it rained so much that the garage it was housed in flooded, wiping out the power to the entire rack.
That was well over a year ago, and I have since not moved or touched any of it, instead deciding to downsize heavily and rely on colo.
Here is a photo I had of the state of things before it all went took a swim, RIP Muffin's Lab.
The Why.
After the above happened, I made sure I had enough storage elsewhere to get by on, which I have been doing since. The server is another home grown machine I built here, but I've decided to retire it and build something new and build for purpose.
This new server will be storing personal data such as photos and other crucial things such as backups and NVR footage. The existing data will be transferred over as you will see later in the post. I also plan on using FreeNAS's built-in rclone GUI to push the data to the cloud as part of my backup strategy, as well as pushing ZFS snapshots to another storage server, but that will be in another post.
The How.
Items marked N/A without a price were already in my inventory and are thus being reused.
SuperMicro X11SSL-F MicroATX
Found this on eBay for a steal really, these types of ‘server grade' motherboards are always pretty expensive in the UK, so getting something moderately modern with support for ECC and the likes was a no-brainer.
I like using motherboards like this because you get what you need and nothing more. No wanky ‘5.1 surround audio' tacked to it, no fucking RGB, this is meant to be used in a server. Plus it's got IPMI so that's always handy.
Now, I don't think that you shouldn't use wanky gaming/consumer motherboards if you have to, but if you can use something like this, then you should be.
Intel Core i3-7100T
The motherboard supports E3-1200 v6/v5, 6th/7th Gen Core i3 processors along with Celeron/Pentium lines. Realistically any processor would be fine for this. Modern processors are so damn efficient that even with heavy encryption and full gigabit throughput I don't think you'd max out any of the available options in most real-world, home/prosumer applications.
I ended up getting the best deal I could find and not necessarily the cheapest option, which ended up being a 7100T. The T variant is more desirable for me as they have a lower TDP, not that this chip will ever be pushed to its maximum TDP doing what it's being tasked with.
Core i3s have become amazing CPUs, with decent clock speeds, hyperthreading and ECC support, these are, imo, the best chips you can get for small servers like this, hell I even used one in post, Building a compact, quiet, low powered ESXi/Storage Whitebox Hybrid.
32GB (2x16GB) Samsung B-die DDR4 ECC Unbuffered
It's memory, it's ECC, I got it for cheap, cool.
be quiet! Pure Rock Slim
I really like be quiet!'s products. In my experience they make really well built, reliable and, believe it or not, quiet equipement. Oh and it was cheap.
PERC H200
Cheapest card I could find that can be flashed with HBA firmware.
2x Mini SAS 36pin SFF-8087 to 4 x SFF-8482 SATA
So, I actually needed this as some of my disks are SAS disks and without a backplane you need these connectors, you'll see in the pictures further down.
2x 4xSATA Power Splitter Adapter Cable
To power the disks, again you will see this below.
Define XL R2
I've owned this case for a long time before I really got into homelabbing and servers. I do believe it was my very first NAS. The guts eventually got transferred into a rackmount case and it was unused for many years, but I really love Fractal stuff and as is perfect for this build.
Back in the day I actually added an additional drive cage to it, bringing it to 12x 3.5″ internal which is nice, I'll only be using 8x slots for now but it's good to know I can slap 4 more disks in with relative ease, I won't even need another HBA as the motherboard has sufficient onboard SATA connectors.
As luck would have it I have some pictures of it in the first flat I moved into with my girlfriend at ripe old age of 17, when I still had hopes and dreams.
Looking at these pictures I was rather confused where that PSU went, as the one currently in the case is not that be quiet! unit, but then I remembered this was transplanted into its rackmount replacement, and then transplanted again into its rackmount replacement a few years down the line in this post.
Corsair CX430M PSU
This PSU was already in the case when I came back to it. Is it the best PSU? Probably not. Will it do the job? Most likely. If it does decide to burst into flames, all I ask is it (quickly) take me with it.
The Disks
I have a lot of disks. I mean a lot. Well over 200TB of disks as previously mentioned, the issue with these disks however is that one day, I will need to resurrect the arrays and get the data from them and, most likely, archive it to the cloud.
What I ended up doing for this is heading over to eBay and finding someone with multiple disks that was accepting offers, and with that, I managed to snag 6x HGST 4TB SAS drives. SAS drives are usually a lot cheaper to buy than SATA owing to their niche market and abundance of decoms.
This is more than enough to store the data I will need to store for myself and my family and at £12.50 per TB it's a pretty good deal, not to mention my prefered vendor for disks is HGST.
The Seagate drives are currently in the NAS running here now, there are 4 in a Z1 (yeah I know) with one failed, so 3 working. I am planning to use 2x of these to fill the chassis to 8 disks, having 1 disk ready as a replacement.
As previous, I am leaving 4x bays unpopulated for anything I may require in the future.
Fractal are pretty dope.
One issue I faced was that I did not have any screws required to install the HDDs in their caddies. I submitted a support ticket with Fractal Design and they sent me over some caddy screws. Unfortunately, there was a misunderstanding and they ended up sending me 12 screws instead of 12 sets of 4, so another ticket and a few days later they sent me over the remaining screws, great!
Big up Tim and William whosever you are from Fractal Design Support.
Another minor issue is that the case is missing one case foot, I probably could have asked for one of those too but completely forgot. It's a non-issue anyway as the case stands up fine, it just rocks if you push it down on that side, nothing some cardboard stuffed underneath can't fix.
The Build.
Looking at the case after I'd gotten it out of storage was quite sad, the old girl was pretty grimey and mould had started to set in from the flooding of the garage.
So the first thing that had to be done was clean the old girl up a bit. I used a lot of antibacterial spray and scrubbed the dirt off the best I could without disassembling the different bits of the chassis, ain't nobody got time for that.
The rest of it is pretty standard build stuff, you know how it goes. Here are some pictures.
I'm not sure how I feel about pre-applied thermal compound on new coolers, on the one hand it's handy and well spread, but on the other hand I kinda just want to use my own stuff. ¯_(ツ)_/¯.
I get that these push clippy things are on the low end because they're cheap and not really ideal for proper cooler pressure, but god damn why can't all cooler installations be this easy? Some of the shit I've had to install in my past have been too damn complicated for no obvious reason.
Although, after putting this thing on the board, there was very obvious warping happening on the board, so, pointless rant by me, clearly.
I really don't mind warping as much as a lot of people seem to, I've used a lot of warped PCBs in my day and they're pretty solid bits of kit. Once I installed the board in the case the motherboard screws seemed to ‘bed it back' as it were, so the board was straight.
Here's the case all cleaned up and ready to inplant the hardware.
You can see the 6 purchased drives here, the 2 Seagate drives destined to go into the 2 slots at the bottom will be put there after the data transfer from the current server.
Internal USB slots are pretty dope, this is where FreeNAS will be installed.
You've also probably noticed that I have yet to install any memory, the memory was due to arrive during the day so I got started anyway, I ended up receiving it around this point, and I'm always surprised how small low profile memory is.
Here's the back of the case with almost everything plugged in and routed. You can see the how the breakout cable here works along with the power splitters I bought, makes life a lot easier.
All the fans are connected to a fan controller at the front of the case. Usually I don't care very much for this but in this case it allows all 3 of the 120mm case fans to be connected to one molex, and, if I ever wanted to, lower their RPM. Best macos virtual machine. This is the only molex used in this system so it's annoying to have to use any at all but it beats running them to the motherboard.
Setting Up FreeNAS.
So all was almost done with the build so I plugged it into a monitor I keep in the corner of the office for builds and servers. Thankfully, all looked well and the system POSTs.
Unfortunately, the system was hanging on the H200's BIOS which isn't ideal, but these cards can be finicky. I always like to reflash cards I get anyway to make sure everything is updated and working properly.
After leaving to go to the shops and returning the system was complaining about no boot drive, so the system did actually boot past the H200's option ROM, it just took ages.
Knowing this I set up the FreeDOS USB I'd need to flash the card, set the USB to boot without the card plugged in, plugged the card in and waited for it to finish doing its thing. I then just did ye olde' sas2flsh in a DOS environment and got the card crossflashed to an LSI 9211-8i in IT mode with firmware P20.
At this point, I installed FreeNAS, setup some basic configuration including a RAID-Z2 of the 6 drives that were in there, and started copying the data over from the old fileserver. This took a few hours but once it was done I brought the old one into the office to start taking out bits I would need.
If you've read my previous posts, you may remember this machine from this post, when I moved out I moved my VMs elsewhere and it has since just been running FreeNAS bare metal.
I needed the 2 drives from the machine I would be taking and the 4x 2.5″ adaptor in the front. The reason I needed this is that I wanted to carry over the SSDs in that into this new build to use as a system dataset.
With FreeNAS, when using encryption, you cannot set a passphrase or recovery key if a datastore is being used as the system dataset, and as this was the only pool that's where the dataset currently resided. I did not want to run the system dataset from the USB boot drive. These SSDs are setup as a mirror pool in FreeNAS.
Once this was all done and the arrays looked like this;
PSU Woes
So after getting the pools setup and data moving I started to hear a horrible grinding noise from the PSU, giving it a good smack seemed to fix it for a bit. After looking online it appears that this is a common failure with these PSUs and a simple fan replacement would do the trick.
I decided to order a new 80+ Gold PSU from Be Quiet! to replace it with as I thought replacing the PSU with something better was the smarter choice in the long run, so the next day I had one sitting on my desk.
I ordered a new fan for the PSU and will fit it and keep the PSU as a spare for another time, I'm sure it will be put to good use once again someday.
Unfortunately, the 24 pin ATX power connector was, and I fuck you not, millimetres (very small in ‘freedom' units) too short and whilst I could just about get it to fit by routing it over the motherboard, which is what I did until an extension I ordered arrived allowing me to at long last, finish this build and put it in the comms room.
Best Prosumer Nas
Donzo.
Data moved from old NAS, Active Directory connected and permissions setup, shares configured, timemachine storage allocations done, NVR (Blueiris) recording to its mirror share, I was finally finished with this project.
It was very nice to finally see the following too, not just on my machine but on the other Macbooks in the house which is about 5 at this point.
I noticed that the Timemachine shares always use the .local domain instead of the proper FQDN of the machine, I've not looked into this yet but it doesn't seem to be an issue.
I've configured storage quotas for each machine/user for timemachine based on how large the storage of the Macbook is. Since mine has a 1TB disk, 2TB will allow me to have a nice history of files in my vault.
All that was left to do now was move it to it's forever (for now) home, on the comms floor room.
What Next?
When I was digging through my old machines I found a machine I had built years ago in a Node 304 which got me thinking I could probably do with a backup of the above machine, but in a tiny form factor, hidden away.
So, my next planned project is going to be (another) another NAS. But that will be more focused on the software side of things rather than the hardware, although the hardware will be awesome as who doesn't love tiny servers?
The plan is to send incremental ZFS snapshots from this NAS, over to this planned NAS via ZFS send.
I am also due a massive internet upgrade in the coming weeks once my road is dug up (35/15 Mb/s > 500/100 Mb/s), so I will probably do a post on connecting FreeNAS with cloud providers (Google Drive, in my case) and sending data off to the cloud as a backup, keeping in sync with your local box.
Best Prosumer Nasal
So, unless the universe delivers our sweet demise via asteroid instead of the inevitable collapse of human society as we know it I will, hopefully, see you again in the next post.
Muffin.
* I use Amazon referral links and donate the little money to my kids' school. You support charter schools when you use these links.
I LOVE my Amplifi Wi-Fi Mesh Network. I've had it for two years and it's been an absolute star performer. We haven't had a single issue. Rock solid. That's really saying something. From unboxing to installation to running it (working from home for a tech company, so you know I'm pushing this system) it's been totally stable. I recommend Amplifi unreservedly to any consumer or low-key pro-sumer who has been frustrated with their existing centrally located router giving them reliable wi-fi everywhere in their home.
That said..I recently upgraded my home internet service provider. For the last 10 years I've had fiber optic to the house with 35 Mbp/s up/down and it's been great. Then I called them a a few years back and got 100/100. The whole house was presciently wired by me for Gigabit back in 2007 (!) with a nice wiring closet and everything. Lately 100/100 hasn't been really cutting it when I'm updating a dozen laptops for a work event, copying a VM to the cloud while my spouse is watching 4k netflix and two boys are updating App Store apps. You get the idea. Modern bandwidth requirements and life has changed since 2007. We've got over 40 devices on the network now and many are doing real work.
I called an changed providers to a cable provider that offered true gigabit. However, I was rarely getting over 300-400 Mbp/s on my Amplifi. There is a 'hardware NAT' option that really helps, but short of running the Amplifi in Bridged Mode and losing a lot of its epic features, it was clear that I was outgrowing this prosumer device.
Give I'm a professional working at home doing stuff that is more than the average Joe or Jane, what's a professional option?
UniFi from Ubiquiti
Amplifi is the consumer/prosumer line from Ubiquiti Networks and UniFi (UBNT) is the professional line. You'll literally find these installed at business or even sports stadiums. This is serious gear.
Let me be honest. I knew UniFi existed. Knew (I thought) all about it and I resisted. My friends and fellow nerds insisted it was easy but I kept seeing massive complex network diagrams and convinced myself it wasn't worth the hassle.
My friends, I was wrong. It's not hard. If you are doing business at home, have a gigabit network pipe, a wired home network, and/or have a dozen or more network devices, you're a serious internet person and you might want to consider serious internet networking gear.
Now, UniFi is more expensive than Amplifi as it's pro gear. While an Amplifi Mesh WiFi system is just about $300-350 USD, UniFi Pro gear will cost more and you'll need stuff to start out and it won't always feel intuitive as you plan your system. It is worth it and I'm thrilled with the result. The flexibility and customizability its offered has been epic. There are literally no internet issues in our house or property anymore. I've even been able to add wired and wireless non-cloud-based security cameras throughout the property. Additionally, remember how the house is already wired in nearly every room with Cat6 (or Cat5e) cabling? UniFi has reintroduced me to the glorious world of PoE+ (Power over Ethernet) and removed a half dozen AC wall plugs from my system.
Plan your Network
You can test out the web-based software yourself LIVE at https://demo.ui.com and see what managing a large network would be like. Check out their map of the FedEx Forum Stadium and how they get full coverage. You can see a simulated map of my house (not really my house) in the screenshot above. When you set up a controller you can place physical devices (ones you have) and test out virtual devices (ones you are thinking of buying) and see what they would look like on a real map of your home (supplied by you). You can even draw 3D walls and describe their material (brick, glass, steel) and their dB signal loss.
When you are moving to UniFi you'll need:
- USG - UniFi Security Gateway - This has 3 gigabit points and has a WAN port for your external network (plug your router into this) and a LAN port for your internal network (plug your internal switch into this).
- This is the part that doles out DHCP.
- UniFi Cloud Key or Cloud Key Gen2 Plus
- It's not intuitive what the USG does vs the Cloud Key but you need both. I got the Gen2 because it includes a 1TB hard drive that allows me to store my security video locally. It also is itself a PoE client so I don't need to plug it into the wall. I just wired it with a single Ethernet cable to the PoE switch below and left it in the wiring closet. There's a smaller cheaper Cloud Key if you don't need a hard drive.
- You don't technically need a Cloud Key I believe, as all the UniFi Controller Software is free and you can run it in on any machine you have laying around. Folks have run them on any Linux or Windows machine they have, or even on a Synology or other NAS. I like the idea of having it 'just work' so I got the Cloud Key.
- UniFi Switch (of some kind and number of ports)
- 24 port UniFi Switch - 24 ports may be overkill for most but it's only 8 lbs and will handle even the largest home network. And it's under $200 USD right now on Amazon
- 24 port UniFi Switch with PoE - I got this one because it has 250W of PoE power. If you aren't interested in power over ethernet you can save money with the non-PoE version or a 16 port version but I REALLY REALLY recommend you use PoE because the APs work better with it.
Now once you've got the administrative infrastructure above, you just need to add whatever UniFi APs - access points - and/or optional cameras that you want!
NOTE/TIP - A brilliant product from Ubiquiti that I think is flying under the radar is the Unifi G3 Flex PoE camera. It's just $75 and it's tiny but it's absolutely brilliant. Full 1080p video and night vision. I'll talk about the magic of PoE later on but you can just plug this in anywhere in the house - no AC adapter - and you've got a crystal clear security camera or cameras anywhere in the house. They are all powered from the PoE switch!
I had a basic networking closet I put the USG Gateway into the closet with a patch cable to the cable modem (the DOCSIS 3.1 cable modem that I bought because I got tired of renting it from the service provider) then added the Switch with PoE, and plugged the Cloud Key into it. Admin done.
Here's the lovely part.
Since I have cable throughout the house, I can just plug in the UniFi Access Points in various room and they get power immediately. I can try different configs and test the signal strength. I found the perfect config after about 4 days of moving things around and testing on the interactive map. The first try was fine but I strove for perfect.
There's lots of UniFi Access Points to choose from. The dual radio Pro version can get pretty expensive if you have a lot so I got the Lite PoE AP. You can also get a 5 pack of the nanoHD UniFi Access Points.
These Access Points are often mounted in the ceiling in pro installations, and in a few spots I really wanted something more subtle AND I could use a few extra Ethernet ports. Since I already had an Ethernet port in the wall, I could just wall mount the UniFi Wall Mounted AP. It's both a wireless AP that radiates outward into the room AND it turns your one port into two, or you can get one that becomes a switch with more ports and extends your PoE abilities. So I can add this to a room, plug a few devices in AND a PoE powered Camera with no wall-warts or AC adapters!
NOTE: I did need to add a new ethernet RJ45 connector to plug into the female connector of the UniFi in-wall AP. Just be sure to plan and take inventory. You may already have full cables with connectors pulled to your rooms. Be aware.
There are a TON of great Wireless AP options from UniFi so make sure you explore them all and understand what you want.
Here's the resulting setup and choices I made, as viewed in the UniFi Controller Software:
I have the Gateway, the Switch with PoE, and five APs. Three are the disc APs and two are in-wall APs. They absolutely cover and manage my entire two story house and yards front and back. It's made it super easy for me to work from home and be able to work effectively from any room. My kids and family haven't had any issues with any tablets or phones.
As of the time of these writing I have 27 wireless devices on the system and 11 wired (at least those are the ones that are doing stuff at this hour).
Note how it will tell you how each device's WiFi experience is. I use this Experience information to help me manage the network and see if the APs are appropriately placed. There is a TON of great statistics and charts and graphics. It's info-rich to say the LEAST.
NOTE: To answer a common question - In an installation like this you've got a single SSID even though there's lots of APs and your devices will quietly and automatically roam between them!
The iPhone app is very full-featured as well and when you've got deep packet introspection turn on you can see a ton of statistical information at the price of a smidge of throughput performance.
I have had NO problem hitting 800-950Mbs over wired and I feel like there's no real limit to the perf of this system. I've done game streaming over Steam and Xbox game streaming for hours without a hiccup. Netflix doesn't buffer anymore, even on the back porch.
You can auto-optimize, or you can turn off a plethora of feature and manage everything manually. I was able to twitch a few APs to run their 2.4Ghz Wi-Fi radios on less crowded channels in order to get out of the way of the loud neighbors on channel 11.
Best Prosumer Nascar
I have a ton of control over the network now, unlimited expandability and it has been a fantastically stable network. All the APs are wire backed and the wireless bandwidth is rock solid. I've been extremely impressed with the clean roaming from room to room while streaming from Netflix. It's a tweakers (ahem) dream network.
* I use Amazon referral links and donate the little money to my kids' school. You support charter schools when you use these links.
Sponsor: Get the latest JetBrains Rider with WinForms designer, Edit & Continue, and an IL (Intermediate Language) viewer. Preliminary C# 8.0 support, rename refactoring for F#-defined symbols across your entire solution, and Custom Themes are all included.
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
Best Prosumer Nasal Spray
AboutNewsletter
