New Home-Freenas user from Germany

[modtta4455]

Servus,

i just started to play around a little bit with freenas. As a german engineer I love to overengineer stuff. So from "I just get a cheap synology" I went to "lets build a 32TB netto Freenas with UPS and RaidZ2" in a bout 2 weeks. Complete overkill for my solution, but hey - its also some kind of hobby...

So here I go:

Current Setup:

ASUS P10S WS C236 Mainboard (8x SATA 2x NVME, 2x Intel GLAN, considerably low power). Snatched a bargain here at 120€
Intel G4560, just runs @20% with full GLAN write.
32 GB ECC RAM (Kingston KVR21E15D8K2/16I x 2), accidently bought on ebay *g*, just wanted to see how high it sells... got 32 GB ECC for 130€
Boot Drive: Old 100gb Laptop HDD I had in the shelf, USB threw a lot of errors on the SLC Sticks I had.
Bequiet PurePower 450W PSU
Generic Case + lots of Fans, running 35°C on the HDDs
APC BK650EI UPS

I was able to do the build for 435€ excluding Data-Drives. Pretty fair for that setup I think. Fully operational I run roughly 70W. Need to configure Spindown and so on later (currently still initial-filling it since 3 days).

DATA: 6x WD80EZZX salvaged from Mybooks I more or less already had / was able to get for just 190-200 EUR per 8 TB, jaja, I know those are not WD Reds, but RaidZ2 + Full Backup should cover that. Also I have a few more for self-warrantying and cold spares...

I initially planned to do the "quick and dirty" lets just start with a degraded RaidZ2 and do a resilver once all is in. Unfortunalety the GUI didnt import the array. It was possible to CLI mount it though. With that idea tanked, I got additional 3 8TBs to start with the setup from the beginning...

Current Config:

RaidZ2 with AES for netto 32 TB, with complete external Backup of all data on a spare set of 4x 8TB.

Prospective Ideas:

Get a LTO-5 or LTO-6 Streamer for Backup, then throw in another vdev with the remaining 4 drives. That would mean I would have to install a HBA, but I need that for LTO anyway (SAS)

Already thinking about 10G-Lan-Equipment :). I started in the Gigabit World 14 years ago with my mighty Thinkpad T40p, but didnt improve ever since. Unfortunately it seems that 10G is way more complicated and flimsy than 1000M.

Other than the Freenas I am only using Windows 7. My last interactions with linux were as a dorm admin during the university, where we ran the servers with debian. Other than that, I am a big fan of Thinkpads (currently 5 here: T40p, T61, W500, T440p, W540).

So. Here I am :)

 

[Spearfoot]

Servus,

i just started to play around a little bit with freenas. As a german engineer I love to overengineer stuff. So from "I just get a cheap synology" I went to "lets build a 32TB netto Freenas with UPS and RaidZ2" in a bout 2 weeks. Complete overkill for my solution, but hey - its also some kind of hobby...

So here I go:

Current Setup:

ASUS P10S WS C236 Mainboard (8x SATA 2x NVME, 2x Intel GLAN, considerably low power). Snatched a bargain here at 120€
Intel G4560, just runs @20% with full GLAN write.
32 GB ECC RAM (Kingston KVR21E15D8K2/16I x 2), accidently bought on ebay *g*, just wanted to see how high it sells... got 32 GB ECC for 130€
Boot Drive: Old 100gb Laptop HDD I had in the shelf, USB threw a lot of errors on the SLC Sticks I had.
Bequiet PurePower 450W PSU
Generic Case + lots of Fans, running 35°C on the HDDs
APC BK650EI UPS

I was able to do the build for 435€ excluding Data-Drives. Pretty fair for that setup I think. Fully operational I run roughly 70W. Need to configure Spindown and so on later (currently still initial-filling it since 3 days).

DATA: 6x WD80EZZX salvaged from Mybooks I more or less already had / was able to get for just 190-200 EUR per 8 TB, jaja, I know those are not WD Reds, but RaidZ2 + Full Backup should cover that. Also I have a few more for self-warrantying and cold spares...

I initially planned to do the "quick and dirty" lets just start with a degraded RaidZ2 and do a resilver once all is in. Unfortunalety the GUI didnt import the array. It was possible to CLI mount it though. With that idea tanked, I got additional 3 8TBs to start with the setup from the beginning...

Current Config:

RaidZ2 with AES for netto 32 TB, with complete external Backup of all data on a spare set of 4x 8TB.

Prospective Ideas:

Get a LTO-5 or LTO-6 Streamer for Backup, then throw in another vdev with the remaining 4 drives. That would mean I would have to install a HBA, but I need that for LTO anyway (SAS)

Already thinking about 10G-Lan-Equipment :). I started in the Gigabit World 14 years ago with my mighty Thinkpad T40p, but didnt improve ever since. Unfortunately it seems that 10G is way more complicated and flimsy than 1000M.

Other than the Freenas I am only using Windows 7. My last interactions with linux were as a dorm admin during the university, where we ran the servers with debian. Other than that, I am a big fan of Thinkpads (currently 5 here: T40p, T61, W500, T440p, W540).

So. Here I am :)

Welcome to the forum!

The forum's "10 Gig Networking Primer" may interest you if you're thinking about going 10G.

Be warned that building these systems is highly addictive... I've built three in the last two years! Sounds as though you may have been bitten by the bug, too. :D

 

[modtta4455]

Well. I just did a short benchmark but i am sure there is a bug. Values seem way to high:

root@G4560:/mnt/hydrogenium # dd if=/dev/zero of=/mnt/hydrogenium/ddfile bs=16384K count=10000
10000+0 records in
10000+0 records out
167772160000 bytes transferred in 38.470195 secs (4361094622 bytes/sec)

root@G4560:/mnt/hydrogenium # dd of=/dev/null if=/mnt/hydrogenium/ddfile bs=16384K count=10000
10000+0 records in
10000+0 records out
167772160000 bytes transferred in 22.079463 secs (7598561761 bytes/sec)

That seems to be 4,3 gb/s write and 7,6 gb/s read. That couldnt be true. 6x best case 200 MB = 1200 mb/s
Any funny compression working here?

 

[Robert Trevellyan]

Any funny compression working here?

LZ4 compression is the default, so you should turn that off for the dataset you use for benchmarking.

 

[danb35]

LZ4 compression is the default, so you should turn that off for the dataset you use for benchmarking.

...and zeroes are highly compressible.

 

[Robert Trevellyan]

...and zeroes are highly compressible.

With compression enabled and all-zero data, ZFS doesn't even bother to allocate storage blocks. That's why all-zero data doesn't show up in the reported compression ratio for a dataset.

 

[modtta4455]

Ahh, ok, got that. Did a retest. About 500 read and write. Way enough for my GLAN cabling ;-)
Thinking of taking out 16GB of RAM... If performance stays over 200 I am ok with it. Any guesses?

I managed to get SSH and UPS running. Any cool ideas what to run as a VM? I have spare computing power...

 

[Stux]

Plex. Might be. Setter as a jail.

Pfsense. If you're running a type 1 hypervisor.

 

[danb35]

Any cool ideas what to run as a VM?

BOINC? Donate your spare cycles to help mankind. Can be done in a jail as well.

 

[Stux]

BOINC? Donate your spare cycles to help mankind. Can be done in a jail as well.

More like donating electricity I guess. There's a case to be made that massive computation can be more environmentally handled in super computers or data centres

 

[modtta4455]

Ok I just tried with 16 GB instead of the installed 32. Almost no change in transfer rates over GLAN. --> i guess i´ll use the remaining 16 for another PC...