Is ZFS for me?

[jdabb]

I have three Hard drives(10tb, 2tb, and a 2tb) in WD my cloud ex4100 that I want to use in a new server build. I already bought most of the components that I will be using for free nas but looking into zfs a little bit more and I am not sure if it is the right choice for me. I want to be able to add drives later on, and it looks like with zfs I should be buying all the wanted drives upfront.

here are the parts I have so far

Supermicro X10SL7-F Motherboard
Intel Xeon E3-1231 v3 CPU
16gb crucial RAM
16gb Samsung DDR3 1.35v-1600 M391B1G73QH0 RAM - coming Soon
Seasonic-550 Power Supply
Antec case
Noctua CPU Cooler
and the three hard drives (10tb, 2tb, and a 2tb)

Would it be smart to go with ZFS?

I plan on using this for media mostly movies and tv shows (plex server), Dashcam video backups, and maybe some photo backups.

 

[Chris Moore]

and it looks like with zfs I should be buying all the wanted drives upfront.

You just need to plan properly. You can't grow a vdev by adding drives and all the drives in a given vdev need to be the same size because it will only use the amount of space (per drive) that is available on the smallest drive.
Here is a link to a good primer on the subject:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://www.ixsystems.com/community...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

You might also want to look into this for the, what does that mean, questions:

Terminology and Abbreviations Primer
https://www.ixsystems.com/community/threads/terminology-and-abbreviations-primer.28174/

Would it be smart to go with ZFS?

ZFS is the best file system there is, bar none, but it does take prior planning to make it do the things you want it to do.

Dashcam video backups, and maybe some photo backups.

If you want to use it to backup thing that you don't want to deteriorate or be lost to a drive failure, you will want to have redundancy. Personally, I am a fan of RAIDz2, but you could use mirror vdevs. To do mirror vdevs with your hardware, you would need an additional 10TB drive. Also, you need to understand that when you put the drives in FreeNAS and establish a pool, the existing data will be wiped when FreeNAS re-partitions and formats the drives for use with ZFS.

 

[Heracles]

Hey jdabb,

You can add drives in a ZFS pool. There are ways to do it that are better than others, but it is possible for sure.

In your case, there is a very big difference between a 10 TB drive and 2 TB drives. Also, with only 3 drives, you would not have many options. You will end up with the limited space of the 2 TB drives.

Should you add a second 10 TB drive, you could do two mirrors, a 10 TB one and a 2 TB one. But to have such a big difference between the two would not be very good. Almost all the activity will end up on the 10 TB one...

I think a little more planning and optimisation is needed here...

 

[jdabb]

since its not an ideal setup I am now looking to get some refurbished drives is this a good idea 6 x 4tb SEAGATE ST4000DM000 and maybe sell the 10 tb?

 

[CraigD]

The heart of a freeNAS server is the motherboard, CPU, and RAM. If you get this right (you have) you only have drives to worry about.

My use case is the same as yours, and I have basically the same motherboard, CPU, and RAM and it works great

The slideshow is really important because it explains how ZFS works, I am also a fan of RAIDz2

If your case can hold 12 3.5" drives I would get 6 data drives now, and add 6 later, this leave a free SATA port. If have less space, if possible max out the drives in your case from the start, planning you pool layout avoids a rebuild of you data

I do recommend a small SSD or Hard drive for boot drive

If you can buy drives that are designed for 24/7 use and support TLER (time-limited error recovery) WD Reds, Seagate Ironwolfs, and Enterprise drives do

Protecting data is something freeNAS does extremely well, I've had 6 drives (old desktop drives) die in my server within the last 2 years and not lost a single byte of data. I've also had the system disk die in my gaming machine, it was operational within 30 minutes using my nighty backup

Have Fun

 

[Chris Moore]

SEAGATE ST4000DM000

I have two servers at home, each running twelve of these drives. The ones I like best, better than the retail model, are the Dell OEM version of the drive. The Dell OEM drives have a different firmware and I have found them to be very reliable. I managed to pickup a bunch of them new a little while back and have had zero trouble from them.

This is a link to the model I am using:
https://www.ebay.com/itm/Seagate-De...s-3-5-SATA-Hard-Drive-Dell-VF3T3/372720600575

 

[jdabb]

That is one option im leaning towards but also was thinking of buying 3 more 10TB(would be shucking them) so I can keep the 10tb that I already own. But then I would have to replace them with 10TB every time one fails and the redundancy is not as widespread correct, is this a good idea or is having 6 * 4TB better (since there would be more failure points)

 

[tfran1990]

Its very common here to see members run raidz2 with a 6 wide vdev and even multiple vdevs.

 

[CraigD]

If @Chris Moore says the drives are good they are

I've not seen them in my part of the world

Have Fun

 

[Chris Moore]

You certainly have options. You might want to run some of the numbers through this calculator to figure out where you stand.
https://wintelguy.com/zfs-calc.pl

Personally, I can't see spending the money on larger drives yet. They are still too expensive for me. I want to have the redundancy (RAIDz2) and I want to have the capacity, but the per-drive cost is too high for me right now.

As an example, five 10TB drives in a single vdev at RAIDz2 would give me about 20TB of usable storage after factoring in the overhead of ZFS checksums and redundancy. This would cost about $1375 though because of the cost of the drives.

- edit - that price on the 10TB pool was based on $275 a drive, the last cost I recalled, I since checked and WD Red drives are $285 while Seagate Ironwolf drives are $250, so your actual cost would vary, and you might even shuck some drives for an even lower cost.

The storage I am using is six 4TB drives times two vdevs (twelve drives total) which give me about 22TB of usable storage, but at a cost of only $780.

So, the 10TB drives are cool, but costly.

 

[jdabb]

I saw in the zfs slideshow that you can use a different number of drives in you vdevs but it is not ideal. I am assuming performance will take a hit ? I plan on doing 4 x 10tb in vdev0 and later adding 6 * 4tb (vdev1)

 

[Constantin]

Some additional factors to consider are speed (more vdevs=more iops) and the cost of electricity. I opted for a single VDEV Z3 because I value integrity above all other considerations and wanted to keep my power load low because I pay 0.25$/kWh.

So, less speed, higher up front cost, but the system runs at less than 100w.

 

[Chris Moore]

I saw in the zfs slideshow that you can use a different number of drives in you vdevs but it is not ideal. I am assuming performance will take a hit ?

Yes. It will have an impact on performance, but it will work.

 

[Constantin]

I’d keep the vdevs similar. A six drive z2 is a very common “building block”, for example.

 

[jdabb]

You certainly have options. You might want to run some of the numbers through this calculator to figure out where you stand.
https://wintelguy.com/zfs-calc.pl

Personally, I can't see spending the money on larger drives yet. They are still too expensive for me. I want to have the redundancy (RAIDz2) and I want to have the capacity, but the per-drive cost is too high for me right now.

As an example, five 10TB drives in a single vdev at RAIDz2 would give me about 20TB of usable storage after factoring in the overhead of ZFS checksums and redundancy. This would cost about $1375 though because of the cost of the drives.

- edit - that price on the 10TB pool was based on $275 a drive, the last cost I recalled, I since checked and WD Red drives are $285 while Seagate Ironwolf drives are $250, so your actual cost would vary, and you might even shuck some drives for an even lower cost.

The storage I am using is six 4TB drives times two vdevs (twelve drives total) which give me about 22TB of usable storage, but at a cost of only $780.

So, the 10TB drives are cool, but costly.

That makes sense just need to figure out what to do with the 10tb I already have. maybe 6 * 4tb is the way to start.

 

[Chris Moore]

That makes sense just need to figure out what to do with the 10tb I already have. maybe 6 * 4tb is the way to start.

You can always add another vdev later. I started with one vdev years ago and added a second when the first got about 50% filled. I have also changed out drives to increase the size of my pool.
To give you an idea of how it can work, I started with one vdev of 1TB drives, added a second vdev of 1TB drives, later replaced the 1TB drives in the first vdev with 2TB drives (one drive at a time) and yet later upgraded the drives in the second vdev to 2TB drives. I repeated that process moving from 2TB drives to 4TB drives, which is where I am now. When the time comes that I need more capacity, I may opt to add another vdev of 4TB drives because I now have a server chassis that can accommodate more drives. I have been looking at the 6TB drives, but they are still a little high priced for me.

If you are willing to invest in SAS hardware, which is not terribly expensive, I have recently picked up some of these drives and been fairly well pleased with them, but there is a bit of difference in how SAS drives behave vs how SATA drives.

https://www.ebay.com/itm/Western-Di...ty-until-Dec-2020/153501252538?epid=216167810

I have about fifty of those drives running in my servers at work and they have been solid. Your mileage may vary. It is always a gamble when you buy used hardware.

If you were to go with SAS drives, you would need a SAS HBA to run them. This is a good model and this is a trusted vendor:
https://www.ebay.com/itm/Genuine-LS...i-P20-IT-Mode-ZFS-FreeNAS-unRAID/162958581156

Also, if you need to go direct from the SAS controller to the drives, you will need forward breakout cables like these:
https://www.ebay.com/itm/Mini-SAS-t...-Breakout-Internal-Cable-19-Inch/371643034851

Just be sure to NOT get the reverse breakout cables because they look the same but are wired differently.

 

[Chris Moore]

PS. My decisions for home are based on my personal low budget. For work I am trying to get a procurement of 80 new 14TB drives completed to upgrade some existing servers. They have money for it. If you have money, please buy what you feel works best, but if you need to keep the cost low, I have searched and found some options.

 

[jdabb]

ahh no money here. I'm taking a gamble on six lower-priced drives now so ill see how that goes, Thanks,