Question about adding 3 more drives

[ruzifan]

Hello,

I currently have 3X10TB HDD with one drive fail redundancy.

I’m planning on buying 3X8TB HDD tomorrow and create another Vdev with one drive fail redundancy.
I just want to if that’s fine without any issues or do I have to keep buying 10TB HDDs

 

[sretalla]

That's fine, you can add another VDEV of the same type (in your case RAIDZ1) of a different size.

Make sure you follow the instructions for extending a volume (although I note that these instructions when navigating by default apply to the legacy UI... @dlavigne ) https://www.ixsystems.com/documentation/freenas/11.2-U7-legacy/storage.html#extending-a-zfs-volume.

Here's the process for the new UI https://www.ixsystems.com/documentation/freenas/11.2-U7/storage.html#extending-a-pool

 

[ruzifan]

That's fine, you can add another VDEV of the same type (in your case RAIDZ1) of a different size.

Make sure you follow the instructions for extending a volume (although I note that these instructions when navigating by default apply to the legacy UI... @dlavigne ) https://www.ixsystems.com/documentation/freenas/11.2-U7-legacy/storage.html#extending-a-zfs-volume.

Here's the process for the new UI https://www.ixsystems.com/documentation/freenas/11.2-U7/storage.html#extending-a-pool

Perfect thanks.
Also, what do u think I should do?

I have 3X10TB Baracudda PRO in there right now.

should I buy 4X8TB WD RED and do a Raidz1 Then move the files from the barracuda to the WD RED? saying this only because people claim the REDs are better

 

[Scharbag]

Perfect thanks.
Also, what do u think I should do?

I have 3X10TB Baracudda PRO in there right now.

should I buy 4X8TB WD RED and do a Raidz1 Then move the files from the barracuda to the WD RED? saying this only because people claim the REDs are better

Depends on your backup strategy or if you can handle data loss.

RaidZ is not a replacement for data backups. RaidZ is intended to increase data availability but should be used with a backup strategy if you cannot tolerate loss.

If you can tolerate possible loss, then if I were you, I would simply buy 3 or 4 new drives and add another vDev to your existing pool. Another option would be to buy more 10TB drives and make a single vDev RaidZ2 pool of 6 or 7 drives. This will require you to destroy your existing pool... But IMHO that would be a better option as it minimizes parity loss while maximizing redundancy.

Cheers,

 

[ruzifan]

Depends on your backup strategy or if you can handle data loss.

RaidZ is not a replacement for data backups. RaidZ is intended to increase data availability but should be used with a backup strategy if you cannot tolerate loss.

If you can tolerate possible loss, then if I were you, I would simply buy 3 or 4 new drives and add another vDev to your existing pool. Another option would be to buy more 10TB drives and make a single vDev RaidZ2 pool of 6 or 7 drives. This will require you to destroy your existing pool... But IMHO that would be a better option as it minimizes parity loss while maximizing redundancy.

Cheers,

in that case, I have 2 types of files stored in mynas.
Media such as movie rips. And videos and photos of my family from trips and other events.
If I lost the movie rips, I don’t care much.
However, I would like to ensure I never lose my personal family video and movies.

so for that, should I setup 5X8TB WD RED and do a raidz2? And leave the raidz1 with the 3X10TB barracuda pro??

or do you think getting 6X8TB RED and do a mirror would be best for my personal files.

put it this way, 8 years ago, I had a Synology
Nas with 2 drives and I didn’t know anything about raid and I stored my WEDDING video in there.. yep, drive crashed and I lost the vidoes.

 

[Scharbag]

in that case, I have 2 types of files stored in mynas.
Media such as movie rips. And videos and photos of my family from trips and other events.
If I lost the movie rips, I don’t care much.
However, I would like to ensure I never lose my personal family video and movies.

so for that, should I setup 5X8TB WD RED and do a raidz2? And leave the raidz1 with the 3X10TB barracuda pro??

or do you think getting 6X8TB RED and do a mirror would be best for my personal files.

put it this way, 8 years ago, I had a Synology
Nas with 2 drives and I didn’t know anything about raid and I stored my WEDDING video in there.. yep, drive crashed and I lost the vidoes.

If you cannot tolerate data loss, then you NEED a backup strategy.

If I were you, I would look into offsite storage solution for my personal files. That is what I do. I use Tresorit for stuff I want to keep and cannot replace. Then for media, I use RaidZ2 for my production pool and RaidZ2 for my backup pool.

If you cannot afford or do not have space for a backup solution, I would make the largest RaidZ2 pool I could and make it RaidZ2 for media use AND I would sync my personal files to an online service - preferably one that provides end based encryption.

Unfortunately, too many people have learned the hard way that RaidZ does not replace backups. Ever.

Cheers,

 

[ruzifan]

If you cannot tolerate data loss, then you NEED a backup strategy.

If I were you, I would look into offsite storage solution for my personal files. That is what I do. I use Tresorit for stuff I want to keep and cannot replace. Then for media, I use RaidZ2 for my production pool and RaidZ2 for my backup pool.

If you cannot afford or do not have space for a backup solution, I would make the largest RaidZ2 pool I could and make it RaidZ2 for media use AND I would sync my personal files to an online service - preferably one that provides end based encryption.

Unfortunately, too many people have learned the hard way that RaidZ does not replace backups. Ever.

Cheers,

Quick question.
If my current pool is made up of 1 Vdev using 3 disc s radiz1. Could I extend the volume by creating another vdev using 5 disc and radiz2? Or let’s say 4 disc mirror? Or do the Vdev have to exactly the same as the original one? 3 disc raidz1.
I was clear on the using different size drive, but wasn’t sure if the number of drive and the raidz mattered

 

[pro lamer]

If my current pool is made up of 1 Vdev using 3 disc s radiz1. Could I extend the volume by creating another vdev using 5 disc and radiz2?

If you can afford 5 disks (and if they are big enough) you might be better creating a separate raidz2 pool, moving all the data from your present pool there and destroy the original pool and reuse the three disks...

Sent from my phone

 

[ruzifan]

If you can afford 5 disks (and if they are big enough) you might be better creating a separate raidz2 pool, moving all the data from your present pool there and destroy the original pool and reuse the three disks...

Sent from my phone

Reuse the 3 10TB disc as what? I’d need to buy 2more 10TB Disc later on and extend the volume with radiz2 after right?
Not like I’d have immediate use of the discs after I move the files

 

[pro lamer]

Reuse the 3 10TB disc as what?

Maybe, as @Scharbag wrote, as a backup/offsite backup...

Sent from my phone

 

[Scharbag]

Reuse the 3 10TB disc as what? I’d need to buy 2more 10TB Disc later on and extend the volume with radiz2 after right?
Not like I’d have immediate use of the discs after I move the files

Yeah, ZFS is a bitch some times. You need to plan ahead - I know I have painted myself into a corner more than once. :)

I find 6 drive RaidZ2 vDevs work great in my system - ~33% parity loss. I agree with @pro lamer - use your old 10TB drives as a 3 drive RaidZ1 backup pool. Then buy new drives for a production pool and set it up as RaidZ2 if you can.

I guess we should ask - how many drives can your system support? Also, how are you connecting your drives? SAS expander?

Cheers,

 

[pschatz100]

@ruzifan Reffering back to the original post: How much space do you need? Also, how about providing your system specs? It is difficult to offer suggestions without knowing what you have and what you are doing with it.

That said, it is not wise to use RaidZ1 with such large discs. I am a believer in using RaidZ2. There is a lot of discussion on the internet why RaidZ1 is not a good solution with large discs.

 

[ruzifan]

Yeah, ZFS is a ***** some times. You need to plan ahead - I know I have painted myself into a corner more than once. :)

I find 6 drive RaidZ2 vDevs work great in my system - ~33% parity loss. I agree with @pro lamer - use your old 10TB drives as a 3 drive RaidZ1 backup pool. Then buy new drives for a production pool and set it up as RaidZ2 if you can.

I guess we should ask - how many drives can your system support? Also, how are you connecting your drives? SAS expander?

Cheers,

@ruzifan Reffering back to the original post: How much space do you need? Also, how about providing your system specs? It is difficult to offer suggestions without knowing what you have and what you are doing with it.

That said, it is not wise to use RaidZ1 with such large discs. I am a believer in using RaidZ2. There is a lot of discussion on the internet why RaidZ1 is not a good solution with large discs.

Ok so I’m running amd Ryzen 5 2400G CPU
32GB 3000MHTZ DDR4 RAM. NON ECC
Using gigabyte b450m mother board.

my setup can take up to 10 drives total.
I have 1SSD for boot. And 3 x 10TB HDD raidz1 now.

I just got delivered 6 X 8TB WD RED.
because I maxed out my expansion and I think this is more than enough space.
I have 2 options.
Extend current volume by creating a new VDev with the 6X8TB raidz2, but then have the risk of having the 3X10TB raidz1 vdev in that pool dying on me and risk losing the entire pool
So option 2, I can move my files to an external which I have. Then kill the pool and start fresh wjrh 9X8TB raidz2 or even raidz3. Yes I know I’ll lose 2TB from those 10TB If I put them in the same vdev.

I also ran out of SATA ports, so I ordered this which is coming Thursday. I need 4 more sata ports.

https://www.amazon.ca/dp/B07QCMHNWG/ref=cm_sw_r_cp_api_i_WITcEb523A83S

I’m assuming freenas supports sata expansion right?

 

[Scharbag]

Ok so I’m running amd Ryzen 5 2400G CPU
32GB 3000MHTZ DDR4 RAM. NON ECC
Using gigabyte b450m mother board.

my setup can take up to 10 drives total.
I have 1SSD for boot. And 3 x 10TB HDD raidz1 now.

I just got delivered 6 X 8TB WD RED.
because I maxed out my expansion and I think this is more than enough space.
I have 2 options.
Extend current volume by creating a new VDev with the 6X8TB raidz2, but then have the risk of having the 3X10TB raidz1 vdev in that pool dying on me and risk losing the entire pool
So option 2, I can move my files to an external which I have. Then kill the pool and start fresh wjrh 9X8TB raidz2 or even raidz3. Yes I know I’ll lose 2TB from those 10TB If I put them in the same vdev.

I also ran out of SATA ports, so I ordered this which is coming Thursday. I need 4 more sata ports.

https://www.amazon.ca/dp/B07QCMHNWG/ref=cm_sw_r_cp_api_i_WITcEb523A83S

I’m assuming freenas supports sata expansion right?

Oof.
SATA expanders are generally considered to be a bad idea. Perhaps the model you bought is better, but in the past, they are a hard no-no.
Best to use a proper HBA card like the 9211-8i with SAS to SATA cables.

Cheers,

 

[ruzifan]

Oof.
SATA expanders are generally considered to be a bad idea. Perhaps the model you bought is better, but in the past, they are a hard no-no.
Best to use a proper HBA card like the 9211-8i with SAS to SATA cables.

Cheers,

OMG so I did some quick research and it turns out that's a no no. I just cancelled my order and bought

Amazon.ca

THat's the 9211-8I. I also got these cables.

LiNKFOR 2 Pcs 1.6ft Mini SAS to 4 SATA Cable SAS Breakout Cable Mini SAS Male SFF-8087 to 4 SATA Female Cable Multi-Lane Mini SAS Host Internal Cable to Target HDD Hard Drive Splitter Cable : Amazon.ca: Electronics

LiNKFOR 2 Pcs 1.6ft Mini SAS to 4 SATA Cable SAS Breakout Cable Mini SAS Male SFF-8087 to 4 SATA Female Cable Multi-Lane Mini SAS Host Internal Cable to Target HDD Hard Drive Splitter Cable : Amazon.ca: Electronics

www.amazon.ca

I guess I have to flash that card right, based on what I read.. it needs to be IT mode or something. I gotta find a guide for that once the card comes in.

Also, since I got 10 drives. Can I plug 8 drives in the HBA and my ssd and the extra drive in the motherboard's SATA. Is that fine?

 

[Scharbag]

OMG so I did some quick research and it turns out that's a no no. I just cancelled my order and bought

Amazon.ca

THat's the 9211-8I. I also got these cables.

LiNKFOR 2 Pcs 1.6ft Mini SAS to 4 SATA Cable SAS Breakout Cable Mini SAS Male SFF-8087 to 4 SATA Female Cable Multi-Lane Mini SAS Host Internal Cable to Target HDD Hard Drive Splitter Cable : Amazon.ca: Electronics

LiNKFOR 2 Pcs 1.6ft Mini SAS to 4 SATA Cable SAS Breakout Cable Mini SAS Male SFF-8087 to 4 SATA Female Cable Multi-Lane Mini SAS Host Internal Cable to Target HDD Hard Drive Splitter Cable : Amazon.ca: Electronics

www.amazon.ca

I guess I have to flash that card right, based on what I read.. it needs to be IT mode or something. I gotta find a guide for that once the card comes in.

Also, since I got 10 drives. Can I plug 8 drives in the HBA and my ssd and the extra drive in the motherboard's SATA. Is that fine?

IIRC, mixing HBA and internal SATA is just fine.

Good call on getting the HBA. Flashing is not too difficult and can be done right in FreeNAS. I would recommend flashing before you connect any drives to it. There are some good guides out there. Make sure you follow them so you are successful. I would recommend you not install the boot bios on the HBA as you will never be booting from it and it slows down the boot cycle of your server something fierce.

Cheers,

 

[pschatz100]

I just got delivered 6 X 8TB WD RED.
because I maxed out my expansion and I think this is more than enough space.
I have 2 options.
Extend current volume by creating a new VDev with the 6X8TB raidz2, but then have the risk of having the 3X10TB raidz1 vdev in that pool dying on me and risk losing the entire pool
So option 2, I can move my files to an external which I have. Then kill the pool and start fresh wjrh 9X8TB raidz2 or even raidz3. Yes I know I’ll lose 2TB from those 10TB If I put them in the same vdev.

You have a third option as well, which would be to leave the existing 3x10Tb RaidZ1 as is, and add a new pool with 6x8Tb RaidZ2. There is no reason why you cannot have two pools. At the end of the day, I think your option 2 will leave you with reliable storage and will also give you the most usable space. But it really depends upon what you want to do and how much tolerance you have for risk.

 

[ruzifan]

You have a third option as well, which would be to leave the existing 3x10Tb RaidZ1 as is, and add a new pool with 6x8Tb RaidZ2. There is no reason why you cannot have two pools. At the end of the day, I think your option 2 will leave you with reliable storage and will also give you the most usable space. But it really depends upon what you want to do and how much tolerance you have for risk.

Yes, i was also thinking about that, but kept asking myself if it's really important to jut have one pool. LIke everyone here seems to love expanding rather than just creating a new pool and minimizing the risk of losing m,ore data. (expanding pool and losing 1vDEV = all data gone) VS create a new pool, and even if you lose that VdeV, ur other pool is still ok. i don't know why people don't create new pools.


my final questions i guess is.. lets say RAIDZ1.. i have 3 drives and 1 drive fail.. How long do i have to replace the drive? Is it basically 1 drive fails.. i repalce it and i'm good, or 1 drive fails and another drive fails before i replace the first bad drive, i lose all the data?? bit confused on that part.

 

[jon_2112]

@ruzifan Reffering back to the original post: How much space do you need? Also, how about providing your system specs? It is difficult to offer suggestions without knowing what you have and what you are doing with it.

That said, it is not wise to use RaidZ1 with such large discs. I am a believer in using RaidZ2. There is a lot of discussion on the internet why RaidZ1 is not a good solution with large discs.

What does "large disk" mean in this context though? 5 years ago, 1 TB was a large disk, but it's not really thought of that way now. ZFS is still ZFS. Was RAIDZ1 not OK for 5x1TB in 2014, but it's OK now?

I think "RAIDZ1 is bad for large disks" is kind of nonsense, and a better way to think about it is to realize that you're more at risk of data loss with RAIDZ1 than RAIDZ2. You need to weigh how much of a problem losing your pool is against things like the cost of an extra disk, how robust your backup strategy is, and how many ports you have available. For me, for instance, I'm comfortable running RAIDZ1 on a 5x5TB array because I have a very good backup strategy, am maxed out on motherboard ports, and downtime is really not a problem at all. For other cases where availability is important, then RAIDZ2 makes a lot more sense even if it means adding an extra controller.

 

[ruzifan]

What does "large disk" mean in this context though? 5 years ago, 1 TB was a large disk, but it's not really thought of that way now. ZFS is still ZFS. Was RAIDZ1 not OK for 5x1TB in 2014, but it's OK now?

I think "RAIDZ1 is bad for large disks" is kind of nonsense, and a better way to think about it is to realize that you're more at risk of data loss with RAIDZ1 than RAIDZ2. You need to weigh how much of a problem losing your pool is against things like the cost of an extra disk, how robust your backup strategy is, and how many ports you have available. For me, for instance, I'm comfortable running RAIDZ1 on a 5x5TB array because I have a very good backup strategy, am maxed out on motherboard ports, and downtime is really not a problem at all. For other cases where availability is important, then RAIDZ2 makes a lot more sense even if it means adding an extra controller.

can you please share your backup strategy?

I'm gonna have a total of 40TB available storage after RADIZ2. I'm wondering what's the best backup option. I seen online backup, but does that get expensive for 40TB?