Home Server Questions - lots of questions....

[LIGISTX]

8 disks in a vdev is fine, even 10 is OK. What starts to happen is rebuild times get really slow with more drives and that can get scary.

Yea, true.

I am now just trying to figure out how to best utilize the setup I have and figure out future state... This just became a very hard optimization problem lol.

 

[LIGISTX]

The other issue is.... trying to get all the data onto the new server.

I have 9 drives in a RAID Z2 array currently... Not the best idea. So I would need to have enough storage space on my serer to copy that over, and then reorganize that array into either a larger disc Z3 array, or make it only 8 drives wide.

This is why this gets tricky.....

 

[joeschmuck]

This is just my opinion but if your large data store is going to be video content such as streaming movies, then RAIDZ2 is sufficient. I originally started to rip every movie I had just so it was available via Plex but eventually I started to delete these movies when I realized that the odds of my watching most of these movies again are very slim. I have certain movies which I know I'll watch again such as Independence Day, Dead Pool, The Incredibles, Salt, Tangled, and Mulan (yes, my daughter and grandkid movies are probably a higher priority than my own and I've own the VHS of Mulan and two DVDs now so putting this one on the server was a smart move).

Lets talk about RAIDZ2 for a quick second, this means you can loose 2 drives and your data is still available. If you loose a third drive then all your data is gone. You can of course go with RAIDZ3 if you desire, it's really up to you. If you have really important data like financial data or photographs, you should also back those up routinely to other media just in case your FreeNAS machine becomes toast (struck by lightning maybe).

If you really want redundancy then you need to go for something like two or more mirrored vdevs in a RAIDZ3 configuration, but I'm not sure it's worth it.

My advice is to plan your storage needs to last 3 years at which point you are likely to start replacing drives and then you can bump those up in capacity if desired. Also, if you pile yourself with a lot of hard drives, while the initial investment it quite a bit, it's going to still be costly to maintain the pool because drives will fail and if they are not under warranty then there is more cost. So what storage size do you think you really will need over the next 3 years?

his just became a very hard optimization problem lol.

Naw, I think you mean how to best utilize your drives. Optimizing isn't in the equation since you are really servicing a slow network connection and you will have no issues flooding it with data.

 

[joeschmuck]

The other issue is.... trying to get all the data onto the new server.

I have 9 drives in a RAID Z2 array currently... Not the best idea. So I would need to have enough storage space on my serer to copy that over, and then reorganize that array into either a larger disc Z3 array, or make it only 8 drives wide.

This is why this gets tricky.....

A large stack of DVD-R media ;)

 

[LIGISTX]

This is just my opinion but if your large data store is going to be video content such as streaming movies, then RAIDZ2 is sufficient. I originally started to rip every movie I had just so it was available via Plex but eventually I started to delete these movies when I realized that the odds of my watching most of these movies again are very slim. I have certain movies which I know I'll watch again such as Independence Day, Dead Pool, The Incredibles, Salt, Tangled, and Mulan (yes, my daughter and grandkid movies are probably a higher priority than my own and I've own the VHS of Mulan and two DVDs now so putting this one on the server was a smart move).

Lets talk about RAIDZ2 for a quick second, this means you can loose 2 drives and your data is still available. If you loose a third drive then all your data is gone. You can of course go with RAIDZ3 if you desire, it's really up to you. If you have really important data like financial data or photographs, you should also back those up routinely to other media just in case your FreeNAS machine becomes toast (struck by lightning maybe).

If you really want redundancy then you need to go for something like two or more mirrored vdevs in a RAIDZ3 configuration, but I'm not sure it's worth it.

My advice is to plan your storage needs to last 3 years at which point you are likely to start replacing drives and then you can bump those up in capacity if desired. Also, if you pile yourself with a lot of hard drives, while the initial investment it quite a bit, it's going to still be costly to maintain the pool because drives will fail and if they are not under warranty then there is more cost. So what storage size do you think you really will need over the next 3 years?


Naw, I think you mean how to best utilize your drives. Optimizing isn't in the equation since you are really servicing a slow network connection and you will have no issues flooding it with data.

Good points, and my optimization I did mean utilization. And actually, no joke tho, a large stack of DVD's legit may be the best idea hahaha

 

[joeschmuck]

Well I hope you don't need DVDs to make the migration but it could be your only option. How much data you you currently have to transfer?

 

[LIGISTX]

Well I hope you don't need DVDs to make the migration but it could be your only option. How much data you you currently have to transfer?

Yea, I know, DVD won't actually work haha.

My current NAS has 17 TB of data on it. And that is across a 9x3TB array in Z2. I would like higher than 20% redundancy though... so I can't just take them out of my current NAS and pop them into my new server. I assume if that 20% Z2 was satisfactory I in theory could just do that, right? Or do I HAVE to create a new array under freenas to get all the ZFS goodies? Can I just bring in a vdev from open media vault? Maybe if I deem Z2 is adequate, I will just build another vdev on top of bringing in my "old" vdev from my NAS and call it a day.

 

[joeschmuck]

You very well might be able to import your current ZFS pool without issues. BUT, if you do this, DO NOT UPGRADE YOUR POOL or you won't be able to go back to your old system if for some reason you didn't like what you were getting into. Once you are happy with FreeNAS and know you are staying, you can upgrade your pool if you like.

But with all this said, you have to know that there is always some risk so ensure your important data is backed up.

17TB, that is a bit of data.

EDIT: I think you will have too much data on that pool and FreeNAS will jump into it's optimization mode and things will slow down. Maybe you could move some of the data off the drives first, maybe a few TBs.

 

[LIGISTX]

You very well might be able to import your current ZFS pool without issues. BUT, if you do this, DO NOT UPGRADE YOUR POOL or you won't be able to go back to your old system if for some reason you didn't like what you were getting into. Once you are happy with FreeNAS and know you are staying, you can upgrade your pool if you like.

But with all this said, you have to know that there is always some risk so ensure your important data is backed up.

17TB, that is a bit of data.

What do you mean upgrade my pool?

Also, my current Z2 array is not ZFS. I am not sure what exactly it is actually.... I think it is EXT4 or something like that. I am not there atm so I can't look, and I don't really pay attention to it... It has been running fine for years, and its a pretty simple OS/software. Didn't take nearly the level of detailed learning freenas and ZFS has/is taking.

 

[joeschmuck]

What do you mean upgrade my pool?

There are different versions of ZFS, mainly extra feature sets. If you upgrade the ZFS pool then you have upgraded the feature set and thus cannot use a previous version of ZFS to mount the pool. I have not upgraded my ZFS pool in years because the feature sets don't benefit me and I have the option to roll back to FreeNAS 8.1 (or around that verion) if I really wanted to.

I am not sure what exactly it is actually.... I think it is EXT4 or something like that.

Sorry, I thought you previously stated it was ZFS. Well then you will not be importing your drives with the data on them. You will be forced to save all your data off and create the pool/vdev and then load your data back onto the system. 17TBs will take a long time. If you save the data to say a USB hard drive (or many) which I believe must be formatted as NTFS or FAT32, you can setup the USB connection to automatically copy all the data on to the pool. It's not a selective process, it is a simple copy and fairly fast. I don't know if you can do that with a SATA connection, never tried it.

I think you do have a lot to think about.

 

[LIGISTX]

There are different versions of ZFS, mainly extra feature sets. If you upgrade the ZFS pool then you have upgraded the feature set and thus cannot use a previous version of ZFS to mount the pool. I have not upgraded my ZFS pool in years because the feature sets don't benefit me and I have the option to roll back to FreeNAS 8.1 (or around that verion) if I really wanted to.


Sorry, I thought you previously stated it was ZFS. Well then you will not be importing your drives with the data on them. You will be forced to save all your data off and create the pool/vdev and then load your data back onto the system. 17TBs will take a long time. If you save the data to say a USB hard drive (or many) which I believe must be formatted as NTFS or FAT32, you can setup the USB connection to automatically copy all the data on to the pool. It's not a selective process, it is a simple copy and fairly fast. I don't know if you can do that with a SATA connection, never tried it.

I think you do have a lot to think about.

Ok, well.... I have figured it out!

I do have another 3 TB RAID array in my HTPC. Long story... but the NAS is shared with some friends, so my HTPC is my own local array...

Without explaining that toooo much, some of the data on my HTPC array is duplicated from the nas. SO, what this means is, we can buy a few more 3 TB drives to create a 8x3 TB vdev on the new server. That will give us 12-13 TB of usable space with the 80% capacity of ZFS factored in. Would be able to copy via local network from the old NAS to the new server. Then the remaining few TB isn't a big deal at all

 

[LIGISTX]

Sent from my jailbroke iPhone using Tapatalk

 

[LIGISTX]

Ok, so.... One final question now.

Our current drives are seagate 3TB's. I don't have the model number off hand, but I pretty much guarantee they are 7200 rpm units.

Knowing what I know now (this Nas was built YEARS ago), I would have done WD greens, specifically firmware modded to reds or at least to change the head parking rate.

I wanted to build the new array with firmware modded greens, but.... I don't know if mixing and matching greens with 7200 rpm drives is a good idea. If it doesn't matter and will just result in the 5900 slower speeds, I don't really care. But if it is a bad idea for other reasons beyond speed or some weird wear factor where the drives have to work extra hard to try and match each other, maybe I should stick with these vdevs being identical type of drives? Any input...?


Sent from my jailbroke iPhone using Tapatalk

 

[LIGISTX]

Damnit. That just isn't THAT much of a space increase. Would it be unwise to do a 9x3TB Z2 array? It would give an upgrade option in the future with 6x5TB Z2, and make the best use of the my current 3TB drives. Just worried if 9xZ2 is a sketchy.

That being said. This isn't mission critical data. If it was all lost I would be sad, for a while. But nothing would ACTUALLY result from that. No clientele to disappoint, no monetary bank data would be lost etc.


Sent from my jailbroke iPhone using Tapatalk

 

[Robert Trevellyan]

Except that won't give me enough space if I use my current 3TB drives, and if I don't use my current 3TB's I am not utilizing over a thousand bucks worth of drives I already own.

Three six-disk RAIDZ2 vdevs?

 

[LIGISTX]

Three six-disk RAIDZ2 vdevs?

Not enough space. I need to get the total usable pool size over 30 TB and have room to grow in the future. Which is why I am now wondering if this is an OK solution:

Sent from my jailbroke iPhone using Tapatalk

 

[Robert Trevellyan]

It's generally recommended to make all vdevs in a pool as similar as possible. I would be more inclined to diverge on disk size than in number of disks per vdev:

6x 3TB
6x 3TB
6x 5TB

would give you very roughly 44TB, with 80% utilization at around 35TB. To expand, you could add one more vdev, or replace all the disks in an existing vdev.

 

[LIGISTX]

It's generally recommended to make all vdevs in a pool as similar as possible. I would be more inclined to diverge on disk size than in number of disks per vdev:

6x 3TB
6x 3TB
6x 5TB

would give you very roughly 44TB, with 80% utilization at around 35TB. To expand, you could add one more vdev, or replace all the disks in an existing vdev.

Is there any particular reason for this? Unfortunately I am finding myself in a bit of a weird situation. I didn't properly plan all of this out years who when the current Nas was created, or a few months ago when the additional 3TB's where purchased.

Is vdev similarity just there to try and improve performance....?




Sent from my jailbroke iPhone using Tapatalk

 

[LIGISTX]

Ok. To summarize this somewhat all over the place thread. Lots of thinking out loud, and trying to figure out a future state plan while maximizing the drives I already own.

I have 12 3TB drives right now. And I have 4 3TB drives to help out the migration process (these 4 will not be in the server, but they will help me augment my migration).

Currently, my plan if it makes sense... Is to run 2 vdevs, each with 9x3TB in Z2. I know this is only 22% redundancy, but that data isn't mission critical. Is this not safe? I know cyberjock has a 10x6TB array, but I also know he has extensive knowledge of ZFS and freenas where I do not. That being said, I plan to follow every guide I can find and setup smart monitoring, email notification, literally everything I can to have a full view of my drives health. Also, if I do lose a drive, I plan to shut down the server until a new one is acquired and in my physical possession to swap out the bad drive, so the server will never be running in a degraded state (except during the rebuild.... Obviously).

My next concern is mixing drives. I am current using seagate drives, if I could do it over I would probably go WD greens, but obviously I can't go back in time... Yet ;)

Would it be a bad plan to mix and match seagate 7200 drives and WD greens? Besides the entire vdev being brought down to the slower speed, is there an issue with this? Speed is not a concern, this Will saturate gigabit which is all I require. I just don't want unneeded wear on the drives trying to keep up or slow down to match the other drives.

Thanks for all the help guys!


Sent from my jailbroke iPhone using Tapatalk

 

[gpsguy]

How many hours do you have on the existing 3 TB drives?

Have you looked at their SMART data? Any problems with them?

I am not there atm so I can't look, and I don't really pay attention to it... It has been running fine for years, and its a pretty simple OS/software.

Like Robert Trevellyan, I'd recommend 6 disk RAIDz2 vdevs. Sure, you'll lose 1/3 of the space to overhead, but you'll only need an additional 6 disks in the future, to add another vdev to the pool.

I'd look at starting with 6x6TB drives and copying your existing data onto it. And, add another 6x3TB vdev using your existing disks. Depending on age, condition, remaining warranty on the 3TB drives, you may need to use your extra disks to cover dying disks.

Save up for 6 additional 6TB drives and either add another vdev or replace the 3TB drives with 6TB drives.

Given your use case, I wouldn't buy any more 3TB drives.