Adding Different Sized Mirrored Sets To Striped Mirrored Pool

[brando56894]

Currently I have two mirrored pools set up:

Storage (6x 4 TB HGST 7200 RPM)
VMs (I'm using FN10), Usenet/Torrent downloads and Multimedia.

SafeKeeping (2x 1 TB WD Red 5400 RPM)
Largely archival storage: pictures, some Windows downloads that were hard to find :), game installers (since I don't have enough room to install them on my pc and store the massive installers), various config backups that I never remember to use and some adult material haha. The pictures are the only really important things, and those are on Facebook, Google+, and Google Drive.

I split out the pools originally when I was learning about ZFS since somehow I would always manage to destroy my Storage pool when adding/removing/replacing a device to/from the pool which would also cause me to lose my "important" stuff. This way, in case Storage was FUBAR I still had my other pool.


I've decided I no longer need to split them apart and would like to have one pool, eventually with all (but 2 with) 7200 RPM drives (I have 13 bays in my case, 11 of which are 3.5"). I also have 2 150 GB WD Raptors which spin at 10K RPM. I was wondering would it slow down my pool over all if I added in the 5400 RPM and 10K RPM drives considering it would have to deal with drives that have 3 different rotational speeds and latencies, not to mention different overall sizes? I have an SLOG attached to the storage pool but I know that's only useful for sync writes.

 

[Spearfoot]

(I'm using FN10)

You do understand that FreeNAS 10 is BETA software? And that you should not use it to store any data which are valuable to you? In brief: do not use FreeNAS 10 to store important data!

Regarding pool layout... IMHO, the 150GB Raptors don't offer enough capacity to warrant adding them to a pool comprised of drastically larger drives such as your 4TB HGSTs. The difference in rotational speed is less troublesome; while not ideal, it's not a show-stopper to mix 5400 and 7200RPM drives in a pool. Adding 10K drives to such a mix? I dunno... I wouldn't do it, again, mainly because the Raptors you have have such small capacity.

If you're considering having 11 disks in a single vdev, you might want to consider using RAIDZ3. The problem, of course, is that you must have all 11 drives installed to create the vdev; you can't add them incrementally as time goes by.

Good luck.

 

[brando56894]

Yes I'm well aware that it's beta software. Nothing I have is "mission critical" and can be reacquired in a few weeks, it's a home server that just largely serves media to myself, my roommate and friends. Like I said all my really important stuff is in the cloud.

I agree that 300 GB is nothing in the grand scope, but I wanted to find a use for them haha. Maybe I'll put them in my Dad's NAS. If I don't use them, not a big loss since I got them for $15 a piece off of NewEgg. I guess I'll add in the 1 TB drives and give it a try.

I decided to go with Mirrors instead of RAIDZ1/2/3 because it's easier to upgrade and doesn't require as much planning, also for read speed benefit. I originally had a Z1 pool and was kind of annoyed to learn that the shiny new 4 TB drive I had just purchased to replace one of the 4 3 TB drives I had in the pool wouldn't show it's true value until a few hundred dollars later when I upgraded the other 3 drives. With Mirrors I can add them in pairs of 2 and see an instant increase in storage space and read performance. By the time I get up to filling the case, I may split it up into two pools, who knows since it will probably be a year or two from now at least.

 

[Spearfoot]

Yes I'm well aware that it's beta software. Nothing I have is "mission critical" and can be reacquired in a few weeks, it's a home server that just largely serves media to myself, my roommate and friends. Like I said all my really important stuff is in the cloud.

Good! So long as you know what you're doing... :)

I agree that 300 GB is nothing in the grand scope, but I wanted to find a use for them haha. Maybe I'll put them in my Dad's NAS. If I don't use them, not a big loss since I got them for $15 a piece off of NewEgg. I guess I'll add in the 1 TB drives and give it a try.

Yeah, 10K stuff is obsolete and soon to be completely replaced by flash storage. It's the old "I've got these gizmos, now what the heck do I do with 'em" conundrum.

I decided to go with Mirrors instead of RAIDZ1/2/3 because it's easier to upgrade and doesn't require as much planning, also for read speed benefit. I originally had a Z1 pool and was kind of annoyed to learn that the shiny new 4 TB drive I had just purchased to replace one of the 4 3 TB drives I had in the pool wouldn't show it's true value until a few hundred dollars later when I upgraded the other 3 drives. With Mirrors I can add them in pairs of 2 and see an instant increase in storage space and read performance. By the time I get up to filling the case, I may split it up into two pools, who knows since it will probably be a year or two from now at least.

Ummm... sorry, I missed that you're using mirrored vdevs. So, as you point out, you can add pairs incrementally to your pool as time goes by. You get the most IOPS with mirrors, and the rotational speed issue pretty much goes away as long as you use pairs matched in speed. Space efficiency - 50% - ain't so great, but you can't have everything, right?

 

[brando56894]

I purchased the raptors specifically to host ESXi when I was using it, I was looking at getting a cheap SSD but the cheapest decent sized one I could find was a lot more than $30 for the two raptors haha

Yea, sorry I forgot to mention that they were mirrored vdevs. Nice to hear that the rotational speed isn't an issue, since they are all paired correctly. I've always wanted to go with RAID10 (back when I was first using RAID) but sacrificing half of my space didn't seem too great to me, especially when I only had like 5 or 6 TB to begin with, now that I have a total of like 23 TB it's not that much of a problem now haha

 

[brando56894]

I did it and seemingly it added about 100 MB/sec to my write bandwidth and about 50 MB/sec to my read bandwidth.

The results I'm getting are counter-intuitive to what I know about Mirrors: I thought there was a read benefit since there were more drives to read from and writes kind of suffered since it had to duplicate the data. I guess it should be said that I do have an SLOG attached to the pool which could be the reason for the increased write performance.

Code:

Storage, 120 GB File using dd

Write: 395 MB/sec 

Read: 235.5 MB/sec 


Write: 502 MB/sec (after adding a 1 TB mirrored vdev)

Read: 276 MB/sec (after adding a 1 TB mirrored vdev)

 

[darkwarrior]

I did it and seemingly it added about 100 MB/sec to my write bandwidth and about 50 MB/sec to my read bandwidth.

The results I'm getting are counter-intuitive to what I know about Mirrors: I thought there was a read benefit since there were more drives to read from and writes kind of suffered since it had to duplicate the data. I guess it should be said that I do have an SLOG attached to the pool which could be the reason for the increased write performance.

Code:

Storage, 120 GB File using dd

Write: 395 MB/sec

Read: 235.5 MB/sec


Write: 502 MB/sec (after adding a 1 TB mirrored vdev)

Read: 276 MB/sec (after adding a 1 TB mirrored vdev)

Hey there,

Edit: Nevermind I just realized that you are using dd....

 

[brando56894]

One odd thing I have noticed is that copying from one dataset to another datatset in the same volume causes the transfer speed to max our at 45 MB/sec. any idea why?