Optimizing mixed disk setup with ZFS

[GaiusMinimus]

I apologize if this question has been answered before. I searched "mixed disk" in the search and didn't see any results.

I currently have a mixture of disks I would like to use in my ZFS setup. I would like some redundancy (be able to lose at least 1 drive) so I believe a raidz setup is ideal, but the mixture of disks is unusual. Can someone suggest some pointers on optimizing my setup?

Here's my disk mixture:

1x 240 GB SSD
1x 4 TB 7200
3x 2 TB 5400 (very slow "green" drives)

How would you suggest I organize my ZFS pools to optimize storage, speed, and redundancy?

Currently I have two pools and the SSD split into separate cache volumes:

OS installed to 4 TB disk
3x 2 TB disk used in a raidz mounted separately
SSD split into two logical volumes and added as a cache to both the OS and the disks.

My problem is there's no redundancy with the OS and I'm not sure about the SSD split. Would it be better to create a 150 GB partition on both the SSD and 4 TB disk, then install the OS in a raidz setup for this pool? Then I could create the raidz with the 3x 2 TB disks. I could use the remaining 70 GB ssd space as a cache for the raidz?

Thank you all in advance for this setup. Please feel free to point out anything wrong in my setup. Your help is greatly appreciated!

 

[pirateghost]

Wow.

You are wasting a 4tb disk for the OS?

You're partitioning drives and using them in zfs pools? You're a brave soul.

Get 2 USB thumb drives, or 2 small ssds. Install the OS to the pair (it's an option when you install or you can mirror the OS drive after the install, again, supported in the GUI).

Throw 2 2tb disks in a mirror
Add to a pool
Throw remaining 2tb and 4tb disk in another mirror vdev, extend the pool with it.

When you start replacing disks, you can replace them with larger disks until they are all 4tb disks(or whatever size you desire) or you can continue to add mirrored vdevs to get more space.

Your SSD cache is pretty worthless at this point. What are you gaining by using it?

 

[joeschmuck]

Wow, what a mess.

My advice is a bit different from @pirateghost so I'm going to post it.

I'm not a fan of mirrored boot devices unless they are driven from a true RAID card nor USB Flash drives as the FreeNAS OS but for your situation I'd recommend either a single USB Flash Drive (8GB or larger) or a SSD. You could use that 240GB SSD but that is such a waste however you could use it and while there are all these crazy holiday sales going on, buy a cheap SSD (32GB or larger, whatever is a good deal) to replace it later. Now since you are using a single boot device you should always practice backing up your configuration file. Heck, you should do that even with a mirror OS. The thing is, having your configuration backup file will allow you to reinstall the FreeNAS OS and then just restore the config file and all is good again. It's the easiest restoration you will ever find.

Next comes how to configure your pool and you could go with the mirrored vdevs as my friend stated above or go with placing all four into a RAIDZ2 vdev. No matter how you slice it the 4TB drive will effectively only be a 2TB drive. If you replace the 2TB drives with 4TB drives then the pool will POP to all drives being recognized as 4TB each but not before. The mirrored version listed above is a bit different but similar.

There are pros and cons to each implementation and you should read about each one and figure the one you like the most. I would recommend you create both versions for the heck of it just to see the differences, and read up on it.

Good Luck!