Dummy-proof or Dumb-proof? - Heterogeneous array creation.

[Alvinator]

I have been using ZFS for over a year and a half now, primarily on Ubuntu, but I cannot for the life of me figure out how to create (using the gui in FreeNas 9.1.1) an array using several different hdd sizes.

Background, I've got about 2.5 TB of data I backed up to a 3TB drive and verified in order to re-create my array with the 4k sectors. I tried using FreeNas to do so using the remaining drives.

My drives:
1 x 3TB full of data
4 x 3TB
1 x 1TB
1 x 500GB

My plan is to create a Raidz1 using the six drives there, then load all data from the 3TB drive onto it, then swap the 500G for the 3TB to increase my array size to around ~5TB (later to be increased to 15TB when I have my next $150 to spend on the last 3TB drive).

Problem is I open the ZFS volume manager, type in a pretty name, and I've tried a vast assortments thereafter to create a 6x500G RaidZ1 array but it only comes up with being able to do a 4x3TB with a 1TB spare and a 500G spare which I can also choose to stripe. Oh and it won't let me do a Z1, it insists on a Z2 to be "optimal". Anything I manually type in messes it up so I can't seem to do that either. Any help would be appreciated, I'd like to see if creating an array 100% on FreeNas would result in faster speedtests than one transferred from a Ubuntu creation. Lastly I've tried creating a volume of 1x500GB and "Extending" it using the "Volume to extend" drop-down, also to no avail. I am unfamiliar to FreeNAS or the underlying FreeBSD kernel and its accompanying terminology or command prompt syntax so I would like to avoid this if at all possible.

The reason I believe this may be a bug is that I presume there was an effort to make this "dummy-proof", but one would think there would at least be an "advanced" button over-ride or something similar? I believe thustly that it would prove to be pretty dumb without said button, thereby becoming 'dumb-proof'.

I've searched quite a bit for an answer for this.

Lastly, and slightly off-topic, (and I didn't search for this), is there a way to force FreeNAS to use more of a /dev/disk/by-id type labeling system rather than the normal ada0, ada1 type? I frequently swap motherboards and other hardware so it would be nice to ensure if I swap it to something it won't go and accidentally wipe out any of my existing drives on said new system.


Thank you all very much for any help and responses.

 

[Dusan]

The easiest solution right now is to install FreeNAS 8.3.2, create the pool and then upgrade to 9.1.1.
You can also add your voice here: https://bugs.freenas.org/issues/3274

 

[cyberjock]

Surprise! You are 100% correct. It was to prevent people from being stupid and adding single disks to a pool and thinking at the vdev would just add the disk. Lost alot of data that way. iX came up with the new volume manager to prevent people from doing stupid things, but at a price.

FreeNAS doesn't use ada0. It uses GPTIDs. If you use the CLI you'll get it wrong, but the GUI is the way to go. You should use the GUI for everything lest you piss off the FreeNAS gods who may deny you your data someday. :)

Your fix for the volume manager denying you your desired configuration is to install 8.3.1 and use that to create your pool, then upgrade to 9.x. I didn't ever upgrade my pool (still v28 for both of my pool) just in case I need to go back and use the old volume manager someday. Unfortunately, I think every moderator on this forum is still on a v28 pool because of the new "idiot proof" volume manager. Sometimes we want to do something that might be idiotic, but know better. The new Volume Manager is very powerful, but what it gives in convenience it takes in advanced features for those that know better. I've talked to one of the main developers and I can understand their decision(even if I disagree with it). I think its a bad plan to have a software package that takes control before you've even created a place to put your data. It has turned off many people in the forums and IRC who leave with the opinion of "let me do what I want or get out of my way". Others just question that if their first interaction with FreeNAS' WebGUI is "broken" then "What else is wrong with this software package?" Then people start questioning if this software package is worth their time to learn. Considering making a pool is step 1 in using FreeNAS that's not a good way to start.

You can see my opinion by reading my ticket asking for it to be fixed: https://bugs.freenas.org/issues/3274

 

[DrKK]

There's no question at all about this. Any legal ZFS configuration for drives should be permitted in FreeNAS. FreeNAS would be within its rights to warn users about wonktastic configurations (and/or to put an advanced button), but the OP is absolutely right. It is not acceptable that a legal (if dumb) ZFS configuration would be prohibited by the GUI/volume manager.

I vote thermonuclear "yes".

 

[joeschmuck]

Voted. My opinion is already in the ticket.

 

[rcstevensonaz]

I'm dumbfounded. Just came over from NAS4Free only to find out I cannot create perfectly legitimate vdevs (e.g., 3-way mirror of 500GB x 2 + 640GB) or choose which drives to use in the vdev (spread across multiple SAS backplanes).

I also agree with cyberjock that there is risk doing this at the CLI (since the GUI does a lot of management including GPT labels).

So the question for me is do I install earlier FreeNAS simply to create zpools or return back to NAS4Free for now.

 

[Alvinator]

I just ran up a live USB of Ubuntu 12.04 LTS with zfs installed, made my silly array, and then imported it. I also noticed in FreeNas that the ashift=12 (4k sector mapping for 2TB+ drives) was slightly lacking as far as I can remember (a lot has happened in the last 2 weeks), but I did find (when changing over to the release version 9.2) the "record size" advanced option in the create a dataset dialogue, that's good :) I changed over to 9.2 after my friend pointed out that I was on 9.1.1 RC (Release Candidate) which means its a candidate for release, not the actual final release version. Anyways, long story short, I solved it easily-ish, but yeah an "I'm sure I know what I'm doing" button would be nice for the array, I'll add my vote on those links too :)

Thanks to all who responded
...now how do I mark this as solved?? lol

 

[cyberjock]

I'd seriously recommend you create the zpool with FreeNAS. FreeNAS makes assumptions like it creates the pool from the second partition on a disk, aligns it properly and such. If you break these expectations you may find that when you upgrade to a future version of FreeNAS your pool will not function. We've seen alot of people create pools outside of FreeNAS 8.x+ and regretted it. The forum has plenty of users that are forever stuck on 8.x because 9.x won't import their pool.

 

[Alvinator]

Okay I've deleted the array, still have my data on the one 3TB drive, and still have the 4x3TB + 1x1TB + 1x500GB expectation until data is transferred and I can then swap out the 500 for the 3TB. I did this because FreeNAS seems to enjoy running out of swap and locking up, after discovering the "amount to use as swap" option under advanced settings I would like to use this. However, I went to use the cli to create the array in FreeNAS (because I still want to make my silly array which the gui won't allow), but when I used the command "ls /dev/disk/by-id/" it would not do it, I then found out there is no 'disk' directory found in the 'dev' directory, presumably a BSD thing.

So my next question naturally is "where is a /dev/disk/by-id/ type referencing available on FreeNAS?" In Ubuntu this function lists disks by their volume's serial number and/or model number, VERY effective to determine which disk to replace whence needed and VERY good for swapping an array across motherboards or multiple arrays across multiple boards, I am strongly against the use of the "ada0 ada1 ada2..." type naming that FreeNAS defaults to as this could lead to catastrophic failure (if adding other drives to the board in different orders). So how would one reference the drives properly?

In a Ubuntu environment this would be my typical creation line, how would I alter this to be effective in FreeNAS, or at least where can I reference zfs to for disk serial numbers?

sudo zpool create -o ashift=12 DJAlvinT /dev/disk/by-id/serial-of-disk1 /dev/disk/by-id/serial-of-disk2 /dev/disk/by-id/serial-of-disk3 /dev/disk/by-id/serial-of-disk4 /dev/disk/by-id/serial-of-disk5 /dev/disk/by-id/serial-of-disk6

And if I were to use the cli to create the array, after setting the "swap" amount per drive, would the swap value take or would the cli ignore this value?

Thanks again for the responses, it is very much appreciated.

 

[warri]

Check those threads for details on creating a pool from CLI:
- http://forums.freenas.org/threads/c...us-disks-or-how-to-add-volumes-via-cli.16963/
- http://forums.freenas.org/threads/about-to-expand-my-zpool.16067/page-2
- http://forums.freenas.org/threads/create-zfs-mirror-by-adding-a-drive.14880/

The essential steps are (for each disk):
- Create a partitioning schema with gpart create
- Add swap partition with gpart add
- Add ZFS partition with gpart add

Then:
- Verify labels with gpart list
- Create pool with zpool create (using the rawuuids listed for the respective ZFS partitions in gpart list, not the device/partition names like ada0p2!)
- Export pool via CLI with zpool export
- Auto-Import pool via Web GUI

More details are in the threads I mentioned above. Not sure if you have to adapt all the ZFS partitions to match up in size, I guess just give it a try without first and see if zpool create can sort it out.

 

[Alvinator]

Thank you so much for the fast response, I'll try it out straight away :)

 

[Alvinator]

Okay I see what they're doing, they're using GPTIDs for the naming schema, is there any way to alternatively use the serial numbers the disks provide?

For example, halfway down the second link you sent me, mjban provides a code excerpt which demonstrates what is shown below, where the zpool status function lists gptid/xxxxxxxx-xxx-xxxx-xxxx-xxxxxxxxx instead of a nice /dev/disk/by-id/SeagateBarracudaXT-3000M001D43552. I sense this will lead to a huge problem in the future in determining which drive need be replaced on a live system.

**Sorry half-way down the *second* page of your second link, there we go :)

Code:

NAME                                            STATE    READ WRITE CKSUM
NAS                                            ONLINE      0    0    0
raidz2-0                                      ONLINE      0    0    0
        gptid/26e40201-47da-11e3-bae2-bc5ff4e79b13  ONLINE      0    0    0
        gptid/29cc96a7-47da-11e3-bae2-bc5ff4e79b13  ONLINE      0    0    0
        gptid/2b0c66db-47da-11e3-bae2-bc5ff4e79b13  ONLINE      0    0    0
        gptid/2be47a60-47da-11e3-bae2-bc5ff4e79b13  ONLINE      0    0    0
        gptid/2c9e0876-47da-11e3-bae2-bc5ff4e79b13  ONLINE      0    0    0
        gptid/2d6290f2-47da-11e3-bae2-bc5ff4e79b13  ONLINE      0    0    0

 

[Alvinator]

I've just searched google for 20 mins,... where can I find the 8.3.2 RELEASE usb image? the sourceforge link only provides 8.04 and says its maintained now at git-hub, which I venture over to and cannot figure out. :/ Anyone have a permalink? Or a link to the archives?

 

[cyberjock]

cdn.freenas.org

 

[Alvinator]

Epic!! Thank you!!

 

[warri]

No, don't use drive serial numbers - the GUI expects GTPIDs. You can always resolve the GTPID to the actual drive with glabel status. Then you can get the drive's model and serial number with smartctl -a /dev/adaX or camcontrol devlist.

FreeNAS 8.3.2 will use GTPIDs as well.

 

[cyberjock]

If I were you I'd just install FreeNAS 8.3.1, make the pool, then upgrade to 9.2.0. This just makes me wonder what's going to bite you later...