Importing FreeNAS created pools on other ZFS systems?

[DaPlumber]

Has anyone tried importing a zpool created on FreeNAS on another non-FreeNAS, preferably non-FreeBSD based system? Obviously encryption isn't going to work, but how "transportable" are FreeNAS pools? I'm especially interested in anyone's tried this with versions of Solaris and/or OpenZFS on OS X? Were there any issues/land-mines?

 

[cyberjock]

This question is asked at least 2-3 times a week. I answered this in detail a few days ago. ;)

 

[DaPlumber]

My search-fu must be broken because I saw plenty of threads about importing pools into FreeNAS, but nothing on the other way around: Importing FreeNAS GPT'ed, formatted, etc pools into other ZFS implementations. I'm guessing for a start the importing system has to deal with the GPT, the swap partition, and be able to find the partition with the ZFS label on it, rather than glom the whole drive the way I'm used to, but is there anything else?

I'm thinking of doing it this way around precisely BECAUSE of all those horror stories, I'm happy to let FreeNAS "drive" the format, but I'm looking for some experience in what I'm going to have to deal with.

 

[cyberjock]

Oh, you're right. I thought it was the reverse. That's what I get for answering a question while putting my shoes on.

It should work, but any ZFS nazi will tell you that you shouldn't as a course of business cross between different OSes except as an emergency measure, and then only short term.

But, in terms of compatibility, as long as the OS supports the ZFS version and feature flags if v5000 then it should be importable. If some feature flags aren't supported you *may* be able to mount read-only, but not always.

In short, don't expect to, and you really shouldn't try to.

 

[DaPlumber]

I AM a (recovering) ZFS nazi. :D

My immediate use case is pretty corner: I have a bunch of archived VM images on various external drives (boo, hiss!) on a MAC and an elderly Solaris Workstation. Not having the luxury of 10GbE I thought I'd use mirrored 2 disk zpool for transport. eSATA both ends to get the files in the more permanent archive on the shiny new FreeNAS. I'm the impatient and/or forgetful sort so multi-hour network melting's don't really appeal.

OpenZFS on the MAC is also v5000/v28 plus attributes and we'll see how "compatible" the Solaris 11.1 box finds it...

 

[cyberjock]

I'm almost 100% sure that no OS out there supports all the feature flags that FreeNAS uses except FreeBSD 10 current.

 

[DaPlumber]

I guess I'm going to find out the hard way how many of those are "features for write".

Hmmm. Nice of them to add it to "zpool upgrade -v":

Code:

This system supports ZFS pool feature flags.
 
The following features are supported:
 
FEAT DESCRIPTION
-------------------------------------------------------------
async_destroy                        (read-only compatible)
    Destroy filesystems asynchronously.
empty_bpobj                          (read-only compatible)
    Snapshots use less space.
lz4_compress                     
    LZ4 compression algorithm support.
multi_vdev_crash_dump             
    Crash dumps to multiple vdev pools.
spacemap_histogram                    (read-only compatible)
    Spacemaps maintain space histograms.
enabled_txg                          (read-only compatible)
    Record txg at which a feature is enabled
hole_birth                       
    Retain hole birth txg for more precise zfs send
extensible_dataset               
    Enhanced dataset functionality, used by other features.
bookmarks                            (read-only compatible)
    "zfs bookmark" command
 
The following legacy versions are also supported:
 
VER  DESCRIPTION
---  --------------------------------------------------------
1  Initial ZFS version
2  Ditto blocks (replicated metadata)
3  Hot spares and double parity RAID-Z
4  zpool history
5  Compression using the gzip algorithm
6  bootfs pool property
7  Separate intent log devices
8  Delegated administration
9  refquota and refreservation properties
10  Cache devices
11  Improved scrub performance
12  Snapshot properties
13  snapused property
14  passthrough-x aclinherit
15  user/group space accounting
16  stmf property support
17  Triple-parity RAID-Z
18  Snapshot user holds
19  Log device removal
20  Compression using zle (zero-length encoding)
21  Deduplication
22  Received properties
23  Slim ZIL
24  System attributes
25  Improved scrub stats
26  Improved snapshot deletion performance
27  Improved snapshot creation performance
28  Multiple vdev replacements
 
For more information on a particular version, including supported releases,
see the ZFS Administration Guide.

As long as I didn't detach the pool in the middle of one of those functions I think I'm OK. Maybe I should try a send|recv test over the network first and see if that throws any errors. It would be nice if I can get that to work locally with the imported pool, it would save a lot of monkeying about with worrying about copying files with rsync. I may have to manually recreate the zpool and back the version off to the lowest common denominator if worst comes to worst.

Portability and compatibility was one of the original design goals, so it's going to be "interesting" to see how well that's survived. Like I said "recovering"... :D

 

[cyberjock]

If you want portability and compatibility you should do what I do. My pool was created with 8.3.1, then upgraded over time to where I am now (9.2.0) and never upgraded my pool. It's still on v28 as I type this with no intention to upgrade anytime soon.

 

[solarisguy]

There are some obvious rules...

If your pool comes from FreeNAS 9.2.1 or later and your target system does not have feature@hole_birth, you are out of luck.

If your pool comes from FreeNAS 9.x or later, then Solaris would not be able to import it.

Please, please share with us results of your experiments.

 

[DaPlumber]

Nope. Definitely had to back out the pool version. Solaris really doesn't like features.

Looks like the most compatible/simplest option for transport pools is a V28 pool, whole disk, no partitioning. FreeNAS bitches about the lack of GPT but imports it anyway. For transport only I think this is acceptable. I can import from the FreeNAS GUI, so no violating the "it's an appliance, D**N it!" rule. Probably could be created on the FreeNAS CLI, but I'll stick to doing that elsewhere.

Unfortunately my play/experimentation time budget has just been drastically reduced so that's going to be it for now. Sigh. Darn bills need to be paid, etc. :(

 

[solarisguy]

Just to clear one bit, there is always a partition or a slice (on SPARC).

 

[cyberjock]

I will warn you that pools not created on FreeNAS from the WebGUI have a habit of "failing" suddenly and permanently. :P

 

[DaPlumber]

Just to clear one bit, there is always a partition or a slice (on SPARC).

Eh, Unless it's a root pool you don't need the SMI (VTOC) or EFI label/partition, see: http://docs.oracle.com/cd/E23824_01/html/E24456/device-3.html I seem to remember back in the day there was a dummy slice number that meant "the whole disk" but I can't remember.

"Core" ZFS doesn't really care about partitions/slices/yada,yada it's quite happy with pretty much any block device. As a fun exercise it's possible to create a RAID-Z where each "disk" is a mirrored VDEV. Practically useless, but a fun demonstration of ZFS' abstraction from underlying hardware. :D In FreeNAS ELI encryption depends on this fact.

 

[DaPlumber]

I will warn you that pools not created on FreeNAS from the WebGUI have a habit of "failing" suddenly and permanently. :p

Enquiring minds would like to know "why?" :p Seriously, is it because the GUI assumes the features are always present on a pool or what? Trying to serve something up from a non-FreeNAS formatted pool would be asking for trouble IMO. When I say "transfer only", I mean it. Call me paranoid...

 

[solarisguy]

@DaPlumber, that Oracle documentation you had referenced, clearly says that ZFS needs either an fdisk partition or a slice (that might take the whole disk) ;) VTOC is now only required for root

@cyberjock, FreeNAS has certain rules and they must be followed. However, there is nothing magical about them. Well..., maybe inability to clone the USB boot device :D is somehow magical

 

[cyberjock]

@cyberjock, FreeNAS has certain rules and they must be followed. However, there is nothing magical about them. Well..., maybe inability to clone the USB boot device :D is somehow magical

I agree, but over 2 years of watching pools fail with no reason tells me you are sadly mistaken. FreeNAS expects pools to work out a certain way, and history has shown if you try to bend or break those expectations things can go very badly for you and your data.

This isn't open for debate either. It's very well documented throughout the threads of people that have been victims. If you think you know better or want to do it, feel free. You'll get zero sympathy from me and most of the more experienced forum posters when you do what we tell you not to do and you end up where we think you'll end up. ;)

 

[titan_rw]

I will warn you that pools not created on FreeNAS from the WebGUI have a habit of "failing" suddenly and permanently. :p

Some of my pools have been created by me, from the freenas CLI. However I partitioned the drives the same way the GUI does before doing the zpool create command.

The last one I actually created with a bunch of /dev/mdX devices supported by sparse files so I could fail them out of the pool and add real disks in later once I migrated all my data.

Here's the actual command I used:

Code:

zpool create -f nas1pool raidz3 /dev/md3.nop /dev/md4.nop /dev/md5.nop /dev/ada0p2 /dev/ada3p2 /dev/da0p2 /dev/da1p2 /dev/da2p2 /dev/da3p2 /dev/da6p2 /dev/da7p2

Then I failed out the 3 'fake' drives so I had basically an 8 drive stripe. Then copied all the data to the pool. Then resilvered in real drives in place of the mdX's.

This pool continues to function perfectly. Auto imports, zpool status shows the drives by their guid, like any GUI created pool does.

I'm not advocating using the CLI, but I'm in no fear of my pool magically disappearing because I created it from the CLI.

 

[cyberjock]

That's not correct for FreeNAS after 8.0-beta or so. ;)

 

[titan_rw]

Worked fine for me on 8.3 and 9.1.

 

[cyberjock]

Great.. fantastic. I'm happy for you. Doesn't make it a smart choice or a choice that won't backfire someday. ;)

(This argument is just like the people that argue that ECC isn't necessary. Bunch of people could probably run non-ECC and get away with it, maybe even for years. But that doesn't make it smart and it doesn't mean that tomorrow will be the day you get stuck down by the ZFS gods.)

We've had lots of people argue against stuff that is said here, and more often then not they do something simple like an OS upgrade and *poof* game over. Me personally, I'm going to give the conservative advice that it's not smart to do it. I'd rather people do it right than lose their data later and be mad at me for giving them bad advice. ;)