Updating from Nexenta to a new, upgraded FreeNAS build

[Ericloewe]

Nexenta derives from Open Solaris and hasn't been updated since October 2012.

What? Nexenta is involved in OpenZFS.

Has anyone tested ZFS interoperability that far back?

Even if that were the case, yes, ZFS is extremely backward-compatible.

 

[rvassar]

What? Nexenta is involved in OpenZFS.

Nexenta the company very much still exists and has other products, but the original Open Solaris derived Nexenta OS is long EOL'ed... October 2012. I guess I'm guilty of assuming the OP is using the Open/Free version.

Even if that were the case, yes, ZFS is extremely backward-compatible.

Honestly, I've never tried mounting a historical "Oracle ZFS" versioned pool on an OpenZFS system. I walked away from Oracle when they RIF'ed me and never looked back. Bryan Cantrill's comment about "then the Nazi's took over" is perhaps a bit hyperbole, but having lived thru it, I can't fault him for feeling that way. Anyhow... The feature flags enabled pool versioning came in at some point around 2012, but was not adopted by Oracle. It should be good going from any historical pool version to the feature flags enabled version, but if I really was planning a "zfs send" from a 6 year old pool, I'd want to do a little smoke testing. It's probably safe to assume iX and/or OpenZFS test against current Oracle ZFS, and recent OpenZFS pools, but as a general rule, EOL is well... EOL.

 

[Chris Moore]

It's probably safe to assume iX and/or OpenZFS test against current Oracle ZFS, and recent OpenZFS pools, but as a general rule, EOL is well... EOL.

I would not expect they test against current Oracle ZFS, because that would require that they buy a system from Oracle. Wouldn't it?
I have done a send | receive between ZFS on Linux and FreeNAS (both directions) at work and it has not given me any trouble.

 

[rvassar]

I would not expect they test against current Oracle ZFS, because that would require that they buy a system from Oracle. Wouldn't it?
I have done a send | receive between ZFS on Linux and FreeNAS (both directions) at work and it has not given me any trouble.

No. I'd set up a use case like that in some kind of Cloud offering, and rent it by the hour. Sun had a Cloud effort all the way back in 2009 at least. I have the tee shirt and wear it work every now and then, just to confuse the AWS & Azure people. Oracle offered Solaris both SPARC & X64 PaaS by the hour at one point at least as recently as 2016. I don't see them at the moment, but I only took a quick look. Oracle may have finally thrown in the towel on SPARC. But I can't imagine there isn't a Solaris x64 VM there buried in their cloud offerings somewhere.

 

[Ericloewe]

Oracle ZFS is essentially a completely different thing starting with pool version 29. There is no expectation of compatibility

 

[Jatrabari]

The fastest way is to connect all the drives (temporarily) to a single system. It is then possible to locally do a zfs send | zfs receive from one pool to the other within the same system and it goes as fast as the drives are capable of doing.

Honestly, I've never tried mounting a historical "Oracle ZFS" versioned pool on an OpenZFS system. I walked away from Oracle when they RIF'ed me and never looked back. Bryan Cantrill's comment about "then the Nazi's took over" is perhaps a bit hyperbole, but having lived thru it, I can't fault him for feeling that way. Anyhow... The feature flags enabled pool versioning came in at some point around 2012, but was not adopted by Oracle. It should be good going from any historical pool version to the feature flags enabled version, but if I really was planning a "zfs send" from a 6 year old pool, I'd want to do a little smoke testing. It's probably safe to assume iX and/or OpenZFS test against current Oracle ZFS, and recent OpenZFS pools, but as a general rule, EOL is well... EOL.

Oracle ZFS is essentially a completely different thing starting with pool version 29. There is no expectation of compatibility

I am very hesitant about mounting old Nexenta ZFS pools to the new server because of the possible compatibility issues that may arise, I want to keep them separate and the configuration of the new server as it is and also I don’t have any cables for extra drives. I am prepared to transfer data file by file using “conventional” transfer methods. I think I could use zfs send receive commands to backup the main pool to the backup pool.

I looked at md5 and checksumming. Do I understand this correctly that I output the checksums a separate file, then transfer it to the new server and verify checksums there? And as there are many files to check and I will put the server to work over night I will output the results to a file again and then check later the results? Is there any benefit md5 vs. sha checksumming? I gather that for example sha256 would take more resources and time to calculate that basic md5.

 

[Chris Moore]

I looked at md5 and checksumming. Do I understand this correctly that I output the checksums a separate file, then transfer it to the new server and verify checksums there?

If you are doing a ZFS send and receive, there is only a minuscule chance that any data might be corrupted as it traverses the network. I don't think it would be needed to do additional checksumming. What you might do is after the initial send and receive, follow it up with a rsync to check that all the files are the same.
How much data are you looking to copy here?

 

[Ericloewe]

send/recv is checksummed, so corruption is massively unlikely if the operation succeeds.

 

[jgreco]

I am very hesitant about mounting old Nexenta ZFS pools to the new server because of the possible compatibility issues that may arise, I want to keep them separate and the configuration of the new server as it is and also I don’t have any cables for extra drives. I am prepared to transfer data file by file using “conventional” transfer methods. I think I could use zfs send receive commands to backup the main pool to the backup pool.

I looked at md5 and checksumming. Do I understand this correctly that I output the checksums a separate file, then transfer it to the new server and verify checksums there? And as there are many files to check and I will put the server to work over night I will output the results to a file again and then check later the results? Is there any benefit md5 vs. sha checksumming? I gather that for example sha256 would take more resources and time to calculate that basic md5.

The likelihood that a corrupt file would match a checksum without nefarious influence is virtually zero. As a result, you can use a fast algorithm. Or two fast algorithms.

If you feel like porting mtree over to Nexenta and compiling - versions are available for Linux, Solaris, etc. - this is by far the best tool to use to generate file manifests, because it is able to check permissions, ownership, etc.

What you then do is run something like

# mtree -c -p /pool -k uid,gid,mode,size,time,type,flags,ripemd160digest,sha1digest > /tmp/original

and then the same on the new server. Nexenta may not have flags (or the same kind of flags) so you might need to omit that. Once you have your two manifest output files, you compare them with diff. Ideally they are identical.

 

[Jatrabari]

How much data are you looking to copy here?

3-4 TB

I don't think it would be needed to do additional checksumming.

send/recv is checksummed, so corruption is massively unlikely if the operation succeeds.

Mainly md5 or other additional checksumming was intended for FTP, ssh or other file transfer method, before and after. I believe that zfs send/receive will take care of data integrity for sure.

If you feel like porting mtree over to Nexenta and compiling - versions are available for Linux, Solaris, etc. - this is by far the best tool to use to generate file manifests, because it is able to check permissions, ownership, etc.

Truth to be told, I am not that proficient to think about compiling something but thanks for the info and option.

I have to read up on zfs send/receive a little so I know what it does and how it does it.

 

[Chris Moore]

3-4 TB

If that is all the data you have; from my perspective, that is not much data at all. I have enough storage to save that in my desktop computer.

What was the question again?

 

[Jatrabari]

What was the question again?

What would be the best and efficient way to copy data only from Nexenta to FreeNAS and verify with checksums that data is ok.

Doesn't a snapshot that zfs send receive makes for the stream, to my understanding, also copy the zfs version information etc. metadata?

 

[Chris Moore]

Doesn't a snapshot that zfs send receive makes for the stream, to my understanding, also copy the zfs version information etc. metadata?

Send and receive will do the job. The properties of the old pool do not override the properties of the new pool. I have used it several times to move data from one pool to another going from RedHat Linux (ZoL) to the ZFS that FreeNAS uses and moving between FreeNAS systems when migrating to newer hardware.

 

[Evertb1]

PSU frying everything in the server, someone kicking the server, drunk/sleepy/inattentive admin destroying everything, fire, etc.

Take this seriuously. About 11 months ago my UPS shut down the power to my server and was shouting all kinds of alarms. When I checked things I discovered that my servers PSU died and there apperently was something really wrong because one of the modulair connectors was partially melted. Luckely none of the components of my server was affected but I am still wondering what would have happened if I did not have a reliable UPS.

 

[Jatrabari]

I did not have a reliable UPS.

What is your UPS make and model? I am planning to get an UPS also.

 

[Jatrabari]

Send and receive will do the job.

From FreeBSD man page:

Example 12 Remotely Replicating ZFS Data

The following commands send a full stream and then an incremental
stream to a remote machine, restoring them into poolB/received/fs@a and
poolB/received/fs@b, respectively. poolB must contain the file system
poolB/received, and must not initially contain poolB/received/fs.

# zfs send pool/fs@a | ssh host zfs receive poolB/received/fs@a
# zfs send -i a pool/fs@b | ssh host zfs receive poolB/received/fs

So if I have a pool called tank and I use a command on Nexenta

zfs send tank/data@snap1 | ssh 192.168.1.5 zfs receive tank/data@snap1

and ssh server has been activated on FreeNAS side (where 192.168.1.5 is the LAN address for it), this would replicate the Nexenta ZFS pool data to FreeNAS server and I will get the same name for the new pool? Do I have to have the file system on the receiving end?

 

[Chris Moore]

zfs send tank/data@snap1 | ssh 192.168.1.5 zfs receive tank/data@snap1

and ssh server has been activated on FreeNAS side (where 192.168.1.5 is the LAN address for it), this would replicate the Nexenta ZFS pool data to FreeNAS server and I will get the same name for the new pool? Do I have to have the file system on the receiving end?

This is the command I use to replicate (exactly) the data on one pool to another pool inside the same server:

zfs send -R Emily@manual-19Aug2018 | zfs receive -F Irene

This duplicates the entire content of the Emily pool to the Irene pool.

It is my understanding that if you use the @ on the receive side, you will have a snapshot in the destination pool. If you have already created a filesystem on the receive side and leave the @ off, you will get the directory structure and files instead of just a snapshot.

 

[Evertb1]

What is your UPS make and model? I am planning to get an UPS also.

It is an APC BR1200G-GR also known as Back-UPS pro 1200VA. The only thing about it that I don't like (not a real big deal) is that it has no LAN interface. It has an USB serial port to connect if with my FreeNAS box. It not only gives backup up power to my FreeNAS server but it also protect other devices (like the switch connected to the server) against network surges etc.

 

[Jatrabari]

Another update to my project

I managed to transfer using zfs send receive my Nexenta pools to FreeNAS without problem. Only thing was that FreeNAS alerted that the zfs version was old, version 3, so I have to update that. I was little suprised about this because Chris said that the old pool wouldn't override the properties of the new pool. Next is then jails, plugins (Plex, Nextcloud, BOINC...), fan controls, smart and scrub scheduling, shares to Windows and MacOS...

When I had transferred the data to FreeNAS I noticed this

Seems like a bug and the percentage indicator has an extra division somewhere cause should it report 67 % used? I tried to find a bug report involving this but didn't find anything. Would this also impact on reporting when pool is becoming full?

 

[Jatrabari]

Oh yeah, for future reference and searches I used these instructions for the over-the-LAN transfer

https://www.ixsystems.com/community...alf-of-smb-transfer-speeds.73664/#post-510948

minus pv which isn't included in FreeNAS. EDIT: pv is included in FreNAS