Replication process stuck on sending?

[theaddies]

I am running FreeNAS 11.2 U3. I am attempting to replicate about 6 TB of information. The Dashboard shows that the disk size is about right. Meaning that the size of the information transfered seems to have happened. But when I look in the shell the files are not there. The replication logo is slowly moving back and forth as is normal. When I click on it I it says sending. It has been this way for about 6 hours. I started the process yesterday about 36 hours ago. Clicking on the replication icon did show different files that it were replicating. Some of the files have been transferred. I am not sure what is going on. Is it stuck do I need to do something to reset it?

 

[PhiloEpisteme]

I'm not sure what you mean by

Some of the files have been transferred.

Considering that earlier you said

But when I look in the shell the files are not there.

6TB is a lot of data and can take a long time on slower connections. I assume you're doing this over a network, what speed are you getting?

 

[theaddies]

Thanks for the reply. When I say some of the files were transfered I mean that some of the snapshots appear to have been processed and copied to the new drive. The new pool was blank and there are now folders and files.

I am actually doing it within the same FreeNAS machine. I am just replicating from one pool to another. In the replicate field I put the IP address of the same machine that has the snapshots. The speed I was getting was around 70MB/s.

The weird thing is that when it transfers the snapshots I could see the file and it would indicate the % transfered. All this was when I hit the replication icon. Now when I select the replicate icon all I see is "Sending <OK>". And as I said it says that for a very, very long time. What exactly is the "sending" step. It seems like the process has quit but perhaps it is still something going on. I just can't tell what and for how long.
Thanks.

 

[PhiloEpisteme]

You may find this documentation on zfs send|recv helpful in understanding what is going on with replication.

Can you send a screen shot of your replication settings? How much space is taken up by the target [ICODE[vdev[/ICODE].

Edit: Actually link to the documentation.

 

[theaddies]

 

[theaddies]

View attachment 30063

The space taken up on the vdev, which I assume means the pool in this case, BigdaddyZFS is 3.12TB.
I should also say that it doesn't appear to be done. The replication icon is still moving back and forth. Perhaps the darn thing is just not done. I wish there was some way I could tell it wasn't locked up.
I tried running

Code:

ps -U root -axwwo lstart,command | grep 'zfs:' | grep -v grep

but got no result.
Thanks for helping me. I don't mean to be a pain.

 

[PhiloEpisteme]

Why are you using replication on the same machine? Are the snapshots on your machine not sufficient? What is your pool layout?

 

[theaddies]

Well, I wanted upgrade my existing pool from 6x2TB RAIDZ2 to 10x2TB RAIDZ2. I didn't want to end up with 2 existing pools because then I would waste some space to add additional redundency. I asked on here the best way to do that and was told to create snapshots of the existing pool, replicate to another drive with sufficient capacity matching the existing pool. Then destroy the old pool and replicate back to the new 10x2TB pool. That is what I am doing. The second pool I am currently using is just 1x8TB drive. I have all the important files backed up elsewhere incase things go awry.

 

[PhiloEpisteme]

The second pool I am currently using is just 1x8TB drive. I have all the important files backed up elsewhere in case things go awry.

Yeah, backups are important especially if you're using a single drive.

 

[theaddies]

Is there anything I can do to figure out if something is happening or the replication is hung? When I click the icon it simply says Sending <OK>. There is no file name, no % done, no nothing. When I run top I see no command like ZFS running. When I run

Code:

ps -U root -axwwo lstart,command | grep 'zfs:' | grep -v grep

The output is nothing. I am no IT guy for sure, but FreeNAS sure is hard going.

 

[Glorious1]

Did you ever figure this out? I've got the same problem - replication of a daily snapshot is stuck for over a day. It's been working fine up till now, and other snapshots before it got replicated on the same day this one got into trouble.

 

[Glorious1]

I deleted (in the GUI) the destination snapshots it was stuck on. Today's replication seemed to go smoothly and it also replicated the ones from yesterday that were missing on destination.