SOLVED Help with recovering from boot drive failure

[Sonte]

Hello everyone,

Recently my boot HD failed for my freenas setup, and dumb me, I didn't have a backup of my configuration. My old setup used ISCI and a extent device in a zpool that was presented to the Windows OS which formatted it into a NTFS volume. I got a new HD and installed the latest version of Freenas on it. I was able to import my pool from the existing raid, and I see the pool as shown in the below screenshot. I should also mention that it prompted me to upgrade my pool to the latest version for Freenas which I did. Now, I don't know how to proceed from this point. I think the next thing that I have to do is somehow recreate the volume or the extent device. However, some of the research that I have done says that you can't recreate that extent, so I started trying to see if I could somehow mount the existing ntfs volume in Freenas itself and copy out the data. I am finding very little to no information on that in my google searches, or I don't know how to ask google the right question. Can anyone one this forum tell me how to proceed from here, or am I out of luck? Any and all help would be greatly appreciated.

 

[Samuel Tai]

Did you host the zvol for iSCSI out of your boot pool, instead of the Storage pool? Usually, the boot pool isn't visible in the GUI for creating zvols, but that's the only scenario I can think of to account for the missing zvol in your screenshot.

 

[Sonte]

I don't think so. Maybe something went wrong with the import? Is there a way to do it again?

 

[Samuel Tai]

Unfortunately, you've already upgraded your ZFS pool flags, so there's no way to go back to an earlier version. I suspect you may have created the zvol under the jails dataset for warden jails. 11.3 has removed all the logic for warden, and now uses iocage for jails. You can try looking in jails to see if your zvol is there, but at this point, you'll likely need to create a new zvol under your pool top-level and start over with the new 11.3 iSCSI wizard to create an extent.

 

[Sonte]

Thank you for your reply. How would I look in jails for the zvol, and is there any way for me to try and mount the NTFS volume?

 

[Samuel Tai]

Thank you for your reply. How would I look in jails for the zvol, and is there any way for me to try and mount the NTFS volume?

You can spelunk around /mnt/Storage/jails in the shell to see if your zvol is there. Hopefully, you'll recognize the name. At that point, you can create a new zvol. For purposes of discussion, let's say the new zvol is named ntfs-zvol. (Make sure this new zvol is equal or larger in size than the old zvol, if you remember the size.) This will appear as Storage/ntfs-zvol, but will actually be created in /dev/zvol/Storage/ntfs-zvol. Then you can copy the contents to the new zvol via dd if=<path to old zvol> of=/dev/zvol/Storage/ntfs-zvol conv=notrunc status=progress. After that, run through the iSCSI wizard. On the PC side, you may need to delete the former iSCSI mount, and attach to the new extent created by the wizard.

 

[Sonte]

Thank you again for your continued help on this. I went into the /mnt/Storage, and I see my volumes!! They were not in the jails, so I wanted to double check to make sure that your above method was still the recommended one. Also, is the number in the below screenshot the size of the volume? I remember that all together my storage took up over half of my pool. I can delete the Server_Backup volume, as I can recreate the backups easily since that server is still online. However, the WinStor has all of the important stuff on it. If that is not the size, is there a way to see the size?

 

[Samuel Tai]

No need to use dd at this point. To see the actual size of the WinStor dataset, run zfs get all Storage/WinStor | grep used.

Now see if WinStor shows up in the GUI, under Storage->Pools. If yes, then you can go straight to the iSCSI wizard.

 

[Sonte]

When I ran

Code:

zfs get all Storage/WinStor | grep used.

I get the error "cannot open 'Storage/WinStor': dataset does not exist

 

[Samuel Tai]

Interesting. Please run zfs list, so I can see how your datasets are structured.

 

[Sonte]

Please see below.

Code:

cannot open 'Storage/WinStor': dataset does not exist
root@freenas[/mnt/Storage]# zfs list
NAME                                                       USED  AVAIL  REFER  MOUNTPOINT
Storage                                                   3.64T  6.84T  3.64T  /mnt/Storage
Storage/.system                                           79.7M  6.84T   128K  legacy
Storage/.system/configs-646f8dae97d646cc8946ddeb0ca79d97   266K  6.84T   266K  legacy
Storage/.system/configs-a421eaccddb44c098d96b72146b5211d  67.3M  6.84T  67.3M  legacy
Storage/.system/cores                                      677K  6.84T   677K  legacy
Storage/.system/rrd-646f8dae97d646cc8946ddeb0ca79d97      9.71M  6.84T  9.71M  legacy
Storage/.system/rrd-a421eaccddb44c098d96b72146b5211d       117K  6.84T   117K  legacy
Storage/.system/samba4                                     176K  6.84T   176K  legacy
Storage/.system/syslog-646f8dae97d646cc8946ddeb0ca79d97    240K  6.84T   240K  legacy
Storage/.system/syslog-a421eaccddb44c098d96b72146b5211d    964K  6.84T   964K  legacy
Storage/.system/webui                                      117K  6.84T   117K  legacy
Storage/iocage                                             842K  6.84T   139K  /mnt/Storage/iocage
Storage/iocage/download                                    117K  6.84T   117K  /mnt/Storage/iocage/download
Storage/iocage/images                                      117K  6.84T   117K  /mnt/Storage/iocage/images
Storage/iocage/jails                                       117K  6.84T   117K  /mnt/Storage/iocage/jails
Storage/iocage/log                                         117K  6.84T   117K  /mnt/Storage/iocage/log
Storage/iocage/releases                                    117K  6.84T   117K  /mnt/Storage/iocage/releases
Storage/iocage/templates                                   117K  6.84T   117K  /mnt/Storage/iocage/templates
Storage/jails                                              117K  6.84T   117K  /mnt/Storage/jails
freenas-boot                                              1022M   199G    23K  none
freenas-boot/ROOT                                         1022M   199G    23K  n

 

[Samuel Tai]

OK, so it looks like WinStor is just a plain directory, not a datastore. What do the contents look like? ls -lh will yield more legible sizes.

 

[Sonte]

Please see below.

 

[Samuel Tai]

Uhh, did you intend to attach something?

 

[Sonte]

Sorry about that. Copy paste is acting weird.

 

[Samuel Tai]

ls -lhR, sorry. You used -f instead of -h.

 

[Sonte]

Wow yeah that's a lot more info

Code:

-rw-r--r--  1 root  wheel  5497558138880 Jun 22 06:55 WinStor
root@freenas[/mnt/Storage]# s -lhR
zsh: command not found: s
root@freenas[/mnt/Storage]# ls -lhR
total 3910676993
dr-xr-xr-x+ 3 root  wheel     3B Nov 18  2018 .zfs
drwxr-xr-x  9 root  wheel    10B Jul  1 08:02 iocage
drwxr-xr-x  3 root  wheel     2B Nov 18  2018 jails
-rw-r--r--  1 root  wheel   150G Jun 27 03:16 Server_Backup
-rw-r--r--  1 root  wheel   5.0T Jun 22 06:55 WinStor

./.zfs:
total 0
dr-xr-xr-x+ 2 root  wheel     2B Jun 30 15:51 snapshot

./.zfs/snapshot:
total 0

./iocage:
total 15
dr-xr-xr-x+ 3 root  wheel     3B Jul  1 08:02 .zfs
-rw-r--r--  1 root  wheel   3.3K Jul  1 08:02 defaults_backup.json
-rw-r--r--  1 root  wheel   3.2K Jul  1 08:02 defaults.json
drwxr-xr-x  3 root  wheel     2B Jul  1 08:02 download
drwxr-xr-x  3 root  wheel     2B Jul  1 08:02 images
drwxr-xr-x  3 root  wheel     2B Jul  1 08:02 jails
drwxr-xr-x  3 root  wheel     2B Jul  1 08:02 log
drwxr-xr-x  3 root  wheel     2B Jul  1 08:02 releases
drwxr-xr-x  3 root  wheel     2B Jul  1 08:02 templates

./iocage/.zfs:
total 0
dr-xr-xr-x+ 2 root  wheel     2B Jul  1 08:02 snapshot

./iocage/.zfs/snapshot:
total 0

./iocage/download:
total 0
dr-xr-xr-x+ 3 root  wheel     3B Jul  1 08:02 .zfs

./iocage/download/.zfs:
total 0
dr-xr-xr-x+ 2 root  wheel     2B Jul  1 08:02 snapshot

./iocage/download/.zfs/snapshot:
total 0

./iocage/images:
total 0
dr-xr-xr-x+ 3 root  wheel     3B Jul  1 08:02 .zfs

./iocage/images/.zfs:
total 0
dr-xr-xr-x+ 2 root  wheel     2B Jul  1 08:02 snapshot

./iocage/images/.zfs/snapshot:
total 0

./iocage/jails:
total 0
dr-xr-xr-x+ 3 root  wheel     3B Jul  1 08:02 .zfs

./iocage/jails/.zfs:
total 0
dr-xr-xr-x+ 2 root  wheel     2B Jul  1 08:02 snapshot

./iocage/jails/.zfs/snapshot:
total 0

./iocage/log:
total 0
dr-xr-xr-x+ 3 root  wheel     3B Jul  1 08:02 .zfs

./iocage/log/.zfs:
total 0
dr-xr-xr-x+ 2 root  wheel     2B Jul  1 08:02 snapshot

./iocage/log/.zfs/snapshot:
total 0

./iocage/releases:
total 0
dr-xr-xr-x+ 3 root  wheel     3B Jul  1 08:02 .zfs

./iocage/releases/.zfs:
total 0
dr-xr-xr-x+ 2 root  wheel     2B Jul  1 08:02 snapshot

./iocage/releases/.zfs/snapshot:
total 0

./iocage/templates:
total 0
dr-xr-xr-x+ 3 root  wheel     3B Jul  1 08:02 .zfs

./iocage/templates/.zfs:
total 0
dr-xr-xr-x+ 2 root  wheel     2B Jul  1 08:02 snapshot

./iocage/templates/.zfs/snapshot:
total 0

./jails:
total 0
dr-xr-xr-x+ 3 root  wheel     3B Nov 18  2018 .zfs

./jails/.zfs:
total 0
dr-xr-xr-x+ 2 root wheel      2B Jun 30 15:51 snapshot

./jails/.zfs/snapshot:
total 0

 

[Samuel Tai]

Can you run file /mnt/Storage/WinStor? I suspect this is your former zvol image.

 

[Sonte]

Yay! It recognizes that it's a windows partition at least

 

[Samuel Tai]

OK. This will take some additional steps to get you going again:

1. First, in the GUI, create a dataset under Storage name zvols. In the GUI, this is at Storage->Pools. Click the 3 dots at the right end of the line where your pool Storage is listed, and select Add Dataset.
2. Now, create a zvol inside the zvols dataset just created. Click the 3 dots at the right end of the line for the zvols dataset, and select Add Zvol. For the name, you could reuse WinStor. For the size, use 5 TiB, matching your current image.
3. Now, in the shell, run dd if=/mnt/Storage/WinStor of=/dev/zvol/Storage/zvols/WinStor conv=notrunc status=progress. Be careful with the syntax for dd, as it's very easy to lose data if the syntax isn't exact.
4. Make sure the iSCSI service starts on boot.
5. Run the iSCSI wizard to connect the newly-created zvol to an extent.
6. Remove the old extent on your PC iSCSI initiator. Connect to the new extent.
7. If things are working, then you can delete the original file to recover space.