Upgraded freenas to FreeNAS-11.2-U8 and lost fiberchannel connection storage to ESX 6.7 host

mst

Explorer
Joined
Aug 17, 2014
Messages
95
I wonder how can I revert the upgrade or fix what is broken now. My Freenas 8TB show inaccessible now.

I will be providing hardware info soon. Thank you
 

mst

Explorer
Joined
Aug 17, 2014
Messages
95
seems like vcenter and 100 hosts sees the freenas storage 8TB, but when I click on view datastore I have error:

an error occurred, please try again
 

Attachments

  • freenas 8t data read error.JPG
    freenas 8t data read error.JPG
    79.6 KB · Views: 191
  • freenas 8tb.JPG
    freenas 8tb.JPG
    68.8 KB · Views: 179
  • freenas storage.JPG
    freenas storage.JPG
    63.3 KB · Views: 182

mst

Explorer
Joined
Aug 17, 2014
Messages
95
root@freenas[/etc]# zpool status -v
pool: TEST1
state: ONLINE
scan: scrub repaired 0 in 0 days 00:00:01 with 0 errors on Sun Jul 26 00:00:02 2020
config:

NAME STATE READ WRITE CKSUM
TEST1 ONLINE 0 0 0
gptid/bd2507c9-4d19-11ea-95a3-90b11c54a041 ONLINE 0 0 0

errors: No known data errors

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0 days 00:01:12 with 0 errors on Mon Aug 24 03:46:12 2020
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
da6p2 ONLINE 0 0 0

errors: No known data errors

pool: sas
state: ONLINE
scan: scrub repaired 0 in 0 days 00:32:54 with 0 errors on Sun Aug 23 00:32:57 2020
config:

NAME STATE READ WRITE CKSUM
sas ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
gptid/0b1a31b3-21a5-11ea-9d0c-90b11c54a041 ONLINE 0 0 0
gptid/0ec26304-21a5-11ea-9d0c-90b11c54a041 ONLINE 0 0 0
gptid/125fb2cf-21a5-11ea-9d0c-90b11c54a041 ONLINE 0 0 0

errors: No known data errors
 

Herr_Merlin

Patron
Joined
Oct 25, 2019
Messages
200
FC is not supported for us "free"NAS users.. I would bet my money on the FC drivers broken.
 

mst

Explorer
Joined
Aug 17, 2014
Messages
95
ok, I guess I have to go with ISCSI cards and switch or just cards if I have just one or two hosts

what budget ISCSI 10G cards you recommend?
 

Herr_Merlin

Patron
Joined
Oct 25, 2019
Messages
200
depend on what you need? I would just go with 10G intel cards..
 

c77dk

Patron
Joined
Nov 27, 2019
Messages
467
Chelsio would be my choice - it's (afaik) the cards used by iX themselves
 

firsway

Dabbler
Joined
Oct 20, 2018
Messages
32
Don't make my stupid mistake of the other week, and order 10G cards that worked fine in the node hardware, but weren't on ESXi compatibility list that I forgot to check! :confused:
Anyway my setup of 1 x FreeNAS (soon to be 2) + 3 Esxi Nodes (DL360p Gen8) + Nexus 3064 now working just fine with a Supermicro AOC-STGN-I2S in the FreeNAS box, and HP NC552SFP (Emulex based) cards in the nodes
 

no_connection

Patron
Joined
Dec 15, 2013
Messages
480
I wonder how can I revert the upgrade
Can't you just boot into previous version? I thought that was the whole idea with snapshots.

I don't see why FC would not be supported. If it worked then stopped then it should have continued to work until something was deprecated in release notes.
 

mst

Explorer
Joined
Aug 17, 2014
Messages
95
yes seems like my fiber channel cards works only with 11.3 and whatever is newest or even True Nas which supposed to support FC out of box - does now. I have no time just troubleshoot why. Whatever works for me is 11.3 and I reinstalled it to bring it back. Now I can see storage, unfortunately I lost all VM stored previously on that storage.
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
Fibre Channel doesn't have its own "driver" in FreeeBSD/FreeNAS. In fact iSCSI and Fibre Channel both use CTL (CAM Target Layer) to present LUNs.
What Fibre Channel (FC) card are you using? You should be covered by the following 'driver' https://www.freebsd.org/cgi/man.cgi...FreeBSD+11.3-RELEASE&arch=default&format=html as its built in to FreeNAS/BSD

Its more likely that the config [ctl.conf] got borked during the upgrade as its not supported in FreeNAS.
Can you tell me what guide you used to configure FC?
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
Now I can see storage, unfortunately I lost all VM stored previously on that storage.
How? If you used zvol backed LUNs and you didn't intentionally blow away your pool, all should be fine. Worst case, ESXi sees it as snapshot storage and you need to resignature the VMFS partition.
 

mst

Explorer
Joined
Aug 17, 2014
Messages
95
I dont know what happened. I wiped all and setup 11.3 from scratch. I tried to import all pools but that did not work. I setu them up from scratch and vioala. I used mainly these two:


for FreeNas 11.3

I tried to use:

1. Frash 11.7 FreeNas
2. Frash Ture Nas 11.7 (whatever is newest) and my FC was not even detected. I was told TrueNas supports FC but dont know.

To be honest there was many more well detailed links in this forum but they dont exist eny more. I dont know if admin or users remove them intentionally. Nice guide for FC with FreeeNas and ESX step by step with even explanation - dont exist any more.

In FreeNas I use quad port FC and in ESX host I use singe port card QLE2564
 

kdragon75

Wizard
Joined
Aug 7, 2016
Messages
2,457
I dont know what happened. I wiped all and setup 11.3 from scratch. I tried to import all pools but that did not work. I setu them up from scratch and vioala. I used mainly these two:


for FreeNas 11.3

I tried to use:

1. Frash 11.7 FreeNas
2. Frash Ture Nas 11.7 (whatever is newest) and my FC was not even detected. I was told TrueNas supports FC but dont know.

To be honest there was many more well detailed links in this forum but they dont exist eny more. I dont know if admin or users remove them intentionally. Nice guide for FC with FreeeNas and ESX step by step with even explanation - dont exist any more.

In FreeNas I use quad port FC and in ESX host I use single port card QLE2564
Ha nice, that's my old guide! If I had to guess, we just need to load the driver (aka kernel module) ISP4 to get it working again. I don't have a fibre channel setup anymore to test.
You could try
Code:
kldstat
or
Code:
dmesg | grep isp
to see if the module is loaded/they are detected. I would have to look and see if the modules are still in the image though.
 
Top