truenas isp0: SRR[0x...] SRR offset 0x... covered by current CCB data range [0x0..0x...]

[jkm]

I have two FC interfaces configured on my TrueNAS.
Generally, I don't have any problems or alerts and everything works fine.
Nevertheless, I am concerned about a message that appears almost every night on the console of TrueNAS.
This message looks like this:

truenas isp0: SRR[0x...] SRR offset 0x... covered by current CCB data range [0x0..0x...]

I have two interfaces configured, but the above message only applies to one - isp0 interface.

Does anyone know exactly what this message is about and should I worry?

Thanks

 

[jgreco]

FC is officially not supported, and there are relatively few people trying to use it. The stuff is sorta old and there are probably caveats we don't know about.

A SRR is a sequence retransmit request.

 

[jkm]

FC is officially not supported, and there are relatively few people trying to use it. The stuff is sorta old and there are probably caveats we don't know about.

A SRR is a sequence retransmit request.

What do you mean 'FC is not supported'? What about TrueNAS Enterprise version?
And what is 'old'? FC?
Could you write some details?

 

[jgreco]

'FC is not supported'?

I thought it self-explanatory.

What about TrueNAS Enterprise version?

What about it? I don't understand what you're asking. iXsystems sells 100% of the hardware solution that TrueNAS Enterprise runs on. To connect disk shelves to the NAS, they have chosen to use contemporary SAS designs.

FC melts under the crushing workloads presented by ZFS. When iXsystems put out FreeNAS 8 back in 2011, top-of-the-line SAS would have been dual SFF-8087 port 9211-8i, capable of 48Gbps per controller aggregate towards disk shelves. It works easily and magically. Two or three such controllers can be placed in a server "no problem", and SAS expanders allow even a single controller to manage hundreds of disks, albeit with that 48Gbps-per-controller limit.

By way of comparison, top-of-the-line point-to-point FC was the newly released 16G stuff in 2011. Expansion involving non-ptp topologies is pricey and performs poorly, and FC involves some configuration challenges not present in SAS topologies.

There's literally no comparison. FC is da $#!+ and I don't mean that in a good way. FC is just garbage for the backend.

On the frontend, using FreeNAS as the target in a SAN application, there's potentially a better argument for it. However, there again, it's basically really easy to poke holes in. Chelsio had a dual port 40Gbps card back in 2012, so 80Gbps from a single controller. It was ethernet, so it supported iSCSI. iSCSI has some warts, but there's a significant upside in that it can scale far beyond FC, and runs on your existing ethernet infrastructure.

iXsystems actually sponsored the kernel mode iSCSI stuff on FreeBSD, so apparently that was quite the winner for them.

So the questions I would present you with are these:

Is it better to do two different kinds of technologies on the backend, SAS and FC, one better, one crappy, and make compromises to support the crappy one, or is it better to just focus on the less expensive and higher quality technology and do that REALLY WELL?

Is it better to do two different kinds of technologies on the frontend, ethernet and FC, one that handles everything, the other that only handles SAN clients and is basically just a long tail has-been technology, or is it better just to focus on the forward-looking better faster cheaper technology and do that REALLY WELL?

I can't tell you what iX or its developer team are thinking. I don't speak for them. But I've been doing this stuff a really long time. I think I can make pretty good guesses.

 

[jkm]

What about it? I don't understand what you're asking

I would like to note that, according to the information on your website, TrueNAS Enterprise has full support for the FiberChannel protocol.
So isn't "not supported".

Is it better to do two different kinds of technologies on the backend, SAS and FC, one better, one crappy, and make compromises to support the crappy one, or is it better to just focus on the less expensive and higher quality technology and do that REALLY WELL?

There is no doubt that SAS-drives has replaced FC-drives long time ago.
Nobody produces FC-based drives anymore.
I don't argue with that.
On backend SAS technology is better then FC.

Is it better to do two different kinds of technologies on the frontend, ethernet and FC, one that handles everything, the other that only handles SAN clients and is basically just a long tail has-been technology, or is it better just to focus on the forward-looking better faster cheaper technology and do that REALLY WELL?

IMHO over 15 years of my experience with storage and servers both FC and iSCSI on frontend, I know that as for performance, it's hard to beat the low latency and high throughput of FC, because FC was built from the ground up to handle storage traffic.
Unlike protocols that run on top of TCP/IP, FCP is a significantly thinner, single-purpose protocol that generally results in a lower switching latency.
From a performance perspective, iSCSI lags behind FC/FCP.
But when iSCSI is implemented properly, the difference boils down to a few milliseconds of additional latency due to the overhead required to encapsulate SCSI commands within the general-purpose TCP/IP networking protocol.
This can make a huge difference for extremely high transactional I/O loads and is the source of most claims that iSCSI is unfit for use in the enterprise.
This is very importand think - especialy now in SSD world.
Both architectures are great for certain tasks.
However, saying that FC is good for enterprise while iSCSI is good for SMB.
You decide what is better for You.
For me, as an storage and server solution integrator and enterprise administrator, in an enterprise - FC technology is better.

 

[jgreco]

to the information on your website, TrueNAS Enterprise has full support for the FiberChannel protocol.
So isn't "not supported".

Well, that's certainly not true. "My" website has no information about TrueNAS at all. My companies do not sell, service, or support TrueNAS or FreeNAS systems for customers in any way that could be considered commercial competition. We do occasionally do stuff like refurbishing gear and HBA's.

Perhaps you're laboring under some misconception that "I" am associated with TrueNAS. I'm not. I woke up one morning conscripted into moderatorship on these forums, many years ago. iX and I fly in related, overlapping circles.

All I know is that every time the topic has come up in the past with iX developers, it's been met with sort of a shrug. Jordan said outright that they had no plans for it, but I can't find a reference to that. Ah crap jkh's account is actually deleted, making search VERY difficult.

FC target support in FreeNAS 9.1.0

FreeNAS doesn't have FC target support by default, but you can make it work if you want to do some hacking and you don't care that you can't manage it from the GUI. 1. gunzip then copy the attached ko to your FreeNAS box. Put it in /boot/modules. (The attached ko is 64 bit only) 2. Run...

www.truenas.com

jpaetzel used to be one of the significant developers. Also account deleted. This is making it really hard to track down previous "official" statements, so I am going to ask you to please just believe me when I say I have firm recollections.

In any case, there is definitely no official support for FC in Core or Scale, which brings us around to what I originally said. If TrueNAS Enterprise has added support, they haven't told me, and it isn't in the free product. But of course they don't tell me anything, because I'm not an employee. If and when they want to send me a nice TrueNAS Enterprise system for free, I'm happy to find a use for it...

More generally, on the FC target side of things ... Lots of people have made progress of varying degrees of success. The bits that exist at the OS level to implement FC have not been lobotomized out of the OS. But by that same measure, UFS/FFS is still supported by the OS as well. What makes TrueNAS meaningful is integration with the GUI and middleware. As you start "off-road adventuring" with these things, there are some handy bits of documentation on the forum about what others have accomplished. Nothing stops you from doing these things.

So we circle back around to my answer.

If you have TrueNAS Enterprise, contact your field engineer and ask about it. We don't really support TrueNAS Enterprise here in the forums; iX expects you to use their support.

That leaves us with TrueNAS Core, TrueNAS Scale, or FreeNAS, which are the things that these forum support, and my original answer of "FC is officially not supported" which remains the correct answer.

You asked for details, you got 'em. Please don't blame me if I don't have a solid answer to your original question, though. There are very few FC users and I wanted you to get *some* answer.

 

[jkm]

I'm truly sorry.
In fact, I was convinced You were working for iX Systems.

I am currently using TrueNAS Core, and believe me, enabling FC on TrueNAS Core is not a rocket science.
You can simply 'click it' from the web interface.

As we know, TrueNAS Enterprise has FC support.
Does storage with TrueNAS Enterprise installed have similar operating system as TrueNAS Core, but have additional paid features like FC enabled?
If it is true, I think can be some people here who have TrueNAS Enterprise with working FC, and maybe they have displayed a similar console message and can say something more about this message.

 

[jkm]

BTW
Below print screen from TrueNAS Enterprise features from iX Systems TrueNAS web site.
Information about Fibre Channel feature marked red circle and red underline.

 

[jgreco]

some people here who have TrueNAS Enterprise with working FC,

As far as I know, iXsystems provides support for TrueNAS Enterprise through their engineering organization. I've been here for over a decade providing community support for FreeNAS, and now TrueNAS CORE/SCALE, and the common question we occasionally see is "what about FC" and the answer that I've seen has never suggested any interest in official support for it.

Ah, I found it.

Fibre channel LUN mapping

Hi! My first post... I've been reading up on different posts discussing fibre channel target implementation in FreeNAS, and there seem to be working to connect via an initiator to a target on FreeNAS. Though, what seems to be missing, is the(cruzial) ability to map the specific LU to the...

www.truenas.com

There is no plans to implement FC LUN mapping. At this point FC target support is mostly a free side effect of the new iSCSI target implementation.

mav@ is literally the guy who wrote the iSCSI stuff and has probably had more to do with block storage than anyone else.

This was later clarified by jkh that this was not to imply FC support was ever going to come to FreeNAS or even TrueNAS Enterprise. He was, at the time, Chief Tech Officer at iXsystems, so that was really the final word. mav@'s new iSCSI work did mean that it was much easier to spin up unsanctioned FC stuff, and there are still guides on this over in the resources section.

TrueNAS Enterprise installed have similar operating system as TrueNAS Core, but have additional paid features like FC enabled?

Yes, they are supposed to share the core framework.

As we know, TrueNAS Enterprise has FC support.

Well, perhaps as YOU know. It's news to me. But then again, they don't bother to tell me anything. It's not like I'm doing their free product support for them for free or anything, y'know. (that's meant to sound annoyed, and to be clear, not annoyed at/with you).

I think can be some people here who have TrueNAS Enterprise with working FC, and maybe they have displayed a similar console message and can say something more about this message.

Well, they're welcome to, if they want. I have not seen TrueNAS Enterprise users wandering around these forums. Certainly not ones using FC. That's why I responded to you in the first place and told you what the message was about.

 

[jkm]

English is not my native language, but I found two key information in this sentence:

mav@ said:
There is no plans to implement FC LUN mapping. At this point FC target support is mostly a free side effect of the new iSCSI target implementation.

First - It's true that TrueNAS Core works with FC target and it is a free side effect of the new iSCSI target implementation.
It works with some harware limitation, but I've enabled this, and for me, it works great.

Second - It's true that TrueNAS Core with manualy enabled FC don't implement FC LUN mapping.
Nevertheless, if someone have connected server via SAN switch with TrueNAS Core with enabled FC, can use SAN switch zoning to restrict other servers access to TrueNAS Core and its volumes.