CTL is now the iSCSI target in FreeNAS

[jpaetzel]

As of a few minutes ago CTL is now the default iSCSI target in FreeNAS.

I've axed things from the GUI that are no longer needed. I'm thinking
the GUI needs a refresh, but for the moment it's usable.

This target includes full VAAI support when using a zvol as the backing
device, and support for w2k12 clustering.

Caveats:

1. There's a NAA field that's displayed next to extents. This value
MUST be unique for every extent that a VMWare host can see. If there
are collisions between two extents (even if they are on different
FreeNAS boxes!) that are connected to the same VMWare hosts the
datastores will get corrupted disastrously. It's easy to ensure there
are no collisions between extents on the same machine. It's impossible
to ensure there are no collisions between extents on different machines.
We use a value derived from uuidgen, so in theory collisions will never
happen.

2. InitiatorGroups are not yet hooked up. The GUI lets you create them
however they are not yet used. This means that any ACLs you have that
deny access based on istgt style initiatorgroups will not work.

3. I may have fumbled the API, API docs, or API tests. It will be a
couple days before we validate that those work properly.

4. In general the iSCSI documentation is just plain wrong due to the
target change. I'll be catching that up this week as well.

https://bugs.freenas.org/issues/5524

 

[jpaetzel]

As soon as builds are done there will be nightlies with this feature in http://download.freenas.org/nightly/x64/

(as of this writing the images in there (FreeNAS-9.3-ALPHA-16f44a7) do not have all of the changes)

You'll want FreeNAS-9.3-ALPHA-670018 or later.

 

[jgreco]

View: http://www.youtube.com/watch?v=VwDs8j3ZS-M


Starting at about :15 .... grin

But seriously, thanks for all the efforts on this. It's been sucky suggesting Nexenta.

 

[Serverbaboon]

So extended copy is working now?

 

[mav@]

So extended copy is working now?

Yes, it does, though only within one storage host or one iSCSI target (depending on settings). Inter-host copy offload is much more complicated, while benefits are not so obvious.

 

[iSCSIinitiator]

Thanks to the FreeNAS team for all the work on iSCSI!

Mr. Hubbard recently wrote:

"Both istgt and CTL will work with either zvols or files, but whereas istgt preferred files, CTL prefers zvols. For one thing, with the latter it can now return zero'd blocks to the filesystem (e.g. "holes" become real holes again, rather than just blocks full of zeros). It will also work fine with files, yes, but the performance and underlying efficiencies won't be the same.

So, in short, in the new iSCSI (CTL) universe the recommendation will be zvols."

While Mr. Hubbard was comparing zvols vs. files, I was wondering: what are the tradeoffs between creating an extent with a raw physical disk vs. a zvol using CTL?

For example, if I have an iSCSI initiator running ZFS and want to connect multiple FreeNAS iSCSI targets to it, what are the penalties for setting up the iSCSI targets to export each attached physical disk as an unformatted extent? My thinking here is that across lots of target systems, I could have "dumber" targets with less RAM/CPU horsepower if the targets don't have to perform any heavy lifting since they're not running ZFS, they're just writing/reading blocks. I could then consolidate all the intelligence in the initiator, load it with tons of RAM, etc. and have it create mirrored vdevs across the attached iSCSI targets.

I understand that setting up iSCSI targets with zvol based extents can:

1. leverage the target's ZFS cache to improve performance
2. increase data integrity on the target through ZFS, especially if the initiator is running a "dumber" file system that doesn't checksum, etc.
3. facilitate delegation of disk scrubbing by having targets perform scrubs on their own directly-attached disks vs. having the initiator orchestrate scrubs across all the iSCSI targets

Are there any other benefits, though, when comparing zvol-based and disk-based extents attached to a "smart" initiator?

The FreeNAS wiki also states:

"In theory, a zvol and a file extent should have identical performance. In practice, a file extent outperforms in reads/writes but this is only noticeable at 10 GB Ethernet speeds or higher. For high performance, file extents are recommended at this time. Future changes to FreeBSD's zvol code will increase its performance."

which I realize was written before and was alluding to the CTL update. I'm just wondering if a disk-based extent with a ZFS initiator would give me the best of both worlds: data integrity + performance. Wouldn't I/O on a disk-based extent be faster than on a zvol? Plus, I'd have the option of direct-attaching the disks and importing the zpool later (if I moved to HBAs instead of iSCSI).

If I were setting up a FreeNAS box to act as an iSCSI target with a Windows initiator, I'd definitely set up ZFS on the target and would likely create zvol extents. However, if I'm connecting to a FreeBSD initiator and will have the initiator run ZFS, am I safe in exporting raw disk-based extents from the FreeNAS targets?

Thanks!

 

[mav@]

Old comments about zvol/file extent performance should be obsolete by FreeNAS 9.2.1.6. Now their performance should be comparable in both istgt and CTL. Same time CTL supports UNMAP on zvol's, so they are preferable now.

I see no problem from CTL point of view in using raw disks for iSCSI targets. I tested that on vanilla FreeBSD while working on CTL, and it was working well and fast. Though I don't remember what FreeNAS GUI thinks about raw device extents. You will definitely loose all features of ZFS: checksums, snapshots, caching, etc, but if you don't need them...

 

[cyberjock]

FreeNAS WebGUI doesn't do raw devices anymore.. that was removed in 8.0.1 or something. Don't know why since that was much before my time. ;)

 

[iSCSIinitiator]

mav@ and cyberjock,

Thanks for the info. Cyberjock, if I have an attached, unformatted disk in FreeNAS 9.2.1.6, the WebGUI lets me create an extent with it under Services->iSCSI->Extents->Add Extent. There, I can select "Device" for Extent Type and can select the disk from the Device pulldown menu. I can then share this disk from the target and connect to it and use it on the initiator. So it looks like it is supported in the GUI, at least in 9.2.1.6.

One thing I noticed in testing after your comment, though: if I "Enable experimental target" under Target Global Configuration, iSCSI will fail to start if I have a disk selected as a device extent, but will load if I create device extents with zvols. I hadn't noticed this before.

This leads me to some questions:

1. Is there any reason to believe that creating disk based extents with the WebGUI in 9.2.1.6 would break, since it seems to work fine out of the box?
2. Mav@ says disk based extents with CTL work well on FreeBSD, but I get an error with "experimental target" on FreeNAS. Is creating disk based extents with CTL ("experimental target") possible in 9.2.1.6?
   
3. What will happen to disk based extents from 9.2.1.6 if I upgrade to 9.3 when it's available?
4. Given that I think I want disk based extents for the reasons I mentioned earlier (e.g., my initiator runs ZFS, so I'd rather my targets just run "dumb" iSCSI), are there any compelling reasons to use zvol extents instead of disk based extents?
5. If it is conceptually OK to use disk based extents instead of zvols, is there a major benefit to running CTL with disk based extents instead of ISTGT with disk based extents?

I really appreciate any help as I'm about to commit lots of disks and data to whatever iSCSI configuration I settle on and I'm trying to plan accordingly.

Thanks and all the best.

 

[cyberjock]

Ok, so here's my opinion...

1. I don't know anyone that has ever used unformatted disks like this. So doing so presents you with pretty serious risk that a bug that nobody else knows about will inflict your data. Just look in the manual and they talk about a data-killing bug that was in all versions until and 8.3.1. That's more than a 18 months of releases with a data-killing bug! It would suck to be "that guy".
2. It seems to be a supported option as the manual does say unformatted disk. I will say that Jordan and I talked about this a few months ago when I was doing data recovery for someone and he didn't even know FreeNAS ever supported iSCSI extents using unformatted disk. To be honest, I didn't know that it was there again. When I went through the manual it seemed to have been removed back in 8.0.2 or something.
3. What the future holds for iscsi extents on unformatted disks is anyone's guess.
4. If I were planning to use this in production I definitely wouldn't do disk-based for one simple reason- you have no redundancy from disk failures. That's kind of the anti-thesis of FreeNAS and ZFS, but it is what it is. If you want redundancy you are talking a RAID controller with a RAID1 or RAID5 or something. Seems weird to consider a RAID card in FreeNAS, but there it is.

So with all of this being said, I'd never consider using disk based extents in production. It might be an interesting curiosity to play with, but its never anything I'd used for my home and DEFINITELY not something I'd use at work.

 

[jgreco]

There are definitely valid usage models that involve raw devices, especially if you're not considering FreeNAS to be just a ZFS storage appliance but rather a focal point for data storage services. I'm actually looking at the reintroduction of tape back into the environment here, but removable disks are an incredibly attractive option. LTO-6 is about 2.5TBish but costs ~$50/tape, so a 4TB drive for $150 is definitely more expensive per TB but has a substantial convenience factor in not having to swap tapes as often.... plus the tape drives are expensive but attaching several temp drives isn't.

The problem is how to attach them to the network. The physical backups machine here is ancient and the new stuff is all on VM's, so a rational SAN or NAS attachment method is necessary. I can totally see making a physical disk available via iSCSI to a Windows VM running Veeam with NTFS on the drive, with an offsite rotational schedule, as being an option for a small VM cluster; since recoverability is a prime consideration, being able to attach a disk containing the backups to a random Windows box is vastly more flexible than requiring a FreeNAS server and ZFS and network/server configuration.

I'm not sure that what the previous poster suggested is a good strategy though.

 

[Fei]

how to config FC target ?
i wanna lean it.

 

[aufalien]

how to config FC target ?
i wanna lean it.

First, read the manual on configuring iSCSI and then check this link;

http://forums.freenas.org/index.php?threads/so-you-want-to-test-fc-target-mode.21568/

 

[Fei]

First, read the manual on configuring iSCSI and then check this link;

http://forums.freenas.org/index.php?threads/so-you-want-to-test-fc-target-mode.21568/

thx,i will test :)

 

[aufalien]

thx,i will test :)

No worries, mine ain't too good neither :)

 

[mroxa]

Hi Josh,

I'm Trying configure iSCSI target in version 9.3-M4 but I'm not able to restrict by initiators or network. There are options on GUI for it, but doesn't work and then anyone have access for all targets.

I seem in your post "... InitiatorGroups are not yet hooked up. The GUI lets you create them however they are not yet used. This means that any ACLs you have that deny access based on istgt style initiatorgroups will not work."

There will be fix for this? Can I make a manual configuration?

Thank you!

 

[mroxa]

I need this configuration in /etc/ctl.conf and is impossible from GUI.

auth-group ag1 {
auth-type "none"
initiator-portal 192.168.56.102
initiator-name iqn.1991-05.com.microsoft:ie11win8-1
}

portal-group pg1 {
discovery-auth-group no-authentication
listen 192.168.56.104:3260
}


target iqn.2011-03.org.example.istgt:iqn.test {
alias iqn.test
auth-group ag1
portal-group pg1
}