FreeNAS 8.2.0 BETA 3 sees 3TB ESXi 5 RDM as 0MB

[jkotran]

Hi,

I'm a new user. I plan to host my FreeNAS on ESXi 5 using Raw Device Mapping to access a three terabyte drive. Unfortunately, FreeNAS reports that the drive is zero megabytes. For example the dmesg output looks similar to:

da1: 0MB (no media?)

The same exact 3TB RDM works fine with Ubuntu 11.10 and NexentaStor 3.1.2. Any ideas or suggestions?

Does FreeNAS 8.2.0 BETA 3 support 3TB drives?

 

[jkotran]

bump

Has anyone ever experienced da1: 0MB (no media?) regardless of VMware hosting or not?

 

[StephenFry]

FreeNAS has no problems with 3TB drives:
http://img825.imageshack.us/img825/3041/1mgmt.png

 

[jkotran]

This is my dmesg excerpt:
...
da1 at mpt0 bus 0 scbus2 target 1 lun 0
da1: <ATA ST33000651AS CC45> Fixed Direct Access SCSI-5 device
da1: Serial Number Z290ZR94
da1: 6.600MB/s transfers (16bit)
da1: 0MB (no media?)
...
GEOM: new disk da1
(da1:mpt0:0:1:0): unsupportable block size 1694498816
Opened disk da1 -> 22
(da1:mpt0:0:1:0): unsupportable block size 0
Opened disk da1 -> 22
(da1:mpt0:0:1:0): unsupportable block size 1634956389
Opened disk da1 -> 22
(da1:mpt0:0:1:0): unsupportable block size 0
Opened disk da1 -> 22
(da1:mpt0:0:1:0): unsupportable block size 0
Opened disk da1 -> 22
(da1:mpt0:0:1:0): unsupportable block size 0
Opened disk da1 -> 22
(da1:mpt0:0:1:0): unsupportable block size 0
Opened disk da1 -> 22
(da1:mpt0:0:1:0): unsupportable block size 0
Opened disk da1 -> 22

 

[jkotran]

I am successfully able to use a 750GB disk drive via ESXi Raw Device Mapping. It seems to me that FreeBSD's support for 3TB via RDM may not be fully baked. :)

 

[da_angelboy]

I just came across this issue myself. I guess you didn't find a way around this, did you?

 

[jkotran]

No, I haven't found a way around this. In the mean time I'm evaluating NexentaStor and ZFS on Linux. Thus far NexentaStor doesn't look good. I'm experiencing high ESXi CPU load when the Nexenta VM is idle. It seems to be a known bug that it is at least a year old.

http://nexentastor.org/

http://zfsonlinux.org/

I just came across this issue myself. I guess you didn't find a way around this, did you?

 

[da_angelboy]

Reading more on the topic of RDMs and VMDK files from various forums, it looks like ESXi has a limitation on the size of the VMDK file (which is 2TB). When creating the RDM using vmkfstools, we're essentially creating a VMDK file that looks like it is 3TB in size (but is actually only pointing to the physical disk). I know that ESXi allows for datastores that are greater than 3TB (up to 64TB, I believe)...but each VMDK can only be < 2TB..so it can't show up in any VM as a drive larger than 2TB.

Hope this makes sense. I'm going to play with this some more to see if I can somehow break the drive down into 2 separate partitions and create 2 RDMs instead to stay below 2TB each.

 

[jkotran]

I understand what your saying. However, I can successfully pass through 3TB drives using RDM to Ubuntu and NexentaStor. I don't think that RDM is my problem. I'm using RDM, in part, because VMFS has a 2TB limit. However, RDM has a much larger limit to support SAN/NAS logical units. (64TB?)

Reading more on the topic of RDMs and VMDK files from various forums, it looks like ESXi has a limitation on the size of the VMDK file (which is 2TB). ...

 

[jkotran]

From the horse's mouth:

VMware vSphere 5.0 Configuration Maximums

http://communities.vmware.com/servl...1-23832/vsphere-50-configuration-maximums.pdf

 

[da_angelboy]

I haven't tried on Ubuntu or NexentaStor...but I was encountering a similar issue in a Windows 7 image as well where it would be reporting a capacity of 0MB. Strange! When you tried using your 3TB for FreeNAS, was it a brand new drive with no partitions on it? Did you have to do anything in particular to get it working in Ubuntu or NexentaStor?

 

[jkotran]

I am successfully using RDM/Ubuntu/NextentaStor with new drives and disks with existing data. No, I did not have to do anything in particular. I setup a physical (vmkfstools -z ...) RDM, attached it to the VM, and voila!

To recap I can successfully use RDM/FreeNAS with 750GB drives.

 

[arryo]

i tried freenas on ESXi and had exactly same problem with 3tb hdd. Have you figured out a way to fix it yet?

 

[Veni]

Hello.

I'm seeing the same thing on FreeNAS, NAS4Free, Windows 2k3 and Windows 2k8.
On Ubuntu, CentOS and gParted(Debian LiveCD i think) i'm not seeing this.

I'm seeing this on Seagate drives.
I'm not seeing this on Hitachi drives.

I'm using onboard Intel SATA for the Seagate and Hitachi drives.

But when not using the onboard Intel SATA, i'm not seeing this problem.
When using LSI 928024i4e i'm not seeing any problems at all.

The Seagate drives are cheap ones for home use, 4k sectors.
The Hitachi drives are enterprise SATA.


It sounds to me that the culprit could be the onboard SATA controller and that the Linux OS's manage the problem presented by the onboard controller.

 

[johandc]

I am successfully using RDM/Ubuntu/NextentaStor with new drives and disks with existing data. No, I did not have to do anything in particular. I setup a physical (vmkfstools -z ...) RDM, attached it to the VM, and voila!

To recap I can successfully use RDM/FreeNAS with 750GB drives.

I have the same issue with NAS4Free (FreeBSD) with all my disks (couple of 60GB Intel SSD's & 5 Seagate 2TB disks). I am however able to successfully pass them through if I use the -r switch instead of the -z switch.

The difference between the two of them is that the -z will pass the disk through as a physical disk while the -r switch will pass it through as a virtual disk, the disadvantage of using a virtual disk is that no hardware info nor SMART status is available in the guest OS, but the advantage is that these disks can be snap shotted with your vm if necessary...

 

[reb00tas]

Old threat. i tried freenas and nas4free with my 3 tb drives and same problem.

But since i have used openindiana with nappit and no problems.

 

[ixdwhite]

Just to confirm, the setup is:

1. FreeNAS 8.3.0 on file server, VMware ESXi 5.1 on client.
2. FreeNAS is exporting a >=3TB HDD Device Extent via iSCSI.
3. iSCSI target is mapped via Physical RDM to a VM.
4. VM reports bizarre drive data when started.

I'm testing this now and will post with an update shortly.

 

[ixdwhite]

Okay, problem does not reproduce on FreeNAS 8.3.1. The way Device Extents are exported on that version is to treat them as big file extents so the remote doesn't have passthrough command access to the device. The device name comes through as "FreeBSD iSCSI Disk" instead of the underlying device name. In the dark FreeNAS past it used to directly map the device name, but since device names tend to move around, that strategy was discarded.

I will boot 8.3.0-R-pX on my test machine and try to reproduce specifically on 8.3.0, but it appears upgrading to 8.3.1 and rebuilding your device extent(s) will avoid the original problem.

 

[oswa]

Okay, problem does not reproduce on FreeNAS 8.3.1.

That is not correct in my case.
I am trying to replace an older 2TB drive that just died with a new Seagate 3TB (STBD3000100) and after recreating the RDM mapping the VM (FreeNAS) throws the "unsupportable block size 0" for that device.

Host is ESXi 5.1.0 (build 799733), FreeNAS is 8.3.1-RELEASE-p2.

The main difference here is that the older 2TB was mapped as a virtual device (vmkfstools -r ... ) vs the new 3TB drive needs to be mapped as physical device (vmkfstools -z ... ) as VMFS does not support files larger than 2TB.

ESXi handle the 3TB drive just fine, but it seems like the VM with FreeNAS is not able to get the correct geometry/information.

diskinfo daX does not work - does not find the device.
camcontrol inquiry daX seems to works:
camcontrol inquiry da4
pass5: <ATA ST3000DM001-1CH1 CC26> Fixed Direct Access SCSI-5 device
pass5: Serial Number Z1F2KLYG
pass5: 6.600MB/s transfers (16bit), Command Queueing Enabled

I'm not very familiar with FreeBSD in general so I'm struggling here.
Is there a way I can check what the OS thinks the geometry is, and/or lock it to specific numbers?

 

[titan_rw]

I am trying to replace an older 2TB drive that just died with a new Seagate 3TB (STBD3000100) and after recreating the RDM mapping the VM (FreeNAS) throws the "unsupportable block size 0" for that device.

Good luck with virtualization and raw device mapping.

It's not recommended for a reason. Freenas likes raw drives to play with, not drives being mapped by esxi.

Even with vt-d passing through the entire controller card, I'd still be wary of virtualizing freenas for production.