Is there a max 'disks' size? (SOLVED - Add a LUN)

[LearnLearnLearn]

I've got an external 12TB storage connected to TrueNAS-12.0-U4.
If I format the storage to one single volume over 10TB, then truenas doesn't see it in the disks.
If I format the storage to a max of 10TB, then that volume shows up in the disks. In fact, when it saw it, it was slightly over 10TB, something like 10,889.

Is there a limitation of some sort I am not finding out about?

 

[sretalla]

I wIll start by saying what you're trying to do here is not "right".

If you read this article:

What's all the noise about HBA's, and why can't I use a RAID controller?

This is relevant to FreeNAS and TrueNAS CORE. Some parts of it might also be relevant to Scale, but I don't really know how reliable the Linux drivers are. 1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with...

www.truenas.com

... and want to continue reading, I have responded to your questions and asked some others.

I've got an external 12TB storage connected to TrueNAS-12.0-U4

How have you connected it? (USB?)

USB controllers can't usually handle the high volume of work which comes from the way that ZFS bundles transactions, so unless you're using this as a backup device which is not written to constantly, this is a really bad idea.

If I format the storage to one single volume over 10TB, then truenas doesn't see it in the disks.

It wouldn't see the individual disks if you have done that, but are you saying it doesn't see any disk?

Maybe the RAID controller that is putting the disks together in the external storage has a limit? (or is using a non-standard block layout to get to total sizes above 10TB?)
If you share some details about that hardware, maybe we won't need to guess.

Is there a limitation of some sort I am not finding out about?

ZFS (the Zettabyte File System) is designed to handle Zettabytes... (Mega, Giga, Tera, Peta, Exa, Zetta, Yotta... http://highscalability.com/blog/201...etabyte-exabyte-zettabyte-or-a-yottabyte.html)

It's able to work with industry standards like SATA, and SAS to access disks of very large size and can aggregate many of them into volumes of virtually unlimited (with currently available equipment) size.

 

[LearnLearnLearn]

Hi,

>I wIll start by saying what you're trying to do here is not "right".

No idea why you would start out like that when you don't know at all what I'm doing.
USB? You jumped to a lot of conclusions in your reply. I'm not really sure why when the question was very specific and simple.
I was not trying to make anyone guess anything and asked a very specific question with details.

I didn't get the chance to come back and say 'ooops, I didn't assign a LUN group' on my external storage so freenas/truenas wasn't able to see it.

I use freenas/truenas in various ways including fibre channel, iSCSI, DAS and local drives in multi-drive chassis.
Yes, no need to remind me that fibre isn't supported :).

Anyhow, it's all working perfectly. Thanks for your reply.

 

[HoneyBadger]

If you're using TrueNAS as a "head" in front of external storage, just make sure you've accounted for redundancy in your disk connection plane (eg: redundant FC HBAs for the connection) and understand the behavior of any of the dependant storage subsystems, especially any caching (I strongly suggest that's set to read-only on the shelf)

Just be aware that you're swimming in largely uncharted waters, so you'll have to be ready to get your hands dirty.

 

[LearnLearnLearn]

Yes, absolutely good point.

No problem, I've been using freenas for a long time with fibre channel storage. I actually replaced OnStor and BlueArc hardware with freenas. Once things are set up and working right, it works perfectly and almost unlimited amounts of storage can be had.

I get the sense the project is trying to remove some features/supported hardware so they can have it on paid versions only. I don't know this, just how it reads when looking through posts, especially from folks asking about FC.

 

[HoneyBadger]

FC target support has always been a "paid support only" Free/TrueNAS option (and the CTL target still doesn't do LUN masking to my knowledge, so it's not exactly ideal either) - I've always just gone with iSCSI for block, but I get it's not an option for integration into every environment.

For the original question, I was envisioning you using a separate, existing spinning-disk SAN and FC HBAs in initiator mode on a 1U TrueNAS machine. You'd present a large volume from the spinning-disk SAN, TrueNAS would pick it up as a "local disk" and then you throw a ZFS filesystem on top of it and hopefully band-aid the random I/O off the old rust with a box of cheap DDR3 RDIMMs.

(Not speaking from experience here or anything, no sir.)

 

[LearnLearnLearn]

No it's not true FC but using the black shares lets me use FC hardware with the open source package.

Even if they finally remove everything that lets folks use it in this way, many will just keep using older versions and that move might cause a fork anyhow. No idea, just talking out of my pie hole :).

 

[sretalla]

No idea why you would start out like that when you don't know at all what I'm doing.

It seemed to me that you were using an external RAID system to present a single disk to TrueNAS... that's not supported (not reminding you, just explaining my answer).

USB? You jumped to a lot of conclusions in your reply. I'm not really sure why when the question was very specific and simple.
I was not trying to make anyone guess anything and asked a very specific question with details.

I've got an external 12TB storage connected to TrueNAS-12.0-U4
Running TrueNAS-12.0-U4 on a 1U IBM server.

That's all you gave me about the hardware... how else was I supposed to formulate an answer? I don't see where you mentioned FC. I have seen many other cases where people buy a USB enclosure that does RAID, so I made that assumption as a starting point... you note that I asked questions and started down a path of answer for an assumed response (rather than only asking questions... it's quicker that way sometimes).

Anyway, all the best with figuring it out.

 

[LearnLearnLearn]

@sretalla
I understand your position in this but I wasn't getting into the hardware because I didn't want to add complexity to the question. I just needed to know if there was some sort of limit to disks.

It's all good now. Thanks for clarifying.