FreeNAS enthusiastic over here!

[chmod5360]

Hi FreeNAS community !!

I am a FreeNAS newbie, looking forward to build a cheap FreeNAS-driven NAS system, ideally built around an ORICO 3.5" 8-Bay USB 3.1 Type-C Hard Drive Enclosure (Model NS800C3 : http://my.orico.cc/goods.php?id=6531).

So far, Orico helpdesk certified that an external OS can see 8 distinct hards disks in the enclosure, making it therefore manageable by a FreeNAS driving OS to create a RAID volumes. Nevertheless, I still do not know:

a) Whether FreeNAS can effectively manage and run a USB3.1-connected bunch of disks, and

b) Whether FreeNAS will get access to each enclosed hard disk's SMART functions via the USB3.1 connection.

I've registered this website in order to get answers to these questions, before buying this enclosure model and I'd be extremely grateful to the FreeNAS community if you could help me get any help and/or links within this forum that might help me sort this problem out.

 

[SweetAndLow]

a) Whether FreeNAS can effectively manage and run a USB3.1-connected bunch of disks, and

b) Whether FreeNAS will get access to each enclosed hard disk's SMART functions via the USB3.1 connection.

No and no, you must use sata or sas connections. I guarantee this device does not allow direct access to the disks.

 

[Basil Hendroff]

Plenty of information on this. Search the forum for USB enclosure.

 

[chmod5360]

No and no, you must use sata or sas connections. I guarantee this device does not allow direct access to the disks.

Thank you so much for your quick answer SweetAndLow !

I have 4 more questions to clarify my doubts :

1) Do you mean that ONLY a direct (SATA or SAS) access to the hard disks is required to make an enclosure compatible with FreeNAS ?

2) Does your answer only concern the Orico hard disk enclosure model I mentioned, or do you mean that FreeNAS is unable to drive and manage any USB3.1-connected hard disk enclosure ?

3) Alternatively, a colleague of mine explained to me that a USB3.1-connected hard disk enclosure could be compatible with FreeNAS provided that:

* The enclosed hard disks are internally wired on a SATA bus, and
* The USB3 chipset in the enclosure exposes the SMART functions of the hard disks through the USB3.1-connection

The resulting enclosure would probably have lower performances compared to a native SATA or SAS-based enclosure (which is not really a concern to me given the fact I am the unique user of the NAS as an archiving system), but it would work reliably in order to provide :

* Capability to create and manage one or more RAID volumes
* Read / Write the disks reliably with concurrent accesses

Was this colleague right or wrong ?

4) If not, do you know of any USB3.1-based hard disk enclosures that are compatible ? Or could you recommend me >= 8-bay eSata HD enclosures that are compatible with FreeNAS ?

Sorry about the count of my questions (I swear these are the last ones !!), but so far, I found extremely contradicting answers (or no answer at all) to these questions and -- unless l missed the concerned pages -- very little or nothing is said about this subject in the FreeNAS documentation I've read so far.

 

[chmod5360]

Plenty of information on this. Search the forum for USB enclosure.

Thank you very much for your quick answer Basil Hendroff.

I am going to search the forum on this subject.

Hopefully, SweetAndLow's answers to my latest paronoaid questions will also help get most of the answers to my problem... :)

 

[SweetAndLow]

Zero external USB enclosures will work. You're going about this wrong and freenas is probably not for you.

 

[Yorick]

Well, that's a bit harsh :).

FreeNAS may or may not be for you. What you are looking for is an enclosure with a SAS expander in it, not eSATA. See https://www.ixsystems.com/community/resources/dont-be-afraid-to-be-sas-sy.48/

You are taking the right steps by looking into this first. Learn a bit about ZFS, and decide whether the inflexibility inherent in the system (can't expand a raidz vdev after the fact, for example) works for you.

 

[jgreco]

eSATA is fine for single drives, but usually requires a port multiplier for a multi-drive chassis, and this can be very dodgy.

 

[chmod5360]

Hello again,

First of all, thank you very much to the all of you for your kind answers.

@Yorik: yes, that was a bit harsh; perhaps I forgot to mention that I am currently trying to find and validate a technical solution that would match my functional requirements. As you've just pointed out, in that perspective, I am exploring every technical solution that could be compatible, and this is simply called a technical feasibilty analysis.

To be quite fair to everyone, I should have first read Basil Hendroff's suggested links before posting my second wave of questions... Indeed, this thread (https://www.ixsystems.com/community/threads/usb-3-1-usb-c-multi-drive-enclosure.83922/) brilliantly initiated by rfca (https://www.ixsystems.com/community/members/rcfa.101650/), asks questions similar to mine in a context that is exactly the same as mine. In terms of USB3-connected enclosure compatibility, and given the technical arguments developed in the answers of that thread, it seems obvious that the USB standard is not reliable enough to handle such an interface. Pity.

@jgreco : yes, you are right. I came to similar conclusions when I studied this standard months ago : altough eSata turned out to be my primary technical connection choice (before USB), I had to eliminate this standard since most of my readings on eSata (port multiplier if more than 4 HDDs, wiring length constraints, etc.) proved that this standard also accumulate quite a large number of technical drawbacks and/or risks.

@Yorik: yes, again, you are right ! SAS seemed to me the technical perfect promise that would solve all my problems at once, provided I am rich enough to buy 9 x 10 (or 14 TB) SAS HDDs !! SAS is definitvely a professional solution that is financially too expensive for my personal needs. Again, pity...

Before I go further, please let me summarize my main functional requirements :

• A storage solution of about 100-130 TB, ideally consisting of 9 hot swappable HDDs
• Data organization would split into a :
  • vdev 1 (or "Backup vdev) : 1 x HDD, only used only to perform local backups of some of the data originating from vdevs 2 and 3, with very little (if none) read accesses. A subset of this backup data (namely "fundamental" data) will also be backed up to some cloud storage solution
  • vdev 2 (or "Data Repositories" vdev) : 4 (or 5) HDDS + 1 Hot spare, that will store my data, and shall be regarded as a static data archive. In clear: data are only archived in this vdev, and only read when published as "live" data to vdev 3. Example : a music repository containing my zipped music albums shall be stored to vdev2.
  • vdev 3 (or "Work" vdev) : 2 (or 3) HDDs + 1 Hot Spare, exclusively used for live and working data. Live data are data published from vdev2's archived repos and working data may contain the documents and any data I am working on. Example:

- Backup vdev used only to perform local backups of some of the data originating from the 2 other vdevs, with very little (if none) read accesses

 

[Yorick]

yes, again, you are right ! SAS seemed to me the technical perfect promise that would solve all my problems at once, provided I am rich enough to buy 9 x 10 (or 14 TB) SAS HDDs !! SAS is definitvely a professional solution that is financially too expensive for my personal needs. Again, pity...

There's a misunderstanding here. SAS controllers / HBAs can handle SATA drives just fine. You use an external enclosure with SATA bays and a SAS expander, and connect to it via SAS HBA. Your drives are SATA, your HBA is SAS. SAS HBAs can actually control SATA drives directly, as well. I get how that might not be intuitive.

You are talking about three pools here, by the way. If you make this three vdevs in one pool, ZFS will just distribute data across all three, which does not sound like what you want. One vdev per pool, in this instance.

Consider the idea of going for raidz2 vdevs instead of a hot spare. You gain resilience thereby, even if you need to wait for a replacement drive to arrive. If you do 4 or 5 HDDs in raidz1 plus a hot spare, then if another drive fails while the hot spare resilvers in, you are done for. If you do 5 or 6 HDDs in raidz2, then a drive can fail, you can wait for the replacement and start resilvering it in, and if during the resilver another drive fails, you are still good to go.

 

[chmod5360]

( continuation of the previous post)

• Data organization in the data storage system would split into :
  • (...)
  • vdev 3 (or "Work" vdev) : 2 (or 3) HDDs + 1 Hot Spare, exclusively used for live and working data. Live data are data published from vdev2's archived repos and working data may contain the documents and any data I am working on. Example: my music albums are unarchived to vdev3 and made publily accessible and exploitable by streaming software
• Architecture principles :
  • The data storage system is separated from the "intelligent" system. In clear: the data storage syste should be restricted to deliver disk and file management only services. Other services, i.e., added-value services including managing the data storage system, should be handlded by one or several dedicated servers.
  • The server Must contain a FreeNas operational system (VM, Docker-containerized) that drives the data storage system.
• Hardware requirements:
  • The data storage system Must exhibit a desktop form factor
  • The data storage system Must exhibit a 9 hot swappable hard disk front bay
  • The data storage system Must feature a connection technology compatible with FreeNAS
  • The connectivity with the driving system Should be USB-3 or eSata.

Conclusion :

Given the incompatibilies and/or difficulties exposed above, it turns out that managing and driving a "simple" USB3-connected data storage system with FreeNAS is impossible. My conclusion is that although I'm reluctant to use an intelligent storage system that embeds both data and added-value services, I'll have to consider making a complete NAS system or buy an existing commercial solution.

 

[chmod5360]

There's a misunderstanding here. SAS controllers / HBAs can handle SATA drives just fine. You use an external enclosure with SATA bays and a SAS expander, and connect to it via SAS HBA. Your drives are SATA, your HBA is SAS. SAS HBAs can actually control SATA drives directly, as well. I get how that might not be intuitive.

You are talking about three pools here, by the way. If you make this three vdevs in one pool, ZFS will just distribute data across all three, which does not sound like what you want. One vdev per pool, in this instance.

Consider the idea of going for raidz2 vdevs instead of a hot spare. You gain resilience thereby, even if you need to wait for a replacement drive to arrive. If you do 4 or 5 HDDs in raidz1 plus a hot spare, then if another drive fails while the hot spare resilvers in, you are done for. If you do 5 or 6 HDDs in raidz2, then a drive can fail, you can wait for the replacement and start resilvering it in, and if during the resilver another drive fails, you are still good to go.

Yorik, thank again for your answer and for the precision.

It seems I missed the point with the SAS connectivity : what you are explaining seems crytal clear !! I am going to deepen my knowledge on this subject, which gives me hope that I could make it using a SAS expander.

Concerning the data organization : indeed, you are (again!) right: I do not want ZFS distribute data across all three vdevs. My initial data organization strategy was precisely to create vdevs that would preserve disks obsolescence as much as possible. Each vdev MUST be therefore be completetly independent from each other. From what you explained, I should therefore create a pool for each vdev.

Concerning the RAIDZ choice : yes, I am perfectly aware and I do agree with you that a RAIDZ2 would be much better for vdev2 and vdev3 in terms of resilience. But for now, I am again stuck by stupid hardware offers limitations on the market : for now I was unable to find a desktop enclosure with a 10 (or 12) hot swappable disk front bay, which led me to reconsider the question of available storage space / system resilience. I am aware I should not handle this problematic under an angle that consists of priviledging storage space over data resilience in case of disk failure. This is dangerous.... I will therefore try again to investigate the enclosure offer....

Thanks again for your great help that made me save at least 2 weeks of infructuous searches on the web !!!

 

[chmod5360]

(continuation of the previous post)

The Hardware Compatibility Document (click orange button to download the PDF file) explicitly covers the "Additional SATA/SAS connectivity" use case on page 15.

 

[Yorick]

for now I was unable to find a desktop enclosure with a 10 (or 12) hot swappable disk front bay

There are some pretty obvious "but why?!?" questions regarding hot swap and front bays and, for the sake of argument, I'll just go with "because reasons" and leave it at that.

Okay. 12 drives, in front bays, hot-swap, for SATA drives, with a backplane that can connect to a SAS HBA.

Won't be as cheap as just having drives in something like a Define R7 (which can hold 18 internally), and, here's one option:

One 5-drive enclosure, https://www.amazon.com/dp/B000HWJGEW
One SC-743 tower case, https://www.supermicro.com/en/products/chassis/4U/743/SC743T-665B

Total 13 drive bays, hot-swap, front-load

Plus room for the motherboard etc. You do not really gain anything by leaving the motherboard, CPU, RAM and HBA separate from the enclosure that holds the drives. The point of external enclosures with SAS expanders is usually to go past 12-13 drives, to 24 or more. Those are rack-mount, though.

 

[jgreco]

It seems I missed the point with the SAS connectivity : what you are explaining seems crytal clear !! :) I am going to deepen my knowledge on this subject, which gives me hope that I could make it using a SAS expander.

We have a great primer on SAS that is oriented towards the things a FreeNAS user needs to know.

https://www.ixsystems.com/community...-sas-sy-a-primer-on-basic-sas-and-sata.26145/

 

[CompuGlobalHyperMegaNet]

• Data organization would split into a :
  • vdev 1 (or "Backup vdev) : 1 x HDD, only used only to perform local backups of some of the data originating from vdevs 2 and 3, with very little (if none) read accesses. A subset of this backup data (namely "fundamental" data) will also be backed up to some cloud storage solution

Just bare in mind that if you don't have disk(s) for parity or mirroring, then ZFS will still be able to detect most forms of data corruption bit it won't be able to repair the corrupt data.

I strongly recommend that you read Cyberjock's Guide for Noobs (which there's a link to in my signature).