SOLVED Can I use External HDDs for extra Storage in TrueNAS Scale?

[Arwen]

What we mean by not reliable, is the interface for USB is not reliable like SATA, SAS or NVMe. Their are many things that can go wrong with USB attached disks, as explained in the Resource I referenced. You might get lucky and none of those problems may affect you. But they HAVE affected others, so we warn against the use of USB attached disks for data drives.

Next, if this external RAID does stripe or concatenate its drive together, loss of one disk means the loss of all data on both drives. Perhaps not a problem for you. One way to avoid that, is to create separate ZFS pools for each disk that is to be used in a non-redundant configuration. That does mean you have to "manage" which files go on which pool & dataset. But, loss of a single disk pool does not affect any other single disk pools.

All this is not meant to stop you, just we want you to know the risks.

 

[Ashvath Sridharan]

What we mean by not reliable, is the interface for USB is not reliable like SATA, SAS or NVMe. Their are many things that can go wrong with USB attached disks, as explained in the Resource I referenced. You might get lucky and none of those problems may affect you. But they HAVE affected others, so we warn against the use of USB attached disks for data drives.

Next, if this external RAID does stripe or concatenate its drive together, loss of one disk means the loss of all data on both drives. Perhaps not a problem for you. One way to avoid that, is to create separate ZFS pools for each disk that is to be used in a non-redundant configuration. That does mean you have to "manage" which files go on which pool & dataset. But, loss of a single disk pool does not affect any other single disk pools.

All this is not meant to stop you, just we want you to know the risks.

Incase if any one or more disks accidentally becomes offline, then I lose all my data? Won't it rebuild it self when I make it online or reconnect everything? Like for example, I accidently remove the SATA cable from one of the drives and reconnect it and reboot the system, will I still get my data back or its corrupted and lost?

 

[MrGuvernment]

Incase if any one or more disks accidentally becomes offline, then I lose all my data? Won't it rebuild it self when I make it online or reconnect everything? Like for example, I accidently remove the SATA cable from one of the drives and reconnect it and reboot the system, will I still get my data back or its corrupted and lost?

Yes, because you did a stripe and not a mirror, being Raid 0 to get the most space out of your disks?

how big is each individual disk?

Raid0 means your data for a single file can be split between multiple drives, thus one drive dies, that file is corrupted and gone forever. A rebuild can only occur when there is data to reference to rebuild the blocks required.

if you pull a cable and power on the system it can corrupt the array yes. If a drive just dies one day and you have no backups,most of your data is gone and not recoverable....

 

[Ashvath Sridharan]

Yes, because you did a stripe and not a mirror, being Raid 0 to get the most space out of your disks?

how big is each individual disk?

Raid0 means your data for a single file can be split between multiple drives, thus one drive dies, that file is corrupted and gone forever. A rebuild can only occur when there is data to reference to rebuild the blocks required.

if you pull a cable and power on the system it can corrupt the array yes. If a drive just dies one day and you have no backups,most of your data is gone and not recoverable....

But what if the drive that accidently gets disconnected still works without any bad sectors. If I replug it back will all my data still be intact?

 

[Arwen]

But what if the drive that accidently gets disconnected still works without any bad sectors. If I replug it back will all my data still be intact?

Yes, theoretically you can re-connect the drive, and most or all the data should still be available.

When I say "most or all" is in reference to data in flight. If you were writing data to the disk(s) when it / they were dis-connected, that data may or may not be available after re-connection. It solely depends on how far the write got.

ZFS was specifically designed for un-expected power losses and not have corrupt data in the ZFS pools. But, as I said, data in-flight might be lost.

However, ZFS was NOT designed for external RAID controllers. (ZFS IS the RAID controller!) If ZFS is used with a caching RAID controller, (regardless of which RAID type, stripe, concat, Mirror or RAID-5/6), this can cause pool corruption if the caching RAID controller does not honor write barriers and write flushes. It's worse, MUCH WORSE, if the hardware RAID controller does elevator seeks, meaning trying to optimize writes irrespective of what the writer, (ZFS), wants.

ZFS simply was not designed for external RAID. Especially non-Enterprise types.


All that said, I routinely use Enterprise Solaris 11 servers, (which use Solaris version of ZFS), with Enterprise grade SAN storage arrays. The only time we had corruption was our fault, (aka human error), involving improper fail-over to another host.

 

[Ashvath Sridharan]

Just a quick update...
I finally added my drives with 2 USB to SATA docks and it's working fine for now. I will update incase anything goes worng. But thank you everyone in this thread for concluding my questions and most importantly being very kind to me :)