My devices are all new, but truenas are not working properly with the hard drive.

[Misaka_15535]

I purchased some equipment and created truenas and openwrt services.
Seagat ST18000NM000J 18T*5
Japan MILLENNION AT03B (8858M) hard drive cabinet with five hard drive bays
mini computer
CPU: N6005
Memory: Samsung 8G*2
Network port: Intel-i226-V *4
Interface: usb3.1*2 usb2.0*2

The mini computer is installed with esxi, and openwrt is connected to four network ports, which are divided into one wan port and three lan ports.
The wan port pppoe dial-up Internet access, the lan port is directly connected to my main computer (msi z390 gameing plus motherboard, onboard network card Intel I219V), and I use a 0.5m cat7 network cable to link.
I used the high-speed usb data cable that came with the hard disk cabinet to connect to the usb3.1 port of the mini computer for transmission.
truenas connects the lan port through the virtual switch of esxi.
I am using the latest version of TrueNAS SCALE downloaded from the official website

The above is the information I can think of to help you answer my question, if there is any missing information, I'm sorry, please bring it up.


The above is my situation. The purchase of these devices cost me a lot of money. I am very worried that due to the insufficient quality of a certain device, the hard disk will be damaged, data will be lost, and economic losses will be incurred (the price of these mechanical hard disks is very important to me. is expensive).

(The window mentioned below is the system of my main computer, and there is nothing other than esxi, truenas, and openwrt installed in esxi)
This is my first time using a nas system, I first configured the window smb share. After writing a little data, accessing the jump folder, window explorer is always stuck and not responding.
But after a short recovery, I wrote some files again. Less than half of the write, window explorer crashed and restarted. The smb shared drive shows that it is not connected. I try to log in to the background, but the ip address cannot be accessed. During this period, I also try to log in to openwrt, it is always loading, and truenas seems to crash. After a short time, the truenas background can be accessed normally.
I logged into the backend and found the Storage page, showing `ONLINE (Unhealthy)`.

But I don't know what unhealthy is unhealthy. I try to restart.
But the shutdown was unsuccessful for a long time, and I ended up cutting the power in the virtual machine.
I also restarted esxi and the hard disk enclosure.

After some tossing, the Storage page shows `ONLINE`, and everything becomes normal.
I again access the written data through the window, but clicking into the drive is very laggy,
Then I found that the drives in the hard disk enclosure started to work slowly. Every time I accessed a different folder, the lights set for the drives in different hard disk enclosures would also flash. This flashing should indicate that the hard disk is being written alone. After all five drives worked, everything worked fine.

Later, I migrated some data, there are hundreds of gigabytes, but all the data stored in the nas is less than one terabyte.

When I was about to go to sleep, the one in the hard disk cabinet always beeped every few seconds, and the indicator light showed that all the hard disks were working, but the indicator light only flashed once, and the light time was less than one second.
I logged in to see if I could figure out what was working, but found an error message in the background. Storage page again, showing `ONLINE (Unhealthy)`.
````
Pool hhd state is ONLINE: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected.
````
I'm very worried if there is a problem with the settings there or if there is a problem with the hard disk enclosure or hard disk.
I'm concerned about irreversible damage to these HDDs of mine (because they're expensive).

Because those sounds interfere with my sleep. So I closed both the truenas and the HDD enclosure last night.
Now while I'm writing this stuff, I'm booting again.
But one more warning.

````
WARNING
Device /dev/disk/by-partuuid/969c462e-e4f6-4363-b0ac-21e2e0ecefbc is causing slow I/O on pool hhd.
````

This screenshot is the current dashboard information.

my question is
- There are some issues between the truenas and the hard drive that are not clear to me and now they are not working. This isn't the first time things have gone wrong. And these devices are also brand new. I'm concerned about irrecoverable damage to these hard drives due to the problem there.
- Both my network card and network cable support gigabit transmission speed, but when migrating which data, the fastest transmission speed is only 100m, which is still some distance from the expected fastest transmission. Why?
- Some sounds when the hard disk is working (not the sound of the hard disk rotating, it seems to be related to the magnetic head) will affect my sleep, and the truenas will work continuously and intermittently after I finish transferring the data. This sound is very regular, about once every few seconds, and the indicator lights of the hard disk enclosure will be all on each time it sounds, and the duration is very short and less than one second. Why is there such a sound?
- After I installed truenas, I did not create a pool. The hard disks in the hard disk cabinet were turned on. When I was about to go to sleep, I heard the sound of hard disk work (the sound in the previous question), why are these hard disks not configured , will also start the job?

 

[diogen]

TrueNAS - both, CORE and SCALE - are appliance-like OSs.
And that means preferred (recommended) ways of doing things.
This applies to what hardware to buy, how to install the OS, how to configure the storage, etc.

When you have succeeded doing it the "right" way, you can try some unsupported changes: ESXi, TrueNAS as VM, etc.

It looks like you decided to do it your own way, that has absolutely nothing to do with the recommended way.
And by the sound of it, you do all this for the first time...

For starters, TrueNAS prefers to have "direct" access to HDDs, preferably through a good HBA controller.
Example of the latter is LSI and some of its derivatives (Dell, IBM, etc.). No USB drives as main storage...

After that works you can explore TrueNAS as a VM by the storage controller passthrough.

Read around on this forum. Look through posters' configurations. You'll notice a pattern...

 

[Misaka_15535]

TrueNAS - both, CORE and SCALE - are appliance-like OSs.
And that means preferred (recommended) ways of doing things.
This applies to what hardware to buy, how to install the OS, how to configure the storage, etc.

When you have succeeded doing it the "right" way, you can try some unsupported changes: ESXi, TrueNAS as VM, etc.

It looks like you decided to do it your own way, that has absolutely nothing to do with the recommended way.
And by the sound of it, you do all this for the first time...

For starters, TrueNAS prefers to have "direct" access to HDDs, preferably through a good HBA controller.
Example of the latter is LSI and some of its derivatives (Dell, IBM, etc.). No USB drives as main storage...

After that works you can explore TrueNAS as a VM by the storage controller passthrough.

Read around on this forum. Look through posters' configurations. You'll notice a pattern...

My electricity bill is relatively expensive, so I chose the mini computer, which consumes very little power. A high-speed data cable is the only way to connect the two devices.

I'm now trying to get truenas to access hhd directly based on what you said "truenas prefers direct access to hhd".

 

[Misaka_15535]

TrueNAS - both, CORE and SCALE - are appliance-like OSs.
And that means preferred (recommended) ways of doing things.
This applies to what hardware to buy, how to install the OS, how to configure the storage, etc.

When you have succeeded doing it the "right" way, you can try some unsupported changes: ESXi, TrueNAS as VM, etc.

It looks like you decided to do it your own way, that has absolutely nothing to do with the recommended way.
And by the sound of it, you do all this for the first time...

For starters, TrueNAS prefers to have "direct" access to HDDs, preferably through a good HBA controller.
Example of the latter is LSI and some of its derivatives (Dell, IBM, etc.). No USB drives as main storage...

After that works you can explore TrueNAS as a VM by the storage controller passthrough.

Read around on this forum. Look through posters' configurations. You'll notice a pattern...

I gave usb passthrough to truenas. Now the pool is normal, and the speed of accessing different folders has not slowed down. I am now doing some data migration. But at the beginning, it can run to 100m, and then the limit can only reach 101m. It seems that there is something limiting it.
Since then, it has been writing at a speed of 20-30m. I have allocated a total of 2 cores and 10g of memory to truenas. From the console point of view, there is still 3g of free memory, and the CPU usage fluctuates around 30%, and this fluctuation is very small. How is this going. The cpu and memory usage has not been full, but the speed has been very slow. It doesn't seem normal either, truenas doesn't seem to be working hard, it's more like slacking off.

 

[ChrisRJ]

Please do not put any valuable data on this system. As @diogen has tried to explain, your hardware and overall configuration is far off anything that is known to work reliably. That it appears to work initially is absolutely no indicator for continued, safe operation.

ZFS is a file system developed specifically for really(!) large enterprise-grade storage systems. As such it has special requirements for hardware that are at least uncommon to people not familiar with it. In other words you cannot simply expect that your existing knowledge applies and/or is sufficient.

I don't want to be rude or appear insensitive to your situation - on the contrary. But the only options I see are to either go for a non-ZFS system (which probably means Linux with XFS/ext4 but not BTRFS), or to get suitable hardware and read up how to operate TrueNAS under ESXi.

Good luck!

 

[Misaka_15535]

I feel very wrong. When my hard drive was not directly connected to the usb device yesterday, the writing speed of the scattered files was 20m. Today, the hard disk is set to pass-through, and the speed of scattered files has even dropped to KB. Now the speed is not as fast as before. The speed of about 30m is unacceptable for me who has a lot of files to store and use.

It can run to 100m at the beginning, which means that the current speed bottleneck is not on the network.

When migrating data, I checked the resource usage of my windows, esxi, and truenas, and I confirmed that I didn't find any resource usage that was full.
The occupancy rate of the window hhd where the data to be migrated is within 20% (checked in the window 10 task manager). Other performance resources are close to the idle state.
The CPU usage displayed by the truenas web console fluctuates at 20-30m, and there is still 3g of free memory. Mechanical hard disk io is less than 10m.

Currently, some relatively large games downloaded on the steam platform are migrated, and there are many single large files.

I also tried copying multiple large files separately and the total transfer speed stayed around 30m.

I've tried searching on google and couldn't find anything that fits my situation. I don't know how to solve this, please help me.

 

[Misaka_15535]

Please do not put any valuable data on this system. As @diogen has tried to explain, your hardware and overall configuration is far off anything that is known to work reliably. That it appears to work initially is absolutely no indicator for continued, safe operation.

ZFS is a file system developed specifically for really(!) large enterprise-grade storage systems. As such it has special requirements for hardware that are at least uncommon to people not familiar with it. In other words you cannot simply expect that your existing knowledge applies and/or is sufficient.

I don't want to be rude or appear insensitive to your situation - on the contrary. But the only options I see are to either go for a non-ZFS system (which probably means Linux with XFS/ext4 but not BTRFS), or to get suitable hardware and read up how to operate TrueNAS under ESXi.

Good luck!

Thanks for your suggestion, at the moment I'm just trying to copy some different data into truenas to see if truenas is reliable enough for me, but I don't know why it's not performing well for me at the moment .

I know my equipment is special, but I see a lot of people build esxi+openwrt with a nimi computer similar to mine.
In my understanding, although my configuration is really not the best match for truenas. Although this configuration is relatively insignificant compared to enterprise-class servers.

However, expensive hard disk cabinets (I didn't buy cheap hard disk cabinets to save money), seven types of network cables. With a gigabit network card and usb3.1, the memory is small but larger than the minimum requirements of a nimi host, I think the equipment I purchased should meet the minimum operating standards of truenas.
I also don't really have constant bulk access like a business. Should be enough.

Currently, I don't know what to do in this situation, maybe I should try a non-zfs system like you said.

 

[elvisimprsntr]

You still have not posed the full details of your system. Read forum rules.

Forum Rules

Welcome to the TrueNAS Community! Updated 11/28/2023 First and foremost I want to thank you for joining the TrueNAS community! My name is JoshDW19 and I'm the Community Manager here at TrueNAS. The TrueNAS Community is a friendly place to discuss open source software, technology, and products...

www.truenas.com

But from your posts, it appears
1. You are running TrueNAS virtualized using ESXI on a Mini-PC with no SATA interfaces/bays <---Not recommended. Bare metal on a system with a non-RAID SATA/SAS controller/bays is recommended.
2. Your drives are connected to a 5-bay external USB expansion chassis <---- Will likely not work with TrueNAS.

Suggest you reconsider your hardware choices if you want to use TrueNAS.

 

[joeschmuck]

As others have said, we are not trying to be rude here but my opinion is you are trying to use USB to communicate to the hard drives and you do not understand how ZFS file system operates. It is very data intensive. Keep in mind that ZFS is designed to ensure data integrity through a lot of parity data. You are also a bit light on the RAM for both ESXi and TrueNAS. Look into SWAP file usage for both, if they are not a zero value then you are using swap space which is going to slow any system down substantially. I've been running FreeNAS/TrueNAS for many years on ESXi, I love it, but I also have a properly designed system capable of handling it.

As for the cost of electricity, you can purchase low power motherboards with low power CPU's, I have one in my test computer, I have 16GB RAM as well and it runs TrueNAS Core and Scale very well. It's not super fast, but for file sharing it will maximize the 1 Gbit Ethernet connection. And I've run ESXi on this as well with an older version of FreeNAS when it had an 8GB RAM requirement.

As you were recommended, maybe a different file system would work better for the hardware you have.

 

[Misaka_15535]

You still have not posed the full details of your system. Read forum rules.

Forum Rules

Welcome to the TrueNAS Community! Updated 11/28/2023 First and foremost I want to thank you for joining the TrueNAS community! My name is JoshDW19 and I'm the Community Manager here at TrueNAS. The TrueNAS Community is a friendly place to discuss open source software, technology, and products...

www.truenas.com

But from your posts, it appears
1. You are running TrueNAS virtualized using ESXI on a Mini-PC with no SATA interfaces/bays <---Not recommended. Bare metal on a system with a non-RAID SATA/SAS controller/bays is recommended.
2. Your drives are connected to a 5-bay external USB expansion chassis <---- Will likely not work with TrueNAS.

Suggest you reconsider your hardware choices if you want to use TrueNAS.

I am also a programmer, and I also know that the lack of a corresponding description will affect the answer, and even ask an unsolved or meaningless question, or even a question that is not a problem.
Before asking the question, the forum has prompted information about the question, and I have carefully checked it.
But I think my description is detailed enough, if there is something missing there, I don't know. Please tell me what information is missing, or how to get some special information you need, and I will cooperate.

For the two points you mentioned,
1. My mini-pc has a SATA interface, but the output data cable of my mechanical hard disk expansion enclosure is type-c to typec, type-c to usb3.2, and the mechanical hard disk enclosure can accommodate a total of 90TB mechanical hard disks. The mini-pc has a usb3.1 port, so my connection scheme is the type-c of the mechanical hard disk expansion cabinet to the usb3.1 port of the nimi-pc. I think this transfer rate is not lower than the SATA port. Combined with the low power consumption of the mini-pc (because my electricity bill is a bit expensive here), this is the best option for me.
2. Currently I have set up esxi usb pass-through to truenas. Currently they can work normally. The only problem at the moment is that the transfer speed is much lower than expected, and the cpu and memory are not running at full capacity. It can reach 100m for a few seconds at the beginning of the transfer, so my hardware connection is no problem. While transferring data, the monitoring of the truenas web page shows that the disk write speed is less than 10m/s.

Our native languages are different, these are all translated by me using Google, and then sent out after checking by myself, there may be some sentences that we want to express different meanings, I'm very sorry.

 

[Redcoat]

@Misaka_15535 , I researched your hardware list - what I could not identify is the specific mini-pc that you have nor what CPU it has.

I also cannot tell where/on what openwrt is running (as your router/firewall).

I found your USB 5xHDD cage on gonetpc but no technical information on it.

Also, please tell us precisley what "100m" and similar mean - what are the units of this apparent transfer rate?

I see you have just sent another answer - perhaps some of this is in there. Certainly I believe that people here will be happy to give you more specific comment with some more specific information.

Based on what I see so far, if I were you I would install TrueNAS on bare metal (get rid of ESXi and its overhead) and test it to determine how well it works for you. Only then, with some benchmark infomation and performance undertsanding with your current hardware, I would install ESXi and virtualize TrueNAS for testing.

Good luck!

EDIT: Can you confirm the share type to which you are transferring data - I am assuming SMB from a windows share? Also, how are you measuring the transfer speed?

 

[Misaka_15535]

As others have said, we are not trying to be rude here but my opinion is you are trying to use USB to communicate to the hard drives and you do not understand how ZFS file system operates. It is very data intensive. Keep in mind that ZFS is designed to ensure data integrity through a lot of parity data. You are also a bit light on the RAM for both ESXi and TrueNAS. Look into SWAP file usage for both, if they are not a zero value then you are using swap space which is going to slow any system down substantially. I've been running FreeNAS/TrueNAS for many years on ESXi, I love it, but I also have a properly designed system capable of handling it.

As for the cost of electricity, you can purchase low power motherboards with low power CPU's, I have one in my test computer, I have 16GB RAM as well and it runs TrueNAS Core and Scale very well. It's not super fast, but for file sharing it will maximize the 1 Gbit Ethernet connection. And I've run ESXi on this as well with an older version of FreeNAS when it had an 8GB RAM requirement.

As you were recommended, maybe a different file system would work better for the hardware you have.

I checked. SWAP is not used. If it is used, then I am very clear that my memory is not enough, and I will increase the memory of the mini-pc without hesitation.

Use these things, find problems, solve problems. I think this is the most fun.
I don't think I'm being rude to allow truenas with a completely non-functional hardware device. Even these devices are indeed not suitable for truenas. But it should also meet the minimum requirements.
I have also considered the low-power motherboard of the low-power CPU you mentioned. I chose this industrial computer type nimi-pc after considering many aspects.

If you think I offended you, then I am very sorry, I have no ill will or anger.

 

[Misaka_15535]

@Misaka_15535 , I researched your hardware list - what I could not identify is the specific mini-pc that you have nor what CPU it has.

I also cannot tell where/on what openwrt is running (as your router/firewall).

I found your USB 5xHDD cage on gonetpc but no technical information on it.

Also, please tell us precisley what "100m" and similar mean - what are the units of this apparent transfer rate?

I see you have just sent another answer - perhaps some of this is in there. Certainly I believe that people here will be happy to give you more specific commwent with some more specific information.

Based on what I see so far, if I were yoi I would install TrueNAS on bare metal (get rid of ESXi and its overhead) and test it to determine how well it works for you. Only then, with some benchmark infomation and performance undertsanding with your current hardware, I would install ESXi and virtualize TrueNAS for testing.

Good luck!

EDIT: Can you confirm the share type to which you are transferring data - I am assuming SMB from a windows share? Also, how are you measuring the transfer speed?

Thank you very much for your reply. My description seems to have "Description Failed" because of language issues.

100m means files transferred per second, 100 MBytes can be transferred per second.
I didn't know before that you can't understand the 100m I said, because we usually describe the network speed as m instead of mb.

1 Byte (B) = 8 bits
1 Kilo Byte (KB) = 1024B
1 Mega Byte (MB) = 1024 KB
1 Giga Byte (GB) = 1024 MB
1 Tera Byte (TB) = 1024 GB

My transfer type is SMB from a windows share.
I didn't use any tools, just copied the files directly.
I have tried large files, and multiple small file transfers. In my understanding, there will be a gap between some speed measurement software and actual use. In actual use, there will be two cases of transmission of large files and multiple small files. The transmission speed of multiple small files will be slow. This is normal and depends on The real transfer speed should be the average speed seen over a period of time when sustained writes are large enough to matter. When presenting to you, I am using average range values. For example, the meaning of 20m-30m should be: about 20mb when slow and about 30mb when fast.

As far as I know, the real rate of the Gigabit network controller interface should be 1000/8=125mb. I know it must be lossy, but the max transfer I've seen is only 101. And the CPU Highest Usage displayed by the truenas web dashboard is 20%~30%. Is this loss a bit large?


The picture shows the transfer of large files.

Below is a screenshot from Amazon displayed in the store where I bought it.

Thanks again for your reply, you can see at a glance that my statement failed, it helped me a lot, it's a good development.

 

[elvisimprsntr]

Bottom line, TrueNAS is a pre-packaged distribution targeted for iXsystems appliances and similarly configured DIY systems at your own risk.

Unfortunately, your system is far from being supported.
1. EXSI can be done, but not recommended.
2. External USB (other any other bus) expansion chassis are not supported since most, if not all, bridge chips do not expose the drives directly to the operating system.

If you want to continue down your current hardware path, you will likely be told by everyone to use a different distribution.

 

[Redcoat]

Thanks again for your reply, you can see at a glance that my statement failed, it helped me a lot, it's a good development.

Thanks - I don't know if what you have added will help get any more specific comment - I am doubtful because your hardware is so far from the "normal" that we have experienced.

BTW, nobody was suggesting that you were being rude with your questions - the responders were fearing that you might find the answers from our side to be rude, possibly given the added impact of the language barrier.

 

[Misaka_15535]

Thank you very much for your reply. My description seems to have "Description Failed" because of language issues.

100m means files transferred per second, 100 MBytes can be transferred per second.
I didn't know before that you can't understand the 100m I said, because we usually describe the network speed as m instead of mb.

1 Byte (B) = 8 bits
1 Kilo Byte (KB) = 1024B
1 Mega Byte (MB) = 1024 KB
1 Giga Byte (GB) = 1024 MB
1 Tera Byte (TB) = 1024 GB

My transfer type is SMB from a windows share.
I didn't use any tools, just copied the files directly.
I have tried large files, and multiple small file transfers. In my understanding, there will be a gap between some speed measurement software and actual use. In actual use, there will be two cases of transmission of large files and multiple small files. The transmission speed of multiple small files will be slow. This is normal and depends on The real transfer speed should be the average speed seen over a period of time when sustained writes are large enough to matter. When presenting to you, I am using average range values. For example, the meaning of 20m-30m should be: about 20mb when slow and about 30mb when fast.

As far as I know, the real rate of the Gigabit network controller interface should be 1000/8=125mb. I know it must be lossy, but the max transfer I've seen is only 101. And the CPU Highest Usage displayed by the truenas web dashboard is 20%~30%. Is this loss a bit large?


The picture shows the transfer of large files.
View attachment 58231


Below is a screenshot from Amazon displayed in the store where I bought it.
View attachment 58232




Thanks again for your reply, you can see at a glance that my statement failed, it helped me a lot, it's a good development.

Replenish.
The network of truenas is connected through the virtual switch of esxi.
This virtual switch uses the network port of the mini-pc connected to my main computer.
The introduction page of the store where the network cable is purchased states that the maximum transmission speed is 1250mb/s.

 

[ChrisRJ]

@Misaka_15535 , if I understand things correctly you mentioned a few times that your hardware may not be powerful enough to completely fulfill the requirements for ZFS/TrueNAS; but that you assume with your requirements things should be ok.

If that is the case, l would like to add that we have a misunderstanding. The lack of performance was not what I meant, when saying your hardware is not suitable for ZFS. Instead, as others have said, a hard disk connected via USB (in any form) does not allow sufficient control between the file system code (running as part of the operating system) and the actual hardware. There is a bridge between SATA and USB in between; this bridge causes the problems. During normal operation it may be possible that things look ok. But when problems arise with the HDDs (and over time they most likely will), there is considerable risk to loose data.

I applaud your enthusiasm , but a safe and reliable system for ZFS/TrueNAS is simply not possible with USB (especially when combined with ESXi). Theoretically it is of course possible that you operate this system for 10 years and never run into problems. But with such an optimistic attitude, it makes no sense to choose ZFS/TrueNAS in the first place.

And no, you have never been rude here!

 

[diogen]

Looking through your system specs, the major part of your investment is storage - five 18TB hard drives.
They must have set you back by the equivalent of $1,600 or more (not sure what the prices are in Japan).
These drives are fine for your project... The rest is not so much...

First, if you want to do TrueNAS, forget about your external USB enclosure.
Second, get a PC case with 5 internal 3.5" slots for drives.

If your minicomputer has a standard motherboard (mATX or ITX) with five SATA connectors onboard,
simply move the internals from your miniPC to the new case; attach drives to the motherboard.
You should also look into getting an LSI HBA card if the motherboard has an empty PCIe slot...

Then install TrueNAS onto an SSD and start configuring...

 

[joeschmuck]

If you think I offended you, then I am very sorry, I have no ill will or anger.

Nope, I was not offended at all. You have not said anything like "I will do it my way no matter what any of you idiots think", although that might make me laugh if it were from some of the regulars. If you offend someone, you will know it.

As far as I know, the real rate of the Gigabit network controller interface should be 1000/8=125mb. I know it must be lossy, but the max transfer I've seen is only 101. And the CPU Highest Usage displayed by the truenas web dashboard is 20%~30%. Is this loss a bit large?

I'd like to try to clear up language barrier stuff...

Do I understand you correctly, you can transfer a large file and reach speeds up to 101MB/sec or 101mbit/sec?

Let me ask you a few questions...
1) What is the transfer rate for a large single file (4GB or better) to the NAS, listed in MB/sec please? (MegaBytes)
2) Reboot TrueNAS then perform the next test, why? To void the cache.
2) What is the transfer rate for a large single file (4GB or better) from the NAS, listed in MB/sec please?
3) Do the same tests above but with a bunch of small files (the smaller the better).

I do not think you have a CPU limitation, in fact if you come back and tell everyone that you have a throughput of over 80MB/sec or better then I'd say you are doing great. My system typically runs at about 100MB/sec Also, when it comes the throughput testing, there is quite a bit involved in doing it correctly. You should also test out the throughput of all five drives through the USB 3.1 connection outside of TrueNAS. You may find you have a bottleneck with the USB interface.

Lastly, Just because everything is working right now doesn't mean the system is stable. You will need to determine that over time and I do wish you luck that the hardware is stable for this application.

 

[Arwen]

It appears you are using a NUC, (Next Unit of Computing), which is an all in one computer not designed for NAS purposes. These tend to have enough CPU & memory capacity for many tasks, but not NAS with more than the 1 internal 2.5" drive.

While it might be tempting to use USB attached disks, as everyone said, this is not reliable in the long run with ZFS. Further, performance is not just USB 3 speeds. It's that the USB to SATA might very well be some horrendous SATA port multiplier. Or worse, a USB to SATA hardware RAID chip. Basically SATA port multipliers, (or hardware RAID), are not suitable with ZFS. Add that to USB, and you have a recipe for disaster waiting to happen.

One of the issues with both USB or SATA port multipliers is that ZFS write transactions, (and ESPECIALLY ZFS scrubs / re-silvers), send a ton of data in a short time. Both USB & SATA port multipliers, if they work perfectly, slow those actions down. Combined, it's worse slow down. If they don't work perfectly, either driver side or interface chip side, then you can loose data. And in rare cases, I can see loosing all the data, un-recoverable, (at least without backups).


All that said, and as others have said, it may work just fine for years. But many of us here are conservative on hardware design, so that our data is safe and recoverable in the event of problems.