SOLVED Terramaster T12-450 + TrueNAS SCALE = Success

[sammael]

Disclaimer: If you buy/have the nas, follow this and fail I cannot help you (I can try, but cannot guarantee success). It also voids warranty (I think?)

I've not seen this model mentioned on the forums, so I thought I'd share my success story.

My 'made from ebay-bought components' nas up and died week before xmas after runnig strong with no issues for ~5 years. I was eyeing up Qnap TS1655-8G (I already run Scale on Qnap TS-873a), but they closed warehouse for xmas early due to illness and the scalpers were asking ~1k on top of RRP. Later, I've found Terramaster T12-450 and seeing many other models mentioned with successful installation of Truenas I took the plunge (it being ~500£ cheaper than the Qnap made it a lot easier). I know nothing of the brand apart form it's Chinese, but Chinese doesn't automatically equal bad. When you get down to it disassembling anything deep enough there's gonna be something Made in China in it. This is also a nas I use for homelab, experimenting, media streaming, downloading and pretty much everything, with the other nas being a backup so even if the hw were to fail, I wouldn't lose any data (in before someone replies with some horrific story about these particular components causing issues).

Upon arrival I was a bit dismayed to see no hdmi output. I tried the default Terramaster OS first and it wouldn't even install. My disappointment deepened. I opened it up and no display output on the soc board either. However, there is 1 shortest pcie slot. I've put my spare gt1030 (which is full-sized) in it and "it just worked". For this I had to unscrew the soc and lift it as the nas enclosure prevented inserting of the card. To my most pleasant surprise there are also 2x full size nvme slots on the soc (my vm app pool rejoices) I've not seen mentioned in the "manual" (which literally is only a set of simple instruction html pages available online, the only downloadable resources are 4 pictures of the nas, so yeah)

After that I just removed the Terramaster init boot usb from soc, plugged in my drives and connected display+kb. It has ami bios, and the most important thing to do in it is to DISABLE fast boot (credit to a reddit thread I lost a link to). With fast boot enabled I couldn't create pools as the partitions weren't written properly. I also set the default performance profile from balanced to performance. The bios has surprising amount of cpu/ram and south/north bridge options (even more so than my old supermicro x11ssl-cf had), but I mostly left them alone. I only changed the "wait for enter bios keypress" in Boot from default 20secs to 3sec to make re/boot quicker.

Boom! Success! Really happy with how it all turned out.

 

[pimposh]

Congratulations. Couple of questions though

1) what's the CPU utilisation at medium to high load on drives? given that C3558 performance on paper looks half of the V1500B you got in the other unit, I wonder if this unit can do anything else during storage tasks (ruining patience sometimes isn't worth any savings...)?

2) What about the cooling fan settings in Scale? Are they purely hardware speed adjusted, or have you lost the native terra-smth-OS ability to auto/manually adjust them per preference?

Cheers

 

[sammael]

1)
I am currently copying few terabytes from the backup over a 1GBps line continuously since the nas was up, so the utilisation of the drives is perhaps not at its maximum, but it is maximum what they gonna be for me as 1Gbps was always enough for me (there's never more than 2 people accessing the data). There is also a VM running which runs several containers, accessing some of the already copied over data. I've been using the containers running on it without noticing any slowdowns (mainly tube-archivist and navidrome). The more demanding apps/containers run on the qnap which also has nvidia p400 available for hw de/code. Both vm's run with passthrough nic's. It feels snappier than the old nas it replaced which was an old xeon you can get for 7£ on ebay nowadays.

this is the current usage (rest of processes show no cpu usage):

Code:

top - 20:53:25 up  9:22,  1 user,  load average: 2.13, 2.53, 2.71
Tasks: 469 total,   2 running, 467 sleeping,   0 stopped,   0 zombie
%Cpu(s): 15.0 us, 45.0 sy,  0.0 ni, 30.0 id, 10.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  32068.6 total,   1045.4 free,  31133.0 used,    365.3 buff/cache
MiB Swap:  16367.0 total,  16207.0 free,    160.0 used.    935.6 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  23910 root      20   0   15496   7308    324 R  30.0   0.0 217:16.57 rsync
  24358 libvirt+  20   0   18.5g  16.6g  55264 S  25.0  53.1 114:00.91 qemu-system-x86
   3815 root       1 -19       0      0      0 S  10.0   0.0  34:01.03 z_wr_iss
   7582 root      20   0 2778204 473064  36164 S  10.0   1.4  54:23.58 k3s-server
  18356 root       1 -19       0      0      0 S  10.0   0.0  34:00.77 z_wr_iss
 371465 sammael   20   0    9520   5092   2820 R  10.0   0.0   0:00.05 top
    343 root      39  19       0      0      0 S   5.0   0.0   3:23.41 dbuf_evict
   3817 root       0 -20       0      0      0 S   5.0   0.0  10:55.56 z_wr_int
   3933 root       1 -19       0      0      0 S   5.0   0.0  34:02.83 z_wr_iss
  23907 root      20   0   15236   7992   1292 S   5.0   0.0  30:02.85 rsync
  23914 root       0 -20       0      0      0 S   5.0   0.0  10:57.27 z_wr_int
  23915 root       0 -20       0      0      0 S   5.0   0.0  10:56.05 z_wr_int

2)
I am not even aware there are any fan related settings in Scale, where would I find those? The fans on both nases are left on default, they are quiet enough, well as quiet as 20 spinners can be, which is to say if there's copying going around you hear the drives before the fans. This all is a moot point for me though as they sit out of the way in the coldest corner of the house with no heating where they don't bother anyone.

 

[sammael]

for comparison this is usage on the V1500B system that sends the data:

Code:

top - 21:29:31 up 4 days, 18:38,  3 users,  load average: 4.47, 4.91, 4.75
Tasks: 738 total,   2 running, 736 sleeping,   0 stopped,   0 zombie
%Cpu(s): 20.0 us, 13.3 sy,  0.0 ni, 43.3 id,  0.0 wa,  0.0 hi,  6.7 si,  0.0 st
MiB Mem :  64255.6 total,   5182.0 free,  59136.2 used,    862.7 buff/cache
MiB Swap:  16367.0 total,  16362.8 free,      4.2 used.   5119.4 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
1081989 libvirt+  20   0   21.0g  17.1g  55332 S 177.8  27.2     73,23 qemu-system-x86
1454980 root      20   0   15940  12492   5696 R  27.8   0.0 149:58.17 rsync
1955492 admin     20   0    9528   5064   2796 R  16.7   0.0   0:00.05 top
1709515 root      20   0 3154324 148272  33720 S  11.1   0.2   4:55.97 python3
   2764 root       1 -19       0      0      0 S   5.6   0.0  10:22.16 z_wr_iss
   2767 root       0 -20       0      0      0 S   5.6   0.0   4:45.49 z_wr_int_1
   2796 root      39  19       0      0      0 S   5.6   0.0   0:45.58 dp_sync_taskq
   2874 root      20   0       0      0      0 S   5.6   0.0   5:18.14 txg_sync
   2877 root       1 -19       0      0      0 S   5.6   0.0  10:21.10 z_wr_iss
   2878 root       1 -19       0      0      0 S   5.6   0.0  10:21.62 z_wr_iss
   2879 root       1 -19       0      0      0 S   5.6   0.0  10:22.41 z_wr_iss
   2891 root       1 -19       0      0      0 S   5.6   0.0  10:22.15 z_wr_iss
   4075 root       0 -20       0      0      0 S   5.6   0.0 129:59.64 z_rd_int_0
   6600 root       1 -19       0      0      0 S   5.6   0.0  10:22.70 z_wr_iss
1432092 root       0 -20       0      0      0 S   5.6   0.0  27:39.58 z_rd_int_0
1432093 root       0 -20       0      0      0 S   5.6   0.0  27:40.04 z_rd_int_1
1432094 root       0 -20       0      0      0 S   5.6   0.0  27:40.99 z_rd_int_1
1548081 root       0 -20       0      0      0 S   5.6   0.0   0:38.46 z_wr_int_0
1548082 root       0 -20       0      0      0 S   5.6   0.0   0:38.54 z_wr_int_0
1561715 root       0 -20       0      0      0 S   5.6   0.0   0:35.35 z_wr_int_1
1711984 root      20   0 9548628 168288 126408 S   5.6   0.3   6:27.02 ffmpeg

 

[pimposh]

1) I was more interested in fio benchmarks as top does not really answer my doubt how C3558 is performing on multi-drive vdev's. As long of vdev's layouts are similiar/the same.

2) I might have not been clear or misunderstood.

In general system cooling fans might be adjusted either by EC/BIOS itself (without need of anything on software side) _or_ by software daemon.

As long as you run QNAP software, fans on your second NAS (TS-873A) are controlled via software daemon which is part of QTS/QuTS.
Since we're on Scale, in this case (TS-873A) fans are ... just spinning at their minimal settings, which won't be sufficient if you move out your device from coldest corner...
There is an easy (CLI) way to get them controlled via external kernel module as long as you feel fine with it, but that is related with different thread that you may find on forums.

However, my question was Terramaster related - how do you control them if at all (you're right, scale does not allow to play with different fans controller on it's own by default) - or they don't need any software daemon and rpm adjustments are handled by EC/BIOS.

Cheers

 

[sammael]

@pimposh I guess that's bit beyond my layman enthusiast needs, once I get my fill of tinkering with vms and trying out new docker containers it mostly sits idle until I stream some media. Energy consumption was a also a consideration, the ebay components nas wasn't exactly efficient. In conclusion, due to where the machines are I do not care about the fans at all. Not 100% sure, but there might've been something in the bios of the T12. They have standard fan header so if it were to come to worst I could just connect them externally and run at full 12V.

 

[sammael]

@pimposh I had a chance to peek into the terramaster nas bios again and it has a dedicated section to fans as I remembered:

It lists 2 fans (system and cpu), the unit has 3 fans, but only 2 headers in use on the mobo leading me to believe one of them is a Y-splitter cable to 2 fans.

I also experimented with stuffing in more/ECC RAM, but sadly no bueno:
- 2x 32G ECC stick, 1x 32G ECC stick - no boot, no video output (I know it does the memory training after cold boot, but I gave it good 15mins and it was usually over in under 2min with 32G non-ECC)
- 1x32G non-ECC stick no boot/video output either so it seems it maxes out at 16G no-ECC per slot
I don't have a 16G ECC sodimm stick to test