Fan speed for 24-bay storage

[Kosta]

Hello,
this is more a general question than TrueNAS specific, but it goes hand in hand with the proposed setup. Right, so:
My new case is Inter-Tech 4724.

4U 4724 - Inter-Tech GmbH

Inter-Tech Elektronik Handels GmbH - Gehäuse, Netzteile und vieles mehr aus der Hardwarewelt

www.inter-tech.de

All is well except the fans being waaaaaaaay too loud. I expected it to be somewhat louder, but this are dBs that I know from our server rooms. Understandable.
However, I am sure I can go down with RPM.
I am aiming toward Noctua NF-F12 industrialPPC, be that 2000rpm or 3000rpm.
I previously had a PC case, and this one had 5 120mm fans, but really, the "server" was barely noticeable. And HDD temperatures were all in normal range, around 36°C.
I have to add that beside 3 120mm fans in the middle, the case also had 80mm fans in the back (which will also be replaced), and a separate CPU fan (which will also be replaced).
Possibly important: motherboard is Supermicro X11SPI-TF. It has some rudimentary fan management, but nothing fancy.

So, couple of questions:
Do you find these fans enough for that storage case, if it were filled with 24 disks? ( I am going to reckon it's enough for 10 disks I have now)
Would you rather go 2000rpm or 3000rpm?
Should I get some kind of fan management?

 

[sretalla]

You should certainly have the 3000 RPM option, you can always spin them slower. You should consider static pressure (and make sure you blank out the empty bays to have airflow moving over your disks and not just through the empty bays), so the fans probably need to be almost the same with any number of disks.

You should look up PID fan control scripts in the forum, there are several out there and it seems your board is probably already ready for it.

 

[Kosta]

Indeed. I forgot that Noctua actually has those 5V/7V adapters to throttle the speed. Ordered 3000rpm ones.

I am reading about the script for Supermicro motherboards, but I am a bit unsure where to run those? I have bunch of VMs, ESXi, do I need to boot some live CD or something? Because otherweise direct motherboard access isn't possible, if I understand it correctly?

Besides, I have another problem: my HDDs aren't showing any temperatures in TrueNAS. This used to work previously. How are the temperatures read about usually?

 

[sretalla]

I am reading about the script for Supermicro motherboards, but I am a bit unsure where to run those? I have bunch of VMs, ESXi, do I need to boot some live CD or something? Because otherweise direct motherboard access isn't possible, if I understand it correctly?

Besides, I have another problem: my HDDs aren't showing any temperatures in TrueNAS. This used to work previously. How are the temperatures read about usually?

OK, complicated...

If you're doing it right, you'll be passing your HBA through to the TrueNAS VM, so your temperatures will be read via smartctl... if you pass virtual disks into the VM from the hypervisor, that isn't going to work.

I haven't seen anyone sharing information on running the fan scripts to attach to a supermicro board that isn't on bare-metal TrueNAS, so I can't confirm how that might be done (other than knowing that the script calls ipmitool, which can call over IP addresses to your IPMI card, so should be doable with an adjustment to the ipmitool variable in the script to include the IP and login). If that's no good, I use a Corsair Commander Pro to control my fans (no supermicro MoBo for me) and a custom version of the script I created to allow the script to get to the fans via the controller which I pass into the VM over USB.

 

[Kosta]

I am passing through both HBAs to TrueNAS. I checked the status of one of disks with smartctl -x /dev/da6, and it says RAW_VALUE 33.
However no temperature in Reporting. What am I missing?

Oh I see, I didn't see the IP/login field. Will certainly try executing that from TrueNAS VM itself.

And what is really bugging me. Previous setup was for some reason "sorted". As in, da0, da1 etc, corresponded to disks 1-8 from top to bottom.
However now, all is mixed up, and I have NO idea which disk is which.

Besides, for some reason, some disks are showing errors, which they didn't do in the previous setup. There is a new HBA currently, and all connections go over SAS connectors. Controllers are LSI 9201-8i.

I really thought this would be waaaay less painful.

 

[sretalla]

I am passing through both HBAs to TrueNAS. I checked the status of one of disks with smartctl -x /dev/da6, and it says RAW_VALUE 33.
However no temperature in Reporting. What am I missing?

I think you need to specify in the additional smart options that your disks are SAS... SATA is expected.

And what is really bugging me. Previous setup was for some reason "sorted". As in, da0, da1 etc, corresponded to disks 1-8 from top to bottom.

Are you talking about the disks list? no idea what you're referring to. You can't rely on the label assignment to remain constant anyway. You need to refer to disks by serial numbers to be sure.

Depending on what hypervisor you're using, maybe there's an option to control the fans directly on it... I think there's something called fancontrol which you may be able to install if your hypervisor is debian based (like proxmox).

 

[Kosta]

I think you need to specify in the additional smart options that your disks are SAS... SATA is expected.

No no, disks are SATA, connections from the backplane to the HBA are SAS.
A thing changed here with the new setup: previously, I used a SAS to 4x SATA cable, which I connected directly.
Now, disks are hot swap, with SAS 3G backplane.

Are you talking about the disks list? no idea what you're referring to. You can't rely on the label assignment to remain constant anyway. You need to refer to disks by serial numbers to be sure.

Yes, I am talking about that. Still, if I see a failing disk, how do I know which disk in the front is it??

Depending on what hypervisor you're using, maybe there's an option to control the fans directly on it...

It's ESXi. Can ESXi do anything?

 

[sretalla]

if I see a failing disk, how do I know which disk in the front is it??

If you wanted enclosure management, you need to have bought the hardware from iX...

You have a few options to do poor man's enclosure management.

1. Excel ( I use that method)... I keep a table in the shape of the enclosure with the details of each disk in the respective cell.

2. dd ... you can either use it positively if the disk is still responding at all... like dd if=/dev/da1 of=/dev/null bs=1024K count=5000 that should make the activity light go on solid for some seconds on the disk in question. If the disk is really dead, you may have to do it by elimination for every disk until you don't see a light.

3. Label your drives physically in a place where you can see the serial number from the front and use that to match to the disks list.

It's ESXi. Can ESXi do anything?

Not that I'm aware of.

No no, disks are SATA, connections from the backplane to the HBA are SAS.
A thing changed here with the new setup: previously, I used a SAS to 4x SATA cable, which I connected directly.
Now, disks are hot swap, with SAS 3G backplane.

So you need to tell smartctl about that in the options for those disks... if you say nothing about the type being SAS, it will assume SATA and give you nothing.

 

[Kosta]

If you wanted enclosure management, you need to have bought the hardware from iX...

Wasn't aware I needed "enclosure management". But: looking at the homepage, I am sure these cost way more than I would be ready to pay (Inter-Tech cost me €500). So fine, I'll live with what I have.
And yes, Excel will do in this case. Small price for the cheap case. Beside, it's not like I change disks daily.

So you need to tell smartctl about that in the options for those disks... if you say nothing about the type being SAS, it will assume SATA and give you nothing.

Thank you. That worked out, however it did not accept smartctl -d scsi, gave me an error. However, after setting smartctl -d ata, it existed OK, and now I can see the temps. Weird, shouldn't it recognize disks as SATA anyway?

 

[sretalla]

I think the default is -d sat, so you needed to push it to the other option.

 

[Kosta]

I thought default sat is same as ata, but apparently not.
I thank you very very much for helping me out!

Now, if I could get SSH working to be able to copy data to TrueNAS and try those scripts, things would be peachy. But for some reason, SSH isn't working any more (did previously).
I posted a separate thread, if you feel like answering:

SOLVED - SSH used to work, doesn't any more

Hello, not sure what changed since I had it working last time, most likely system update (inside of 12.x version). I would usually create keypairs, give the public key to the root, then set up ssh connection, select the private key... It starts the connection up, I can input my credentials, but...

www.truenas.com

 

[Kosta]

So, I implemented the mentioned script, running spinpid2.sh successfully in a SSH session, fans are being controlled well and according to set temperatures.

I now would like to implement an postinit task, however it doesn't work.
I made it +x of course, otherwise I would not have been able to run it in the first place.
I tried setting the script as a Script in POSTINIT or as a command. For script, I would simply give it a path, or with command I would say ./pathblahblah/spinpid2.sh.
And I tried different locations, like /root/spinscripts or /mnt/myvdev/mypool/scripts/spinscripts.
Nothing helps apparently.
I am most likely missing something. Any idea what?

 

[Kosta]

Small update:
I got it to work, I had to put the script on the /mnt AND I had to use a command, without ./ in the front.
Additionally, Timeout has to be higher than 0 - my thinking was 0 is disabled. Oh well. Good that it works.

But, here's the problem now:
Do I set the timeout in seconds for a year or more, or how do you go about this? If I leave it low, it will stop the script, right?

 

[sretalla]

I have had very good success using tmux for that... like this:

OpenCorsairLink in a jail to control fans?

I have had another look at this and was finally able to compile the official project (https://github.com/audiohacked/OpenCorsairLink) on a stock FreeBSD 12 install which I will do a bit more to document in the next little while and will also update the script to use the latest updates to @Kevin...

www.truenas.com

tmux new-session -d -s fanscript '/mnt/tank/scripts/MASTER_PID_fan_controller.pl'

To make sure that you get the interpreter you want, you may also want to specify like /bin/sh in front of your script (I don't need to do that with perl for mine).

Also you could just put & at the end of your command to have it detach immediately to avoid the timeout.

In addition though, with the tmux method, you can start and stop the script by attaching to the tmux session at any time without needing to reboot.