Recommendation for 12 or 24 bays

[john60]

A year ago, this forum help me pick the HW for my budget build. My goal was to get to know Truenas. A year later, I am a Truenas fan. Actually Truenas Scale fan. Thank you. If I may, I would like your advice for upgrading my system.

I have 2 Truenas systems (Dell PowerEdge T320 Tower E5-2440, each with a For LSI 6Gbps SAS HBA 9240-8I IT Mode ZFS FreeNAS unRAID + 2*SFF-8087 SATA and each of these Truenas and my server has a 10Gb SFP+ PCI-E Network Card NIC, with Broadcom BCM57810S Chip, Dual SFP+ Port, PCI Express X8, Support Windows Server /Linux/VMware) card. The disks are qty 6 of 4TB in each of the Truenas systems. The disks were the cheapest no-name brands I could find on eBay arranged in a Z2 configuration.

My server -> truenas1 -> truenas2
-> backblaze

I am now getting rather attached to my data, so am getting a little scared with the no-name brand disks. Also, the Z2 pools are 80% full.

So I purchased qty 6 Western Digital 16TB WD Red Pro NAS Internal Hard Drive HDD - 7200 RPM, SATA 6 Gb/s, CMR, 256 MB Cache, 3.5" - WD161KFGX. I am in the process of replacing the disk in Truenas1. I would like to use the non-name disk removed from Truenas1 and place them in Truenas2. This way I get 4x capacity in Truenas1, and 2x capacity in truenas2.

Can someone recommend a low-budget 12 or 16 bay server on ebay that I could use to replace Truenas2?
It will sit in my office, so it can't be a screamer.
Cannibalizing Truenas2 is an option if it helps.

 

[MisterE2002]

More disks = more heath. Some descent cooling is preferred. With about 10 drives you can use a "normal" desktop case. But if more, i would recommend a rack case. Second hands supermicro cases are nice.

But the fans are loud. Personally i bought the SC846 (24 disks). Connected the fans to my (supermicro) motherboard and now it will control the speed. Was still a bit loud for my tasting and bought quieter ones.
2 x FAN-0104L4 -> 672042023585 - San Ace 80 - Model: 9S0812P4F051
3 x FAN-0074L4 -> 672042020270 - San Ace 80 - Model: 9G0812P1F03

That said, my nas is on another room and with the door closed it is acceptable.

Another big advantage is the internal SAS expander. If have 2 cables (but only one is required) to my hard drives.

Also check the community hardware guide.

 

[john60]

Thank you for the good information.

Do you mean like this ebay listing ? It is a lot more than I was expecting.
Is this 12 bay ebay listing similar but cheaper?

The vendor said "361-8i up in JBOD mode" for running truenas. Do you know if this is true and how does this relate to IT mode?

 

[MisterE2002]

if you want the expander route please read https://www.truenas.com/community/t...-sas-sy-a-primer-on-basic-sas-and-sata.26145/

For the "expander" you probably want the version with the SAS2 backplane (BPN-SAS2-846EL1). Validate this with a seller!

I bought the everything second hand. All prices in Euro's (Supermicro SuperChassis 846BE16-R920B: 300, rails: 50, Supermicro AOC-S2308L-L8e: 50)


Note: the expander also has a firmware. Retrieved a newer version of Supermicro but decided not to take the risks. HBA updated with the official SM IT software.

Using it with Toshiba MG09 disks without problems. After initial data transfer i even disk checksum compares using the old and new nas. Everything correct as expected.

 

[john60]

Again, thank you.

Sorry for being slow; what did I say that indicated "if you want the expander route"? I may not have the terminology right.

The first ebay listing of the 24 shelf is about 2000 euro. So it must be either a ripoff of something much different from yours.

 

[MisterE2002]

The expander is what is a switch for your network. The jgreco link i supplied is really good. But i had to read it multiple times to sink in :)

You also have options with 4-to-1 explained in part "Supermicro backplanes (TQ, A, BE16, BE26)"


But if you want the SuperMicro route. Know thing works the best with everything from them. Although most cases can handle the usual motherboard formats the case header connector (power/reset/led) is proprietary. But they also sell a adapter to normal pin headers.

The case fans are no consumer fans. So a bit louder and using more power. Not sure if a normal consumer motherboard can power this. If you replace the SM fans in the case with normal ones you can get troubles with the automatic speed control (it will reduce the speed, thinks it is broken and will ramp up the speed to max. Reinse repeat)

The SM board also has a Power i2c connector. So the IPMI knows the health of the (2) PSU's.

About the PSU's. Make sure you get the SQ versions!.

The remote desktop feature is also nice.

I bought from this guy in the Netherlands

 

[MisterE2002]

The first ebay listing of the 24 shelf is about 2000 euro. So it must be either a ripoff of something much different from yours.

Yeah ripoff. "chia" coins miner pushing the price.

And not clear what motherboard is used. Probably consumer/gaming stuff.
The 760 (i have 2x 920) Watt SQ PSU is more than adequate.

JBOD is probably a TQ backplane (24 sata cables). I think i was wrong here. It should be a case with a expander (BE2C). But *only* if the original backplane is not switched. Therefore always verify with the seller.

 

[john60]

The system I am leaning on is ebay listing that includes the case, controller, psu, CPU and memory.
All that is needed is disks (I have qty 12 4TB SATA) and to program the 9361-8i in IT mode.

Way too much memory and CPU, but maybe I will find a use for this capacity.

Then sell my existing Truenas system.

Does anyone know if the 9361-8i can be programmed in IT mode?

If the PSU are too noisy, then I will need to purchase some SQ ones.

 

[MisterE2002]

SAS3 is probably overkill (expensive) for spinning rust. Also ebay is unclear. Is stuff defect? (red)

For HBA check https://forums.servethehome.com/ind...and-hba-complete-listing-plus-oem-models.599/

 

[john60]

I have a 9361-8i and can't find a HBA IT firmware for it, but it has JBOD mode.

Is there a comprehensive test to confirm if this card's JBOD actually implements the same as the typical HBA IT mode?

Otherwise, we are stuck in the scenario of "these old HBA in IT mode believe to work" but no way to actually test, and these new cards may appear to work, but we fear they are missing something but may actually be more complete and robust as compared to the old cards.

Oops, this comment went on the wrong thread.

 

[MisterE2002]

I am not really experienced in HBA's (and al the numbers, etc.) but i think the sas3 card has no advantages with your plan (normal hard drives). And only will use more power.
For the 2308 (https://www.supermicro.com/wdl/driver/SAS/Broadcom/2308/Firmware/) i can find official IT firmware.
The 3108 is unclear (https://www.supermicro.com/wdl/driver/SAS/Broadcom/3108/Firmware/).

Maybe you better can create another thread specific for the HBA question.

 

[john60]

I purchased the SuperMicro 6028U-TR4T+ 2x E5-2650v4 128Gb 9361-8i RAID 12LFF Server X10DRU-I+ A4.
Hope it is not too noisy.
Hope the JBOD mode on the 9361-8i works.

 

[jgreco]

Is there a comprehensive test to confirm if this card's JBOD actually implements the same as the typical HBA IT mode?

It doesn't and of course there isn't. Drive failures, swaps, and some disk errors aren't handled consistently. However, this has already been explained to you in another thread. Please limit your discussion of uncertainty to that thread; while we do not mind explaining it to you in one place with appropriate context, misinformation presented as "just asking questions" confuses the new users.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com