Gifted a SM846 system which I am upgrading. Looking for feedback, suggestions, and guidance as a first time TrueNAS user and homelab builder

[Koop]

Hello everyone,

Been lurking and reading for a while while doing my best to learn as much as possible as I slowly work my way through this home lab building process.

My primary use case for this system is to first and foremost to experiment with and have fun learning and exploring all the configuration and features of TrueNAS itself.

After that however I'd like it to be my main repository for media such as photos, music, movies, documents and whatever else would normally end up on large single disk storage or external drives scattered around my home for my family. I have a ton of content strewn about and I wish to consolidate it all to a single organized platform.

I'd like to experiment with apps such as Nextcloud, Plex, HomeAssistant- there's a massive catalogue to comb through and play with but those are just the few I feel I'd use long term. I'd also like to out running some VMs just because Scale can do it. Eventually I'd like to have a separate Proxmox server dedicated to virtualization but for now I'm trying to make my TrueNAS build my one stop shop.

I know that's a perhaps an ambigiouis use case of "I just want to do and try everything!" but it is sort of the truth as of now. The only data that will truly be critically importan to me will be personal/family photos, video and documents. I will be backing up that particular data outside of this system to ensure it's not lost during any 'experimentation'.

I've been making decisions and was looking to see if I have any misunderestandings, made any oversights, or if anyone wanted to generally provide feedback on my decisions thus far. Being new to TrueNAS I am sure there are things I may not realize I am doing wrong. Thank you in advance to anyone who provides feedback.

I was gifted a 4U SuperMicro 846 system. This is what kicked off the entire idea to build out a home lab. I suspect it specifically was a SC846TQ-R1200B based on the backplane and PSUs it came with- but I don't really know for sure. Regardless, inside was the following:

Supermicro X8DTH-i motherboard
64GB DDR3 ECC Memory
2 * Xeon E5620s
BPN-SAS-846TQ backplane
3x SAS9211-8i IT Mode SAS HBA spread across all individual connections
1x Intel SSD as a boot drive "casually" hangin' loose in the chassis behind the DVD drive
2x PWS-1K21P-1R (1200W)

My goal was to take the above system and begin 'upgrading' it. After reading the SAS Primer the first thing I did was replace the current backplane with a BPN-SAS2-846EL1. The goal here was to cut down on the need for 3x HBAs and all the wiring cluttering the fanwall. I paired the backplane with a Supermicro AOC-S3008L-L8E. While I understood there is a SAS3 backplane option my goal so far was only to use spinning disks so I most likely would not encounter any type of bandwidth limitations. The AOC-S3008L-L8E was also relatively inexpensive and I figured if I ever had a need it could be leveraged in the future if I were to get a SAS3 backplane. Also the backplane I got was cheap as dirt ($40) vs the hundreds of dollars for a BPN-SAS3-846EL2. I flashed the HBA into IT mode, swapped the backplanes connected my two SFF-8643 to SFF-8087 cables and everything is working without issue.

Next I planned to shrink down to a single CPU system. I had honed in on going with either a single CPU X10 or X11 board- and was narrowing it down to the X10SRI-F. I was skimming ebay and forums for memory and a CPU choice when I came across a 8x Hynix 64GB LRDIMM PC4 2400 lot (HMAA8GL7MMR4N-UH) which would be the maximum capacity for the board. It looked like a great deal so I scooped it up. Maybe a buy before thinking situation but with my very loose use case of "try everything" I figured this would give me the most flexibility for deploying VMs, etc... Plus every time someone asked on the forums here how to improve their performance the first answer was usually "Can you add more memory for ARC?" so I figured there wasn't really a negative conscequence of having too much memory other than not leveraging it all- which means I just need to find all the ways I can!

Before purchasing my planned X10SRI-F however I came across someone selling motherboard+memory+cpu+cooler combo at a fantastic price. While I originally was hesitent in the price jump to an X11 board while considering them on an one by one basis, this deal felt too good to pass up. What I got was the following:

Supermicro X11SPi-TF motherboard
128GB DDR4 ECC memory (SK Hynix 4x32GB RDIMM, HMA84GR7MFR4N)
Intel Xeon Platinum 8153
Noctua NH-D9 DX-3647 4U active CPU cooler

I know from all my reading that it's best to avoid 10GBase-T, but it's here now. I did already own a small 10GBase-T switch as well so I'll be able to leverage my connectivity there not that I think i will ever be able to saturate that connectivity with the drives I have. If I add additional networking to my system though I know to refer back to the 10 Gig Networking Primer and go the route of SFP if there arises a need.

Obviously I have an issue of exccess memory between the two kits. Once I'm done testing it all I may consider selling some- or even better- keeping it to use in that eventual proxmox build.

For now I plan to use the Platinum 8153 but I know there are many inexpensive options I can jump between. If anyone had a particular CPU reccomendation I'd very much like to hear opinions. I was looking at the Gold 6132 as a good option but there's about a zillion xeon SKUs so reccomendations would be appreciated.

Finally I wanted to talk about my storage media and layout plans. I have a total of 26 10TB 7200RPM drives. My plan is to slot in 24 drives and keep two on hand as cold spares. I know that this gives me a few options and from my limited knowledhe thus far I could consider 3x 8 wide RAIDZ2, 4x 6 wide RAIDZ2 or for maximum capacity 2x 12 RAIDZ2- is that corret? I assume the right place to land here is with either the 8 or 6 wides choices.

However another thought I had was if I want to experiment with different layouts I should consider only using a portion of my disks for a 'media' pool and create a second pool with something like, not sure, mirror vdevs? Just for fun and to see how it works out. Maybe that makes sense to use for VM data? Then I can have that be my experimental pool of disks. Still learning so forgive me if that doesn't make sense. I'd be interested to hear opinions on how others would break these disks up given my, again, understanbly vauge use case.

Some other things I've done for the system itself:

• Replaced the stock PSUs with a pair of PWS-920P-SQ PSUs
  
• Aquired 2x Supermicro SSD-DM128-SMCMVN1 for the use of mirror boot drives
  
• Experimented with using FAN-0074L4 fans on the fan wall (this wasn't a smart idea. Those drives get a bit toasty with only these)
  
• Tried to find a replacement top panel as mine is damaged. I found part number MCP-230-82501-0N via newer 846 chassis models but wasn't sure if it would be the same exact fit for mine
  
• I wish to remove the DVD drive from my chassis but have no idea what part would be used to blank out the slot

Some things I've done to create set up my own home lab rack:

• Aquired a 4-Post 25U StarTech open frame rack
  
• Aquired 2x refreshed Tripp Lite SMART1500RM2U UPS systems
  
• Aquired a Rosewill RSV-L4412U Server Chassis for use as my potential future proxmox server location

Some things I'd like to look more into for my homelab:

• A top of rack switch (brocade?)
• A dedicated hypervisor (proxmox)
• Rack PDUs (maybe unnesissary and expensive?)
• Cable management solution that works well with my open frame rack
• Solutions for more airflow/cooling to front of rack?
• Placing my rack in a sudo-cabinet by putting it in a literal closet?
  • Then adding discreet vents to the top and bottom of the door

That's everything I can think of. Sorry for such a massive post. If anyone has feedback or opinons please feel free to share so I can continue learning. Appreciate all the resources here and all the knowledge I've been able to thus far accumulate becuase of it.

 

[artlessknave]

BPN-SAS3-846EL2

note that that is a dual path expander model number. it would be more expensive. you would likely want a BPN-SAS3-846EL1

I wish to remove the DVD drive from my chassis but have no idea what part would be used to blank out the slot

doesn't really matter. the chassis model you have in the post doesnt appears to have a dvd drive. I have a 3u and replaced a dvd drive and internal hdd bay with icy dock slim HDD parts to give me 2 extra hotswap drive bays.

https://www.amazon.ca/gp/product/B01998GYJO/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

3x 8 wide RAIDZ2, 4x 6 wide RAIDZ2 or for maximum capacity 2x 12 RAIDZ2- is that corret?

for a 12 wide you might want to consider raidz3, as its getting close to that point. otherwise correct.

portion of my disks for a 'media' pool and create a second pool with something like, not sure, mirror vdevs?

absoolutely possible. depends on your needs. i find managing multiple pools annoying so I generally tend toward as few pools as possible.

other than that, you seem to have everything pretty much under control.

 

[danb35]

I assume the right place to land here is with either the 8 or 6 wides choices.

Yeah, 12 disks are kind of wide for a vdev. Given that you have all the disks now, I'd probably go for 8-wide vdevs, and whack in a couple of SSDs for VM and/or app storage.

As for CPU, if you have the 8153, might as well use it--I don't see that downgrading it buys you anything. Idle power consumption between that and, say, a 6132 doesn't vary appreciably.

A top of rack switch (brocade?)

I trust you've already run across the servethehome.com thread on these? They're very featureful for the cost; I was able to replace four switches with one of the Brocade units.

A dedicated hypervisor (proxmox)

I like and mostly use Proxmox (I'm running a total of six PVE servers), but xcp-ng may also be worth checking out. And Xen Orchestra (its management UI) is available as a TrueCharts app for SCALE.

Cable management solution that works well with my open frame rack

A definite weak spot for me.

Aquired 2x refreshed Tripp Lite SMART1500RM2U UPS systems

If these allow for external battery packs like my APC unit does, you might want to consider using lithium batteries instead. I just did that on my APC UPS because I got sick of lead-acid batteries dying all the time. It's ugly, but it works:

Same charge profile as lead-acid, lifespan of thousands of cycles, and over an hour of runtime for my full rack (which draws about 1.2 KW).

 

[joeschmuck]

consider using lithium batteries instead.

But there is a safety consideration to be aware of. Any overcharging or even dropping the voltage too low on a lithium battery can lead to really bad things. Your batteries should be placed outside in a proper fire-proof area. I woke up in the middle of the night last night and went out to the garage to unplug a lithium battery I was charging earlier for my impact driver. Yes, I was a bit paranoid you could say, but way too many fires are in the news.

Same charge profile as lead-acid

Not exactly the same. Lead-acid is a slow low current charge rate, Lithium is a fast high current rate. The final voltage is different as well. None of these would harm the Lithium battery.

There are articles on the internet worth reading if you venture down this path.

 

[Koop]

note that that is a dual path expander model number. it would be more expensive. you would likely want a BPN-SAS3-846EL1

Ah thanks for pointing that out. You'd use dual path for failover between two HBAs, yeah? Would there be any other reason or things you could do with it?

doesn't really matter. the chassis model you have in the post doesnt appears to have a dvd drive. I have a 3u and replaced a dvd drive and internal hdd bay with icy dock slim HDD parts to give me 2 extra hotswap drive bays.

https://www.amazon.ca/gp/product/B01998GYJO/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

Actually mine has the DVD drive. I might be wrong on my exact model number then? I had found the SM part MCP-220-84610-0N which is stacked dual SSD part that first appeared like I could use it in the same position as the DVD slot but upon measuring the exact length I saw it wouldn't fit in the same slot size as the DVD drive. Here's what mine looks like for refrence:

Would love to use that slot for SSD(s) though if there's something I could fit there.

for a 12 wide you might want to consider raidz3, as its getting close to that point. otherwise correct.

Makes sense. I figured that was a stretch to go that far.

absoolutely possible. depends on your needs. i find managing multiple pools annoying so I generally tend toward as few pools as possible.

other than that, you seem to have everything pretty much under control.

Fair enough, I obviously don't know any better so I'm just spitballing. What do you feel makes managing multiple pools annoying? And thank you- at least better control than if I started to do things with 0 research haha. Still learning of course.

Yeah, 12 disks are kind of wide for a vdev. Given that you have all the disks now, I'd probably go for 8-wide vdevs, and whack in a couple of SSDs for VM and/or app storage.

I figured as much on the 12 wide. Regarding the SSDs I had the same thought- or at least the general idea of "Maybe I could have an SSD pool for something one day". My only issue with that is I'd have to have all my SSDs inside the chassis. I know there's ways to mount them to the inside of the chassis, like using part MCP-220-84603-0N, but I only have one molex connection available to me. I'm not 100% sure on how many SSDs I could power from that but I'm also weary of splitters and since I know cheap molex to sata plugs can be a melting/fire hazard. I don't think there's a different power distributor I could use to solve this, is there? Any ideas?

As for CPU, if you have the 8153, might as well use it--I don't see that downgrading it buys you anything. Idle power consumption between that and, say, a 6132 doesn't vary appreciably.

Makes sense when you put it as downgrading. I just know there's a lot of options to jump between core count and clock speed and wasn't sure if perhaps it may be in my favor to drop cores to get a higher base clock- or if should consider looking at CPUs with lower TDP perhaps. I guess if I was worried about power consumption I would look at why I have 24 7200RPM disks first though hah.

I know that things like SMB are single threaded per client, but I don't believe with my small number of local users in my home that I would need to worry about that? I'm obviously lack knowledge for when such things would, if ever, matter in my use case.

I trust you've already run across the servethehome.com thread on these? They're very featureful for the cost; I was able to replace four switches with one of the Brocade units.

Yeah exactly where I got the specific brocade idea from. I actually do have (old) education on Cisco IOS but that thread had me convinced.

I like and mostly use Proxmox (I'm running a total of six PVE servers), but xcp-ng may also be worth checking out. And Xen Orchestra (its management UI) is available as a TrueCharts app for SCALE.

That's fair on xcp-ng. I'm not too knowledable on the world of hypervisors and the options out there. The only experince I've had was with VMWare and well... Yeah don't think that's going to be an option.

A definite weak spot for me.

Not going to win the clean rack award huh? To quote blade runner "I've seen things you people wouldn't believe...".
"Why is all my equipment overheating!?" only to head around back to see a waterfall of cabling blocking any semblemnce of airflow. "Well sir..."

If these allow for external battery packs like my APC unit does, you might want to consider using lithium batteries instead. I just did that on my APC UPS because I got sick of lead-acid batteries dying all the time. It's ugly, but it works:

Same charge profile as lead-acid, lifespan of thousands of cycles, and over an hour of runtime for my full rack (which draws about 1.2 KW).

Interesting. I'll have to keep that in mind as somethinig to researh.

But there is a safety consideration to be aware of. Any overcharging or even dropping the voltage too low on a lithium battery can lead to really bad things. Your batteries should be placed outside in a proper fire-proof area. I woke up in the middle of the night last night and went out to the garage to unplug a lithium battery I was charging earlier for my impact driver. Yes, I was a bit paranoid you could say, but way too many fires are in the news.

Thank you for the warning.

Not exactly the same. Lead-acid is a slow low current charge rate, Lithium is a fast high current rate. The final voltage is different as well. None of these would harm the Lithium battery.

There are articles on the internet worth reading if you venture down this path.

Thanks, will be sure to research if I need it. My hope is just to be able to power down things quickly during a power outage rather than keep things running for any signifigant amount of time. I believe I should be good with my two units but I haven't fully tested it out. Not like it would be the end of the world but I figured I should at least make a proper attempt.

Appreciate all the feedback. Thanks.

 

[Ericloewe]

Would love to use that slot for SSD(s) though if there's something I could fit there.

Icy Dock makes SSD enclosures in the slim optical drive form factor.

 

[danb35]

But there is a safety consideration to be aware of. Any overcharging or even dropping the voltage too low on a lithium battery can lead to really bad things.

There are lots of lithium chemistries out there, and I should have been more specific--LiFePO4 (or LiFeMgPO4, which is what I'm using) doesn't have the risk of fire of some of the others. It also doesn't have quite the energy density, though it's still far better than lead-acid. The batteries I'm using have an integrated battery management system that will shut them off if the voltage drops too low or rises too high, and that's definitely something you'd want with any sort of Lithium battery (either built-in or external).

LiFePO4 and LiFeMgPO4 are also convenient in that four cells in series are a very close match for a 12V lead-acid battery in terms of voltages--charging, resting, under load, etc. With an appropriate BMS, they really can be a drop-in replacement, and many are now sold that way. They can, as you say, take a much faster charge than lead-acid, but a slow charge doesn't harm them in the least--it's just slower. I'd like to be able to tweak my UPS to charge a bit faster (I measured 2.7A charge after my runtime calibration cycle the other day; the batteries could easily take a 30-40A charge), but that really isn't that critical.

So, @Koop, no, you shouldn't just whack in some random lithium batteries. But a good set of LiFePO4 and LiFeMgPO4 that can tolerate series connections to whatever voltage your UPS takes (mine takes 48V, hence my using four of them in series) will likely last you much longer than the standard lead-acid batteries. But they'll probably look kind of ugly.

 

[joeschmuck]

There are lots of lithium chemistries out there, and I should have been more specific--LiFePO4 (or LiFeMgPO4, which is what I'm using) doesn't have the risk of fire of some of the others.

Thanks for the clarification. There are a few of them as you said.

To whoever wants to use one of these batteries, please do your research, make sure you purchase the correct version, don't let it be the reason your home burns down or worse. I may be emphasizing this but it is a serious safety issue. As I get older, safety has become more important. I think I realized I'm not immune to injury anymore. And when we get into a car and get on the road, that in itself is a huge risk. Idiot drives out there holding a smartphone in one hand while driving down the road. So many accidents, so much death. If I sound like I'm a Medic in an Ambulance, I'm not, I slept at a Howard Johnson's last night . Okay, trying to add a little humor to a serious topic.

 

[artlessknave]

Actually mine has the DVD drive

ohh, its in the back. i haven't seen that before.
if you have a way to get occulink to it (this is new, last time I saw it it was only a concept):

MB852M2PO-B_2 Bay M.2 NVMe SSD PCIe 4.0 Mobile Rack Enclosure for 9.5mm Ultra Slim ODD Bay (2 x OCuLink SFF-8612 4i, no Tri-mode support)

global.icydock.com

otherwise any of these should work.

MB411SPO-2B_2.5" SSD / HDD Hot-Swap SATA Mobile Rack for 9.5mm Ultra Slim CD/DVD-ROM Optical Bay

global.icydock.com

MB411SPO-1B_2.5" SSD / HDD Hot-Swap SATA Mobile Rack for 12.7mm Slim CD/DVD-ROM Optical Bay

global.icydock.com

MB411SKO-B_2.5” SATA/SAS HDD/SSD Rugged Full Metal Mobile Rack for Slim ODD or Slim FDD Bay - With Key Lock

global.icydock.com

MB411SPO-B_2.5” SATA/SAS HDD/SSD Rugged Full Metal Mobile Rack for Slim ODD or Slim FDD Bay

global.icydock.com

a molex will run multiple SSDs just fine. they dont use that much power, nor do they have bug surges in power for spinup like HDDs do. I wouldnt go much over like 10, but less than that should be fine.

 

[Koop]

Icy Dock makes SSD enclosures in the slim optical drive form factor.

Ahhh ok, yes, @artlessknave refereed to them as well. My only objection is only getting a single drive in that space- it seems wasteful. I can see right behind the DVD drive on the inside wall of the chassis is multiple sets of threaded insert. I can see that using MCP-220-84603-ON would allow me to mount at least a total of 4x SSDs but truthfully they are just chunks of metal to secure drives internally. I have seen there are even more novel ways to mount a higher number of drives with 3D printing so I think taking it out completely would be nice. My only problem is I don't know is a blank plate I can get to put in it's place. I couldn't find one via SuperMicro directly from my research but digging through part numbers on their page is... Difficult to say the least haha.

I think this feels like the most logical approach to mounting a decent number of SSDs. I do also have PCIe slots. I read a lot about using Intel Optane drives from the SSD performance resource so I was curious of exploring my options there. Pricing seems to be all over the place though and I apparently missed out when a lot was being sold off cheaply from what I've read.

So, @Koop, no, you shouldn't just whack in some random lithium batteries. But a good set of LiFePO4 and LiFeMgPO4 that can tolerate series connections to whatever voltage your UPS takes (mine takes 48V, hence my using four of them in series) will likely last you much longer than the standard lead-acid batteries. But they'll probably look kind of ugly.

Gotcha, that's a lot of great info. I had no idea about LiFePO4 and LiFeMgPO4 options so it's very informative, thank you for sharing it all. I'll see how fast these lead acid boxes die out on me and when I get annoyed enough to revisit the idea I'll be referring back to all you shared, haha.

Okay, trying to add a little humor to a serious topic.

It's all good! It's good to be cautious especially dealing with lithium batteries. I've had my fair share of puffy phone and laptop batteries to know what dangers are just waiting to be opened up.

ohh, its in the back. i haven't seen that before.

Yeah it seems to be pretty standard for the 4U chassis options. Newer chassis have more verticle space slots though. Either way per my observations above I think it'll be beneficial to make more overall open space to allow for more internal drive mounting whenever I want to tackle that idea.

if you have a way to get occulink to it (this is new, last time I saw it it was only a concept):

MB852M2PO-B_2 Bay M.2 NVMe SSD PCIe 4.0 Mobile Rack Enclosure for 9.5mm Ultra Slim ODD Bay (2 x OCuLink SFF-8612 4i, no Tri-mode support)

global.icydock.com

Oh wow that's pretty cool. I actually did not think of OCuLink at all- been a while since I've seen it even come up. There are definitly expansion cards I could use specifically for OCuLink connectivity. Just a quick google search brough up this SM branded HBA. I'm sure there's even more novel options I can look into as well to use my available lanes. I just have those eight internal motherboard SATA connections and it would seem silly to not leverage them somehow eventually.

otherwise any of these should work.

MB411SPO-2B_2.5" SSD / HDD Hot-Swap SATA Mobile Rack for 9.5mm Ultra Slim CD/DVD-ROM Optical Bay

global.icydock.com

MB411SPO-1B_2.5" SSD / HDD Hot-Swap SATA Mobile Rack for 12.7mm Slim CD/DVD-ROM Optical Bay

global.icydock.com

MB411SKO-B_2.5” SATA/SAS HDD/SSD Rugged Full Metal Mobile Rack for Slim ODD or Slim FDD Bay - With Key Lock

global.icydock.com

MB411SPO-B_2.5” SATA/SAS HDD/SSD Rugged Full Metal Mobile Rack for Slim ODD or Slim FDD Bay

global.icydock.com

I feel silly having never heard of Icy Dock before now. Thanks for all the potential options. Thankfully I'm in no rush I'll be taking my time to explore all my options.

a molex will run multiple SSDs just fine. they dont use that much power, nor do they have bug surges in power for spinup like HDDs do. I wouldnt go much over like 10, but less than that should be fine.

Hm ok fair enough. I'll need to look into safe cables options and not cheap junk that's going to melt on me is all. With the DVD drive disconnected there's also a small 5V power connection that was used for that as well as seen connected here:

Wonder if I can use that power along with the molex.

Anyone selling intel optane drives? haha.

Again thanks for the feedback. I'm going to keep monitoring heat with the stock fans and active CPU cooler. I may entertain the idea of finding more ways to cool my spinners and also ensure my HBA temps are good. Part of why I was curious for suggestions on overall rack cooling since I am worried my drives are going to get hotter once spring and summer roll along.

I haven't done a full burn-in yet of everything but thankfully there are resources upon resources upon resources to help out with that as I follow through with testing everything as it stands now.

If anyone has any ideas for tweaking or adding more cooling to my chassis or rack overall I'd appreciate ideas. I'd ideally not want to have that fan wall going as full blast all the time if I can help it. Obviously if it needs the cooling I'm not looking to circumvent that but if I can add more cooling somehow to lighten their load (and noise) that would be great.

Thanks again everyone!

 

[Ericloewe]

Lead-acid really is terrible

My only objection is only getting a single drive in that space- it seems wasteful. I

Icy Dock has a few models that take two M.2 SSDs. SATA is more reasonable, cabling-wise, but the SSDs themselves are less available than NVMe SSDs.

anyone has any ideas for tweaking or adding more cooling to my chassis or rack overall I'd appreciate ideas. I'd ideally not want to have that fan wall going as full blast all the time if I can help it. Obviously if it needs the cooling I'm not looking to circumvent that but if I can add more cooling somehow to lighten their load (and noise) that would be great.

Besides using SQ PSUs, the only thing you can really do is reduce the thermal load in the disk bays, i.e. Use lower-power disks. That mostly means Helium disks.

 

[Koop]

Lead-acid really is terrible

As long as it can survive long enough to let things power down I'll be happy. I'll need to monitor the battery heath carefully though with a NUT server or something.

Icy Dock has a few models that take two M.2 SSDs. SATA is more reasonable, cabling-wise, but the SSDs themselves are less available than NVMe SSDs.

Understood, makes sense and I agree. It's nothing something I'm looking to do ASAP but just trying to scope out what may be possible in my chassis.

Besides using SQ PSUs, the only thing you can really do is reduce the thermal load in the disk bays, i.e. Use lower-power disks. That mostly means Helium disks.

I did actually pick up the SQ PSUs, the 920Ws. But I have those and the originals if I need to swap. Understood on thermal load options- I was thinking more of an external rack based cooling solution where I could help facilitate better airflow to the front of the rack from below and thus when the fans pull the air over the drive- yeah. Just the idea of using rack mount solutions to increase air circulation in general and then a fresh air intake into my enclosed rack area with an exhaust out via the top.

I think there may be rack mounted fans that could help with this circulation, but not sure if it would really help for what I am trying to achieve. More research to be done.

 

[artlessknave]

I have used these to add drives (I prefer external hotswap so my choices tend very much to that). these might give you some ideas.

https://www.amazon.ca/gp/product/B002MWDRD6/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

MB840M2P-B_Removable M.2 NVMe SSD to PCIe 4.0 x4 Mobile Rack Enclosure for PCIe Expansion Slot

global.icydock.com

others that could be interesting

MB842MP-B_Removable 2 Bay M.2 NVMe SSD to PCIe 4.0 x8 Mobile Rack Enclosure for PCI Express Slot (PCIe Bifurcation required)

global.icydock.com

MB839SP-B_2.5" SATA SSD/HDD to PCIe 2.0 x1 Hot-Swap Mobile Rack for PCIe Expansion Slots

global.icydock.com

MB111VP-B_Removable 2.5" U.2/U.3 NVMe SSD to PCIe 4.0 x4 Mobile Rack Enclosure for PCIe Expansion Slot

global.icydock.com

concept (doesnt exist yet) but is very interesting. usually you can give them feedback on if you would use it and how.

CP074-2-8 x M.2 NVMe SSD to PCIe x16 Mobile Rack Adapter Card for PCIe Expansion Slot with PCIe Bifurcation Function (FHFL)

global.icydock.com

I have seen there are even more novel ways to mount a higher number of drives

technically you can doublesided tape ssds to any surface. makes them hard to replace if they have issues though. they have to be able to breath on one side.

I'll need to look into safe cables options and not cheap junk that's going to melt on me is all.

melting isnt very likely. there isnt that much volts or amps. the biggest problem I have with molex connectors is how the damn pins dont line up, preventing the plugs from connecting, or pushing the damn pins right out. so i look for ones that look well built for the plug itself. the wires dont have to be too beefy, as long as they arent fishingline thin.

Wonder if I can use that power along with the molex.

I believe thats the exact same thing powerwise, just a different connector. was used for floppies long ago, got adapted for other things later.

 

[HoneyBadger]

I think there may be rack mounted fans that could help with this circulation, but not sure if it would really help for what I am trying to achieve. More research to be done.

If you're planning to make a "pseudo-datacenter" room out of a closet, ventilation will be important - but can be as simple as a vented panel at both the top and bottom of the door with some low-noise airflow-focused fans behind them. Bear in mind that it will allow sound to escape as well as air, but baffles can help mitigate that.

 

[Koop]

If you're planning to make a "pseudo-datacenter" room out of a closet, ventilation will be important - but can be as simple as a vented panel at both the top and bottom of the door with some low-noise airflow-focused fans behind them. Bear in mind that it will allow sound to escape as well as air, but baffles can help mitigate that.

Yeah absolutely. I was looking around and came across this for home theater setups:

https://www.amazon.com/dp/B017PMM990?ref=emc_s_m_5_i_n

Maybe could be of use a pair of these going right through the door- on the bottom and top if I ended up wanting to keep the door closed. Hmm something something static pressure of the closet space... I suppose a dehumidifier could be a good idea. Thinking creatively here, it's fun.

I have used these to add drives (I prefer external hotswap so my choices tend very much to that). these might give you some ideas.

https://www.amazon.ca/gp/product/B002MWDRD6/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

MB840M2P-B_Removable M.2 NVMe SSD to PCIe 4.0 x4 Mobile Rack Enclosure for PCIe Expansion Slot

global.icydock.com

others that could be interesting

MB842MP-B_Removable 2 Bay M.2 NVMe SSD to PCIe 4.0 x8 Mobile Rack Enclosure for PCI Express Slot (PCIe Bifurcation required)

global.icydock.com

MB839SP-B_2.5" SATA SSD/HDD to PCIe 2.0 x1 Hot-Swap Mobile Rack for PCIe Expansion Slots

global.icydock.com

MB111VP-B_Removable 2.5" U.2/U.3 NVMe SSD to PCIe 4.0 x4 Mobile Rack Enclosure for PCIe Expansion Slot

global.icydock.com

concept (doesnt exist yet) but is very interesting. usually you can give them feedback on if you would use it and how.

CP074-2-8 x M.2 NVMe SSD to PCIe x16 Mobile Rack Adapter Card for PCIe Expansion Slot with PCIe Bifurcation Function (FHFL)

global.icydock.com

Nice those all look great., thanks for sharing- I'll definitely be giving all their products a look whenever I get around to considering flash expansion for sure. Concept product looks crazy.

technically you can doublesided tape ssds to any surface. makes them hard to replace if they have issues though. they have to be able to breath on one side.

Yeah of course- obviously trying to keep up at least a certain level of standard if possible haha. Some of the 3D prints I've found are pretty cool. Like this mount to hold four drives using the posts in the chassis: https://www.thingiverse.com/thing:4775186

I'm not planning on adding any additional storage yet- but I'm glad to be able to brainstorm future ideas.

melting isnt very likely. there isnt that much volts or amps. the biggest problem I have with molex connectors is how the damn pins dont line up, preventing the plugs from connecting, or pushing the damn pins right out. so i look for ones that look well built for the plug itself. the wires dont have to be too beefy, as long as they arent fishingline thin.

I've just heard about instances like this due to poor quality (lack of shielding?) cables:

Last thing I want is to smell or see that.

I believe thats the exact same thing powerwise, just a different connector. was used for floppies long ago, got adapted for other things later.

True it actually comes chained right off that molex connection so that makes sense.

 

[joeschmuck]

Those melting connectors can also be the result of someone connecting the thing with power turned on. Been there, Done that. Scared the shit out of me.

Maybe could be of use a pair of these going right through the door- on the bottom and top if I ended up wanting to keep the door closed. Hmm something something static pressure of the closet space... I suppose a dehumidifier could be a good idea. Thinking creatively here, it's fun.

If you took a small closet, removed the door, and put in a system with 24 HDDs, it will still cook the system. You must have positive airflow and enough of it.

The little cabinet coolers you listed above are great for a DVR and an amplifier but those don't generate the heat a computer does, nor a lot of hard drives. Back in the day I placed a small computer in a closet and closed the door. I was using it for hacking DirecTv way back and the computer was really on the low end of power as it only needed to help provide the proper ASIC key. Back to my point, the closet warmed up a lot and I was booting from a Floppy Disk Drive, no HDD.

So airflow is important and you need to have enough and to form a good airflow path. Before cutting your door, just remove it and see what happens. If it gets too warm, place a house fan pushing air into the closet. You will find that small places heat up fast.

The only real solution will likely be running an HVAC duct into the closet and open up an exhaust hole. It's a bit of work. If you have a basement, that is the solution if you can use it.

 

[Koop]

f you took a small closet, removed the door, and put in a system with 24 HDDs, it will still cook the system. You must have positive airflow and enough of it.

The little cabinet coolers you listed above are great for a DVR and an amplifier but those don't generate the heat a computer does, nor a lot of hard drives. Back in the day I placed a small computer in a closet and closed the door. I was using it for hacking DirecTv way back and the computer was really on the low end of power as it only needed to help provide the proper ASIC key. Back to my point, the closet warmed up a lot and I was booting from a Floppy Disk Drive, no HDD.

So airflow is important and you need to have enough and to form a good airflow path. Before cutting your door, just remove it and see what happens. If it gets too warm, place a house fan pushing air into the closet. You will find that small places heat up fast.

The only real solution will likely be running an HVAC duct into the closet and open up an exhaust hole. It's a bit of work. If you have a basement, that is the solution if you can use it.

Yeah true, I didn't really think about the power of the fans in that actual product. just made the assumption of airflow = good but perhaps you're right in that it would not move the air fast enough. Perhaps there is a similar way I could move more air and have it cycle faster.

And oh no, yeah, I planned to do it with the door off first- no way I'll be cutting holes as a first idea. I may not even put in the closet at all.

You know speaking on HVAC though there is a window right next to the closet that could be used for dumped hot air. Got me thinking too much hah.

 

[artlessknave]

I've just heard about instances like this due to poor quality (lack of shielding?) cables:

I have plugged and unplugged and messed with these things for like 2 decades and never once have I had that, even with cheapo connectors and PSUs. I'll still get good stuff for "prod" builds but I have to have enough parts parts lying around to build 15-20 full computers....never have I had any of it burn, with one exception, entirely my fault: i shorted something on a motherboard once trying to take a shortcut with a screwdriver by not turning it off. it did not survive, but neither did it burn or smoke; it just zapped once and was dead (screwdrivers are insulted so it just jumped one the board, so 12v at the most. was very dumb. about 130$ for a new board.

 

[Koop]

Alrighty all, I've had quite the journey since my last postings.

I got my sas2 expander backplane and the thing would just not work! All red light. Some red lights. Blinking red light! Sometimes red lights into a normal happy boot and everything works flawlessly! Except when I'd attach and remove and drive and suddenly TrueNAS would vomit so many errors messages about how it couldn't find that drive that I had to hard reset.

I spent probably a week trying to figure out what the issue was. Was it my HBA? My Cables? The Backplane? Was I overloading it with too many drives and not enough PSU? I entertained and tried everything I could.

I think I even stared directly into the PCB's soul and tried to communicate. It had a binary personality though, not my type.

To cut to the chase I got a replacement backplane and boom... No issues.

For the life of me I can't see what is wrong with this expander backplane though. I thought maybe the expander chip was overheating perhaps? I strapped one of the jet engine wall fans to it and plugged it into the backplane fan connection to go full blast right over the expander heatsink to try and keep it cool

By the way let me comment on the term "hot swappable"

What a smart idea. Being able to swap fans when they die. Quick and easy maintenance.

You know what's not a smart idea? Pulling one of those wall fans out all the way without a care in the world when it's going max RPM. The thing actually BIT me. I was assaulted. Attacked. Anyway it really hurt. As the blood came gushing from my finger tip I looked at the fan to see it perfectly fine- now sated with my blood sacrifice.

My finger is almost fully healed two weeks later. I thought for a moment I may have even needed stitches I'm not kiddin'! So yeah word of wisdom. Don't yank out 9k+ RPM fans when they're on full blast. Or at least do it SLOWLY.

I also somehow stumbled over double the amount of memory. All tested and passed mem test with no issue.

I now finally begin the lengthy drive burn-in process. Worried that my drives may run hot I've devised an ingenious solution. Behold.

The extra power supply for load weight purposes is required.

Stay cool my little drives, keep working hard!

 

[Ericloewe]

You know what's not a smart idea? Pulling one of those wall fans out all the way without a care in the world when it's going max RPM. The thing actually BIT me. I was assaulted. Attacked. Anyway it really hurt. As the blood came gushing from my finger tip I looked at the fan to see it perfectly fine- now sated with my blood sacrifice.

Happens to everyone at some point. Worst I've had was a Noctua NF-F12 iPPC at 2-3k RPM. Less linear velocity, but a very sharp leading edge.

To cut to the chase I got a replacement backplane and boom... No issues.

For the life of me I can't see what is wrong with this expander backplane though. I thought maybe the expander chip was overheating perhaps? I strapped one of the jet engine wall fans to it and plugged it into the backplane fan connection to go full blast right over the expander heatsink.

If the old one is still around, you can try re-pasting the heatsink on the expander, SAS2 expander backplanes are all probably 10+ years old by now. Also be sure to update the firmware on the backplane, if you can get it that far along - it's easy to miss, but high impact. Also, is this with a SAS2 or SAS3 HBA?