Cannot Install Truenas on SSD

[bartuc]

Hello,

I'm trying to do a fresh install. Server have 12 SAS drives(all are set as JBOD) and 1 SSD(Also JBOD) and 1 NVMe. I want to install it on SSD. I'm using a flash drive as installation media.

When it boots, it can see all the drives here:

But after that step, when installer launches, it only allows install to NVMe drives. No other drives are avaible to select as installation destination.

Can you please help? I have installed with no problems on the same exact server configuration around a month ago. I was able to see and select all drives on installer.

 

[jgreco]

Welcome to the forums.

When you say "set as JBOD", this implies you have a RAID controller. Please note that RAID controllers are incompatible with TrueNAS.

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

For better help, please consider following the Forum Rules, conveniently linked in red at the top of every page, which will help you form a better problem report, specifically including the details of the hardware you are trying to install on. Without these details, you have basically said "my vehicle won't start" without bothering to tell us if it's a car, truck, motorcycle, train, airplane, rocket, etc., and whether it's powered by gas, diesel, electricity, hydrogen, etc. As a result, the answers you might receive will tend to be low-quality answers that will be very general.

 

[bartuc]

Hello,

Thank you for your reply. The problem is that I was able to install the exact server with exact raid card and exact disks about a month ago. I don't have the same installation media right now. Only different thing is the installation media.

I have a SAS controller which I'm able to set drives as JBOD. I'm able to see SMART data, serial number and all other data on these drives through TrueNAS. This implies that it runs as true HBA and I have no problems with it after all. But this new installation have this problem. Thats why I wanted to consult on the forums.

 

[jgreco]

This implies that it runs as true HBA

The article I linked to dispels that implication with extreme prejudice.

SAS controller which I'm able to set drives as JBOD.

So, which is it, "true HBA", or RAID controller? There is no such thing as setting drives as JBOD on an HBA.

I have no problems with it after all. But this new installation have this problem.

You didn't have any problems with it in the short amount of time you tried it. I have more than ten years of experience providing support for RAID cards and HBA's on these forums, and my experience, along with that of many others, is that RAID cards might sometimes appear to work, but ultimately they end up causing a variety of problems.

The problem is that I was able to install the exact server with exact raid card and exact disks about a month ago.

Quite frankly, I don't care. I can install TrueNAS on all sorts of things where it ultimately will not work out well. Being able to install software is not a guarantee that it's compatible or will work well. I promise you that I can install TrueNAS on a CCISS-based AMD Opteron server from the mid 2000's and make a pool, yet because of the RAID controller and the failing disks, even though it seems to install fine, it will be corrupt or lock up in mere days once I start trying to use it.

This is likely to be a frustrating and pointless discussion. If you feel that

I have no problems with it after all.

then by all means, enjoy your server, have a nice day. Otherwise, please do circle around to the link I previously provided, read it, take it to heart, realize with despair that you will need to replace that RAID controller with an actual LSI HBA crossflashed to IT mode, and you'll end up having a much better experience with TrueNAS, both now and down the road.

 

[joeschmuck]

You have still not followed the advice given you. Read the forum rules and tell us what TrueNAS software you are using, was using, be specific and do not assume we know what you are talking about. Assumptions do not work here when trying to help someone troubleshoot a problem.

Also I have to agree, based on the limited information you provided, I don't see an HBA being set as JBOD either and I'm not saying you couldn't get it working that way but it won't have any fault tolerance which is not what ZFS is about.

 

[bartuc]

Wow, aggressive answers :) I know you are trying to help so no bad feelings about it.

Since I'm a newbie on this, whatever I can learn is good. I've read the linkand trying to be on right direction.

Anyways, here is what is going on.

I'm trying to install Truenas Core.

Server is SUN SERVER X4-2L. RAID controller is LSI M5110. Which is a 12G controller and have a JBOD mode for disks.

On a server installed about a month ago, I had no problems after all. I was able to install it with no problems, installer was able to list all disks. I'm using it. I can get a very good read/write speed out of it. I can see SMART data and disk serial numbers through Truenas Core web interface. 40TB data has been written, no I/O errors. Everything seems to be perfect. Card firmware is 23.7.1-0010 - no idea if it is IT/IR firmware. I didn't flash it. It is as I purchased it. Do you think that everything will work fine on this card?

On the same server today I'm trying to install, I have same card. Card firmware is 23.34.0-0017 - also no idea if it is IT/IR firmware. No disks are avaible to install on installation screen.

I've also installed Truenas Core on a Dell R520 server. Controller is PERC H310 Mini and I was able to set disks to non-raid and had no problems on install. But performans is really bad. No SMART data nor disk serial numbers on Truenas Core web interface. I can tell that there is something wrong with this installation and I'll have problems with it only judging by the performance and the link that you've posted made me understand why it is performing really bad and I'm pretty sure that I'll have problems with this in the future. Thats the reason I'm trying to install a new server with Truenas Core today actually.

Judging by all this information, is it possible for you to tell me which firmware I should be flashing to the card? I'm thinking about flashing "23.7.1-0010" since it is working fine(or looks like it). And is this card/firmware OK? If not, can you tell me what to flash to this card?

And I'm curious about Dell server. H310 seens to be OK so should a firmware flash resolve my current issues with it?

Thank you so much for sharing all your experiences in advance. I know it really hard to try to explain things to newbies. I'm trying to avoid spending hours and days trying different things.

 

[jgreco]

aggressive answers :) I know you are trying to help so no bad feelings about it.

When you insist on proceeding down the road after someone has already tried to give you the information you need, well, no good comes of it.

have a JBOD mode for disks.

This is unacceptable for TrueNAS.

Card firmware is 23.7.1-0010 - no idea if it is IT/IR firmware.

It is not. It is likely to be MFI/MRSAS firmware.

Do you think that everything will work fine on this card?

No, I do not. And clearly you've had the experience of incorrect firmware directly (H310):

performans is really bad. No SMART data nor disk serial numbers on Truenas Core web interface. I can tell that there is something wrong with this installation

And that article I linked to explains this in more detail. It's still a general explanation, not specifically targeting your exact issues.

is it possible for you to tell me which firmware I should be flashing to the card?

To the M5110? Probably the sledgehammer update, that's where you remove the card and beat it with an eight pound sledgehammer, and then get an HBA.

We've often discussed the fact that the LSI RAID controllers all use the same CPU architecture, and while theoretically you might be able to bodge LSI HBA IT firmware onto a high-end RAID controller, it leaves open-ended questions such as what happens to the battery charging circuit. If you're electronically competent, you can probably detach the battery. If you're competent with embedded processors, you might be able to get HBA firmware installed. But these are big maybes.

And the thing is, this is frequently discussed here on the forums. The M5110, two weeks ago:

HDD's connected to HBA don't show up

I'm trying to build a NAS and the recommendation was to use an HBA if I want to use more hard drives. I bought a 9211-8i, connected 8 SATA hard drives to it but I don't see any signs of it working. According to the seller "It has been flashed to the latest IT Mode firmware". Am I supposed to do...

www.truenas.com

As for the PERC H310 Mini, yes, it can be crossflashed to IT mode and then it should work fine. Probably. Dell is a bit weird and some of their stuff likes to "work together" in odd ways.

So this basically ends up with me having to explain this stuff over and over and over and over and over again, and it's really not that pleasant, and I don't get paid to do this. Please excuse me if I sound exasperated. It comes across poorly when it seems like I provide links to things and yet need to write bespoke explanations that cover the same material. It's best to make use of available resources and then ask questions, if you can. We'll get you sorted out either way though.

 

[bartuc]

I've followed the procedure here: https://lazymocha.com/blog/2020/06/05/cross-flash-ibm-servraid-m5110-and-h1110-to-it-hba-mode/ - and everything turned out great. With crossflash, I was able to make it work as an HBA and I was able to install Truenas Core, create pool. I believe this setup won't have any problems in the future at all, according to this documentation: https://www.truenas.com/community/r...bas-and-why-cant-i-use-a-raid-controller.139/ - "If you can figure out the black magic to crossflash one of these to IT-mode and it actually works (be sure to test extensively!), then from the perspective of this article, congratulations, you have an HBA, and it's actually kinda likely to work correctly with FreeNAS." Only thing left is to testing it out by reading/writing data and I hope everything will be perfect.

This document opened my eyes about Truenas, I had no idea about the need to have an HBA. I figured that making all disks Raid0 won't be a good idea since raid controller will take place but I was thinking that JBOD would be fine and my first install with JBOD worked well as I mentioned on my previous post(which I'll also crossflash the raid card to turn it into a HBA after testing the current one).

Thank you so much for all the help, hopefully this thread also helps other newbies like me.

 

[jgreco]

Hey, a link. That's good, I can always make use of actual links to actual information. You even quoted a good bit of text at which to place such link.