10Gbe - 8Gbps with iperf, 1.3Mbps with NFS

[HoneyBadger]

Understood that the "permanent home" of the VMs is on each host, and this is just a central store. If everything stored here will be "cold" or "async" then you can probably disable sync, but again - this opens you up to some risk of data loss. VMFS specifically doesn't handle it well if it loses power mid-write.

For booting, you can stick with the physical drives you have now. A pair of spinning disks is fine for boot, and a 1U with 10 bays is likely pretty tight in terms of potential expansion options for PCIe.

Yes, you won't be needing the battery for the PERC any longer. Might be useful to have on hand if you have other servers though.

Item 3, yes, you will simply select both of the two drives during the TN installation process.

Item 4 is done after the install when you reboot into the running TN OS, and log into the webUI the first time. It will likely see the unused disks and prompt you to create a pool on them.

 

[Kailee71]

Excellent! Your on your way to putting a trustable solution together. Yes flash your perc, a mirror of ssds for your os is fine, more than fine i'd say. If you're using SSD and you're worried about the half life of the disks then overprovision, this is easily done, and will extend the life expectancy by however much your overprovisioning.

Don't forget to back up your setup once you've configured it , from that point on a complete fail of your os is no more than a 15 minute install-and-restore-config away. Then seriously consider striped mirrors instead of raidz if you can afford the small cost in capacity.

All the best - and enjoy the experience ;-)

Kai.

 

[NugentS]

You can use mismatched drives to boot from. Just make sure you either:
1. Build in the first instance as a mirror
2. Install on the smallest - you can add the mirror later

The popcorn was possibly a bit unkind

32GB SLOG on Optanes (single or mirrored) would be ideal as SLOG doesn't need to be big. It just needs to be very fast (and retain data when it loses power - which it does)
Sizing a SLOG - its about 5 seconds of maximum network transfer + a bit for comfort. Under steady state its only ever written to and never read (which is why you need the long life)

 

[LearnLearnLearn]

I read the above but right now, I'm stuck with the flashing of the card. I was able to complete all steps but this one.

root@debian:~# setsas 5c81f660d6350500
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

No LSI SAS adapters found! Limited Command Set Available!
ERROR: Command Not allowed without an adapter!
ERROR: Couldn't Create Command -sasadd
Exiting Program.

There were no other obvious errors other than the ones mentioned in the doc
(https://fohdeesha.com/docs/H710-D1.html)

I have to do some other work but will get back to this later today. I'm excited to be learning more about truenas. My past is using crazy proprietary hardware like OnStor and BlueArc and being stuck in their sometimes non flexible methods. This is a breath of fresh air.

 

[LearnLearnLearn]

Well, there are plenty of posts on the problem but the solutions are all over the place so I cannot just jump at one and try it without knowing if it will work or cause further issues. Until I solve this, I cannot continue. Back to searching unless someone here know what I need to do next.

Steps taken;

Used the DOS disk first. Seemed to work ok. Rebooted using the Linux disk and did the rest over ssh.

Code:

root@debian:~#
Using username "user".
user@192.168.1.150's password:
Send automatic password
Linux debian 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
user@debian:~$ sudo su -
root@debian:~# D1-H710
rmmod: ERROR: Module megaraid_sas is not currently loaded
rmmod: ERROR: Module mptctl is not currently loaded
rmmod: ERROR: Module mptbase is not currently loaded
Errors above are normal!
Trying unlock in MPT mode...
Device in MPT mode
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Setting up HCB...
HCDW virtual: 0x7f33c2000000
HCDW physical: 0x129a00000
Loading firmware...
Loaded 809436 bytes
Booting IOC...
IOC is RESET
IOC failed to become ready
Device in MPT mode
Removing PCI device...
Rescanning PCI bus...
PCI bus rescan complete.
Pausing for 30 seconds to allow the card to boot

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

0 MPT Ports found

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

0 MPT Ports found
All Done! Continue following the guide to set SAS addr
root@debian:~# reboot

Using username "user".
user@192.168.1.150's password:
Send automatic password
Linux debian 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

user@debian:~$ sudo su -

root@debian:~# info
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        No LSI SAS adapters found! Limited Command Set Available!
        ERROR: Command Not allowed without an adapter!
        ERROR: Couldn't Create Command -c
        Exiting Program.

root@debian:~# setsas 5c81f660d6350500
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        No LSI SAS adapters found! Limited Command Set Available!
        ERROR: Command Not allowed without an adapter!
        ERROR: Couldn't Create Command -sasadd
        Exiting Program.

 

[LearnLearnLearn]

Success!
I found this post after hours of searching;

Guide: Flashing H310/H710/H810 Mini & full size to IT Mode

I had no error until the last step when I enter the setsas command. I think that is different then what you experienced.

forums.servethehome.com

The suggestion is to disable virtualization and a couple of other things. On my system, only virtualization was enabled so I disabled that and was able to update the card.

Because of the amount of time I had to spend finding the info, I figure I should share it here.
Ok, so, tomorrow, I'll re-install using the suggested methods. Once I better understand this, I'll also change my other systems since they would be wrong also.

Code:

root@debian:~#
Using username "user".
user@192.168.1.150's password:
Send automatic password
Linux debian 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
user@debian:~$ info
/usr/local/bin/info: 2: /root/sas2flash: Permission denied
user@debian:~$ sudo su -
root@debian:~# info
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        No LSI SAS adapters found! Limited Command Set Available!
        ERROR: Command Not allowed without an adapter!
        ERROR: Couldn't Create Command -c
        Exiting Program.
root@debian:~# D1-H710
rmmod: ERROR: Module megaraid_sas is not currently loaded
rmmod: ERROR: Module mptctl is not currently loaded
rmmod: ERROR: Module mptbase is not currently loaded
Errors above are normal!
Trying unlock in MPT mode...
Device in MPT mode
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Setting up HCB...
HCDW virtual: 0x7f379b400000
HCDW physical: 0x441e00000
Loading firmware...
Loaded 809436 bytes
Booting IOC...
IOC is READY
IOC Host Boot successful.
Device in MPT mode
Removing PCI device...
Rescanning PCI bus...
PCI bus rescan complete.
Pausing for 30 seconds to allow the card to boot

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

1 MPT Port found

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
 1.  ioc0              LSI Logic SAS2308 D1      200      14000700     0

 1.  NVSRAM
 2.  SEEPROM
 3.  FLASH
 4.  BootLoader
 5.  Firmware (backup copy)
 6.  Firmware (current copy)
 7.  Persistent non-manufacturing config pages
 8.  Persistent manufacturing config pages
 9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit] 3

Erasing...

 1.  NVSRAM
 2.  SEEPROM
 3.  FLASH
 4.  BootLoader
 5.  Firmware (backup copy)
 6.  Firmware (current copy)
 7.  Persistent non-manufacturing config pages
 8.  Persistent manufacturing config pages
 9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit] 8

Erasing...

 1.  NVSRAM
 2.  SEEPROM
 3.  FLASH
 4.  BootLoader
 5.  Firmware (backup copy)
 6.  Firmware (current copy)
 7.  Persistent non-manufacturing config pages
 8.  Persistent manufacturing config pages
 9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit]

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

1 MPT Port found

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
 1.  ioc0              LSI Logic SAS2308 D1      200      14000700     0

Firmware image's version is MPTFW-20.00.07.00-IT
  LSI Logic
  Not Packaged Yet
  MPTFW-20.00.07.00-BL
  LSI Logic
  Not Packaged Yet

Downloading image...
Download succeeded

Verifying download...
Verification succeeded
All Done! Continue following the guide to set SAS addr

# reboot

Using username "user".
user@192.168.1.150's password:
Send automatic password
Linux debian 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
user@debian:~$ sudo su -
root@debian:~# setsas 5c81f660d6350500
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        Advanced Mode Set

        Adapter Selected is a LSI SAS: SAS2308_2(D1)

        Executing Operation: Program SAS Address

        SAS Address Successfully Programmed!

        Resetting Adapter...
        Adapter Successfully Reset

        Finished Processing Commands Successfully.
        Exiting SAS2Flash.
root@debian:~# info
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        Adapter Selected is a LSI SAS: SAS2308_2(D1)

        Controller Number              : 0
        Controller                     : SAS2308_2(D1)
        PCI Address                    : 00:03:00:00
        SAS Address                    : 5c81f66-0-d635-0500
        NVDATA Version (Default)       : 14.01.00.06
        NVDATA Version (Persistent)    : 14.01.00.06
        Firmware Product ID            : 0x2214 (IT)
        Firmware Version               : 20.00.07.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : SAS9207-8i
        BIOS Version                   : N/A
        UEFI BSD Version               : N/A
        FCODE Version                  : N/A
        Board Name                     : SAS9207-8i
        Board Assembly                 : N/A
        Board Tracer Number            : N/A

        Finished Processing Commands Successfully.
        Exiting SAS2Flash.

 

[fohdeesha]

Success!
I found this post after hours of searching;

Guide: Flashing H310/H710/H810 Mini & full size to IT Mode

I had no error until the last step when I enter the setsas command. I think that is different then what you experienced.

forums.servethehome.com

The suggestion is to disable virtualization and a couple of other things. On my system, only virtualization was enabled so I disabled that and was able to update the card.

Because of the amount of time I had to spend finding the info, I figure I should share it here.
Ok, so, tomorrow, I'll re-install using the suggested methods. Once I better understand this, I'll also change my other systems since they would be wrong also.

Code:

root@debian:~#
Using username "user".
user@192.168.1.150's password:
Send automatic password
Linux debian 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
user@debian:~$ info
/usr/local/bin/info: 2: /root/sas2flash: Permission denied
user@debian:~$ sudo su -
root@debian:~# info
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        No LSI SAS adapters found! Limited Command Set Available!
        ERROR: Command Not allowed without an adapter!
        ERROR: Couldn't Create Command -c
        Exiting Program.
root@debian:~# D1-H710
rmmod: ERROR: Module megaraid_sas is not currently loaded
rmmod: ERROR: Module mptctl is not currently loaded
rmmod: ERROR: Module mptbase is not currently loaded
Errors above are normal!
Trying unlock in MPT mode...
Device in MPT mode
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Device in MPT mode
Resetting adapter in HCB mode...
Trying unlock in MPT mode...
Device in MPT mode
IOC is RESET
Setting up HCB...
HCDW virtual: 0x7f379b400000
HCDW physical: 0x441e00000
Loading firmware...
Loaded 809436 bytes
Booting IOC...
IOC is READY
IOC Host Boot successful.
Device in MPT mode
Removing PCI device...
Rescanning PCI bus...
PCI bus rescan complete.
Pausing for 30 seconds to allow the card to boot

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

1 MPT Port found

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
 1.  ioc0              LSI Logic SAS2308 D1      200      14000700     0

 1.  NVSRAM
 2.  SEEPROM
 3.  FLASH
 4.  BootLoader
 5.  Firmware (backup copy)
 6.  Firmware (current copy)
 7.  Persistent non-manufacturing config pages
 8.  Persistent manufacturing config pages
 9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit] 3

Erasing...

 1.  NVSRAM
 2.  SEEPROM
 3.  FLASH
 4.  BootLoader
 5.  Firmware (backup copy)
 6.  Firmware (current copy)
 7.  Persistent non-manufacturing config pages
 8.  Persistent manufacturing config pages
 9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit] 8

Erasing...

 1.  NVSRAM
 2.  SEEPROM
 3.  FLASH
 4.  BootLoader
 5.  Firmware (backup copy)
 6.  Firmware (current copy)
 7.  Persistent non-manufacturing config pages
 8.  Persistent manufacturing config pages
 9.  Boot services (BIOS/FCode)

Select what to erase:  [1-9 or RETURN to quit]

LSI Logic MPT Configuration Utility, Version 1.72, Sep 09, 2014

1 MPT Port found

     Port Name         Chip Vendor/Type/Rev    MPT Rev  Firmware Rev  IOC
 1.  ioc0              LSI Logic SAS2308 D1      200      14000700     0

Firmware image's version is MPTFW-20.00.07.00-IT
  LSI Logic
  Not Packaged Yet
  MPTFW-20.00.07.00-BL
  LSI Logic
  Not Packaged Yet

Downloading image...
Download succeeded

Verifying download...
Verification succeeded
All Done! Continue following the guide to set SAS addr

# reboot

Using username "user".
user@192.168.1.150's password:
Send automatic password
Linux debian 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
user@debian:~$ sudo su -
root@debian:~# setsas 5c81f660d6350500
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        Advanced Mode Set

        Adapter Selected is a LSI SAS: SAS2308_2(D1)

        Executing Operation: Program SAS Address

        SAS Address Successfully Programmed!

        Resetting Adapter...
        Adapter Successfully Reset

        Finished Processing Commands Successfully.
        Exiting SAS2Flash.
root@debian:~# info
LSI Corporation SAS2 Flash Utility
Version 20.00.00.00 (2014.09.18)
Copyright (c) 2008-2014 LSI Corporation. All rights reserved

        Adapter Selected is a LSI SAS: SAS2308_2(D1)

        Controller Number              : 0
        Controller                     : SAS2308_2(D1)
        PCI Address                    : 00:03:00:00
        SAS Address                    : 5c81f66-0-d635-0500
        NVDATA Version (Default)       : 14.01.00.06
        NVDATA Version (Persistent)    : 14.01.00.06
        Firmware Product ID            : 0x2214 (IT)
        Firmware Version               : 20.00.07.00
        NVDATA Vendor                  : LSI
        NVDATA Product ID              : SAS9207-8i
        BIOS Version                   : N/A
        UEFI BSD Version               : N/A
        FCODE Version                  : N/A
        Board Name                     : SAS9207-8i
        Board Assembly                 : N/A
        Board Tracer Number            : N/A

        Finished Processing Commands Successfully.
        Exiting SAS2Flash.

Disabling virtualization is literally the first step in my guide - in fact, I even put it in bold, and state twice that it must be done

 

[LearnLearnLearn]

Yes, I now see that you had mentioned that in the initial page but once I knew I needed to get to the second page, I guess I moved on to that one and missed this important step.

This is the page I followed.

H710 D1 Mini - Fohdeesha Docs

fohdeesha.com

I think you should mention it in each of the sub pages like the one I share above to make sure it's not missed.
Anyhow, I appreciate that you make this available. Thanks.

 

[Samuel Tai]

It's in the Introduction section, @NasProjects:

Preparation​

Ensure there is only one LSI-based adapter in your system. If there are others besides the adapter you intend to flash, remove them! You also need to disable a few BIOS settings. This step is not optional. In your server BIOS, disable all of the following:

• Processor Settings > Virtualization Technology
• Integrated Devices > SR-IOV Global Enable
• Integrated Devices > I/OAT DMA Engine

Note: If you're flashing a full size card on a non-Dell system, such as an AMD based desktop or server, make sure you find any BIOS settings related to IOMMU and Virtualization, and disable them

When you're finished with this guide, don't forget to go back and enable Virtualization, as well as SR-IOV if you plan to use it.

 

[fohdeesha]

Sorry but I don't see any mention of it in the doc I was following which is this.

H710 D1 Mini - Fohdeesha Docs

fohdeesha.com

that is a subsection of the guide that should only be followed after using the "preparation" intro - https://fohdeesha.com/docs/perc.html

this prep page is what is linked to by all the articles here, and is what has the ISOs you used. How did you follow the D1 section if you hadn't even grabbed the ISO?

 

[fohdeesha]

I think you should mention it in each of the sub pages like the one I share above to make sure it's not missed.
Anyhow, I appreciate that you make this available. Thanks.

You mean like this text at the top of every subsection including the one you linked?

H710 D1 IT Mode Flashing​

Continued instructions for flashing the H710 Mini Mono (D1 Revision). You should only continue here after following the Introduction Page. If you haven't done so already, go back.

 

[LearnLearnLearn]

This is side tracking my original post. I'm not asking to get into a debate about your documentation, I'm simply pointing out that I had the same problem. Sometimes, not every word is read in a document so I followed most of the steps after downloading the zip file and ISO's.

There is no point in chastising me about it, just take it as input or feedback that the info should be in every page. While trying to figure out what was not working, I came across many posts from others having the exact same problem.

Can we move on now so my question doesn't get wasted.

 

[LearnLearnLearn]

Again, there is no mention about it in this sub-page. For me, I thought I had read all I needed to move on to the next page.

H710 D1 Mini - Fohdeesha Docs

fohdeesha.com

Again, my fault but it's still a point to be made that if I missed it and made this goof, others could too and that could cause a lot of additional support for no good reason. Just take it as input from someone that made the same mistake. I'd like to move on with the rest of the suggestions now that I have this working.

 

[fohdeesha]

Again, there is no mention about it in this sub-page.

H710 D1 Mini - Fohdeesha Docs

fohdeesha.com

Do you see the very top text, that says explicitly do *not* follow that guide until you've gone through the introduction page, which includes the BIOS settings? I suppose I should just copy/paste the entire introduction page there for people who can't read?

There's been a few thousand people who have followed the guide, and I've certainly gotten some emails about things that could be streamlined (and they were), but I have to say in two years not a single person that I know of just skipped the entire intro page after grabbing ISOs lmao. You got pretty lucky actually, there's extra details towards the end of the Intro page regarding card variants - there's more than one kind of D1 for starters, you're lucky you skipped to the right one or you'd have a bricked card

 

[HoneyBadger]

<ignores the previous posts in an attempt to return to the original subject>

So we've definitely gotten past steps 1 and 2 then. Install TrueNAS onto both of the mirrored boot devices and let's see when it's done.

Once I better understand this, I'll also change my other systems since they would be wrong also.

Do you mean they're also using ZFS on hardware RAID? I'd be making fixing that a fairly high-priority item once this is wrapped up.

 

[LearnLearnLearn]

Yup, I saw the top and thought I had read all I needed to move on to the next step. Thanks for your guide and your code. Thousands fine, but plenty of threads from folks that did get caught in the same problem. Yup, some bricked their cards too. I was lucky. Now I want to move on to the next step of re-installing the OS.

 

[LearnLearnLearn]

<ignores the previous posts in an attempt to return to the original subject>

So we've definitely gotten past steps 1 and 2 then. Install TrueNAS onto both of the mirrored boot devices and let's see when it's done.



Do you mean they're also using ZFS on hardware RAID? I'd be making fixing that a fairly high-priority item once this is wrapped up.

One system is a TN with some IBM 3524 storage chassis connected to it. Those chassis have RAIDx configured storage which is shared by TN.
On TN, the different LUNs are seen as pools then shared. It's worked out well, never a problem for many years but obviously, not the way of doing things so will have to look at that.

 

[LearnLearnLearn]

Alright then, back to this, on and off. One of our huge pine trees fell on a neighbors house and caused a lot of damage but no injuries thankfully so I'm in and out dealing with that and on a schedule to get this system up and sent off.

The steps suggested were this;
>1. Destroy the existing pools and virtual disks on the PERC H710.
>2. Use Fohdeesha's guide and ISO to flash your H710 to an LSI IT mode HBA

Done and the card has been flashed.

>3. Reinstall TrueNAS to a mirror of the small boot devices
>4. Configure a RAIDZ2 (if you won't be running VMs from there and/or don't care about performance) or a set of mirror vdevs
>(better for VMFS performance)

Someone mentioned installing the OS but letting the system find the drives after that. Item 4 suggests configuring the storage drives as RAIDZ2, presumably during the re-install.
>5. If you won't be running live machines or can't get there to install a proper SLOG device, disable sync on the dataset.

In terms of a SLOG, I have these on hand though not 100% sure what options the server offers so will have to look. Would any of these do the trick? 1 Samsung SSD SM841 128G, 1, Lite-On LMT-256M6M and an M.2 2242 Kingdata 64g solid state drive.

I also have one SD vFlash card slot and various SSD (all enterprise) but no free bay but might be able to install it on top of the Ethernet controller in the chassis.

 

[HoneyBadger]

Someone mentioned installing the OS but letting the system find the drives after that.

That would have been me. :) Definitely do the install, check off the two "boot devices" and let TN install there.

Item 4 suggests configuring the storage drives as RAIDZ2, presumably during the re-install.

Do this after the installation completes. Don't check off any of the "capacity drives" during the installation itself or they'll be unusable for storage afterwards.

In terms of a SLOG, I have these on hand though not 100% sure what options the server offers so will have to look. Would any of these do the trick? 1 Samsung SSD SM841 128G, 1, Lite-On LMT-256M6M and an M.2 2242 Kingdata 64g solid state drive.

None of those will be a viable SLOG device, sorry. You'll have to YOLO it and run sync=disabled if you want anything resembling decent storage performance, even for non-live VMs.

I also have one SD vFlash card slot and various SSD (all enterprise) but no free bay but might be able to install it on top of the Ethernet controller in the chassis.

The SD vFlash (Dell IDSDM) is probably not fit for purpose for anything here unfortunately, they aren't even recommended for vSphere installs any longer and TrueNAS hits the boot device harder than that. You might be able to rig up an internal USB-to-SATA bridge though off an actual SSD.

For the "Enterprise SSDs" they might be viable SLOG devices if they're focused on write-intensive workloads (eg: rebranded Intel DC S3710 units, or WD SS530 SAS drives) but general read-focused and even some "mixed use" enterprise SSDs aren't capable of the speeds/endurance for SLOG duties. It's a heavy 100% write workload at low queue depths, and needs to be fed by the lowest-latency device possible.

One system is a TN with some IBM 3524 storage chassis connected to it. Those chassis have RAIDx configured storage which is shared by TN.

You'll need to replace the external RAID card with an external SAS HBA like the 9211-8e or similar. Key is you want an HBA, not a RAID card of any stripe. Many OEM cards can be reflashed for a lower-cost option.

 

[LearnLearnLearn]

Thanks for the guidance. Once I've gone through all of this, I will definitely better understand how I've been using this wrongly.

Turns out, I don't have any enterprise SSD's on hand other than some old stuff.
The two SSD I have on hand that I was thinking about are Lenovo mz-7pc1280/0l5 (3Gbps) and an Intel ssdsa2cw1160g3 that is 6Gbps.
I found an IBM SSD 15.8GB (1.5 Gbps) burst rate, (5000 IOPS) and will take the top off the server to see if a 10K or 15K would fit in there because that should work nicely even if not an SSD.