HELP with my 1st TrueNAS server

elove7300

Dabbler
Joined
Aug 16, 2021
Messages
14
Hello, so I'm a newbie who has been researching this site and I have read up on the hardware recommendations.
Usage: Plex with upwards of 20 possible connections and File Storage for Plex content

I have a computer in mind but I'm not sure if the specs are over kill or not enough

System Model: Supermicro 6028U (SAS3)
Chassis: CSE-829UAC-R1K02-T
Motherboard: Supermicro X10DRU-i+
Backplane: BPN-SAS3-826EL1 Expander backplane 12GB/s
CPU: 2x E5-2650 V3 2.3Ghz 10 Core
RAM: 64GB (4x 16GB DDR4 ECC REG)
Hard Drives: None Installed
Storage Controller: PCI-E Card AOC-S3008L-8Le 12GB/s IT Mode HBA controller for Freenas/ UNRAID
Onboard NIC: 4x 10GBase-T ports via AOC-2UR6-i4XT
BMC: IPMI 2.0 Dedicated onboard Port
PCI-E Slots: 8x PCI-E Slots

another question which I'm not sure is possible would be could I us my Nvidia Quadro p2000 in a setup for TrueNAS?
I'm just looking for any guidance as this would be my first server and I'm leaving the Synology world Thank you!
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,943
"could I use my Nvidia Quadro p2000 in a setup for TrueNAS? " - No. As a video card - yes - but not as a hardware offload for video transcoding. TrueNAS doesn't do passthrough well. This is why I switched to a separate machine for Plex with a Quadro but leave the storage on TrueNAS. That's my understanding - someone more knowledgeable might correct me.

[Tongue in Cheek Comment]
I think you are going to have space limitations on that setup. and may have issues booting

[Real Comment]. Transcoding - will be CPU bound. If you can limit transcoding you will be fine - but with 20 connections a few people will transcode and you could quickly max out your available CPU's. My view is use a separate server for Plex and cut down on the CPU's in the NAS. Alternatively wait for Scale (or use the Beta) which apparently will let you offload to the video card
 
Last edited:

elove7300

Dabbler
Joined
Aug 16, 2021
Messages
14
Thank you for the response. I currently have a windows 10 machine with my Nvidia Quadro doing the transcoding and have had no trouble transcoding. I had planned on using the server for Storage and in case something did go wrong with my current system I could run Plex on it.

"I think you are going to have space limitations on that setup. and may have issues booting"

The supermicro case holds 12 bays and I believe supports up tp 16TB each. Can you explain why there may be space limitations as well as why I may have boot issue ?

Also do you have a setup that you would recommend over this or a tweak to this current setup?

Thank you!
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,943
"The supermicro case holds 12 bays and I believe supports up tp 16TB each. Can you explain why there may be space limitations as well as why I may have boot issue ?"
Because you didn't list any discs

Leave the Win10 running Plex and just use the NAS as a NAS. I suspect you will be disappointed if you try to use the NAS as a Plex Server either in a Jail or as a VM - due to the 20 clients

Incidentally I would not, from personal experience, run Plex from a Jail - it was flakey and tended to crap itself when ever I did a TrueNAS upgrade. A VM would be a better case. Plex doesn't use much memory but will use all the CPU it can get its hands on. At least in a VM you can reserve some CPU time for the NAS. However definitely stick to the separate Plex Server with the Quadro

No other real comments on the build - its good (barring the complete lack of disk space...) You might argue its got too much CPU for a NAS - but if you need it to run as a backup PLEX then you need a lot of CPU. Just using the above build as a NAS will have the CPU's idling along doing very little I suspect - but it won't do any harm.

One thing to consider is 20 clients each pulling a large 4K file from the NAS to the Plex server. The 10Gb NIC is great for that - just consider the required speed of the disk system. If the files are big then your ARC (read cache) may only hold 2 of them. Obviously they won't all hit the server at the same time but at the edge case you may run into issues that might require some tuning and or the implementation of an L2ARC / more ARC.

What RAID are you planning (on your non-existent disks)? :tongue:
 
Last edited:

elove7300

Dabbler
Joined
Aug 16, 2021
Messages
14
Thank you for the help the 20 connections was just a test I did with my Quadro so I figured I would see if this server could match it. Since I'm new to TrueNas I'm not sure what the best raid would be. I planned on maybe doing 3 vdev's with 4 drives and running a z1 raid. ...maybe. So lets say the worst happens and I use the above options for the server (but with drives) what level of streams do you thin I could get?
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,943
If base capacity = 10 (and ignoring that 10TB is nearer 9.5)
So each vdev would be 4 drives in Z1 = 3 * 10 = 30 TB + 1 Parity
3 vdevs of 30TB = 90TB useable

12 Disks Total in the pool
Is that what you meant?

You would have the effective IO of 3 disks - and could lose 1 disk in each vdev and the pool would stay up (if degraded). I would suggest that would be more than fine as the reads would be sequential mostly. Lose two disks in one vdev and the whole pool is toast.

Thats a lot of movies BTW

12*10 (-3) = 90TB. Slower - but you could lose any 3 disks and keep running

Actually for your work load I would probably run RAIDZ2 or Z3 for a more resilient pool - you could stand more failures. IO would be slower but I think you could counter that (if needed) with the aforementioned L2ARC on a large SSD (basically a bloody great read cache although it does need some memory (ARC) to operate in.

YMMV

I would point out that that is a 12 disk chassis and the above is 12 disks. Does it have extra space for boot SSD's, maybe in the back. Do not use USB.
 

elove7300

Dabbler
Joined
Aug 16, 2021
Messages
14
"12 Disks Total in the pool Is that what you meant?"
Yes that is exactly what I was trying to convey

I'm not 100% versed on the raids it sounds like there a raid where if all 12 bays were full I could lose any 3 drives is that correct?

There are no other drives in the back but this chassis is not yet finalized as your giving me great ideas here. I did plan on possibly getting a card that would allow me to either connect an Nvme or dual Samsung SSD. I read up a bit on L2ARC and it sounds like you feel I should have it so I would plan on adding that to any system I build.

Would your RaidZ2 and a large L2ARC recommendation change if I didn't put all 12 drives in at once? Odds are I'll buy 4 at a time.

"That's a lot of movies BTW "
2500 and counting
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,943
And what is the average size of the movie - full BR (30+GB)? Or compressed down to ............

OK - here you are running into issues with TrueNAS.
TrueNAS works in vdevs - you can't add individual discs - you add a vdev to a pool. A vdev can be a single disk - but if you loose the disk, you loose the vdev and and if you loose the vdev you lose the pool.
[Caveat: I did read something about expanding RaidZ by disks - a new feature - but I don't remember where - it may or may not be in TrueNAS yet - or I am getting confused]. See below - but TN does not support it yet
RAIDz Expansion

As well as your datapool disks you need at least one extra port to boot from. Do not be tempted by a cheap SATA card. Either use LSI in IT mode OR use a sata port on the mobo OR use a moderately expensive SATADOM with one of the yellow ports on the motherboard (its expensive for 16GB). You only need 16GB for TN CORE (you can also use two mirrored). This is what I do. You can then use the full 12 bays for Data

RAIDZ3 = 9 data disk and 3 parity
RAIDZ2 = 10 data disks and 2 parity
RAIDZ1 = 11 data disk and 1 parity - Just No

Buying 4 disks = RAIDZ3 (1+3=25% useful), RAIDZ2 (2+2=50%useful), RAIDZ1 (3+1=75%useful) which would be a single vdev or mirrored ((1+1)+(1+1)) =50%useful and 2vdevs.
The RAIDZn have the IOPS of a single disk (simplistically)
The Mirrored would have the IOPS of 2 discs

To expand any of the arrays you need to add a new vdev so the next 4 discs would logically be the same RAID type and in the same configuration. You expand by adding vdevs - but any resiliency is limited to the vdev its in - and not the whole RAID Pool. So adding any of the RAIDZ vdevs would increase your IOPS by one disk, and increase space by the amount of useable space in the vdev. If you pick mirrored the your start with 2 disks worth of IOPS but only at 50% useful space and losing 2 disks in the same vdev would be a disaster

Sorry - I am very bad at explaining it.

As for L2ARC - do NOT start with it. But if (and only if) you are having performance issues with data streaming from the TrueNAS then consider that a cheap upgrade may be an L2ARC which is a fast (not RAM speed) read cache that will enable TN to pull movies off the HDD's and onto the L2ARC and deliver them from a fast SSD rather than a slow HDD. But only if you need it.

As with all things it depends on your use case, size of files, how many users are on at the same time etc

RAIDZ2or3 is I think a better option for you - but unless everytime you add disks you can blow the array away and then restore from backups its not very flexible. RAIDZ(x) is good at consecutive reads and writes - which is the sort of use pattern you will be generating here. Its just inflexible when it comes to upgrades and expansion.

If you need L2ARC then a reasonable consumer grade SSD of a TB or so would be good. Use a motherboard SATA port and velcro it to the side, inside the case - just make sure when you build the machine that you have a few spare power cables hanging around after the 12 bays are powered. If the drive dies, it doesn't matter and can just be replaced. Its not an important vdev, its just a read cache
 

elove7300

Dabbler
Joined
Aug 16, 2021
Messages
14
Thank you NugentS I'm thinking whatever I build will replace my current windows 10 plex server. I'm researching using TrueNas scalable not sure how that will work but I believe it will let me use hardware acceleration. Plus I just found 2 Quadro P400 cards that will probably just sit on a shelf.
 

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,943
Scale is probably a better option although you will still have the same ZFS Pool issues and how to build it. However in theory it is possible to use the Quaddro cards for passthrough in Scale.

Remember it is Beta though (at the moment) - so things may not work quite as well as planned at the moment. Of course if you are using Quaddro's then you will still be over CPU'd - but honestly who cares
 
Top