New Build (Not New to FreeNAS) - A few clarification questions

[walk1355]

Greetings everyone. Firstly, for those that take the time to read through this trying to help answer my questions, thank you in advance! I know it's a lot of reading, but I'd rather be more detailed than not.

I repurposed an old "gaming" rig last year and turned it into a freenas box. I more or less wanted to play around with something, and freenas is where I landed. I LOVE how stable it is, despite my personal learning curve. And yes, I realize consumer grade hardware is NOT ideal or secure for FreeNas, but it was free and I was just playing around.

The reason I needed a 2nd machine at all was to remove the tasks of torrenting and plex server off of my primary machine, so I didn't have to leave my high end gaming rig on 24/7.

So here I am today. I have purchased some older server grade hardware (what I would consider server grade anyway). I am looking for validation that the hardware will work well to fit my needs as well as get some answers to some questions, so I will break this post down into a couple of categories, hardware I have purchased for the build and what I am using, what I need the machine to do, and what my questions are.


1. Hardware I have purchased or am repurposing:
a. Intel Xeon E5-2670v2 lga 2011
b. SuperMicro X9SRi-F
c. 8x8gb Hynix HMT31GR7CFR4A-H9 - ECC RDIMMS
d. 2x60gb Patriot Flare 2.5" SATA3 SSD (for OS mirror)
e. Hyper 212 EVO cpu cooler - repurpose
f. 6x4tb WD RED WD40EFRX HDD - 4 new, 2 <3 years old. Planning on raidz2.
g. An older Rosewell Legacy QT01 ATX case (I have ordered adapters to make sure I can fit the 6 hdds and the 2 ssds inside the case)


2. What I need this build to do:
a. PLEX - serve up to 3 TV's with up to 4k resolution videos/movies/ shows.
b. Torrent - my current FreeNAS machine runs deluge and I tend to share for lengthy periods, so I somtimes have 1000+ torrents sharing at once (thank you ATT fiber for my 1/1gbps connection)
c. Sickrage - I use it to auto download some of the stuff I want
d. File storage - I use the machine to house some file backups as well as an disk image of my main gaming rig (so I can do a quick restore if I ever have to reinstall the OS or someting like that).


3. Questions I have and clarifications I need:
a. In general, does the hardware look ok? The RAM showed to be directly compatible with that specific motherboard on SuperMicro's website.
b. SuperMicro's website says the board has 2 SATA 3.0 ports, 4 SATA 2.0 ports, and 4 SATA ports via SCU. My confusion is around the SCU ports. I have used the search function here and trusty google, but I am still confused (or at least I'm not 100% positive in my knowledge). Can I plug this into one of the "SCU" ports on the motherboard and then plug the other ends of that breakout cable into 4 sata drives? Or do I need to purchase a HBA like this to be able to plug in all 8 of my SATA drives? Of course I need to make sure the HBA is crossflashed into IT mode so it's just acting as a bus adapter and not a JBOD or RAID card (see, I told you I have read some of the forums here). My confusion also is the physical SCU plug, if you look at photos of the board, it looks exactly the same as a SATA plug...is that correct? If I can just use the SCU ports to hook up 4 SATA drives directly with a breakout cable, does the motherboard have 2 SCU ports and would that enable me to use 2 breakout cables to hook up my 8 SATA drives (I like the clean look of the breakout cables)? I'm confused whether that motherboard has 4 SCU ports or just 1 SCU port for 4 SATA drives and I'd like to plug in two breakout cables into my 8 drives for a clean look.


I think some of the hardware may be overkill for what I need it to do, but who knows what I will want it to do in the future and I had room in my budget.

Also, my knowledge of FreeNAS is still pretty limited. I haven't had to make many adjustments since I finished setting it all up about a year ago. It took me a month to figure out how to set up a couple of the jails that I had to do manually.

 

[Nick2253]

In general, does the hardware look ok?

Everything looks fine to me. However, your hardware is probably overkill for your use case. The most demanding part of your use case is the 3x 4k streams that you're looking for. I'm not super familiar with the demands of 4k, though I know that it can be pretty demanding. You know your use case better than I do, so if you really do need 3x simultaneous streams, you might want to stick with your platform. If you only have one or two simultaneous, you can probably get away with an E3.

My confusion is around the SCU ports.

SCU is storage control unit. You can only use HDDs on the SCU ports. I believe you'll have to make sure you put your SSDs on the 0-5 SATA ports in order to boot, but will have no problem with the other hard drives. There is no need for an addon card.

You are confusing SCU with SFF-8087 SAS ports. There is nothing special about SCU ports, other than the way the controller underneath them interfaces with the motherboard. For a SATA hard drive, there is absolutely no difference (other than the possible issue about booting from SCU ports, which is not relevant for your use case).

https://www.supermicro.com/support/faqs/faq.cfm?faq=16820

 

[walk1355]

Everything looks fine to me. However, your hardware is probably overkill for your use case. The most demanding part of your use case is the 3x 4k streams that you're looking for. I'm not super familiar with the demands of 4k, though I know that it can be pretty demanding. You know your use case better than I do, so if you really do need 3x simultaneous streams, you might want to stick with your platform. If you only have one or two simultaneous, you can probably get away with an E3.


SCU is storage control unit. You can only use HDDs on the SCU ports. I believe you'll have to make sure you put your SSDs on the 0-5 SATA ports in order to boot, but will have no problem with the other hard drives. There is no need for an addon card.

You are confusing SCU with SFF-8087 SAS ports. There is nothing special about SCU ports, other than the way the controller underneath them interfaces with the motherboard. For a SATA hard drive, there is absolutely no difference (other than the possible issue about booting from SCU ports, which is not relevant for your use case).

https://www.supermicro.com/support/faqs/faq.cfm?faq=16820

I have seen the light! Thank you brother. So the SCU port is a SATA form factor plug (is that a thing)? So I have a board with 10 SATA form factor plugs, 4 of which I can't boot from?

Also, I don't need to do 3x 4k streams, but it could happen. 95% of my plex streaming is to 1 tv at a time, and only about 10% of that is even in 4k. So it's extremely unlikely I need 3x 4k streams at once. I'd rather be on the overkill side of the fence than on the side in which my hardware struggles. Especially since it allows me to do something else with it in the future. And since I had room in my budget, I went a little overkill.

 

[walk1355]

Also, would a

Everything looks fine to me. However, your hardware is probably overkill for your use case. The most demanding part of your use case is the 3x 4k streams that you're looking for. I'm not super familiar with the demands of 4k, though I know that it can be pretty demanding. You know your use case better than I do, so if you really do need 3x simultaneous streams, you might want to stick with your platform. If you only have one or two simultaneous, you can probably get away with an E3.


SCU is storage control unit. You can only use HDDs on the SCU ports. I believe you'll have to make sure you put your SSDs on the 0-5 SATA ports in order to boot, but will have no problem with the other hard drives. There is no need for an addon card.

You are confusing SCU with SFF-8087 SAS ports. There is nothing special about SCU ports, other than the way the controller underneath them interfaces with the motherboard. For a SATA hard drive, there is absolutely no difference (other than the possible issue about booting from SCU ports, which is not relevant for your use case).

https://www.supermicro.com/support/faqs/faq.cfm?faq=16820

Also, would an addon card help throughput in any way?

 

[Nick2253]

So I have a board with 10 SATA form factor plugs, 4 of which I can't boot from?

Bingo! Again, I'm not exactly sure of all the SCU limitations are, but it's safe to assume that all those ports can do is provide HDD storage to your OS.

I'd rather be on the overkill side of the fence than on the side in which my hardware struggles.

My biggest advice/warning to this is: don't forget your power bill. A back of the envelope calculation is that 1W costs you $1.00/year. If you have an older, super power hungry server, when you could have purchased newer, more energy efficient hardware, you can save yourself multiple hundreds of dollars over the years you'll own the machine.

would an addon card help throughput in any way?

No. In any event, your biggest bottleneck is the 1Gbps ethernet link.

 

[walk1355]

I have an additional question and it may not be the right section for the question, so let me know if I need to post somewhere else.

I am currently running a few jails, some of which seemed to have taken me forever to get set just the way I wanted them. I would like to avoid that when putting together this new system. I am familiar with copying over my system settings for freenas, but I am not 100% sure if this would include jails. I assume it doesn't since my jails currently reside on my storage disks and not my boot disks.

I am running 9.10 still and don't really have a reason to upgrade to 11 (if it ain't broke, don't fix it).

I read through this thread, but I don't understand if that will work if you are moving over to new build a new raidz set up. I also don't understand fully the commands they are saying to use.


Could I simply just copy and paste the jails folders from my old build into my new build?

 

[Nick2253]

Everything you need for your jails is captured in the FreeNAS config, and the specific jail dataset(s). You should have no problem migrating the data.

However, you need to make sure that everything is set up the same: same paths, etc. When you say "copy and paste", I'm nervous that you're using some intermediary for the data that won't preserve formatting, or won't capture all the files. I would recommend a tool like rysnc to make sure your source and destination datasets are exact copies of one another.

 

[walk1355]

Ok. I will check out rsync.

But as long as my pool is named the same, I should be ok? My current machine the pool is titled "TANK" or something like that...as long as I name the new one the same, I should be ok?

Just want to make sure this is possible since I am using the 2 disks in my current build and having to pull them and put them in my new build.

 

[walk1355]

Everything you need for your jails is captured in the FreeNAS config, and the specific jail dataset(s). You should have no problem migrating the data.

However, you need to make sure that everything is set up the same: same paths, etc. When you say "copy and paste", I'm nervous that you're using some intermediary for the data that won't preserve formatting, or won't capture all the files. I would recommend a tool like rysnc to make sure your source and destination datasets are exact copies of one another.

When I say copy and paste, I am going to copy directly off of my freenas drives onto a windows machine. build the new machine and copy the files directly into the new pool on the new machine. would that work?

 

[Nick2253]

When I say copy and paste, I am going to copy directly off of my freenas drives onto a windows machine. build the new machine and copy the files directly into the new pool on the new machine. would that work?

This scenario was exactly what I was referring to.

Moving the files to an NTFS volume will hose the permissions, potentially lose stream data (though I'm not sure that matters in this case), screw up symbolic and hard links, and unless you are using the correct settings, will leave data behind. Doing file-level backups of operating systems is complicated for a reason, which is why tools like rsync or zfs send/receive are used. And even then, as you move files between different filesystems, you run into problems where different filesystems only support different features (permissions are a big issue here), so data can be lost or changed.

Furthermore, since there's no Windows ZFS driver, you can't actually copy "directly". Instead, what you'll be doing is copying over a file sharing protocol. Given that you're using Windows, that probably means SMB, which means another layer to cause problems.

It may be better in this case to make a snapshot of your pools, and save that snapshot stream to a file using zfs send and then restore using zfs receive. The Oracle documentation describes this process (see Sending a ZFS Snapshot and Receiving a ZFS snapshot): https://docs.oracle.com/cd/E53394_01/html/E54801/gbchx.html

 

[walk1355]

This scenario was exactly what I was referring to.

Moving the files to an NTFS volume will hose the permissions, potentially lose stream data (though I'm not sure that matters in this case), screw up symbolic and hard links, and unless you are using the correct settings, will leave data behind. Doing file-level backups of operating systems is complicated for a reason, which is why tools like rsync or zfs send/receive are used. And even then, as you move files between different filesystems, you run into problems where different filesystems only support different features (permissions are a big issue here), so data can be lost or changed.

Furthermore, since there's no Windows ZFS driver, you can't actually copy "directly". Instead, what you'll be doing is copying over a file sharing protocol. Given that you're using Windows, that probably means SMB, which means another layer to cause problems.

It may be better in this case to make a snapshot of your pools, and save that snapshot stream to a file using zfs send and then restore using zfs receive. The Oracle documentation describes this process (see Sending a ZFS Snapshot and Receiving a ZFS snapshot): https://docs.oracle.com/cd/E53394_01/html/E54801/gbchx.html

Holy cow. I'm in over my head. I need something that's point and click pretty much.

I may be better off redoing the jails in this case.

Would thus also be a good time to update to 11?

 

[Nick2253]

You might as well upgrade. There's some new stuff happening with jails/VM with 11, and there will be further changes with 11.2 and 11.3. I'm still on 9.10, so I'm not super familiar with these new features.