10th Build - Needing Guidance

[darowley]

I have lots of little questions.... I certainly don't expect any one person to answer to all of this. I know everyone has different recommendation and options. I'm open to hearing it all! - THANK YOU IN ADVANCE!


Basic Specs and Usage:

Mostly used by just me (occasionally maxed by 2 or 3 to stream via Plex).

Raidz2 - Probably 10 drives total
I store lots of backup data. Personal Pictures / documents / music, etc. (I add about 20Gb of new data each day, tons of word / excel documents, music, some videos, program installer, etc).
I stream movies via SMB locally and VIA Plex (local and remote) - Mostly 720p / 1080p content, but I know that will change to more and more 4K content)
I replicate (nightly) all of this to another local FreeNAS server via rsync.
I use snapshots (just in case of virus or accidental deletions). Online backup for the super important stuff.

My current servers running similar hardware (and sometimes slower / older gen i7 processors) all seem to run fine (just a little warm / hot due to poor circulation). I assume that my processor usage is rarely maxed out and doubt it even comes close. I think my biggest bottleneck (besides 1G Ethernet) might be the SATA adapter (listed below).

I have built many FreeNAS servers since 2011. Most of them have been using almost new or "used" Dell XPS towers (8300, 8500, 8700, and 8900). I feel fairly comfortable with the basics. For my personal servers, I have managed to secure 8 spinning drives in these desktop towers, which I think run a littler warmer than normal due to poor circulation.

For this new server, I purchased a 12 Bay Rosewell server chassis ($250): https://smile.amazon.com/gp/product/B00N9CXGSO/

I was going to buy a new motherboard with 10-12 SATA ports but upon searching tonight, I gave up as really don't know what's compatible. I know that the boards I was looking at probably would have worked, but I would LOVE to hear a few recommendations on motherboards (I'm not planning on using ECC) (build info and usage below). While I will probably use a motherboard from a Dell XPS8920, I may change my mind based on recommendations).

RAM: NON-ECC (for my existing board at least), Probably 32GB PC4-2133p (what is compatible with my board). I'm not against ECC, but if I'm using an existing computer I already own, then it's just not compatible. I'm open to buying a motherboard that supports it - I just don't know what to look for. My thought was to buy a board with all the needed SATA ports built in to reduce / eliminate heat from the card. For years, I have assumed that motherboard SATA ports were better / more reliable, but after research tonight, I see that that might not be true. What is recommended? Using the motherboard ports or buying enough ports on a card to have all drives on 1 or 2 SATA expansion cards?

SATA Adapter:
I have been using these IO Crest SATA cards as these Dell XPS desktops only have 4 or 5 SATA ports ($30): https://smile.amazon.com/gp/product/B00AZ9T264
But tonight (for this build) I purchased a SAS9211 8i ($90) / LSI00194 - https://smile.amazon.com/gp/product/B0056FIJP2/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1
These are / will be plugged into an x16 port (the graphics slot). What is the approximate max theoretical rate of a raidz2 with 10 or 12 drives? I saw that these WD Drives pushed about 185 MB/s max. I ask because 1: I have no clue and 2: I may want to get 10GBE someday...

I have purchased quite-a-few WD RED Drives (about $150 each/ normally $200-300) WD80EFAX SATA 256MB Cache (Shucked from: WDBCKA0080HBK-NESN / https://www.bestbuy.com/site/wd-eas...-3-0-hard-drive-black/5792401.p?skuId=5792401 ).

Drive 4k / 512b Alignment: I have read that some drives "LIE" to the OS about their sectors. What is the process to check my existing server that uses these same drives. What is the process to force the drives into 4k, (which I understand is the best for performance reasons), before / during FreeNAS setup / Pool Creation.

I have read in several places that I should use Raidz2 with 10 drives for optimal performance and alignment. I have 12 bays in this chassis. If I setup raidz2 with 12 drives (vs 10 drives), what kind of a real world performance hit am I looking at? Also, is this bad for the drives? Does it wear them out quicker?


Power Supply: What rating power supply should I use? Currently I've been using the PSU that came with the XPS computers (460 Watt Max) PSU Specs: https://http2.mlstatic.com/fonte-de...mmv-D_NQ_NP_19541-MLB20172509054_102014-F.jpg

Is that PSU powerful enough for 10-12 drives?
8TB Drives Spec: Rated: 5V 400mA, 12V 550mA

Network Speed: I have a gigabit switch and have setup some of my servers to use LAACP with 2 or 4 Ethernet cables. After I installed them and setup LACP on the server and switch, I realized that while the cards were capable of 2Gb/s or 4Gb/s, SMB only transfers using one of the lines, maxing out at 1Gb/s. (Please Correct me if this is not true). But, even so, this allows my servers to backup to each other at 1Gb/s and I can push data to my server from my computer or another at a full 1Gb/s. - I'm ok with 1Gb/s as the price for 10G seems too much for me. I can see multiple $300 cards plus a switch adding up very quickly $$$).

With that said... Any obvious bottlenecks or performance issues that I could fix? Once of my biggest concerns is wasted processing power / over usage due to the 10/12 drive raidz2 and the 4k drive alignment issue.

 

[Ericloewe]

Half of your questions (motherboards, SATA connectivity, power, etc.) are answered by the Hardware Recommendations Guide. Link is in my sig.

What is the process to check my existing server that uses these same drives

You need to check the pool's ashift. The value of ashift should be the base 2 logarithm of the sector size - in other words, 12 for 4kB sectors. Changing it is... complicated, let's just hope you have 12 and deal with the other possibility if it arises.

What is the process to force the drives into 4k (which I understand is the best for performance before / during setup of the drive).

Depends on the drive and is generally not possible. A few drives are available in native 4kB and 512e versions, but they're meant for big customers.

I have read in several places that I should use Raidz2 with 10 drives for optimal performance and alignment. I have 12 bays in this chassis. If I setup raidz2 with 12 drives (vs 10 drives), what kind of a real world performance hit am I looking at?

Good question. Increasing suckiness. A RAIDZ vdev has about the same IOPS as a single component drive, so performance tanks with increasing vdev width.

Also, is this bad for the drives? Does it wear them out quicker?

No, but, in extreme cases, we're talking pools so slow that they can't safely resilver in case of a drive failure.

Network Speed: I have a gigabit switch and have setup some of my servers to use LAACP with 2 or 4 Ethernet cables. After I installed them and setup LACP on the server and switch, I realized that while the cards were capable of 2Gb/s or 4Gb/s, SMB only transfers using one of the lines, maxing out at 1Gb/s. (Please Correct me if this is not true). But, even so, this allows my servers to backup to each other at 1Gb/s and I can push data to my server from my computer or another at a full 1Gb/s. - I'm ok with 1Gb/s as the price for 10G seems too much for me. I can see multiple $300 cards plus a switch adding up very quickly $$$).

Link aggregation is mostly a pipe dream. Samba has experimental support for SMB multichannel, which does fulfill the dream. The Resources section has documents for both topics.

 

[Chris Moore]

I add about 20Gb of new data each day

That is a massive amount to add daily, it works out to 7.3TB (terabytes) a year. (or did I math that up)

I replicate (nightly) all of this to another local FreeNAS server via rsync.

I do that too.

I have built many FreeNAS servers since 2011.

This also.

I was going to buy a new motherboard with 10-12 SATA ports but upon searching tonight, I gave up as really don't know what's compatible.

For the number of drives, I would suggest a SAS controller with a SAS expander. I can link you to the exact parts to buy if you want.

While I will probably use a motherboard from a Dell XPS8920, I may change my mind based on recommendations).

Why would you want to do that? You can get a second-hand Supermicro server board from eBay really cheap (around $80) I will link you to a good one if you want. The advantage is that you get remote kvm and the ability to remotely mount an ISO so you don't ever need to connect anything but the network and power to the server.

What is recommended? Using the motherboard ports or buying enough ports on a card to have all drives on 1 or 2 SATA expansion cards?

Someone might suggest SATA, but I would suggest SAS because you can run SATA drives on a SAS controller but it gives you the added ability to control many drives with the SAS expanders.

But tonight (for this build) I purchased a SAS9211 8i ($90) / LSI00194 -

You should have come here to ask questions first because you paid almost double what you should have for that.

What is the approximate max theoretical rate of a raidz2 with 10 or 12

About 530 to 630 MB/s depending if it is 10 or 12 and other random factors that have to do with the kind of data that is bing handled.

Drive 4k / 512b Alignment: I have read that some drives "LIE" to the OS about their sectors. What is the process to check my existing server that uses these same drives. What is the process to force the drives into 4k, (which I understand is the best for performance reasons), before / during FreeNAS setup / Pool Creation.

I have read in several places that I should use Raidz2 with 10 drives for optimal performance and alignment. I have 12 bays in this chassis. If I setup raidz2 with 12 drives (vs 10 drives), what kind of a real world performance hit am I looking at? Also, is this bad for the drives? Does it wear them out quicker?

I leave this for someone else to comment on.

Power Supply: What rating power supply should I use? Currently I've been using the PSU that came with the XPS computers (460 Watt Max) PSU Specs: https://http2.mlstatic.com/fonte-de...mmv-D_NQ_NP_19541-MLB20172509054_102014-F.jpg
Is that PSU powerful enough for 10-12 drives?

I would say no, but others might argue with that. It is down to the starting load of the drives at spinup. I have a server with 16 drives and it only pulls 250 watts under load with Plex stream being transcoded. I have a 750 watt power supply to ensure that there is enough current to start the drives at power up. I would rather have a bit extra available than burn my power supply out from overloading it.

I hate to spend time putting together recommendations if you have already decided on your build, but you can save some money if you are willing to consider used enterprise server gear and still have a very, very capable system. For example, I bought my 10GB network cards for around $50 each and it works great.

 

[darowley]

I hate to spend time putting together recommendations if you have already decided on your build, but you can save some money if you are willing to consider used enterprise server gear and still have a very, very capable system. For example, I bought my 10GB network cards for around $50 each and it works great.

Chris, THANK YOU! My plans are not at all set in stone. Starting from scratch, this is new territory for me. I don't expect this server to be perfect, I just don't wan't to make any major mistakes. I do feel fairly comfortable with what I've done in the past and know that the equipment has worked. I would love links and or info (make / model) of parts to any and all recommended (freenas compatible) parts.

I will build more servers, so any recommendations here will be used to build another server.

You need to check the pool's ashift. The value of ashift should be the base 2 logarithm of the sector size - in other words, 12 for 4kB sectors. Changing it is... complicated, let's just hope you have 12 and deal with the other possibility if it arises.

Thank you with that info, I was able to google and find:
https://charsiurice.wordpress.com/2016/05/30/checking-ashift-on-existing-pools/

That is a massive amount to add daily, it works out to 7.3TB (terabytes) a year. (or did I math that up)

Your math is probably correct. But I know I add more than that each year.

For the number of drives, I would suggest a SAS controller with a SAS expander. I can link you to the exact parts to buy if you want.

Interesting. I was thinking that I bought some sort of a SAS Card / Expander. - I thought I was a little lost, now I KNOW that I'm even more lost.

About 530 to 630 MB/s depending if it is 10 or 12 and other random factors that have to do with the kind of data that is bing handled.

Thank you! That's helpful. So if I understand correctly, then even with 10G Ethernet, I would not be able to saturate the connection...

For the number of drives, I would suggest a SAS controller with a SAS expander. I can link you to the exact parts to buy if you want.

YES PLEASE. I really don't know where to start.

Why would you want to do that? You can get a second-hand Supermicro server board from eBay really cheap (around $80) I will link you to a good one if you want. The advantage is that you get remote kvm and the ability to remotely mount an ISO so you don't ever need to connect anything but the network and power to the server.

Yes... what model should I look for? I get lost with what socket(s) to choose. Remember, I have many used computers at my disposal, lots of processors that I can pull from them (or I can buy a new processor. Used computers are mostly i5's and i7's, most about 2-5 years old. I know my used computers use / have different sockets based, but I'm hoping that you / someone can narrow down a few (sockets) to look at. I may also buy a brand new supermicro motherboard. I remember seeing these boards a few years ago, but forgot the name and I didn't see them searching last night.

You should have come here to ask questions first because you paid almost double what you should have for that.

Hmmm darn... Ok, it's returnable... What do you recommend?

You are right, I should have asked first. But I was worried about being told to read the forms first and search it out myself. So I did... and after a few hours of searching for motherboards, gigabit cards (this was like the 4th time), and SATA/SAS controllers, I also found the Hardware 2016 R1e.pdf file and got a little lost in it, then found this page on HBA: https://www.servethehome.com/buyers...s-freenas-nas-servers/top-picks-freenas-hbas/
So I found one on amazon and saw in the comments that it worked with freenas, then I wrote the above for help.

I generally purchase items on Amazon, but I can buy elsewhere If I know it's compatible.

I would say no, but others might argue with that. It is down to the starting load of the drives at spinup. I have a server with 16 drives and it only pulls 250 watts under load with Plex stream being transcoded. I have a 750 watt power supply to ensure that there is enough current to start the drives at power up. I would rather have a bit extra available than burn my power supply out from overloading it

So, I have read that I should assume 25 watts per drive x 12 = 300 watts. That's less than the 460, but I assume that isn't how one knows, as it's 460 watts TOTAL - the picture above shows the split between the 5v and 12. 142 and 385... which add up to 527 watts.


This now has me thinking even more (SERIOUSLY CONSIDERING): Maybe I should get a different case and motherboard that supports enough for 18-24 drives. That would give me enough space for a few years. Is this practical for raidZ2 - I assume this is were I'm told to use two vdevs of 10 drives or something like that.

Once again, thank you. Any additional help / links are greatly appreciated.

 

[Chris Moore]

Your math is probably correct. But I know I add more than that each year.

The reason I thought I might have mixed up the math is because I generally only add a couple of terabytes a year..

Interesting. I was thinking that I bought some sort of a SAS Card / Expander. - I thought I was a little lost, now I KNOW that I'm even more lost.

I am going to give you a reading assignment to help you understand some of the technology I will throw at you:
https://forums.freenas.org/index.ph...imer-on-basic-sas-and-sata.26145/#post-165190

Thank you! That's helpful. So if I understand correctly, then even with 10G Ethernet, I would not be able to saturate the connection...

This is true and I will give you the, 'sum up,' instead of the full detailed, technical explanation. Each vdev (virtual device) is limited to roughly (very roughly) the performance of the slowest drive in the vdev. So, no matter how many drives you put in your pool, if they are all in one vdev, it is going to be relatively slow. The solution is to use more vdevs. For systems that need really fast data, they use pools made of all mirrors, but that is comparatively expensive. For my money (what I use) is RAID-z2 gives good reliability with a lower total cost. I configured my storage pool with 2 vdevs, each a 6 drive RAID-z2. That gives me more IOPS because of having more vdevs. I get between 600 and 800MB/s depending on the type of data. Small files go slow for example. If you want optimal speed for 10GB networking, the easiest way to get there might be with a pool of mirrors. To have enough vdevs to do it with RAID-z2 will take a lot of drives. You could probably do it with 24 drives in 4 vdevs of 6 drives each.
If you did that, you probably wouldn't want to use 6TB drives though because it would give you about 83TB of raw storage with about 67 TB usable. Honestly, that was the short version.

Yes... what model should I look for? I get lost with what socket(s) to choose. Remember, I have many used computers at my disposal, lots of processors that I can pull from them (or I can buy a new processor. Used computers are mostly i5's and i7's, most about 2-5 years old. I know my used computers use / have different sockets based, but I'm hoping that you / someone can narrow down a few (sockets) to look at. I may also buy a brand new supermicro motherboard. I remember seeing these boards a few years ago, but forgot the name and I didn't see them searching last night.

I will look into that tomorrow. I have to get some sleep. But someone else may come along to comment. There are lots of smart folks on here.

Hmmm darn... Ok, it's returnable... What do you recommend?

You are right, I should have asked first. But I was worried about being told to read the forms first and search it out myself. So I did... and after a few hours of searching for motherboards, gigabit cards (this was like the 4th time), and SATA/SAS controllers, I also found the Hardware 2016 R1e.pdf file and got a little lost in it, then found this page on HBA: https://www.servethehome.com/buyers...s-freenas-nas-servers/top-picks-freenas-hbas/
So I found one on amazon and saw in the comments that it worked with freenas, then I wrote the above for help.

I generally purchase items on Amazon, but I can buy elsewhere If I know it's compatible.

It isn't that the part is wrong, so if you prefer to use Amazon, I don't want you to feel that you can't, it is often more expensive to get parts there though. Here is why. When you are building a server, you can pick certain items from the used parts bin at eBay and get great enterprise grade gear that is being replaced for something faster but still has plenty of guts to get the job done at home. The servers I am using are probably 5 years old and they are doing the job for me and I anticipate being able to use them another 3 to 5 years.

So, I have read that I should assume 25 watts per drive x 12 = 300 watts. That's less than the 460, but I assume that isn't how one knows, as it's 460 watts TOTAL - the picture above shows the split between the 5v and 12. 142 and 385... which add up to 527 watts.

This now has me thinking even more (SERIOUSLY CONSIDERING): Maybe I should get a different case and motherboard that supports enough for 18-24 drives. That would give me enough space for a few years. Is this practical for raidZ2 - I assume this is were I'm told to use two vdevs of 10 drives or something like that.

If you are interested in getting a rack mount server, many of the ones that can be had include redundant hot-swap power supplies, so I wouldn't worry too much about the power situation. Most drives pull the vast majority of their power from the 12 volt rail and the power supply your are looking at, only having 385 watts on the 12 volt rail (and it is probably a split rail) just doesn't have the guts to to start that many drives all at once. It is the startup current you have to satisfy because they idle down to a lower power level after start.

 

[Johnnie Black]

So, I have read that I should assume 25 watts per drive x 12 = 300 watts. That's less than the 460, but I assume that isn't how one knows, as it's 460 watts TOTAL - the picture above shows the split between the 5v and 12. 142 and 385... which add up to 527 watts.

That is a multi rail PSU and so not recommended for a storage server, as the 12v are split in 3 rails, you want a single rail power supply.

 

[Chris Moore]

what model should I look for?

Please don't feel I am saying you must rush right out and get this. We are still talking and looking yet and it is ultimately your decision.
This is actually a fully configured server, except for the drives. You might have to update firmware on the system board or change the firmware in the SAS controller to the IT (initiator target) version but it is a pretty nice package. It would give you almost everything you need right in one package.
https://www.ebay.com/itm/Supermicro...-2-6ghz-8-Core-128gb-24-Bay-JBOD/372154285756
This one is described the same and up for auction by the same seller. If you could get it at a good price, it would be a real deal:
https://www.ebay.com/itm/Supermicro...-2-6ghz-8-Core-128gb-24-Bay-JBOD/292353346542
You can sometimes find really good deals, but right now it is like they need money to buy gifts or something. All the prices are higher than they were last time I looked.
These servers use a backplane like this that the drives plug into:
https://www.ebay.com/itm/Supermicro...A-Expander-Backplane-SAS2-846EL1/352203219801
That backplane has an integrated SAS expander. What that means is that you can connect a single 8 lane (or even 4 lane) SAS card to the backplane to access all the connected drives. This is a desirable feature as it saves having to use multiple drive controllers. You could think of the SAS expander kind of like a network switch for the data traveling between the drives and the controller.
You don't really need the dual socket system board to run FreeNAS, but many of these retired servers come equipped that way by default.
An option would be to buy an empty (bare bones) chassis, and build it out the way you want.
Here is an example:
https://www.ebay.com/itm/Supermicro...BPN-SAS2-846EL1-24x-TRAYS-NO-PWS/382277186258
If that interests you, I will point you at some items of interest?

 

[darowley]

I will wait until after the holidays to buy anything more.

I'd love more info recomendations on (new and used) motherboards for this larger 24 drive build.

 

[Chris Moore]

What kind of price range are you looking to be in?
I like the idea of having two physical servers, a primary and a backup, but you could have a primary and backup pool inside a single server. It would give you data redundancy if you had a pool failure but not full hardware redundancy. It might be a little more economical that way.
It is a choice.

This is a really good server for FreeNAS:
https://www.ebay.com/itm/Supermicro...-24-x-HDD-Storage-Server-W-Rails/173688157339

It is not perfect though. This system has a hardware RAID controller in it that would need to come out and could easily be replaced by one of these:
https://www.ebay.com/itm/HP-H220-6G...9205-8i-P15-IT-Mode-From-US-Ship/192639052923

Then turn around and sell the hardware RAID controller on eBay and get some money back.

Take a look at this post where I was telling someone else about mounting boot drives inside the chassis next to the system board:
https://www.ixsystems.com/community/threads/used-supermicro-4u-server-or-homebuilt.73790/post-518974

I have a chassis like this myself and I like it.

 

[darowley]

Price Range: about $700-$1200 for one server. It's for work and I can spend more if needed.
I'd be more willing to spend more on a 36 bay server especially if it had more RAM and a faster processor.

I agree about the multiple pools inside the one server. I think that will be enough since I have a 12 bay server I could use for backup also.

I also purchased a 9211-8i IT flashed, last week from ebay. Can I use that instead of the 9205 or is the 9205 better?

Take a look at this post where I was telling someone else about mounting boot drives inside the chassis next to the system board:

WILL DO THIS... Thanks!

Does this server come with ECC memory? I couldn't tell from the site but I'm assuming that it would be since it's a server.

 

[Chris Moore]

Does this server come with ECC memory?

Yes, it is ECC, in fact it is registered ECC memory.

This also might be a good option for you:
https://www.ebay.com/itm/Supermicro-4U-48-Bay-FREENAS-Storage-Server-2x-Xeon-Low-Power-L5640-6-Core-32GB/142950808625

 

[Chris Moore]

I'd be more willing to spend more on a 36 bay server especially if it had more RAM and a faster processor.

It is incredibly easy to upgrade one of these systems. I wouldn't hesitate to buy one of the inexpensive chassis and pull the system board, processor and memory to put in my own. Something like this for example:
https://www.ebay.com/itm/SuperMicro-CSE-846-24-Bay-SAS2-BP-Server-w-X9DRi-F-2x-6-Core-E5-2620-2Ghz-Sleds/183735741514
If you didn't want to swap system boards, in the specific item in this link, you could add the amount of RAM you want, upgrade to a faster processor and change that hardware RAID controller for a good SAS controller using parts like these:

CPU: Intel Xeon E5-2650V2 8-Core Socket LGA2011 CPU Sever Processor SR1A8 2.60GHz
https://www.ebay.com/itm/372624706969
Price: US $64.99

RAM: Samsung PC3-12800 16GB RDIMM 1600MHz PC3-12800R DDR3 ECC REG RAM Server Memory
https://www.ebay.com/itm/292973301352
Price: US $30

SAS HBA: HP H220 6Gbps SAS PCI-E 3.0 HBA LSI 9205-8i P15 IT Mode From US Ship
https://www.ebay.com/itm/192639052923
Price: US $42.00

 

[darowley]

SAS Stuck on "initializing"

This is a really good server for FreeNAS:
https://www.ebay.com/itm/Supermicro...-24-x-HDD-Storage-Server-W-Rails/173688157339

I got that one! Thanks!

I also purchased the HP H220 SAS.

I received it today. I just got my drives transferred over, tried to boot, but it kept getting stuck on the SAS "Initializing". I gave up after 15-25 minutes. So I disconnected the two SAS cables, then was able to boot into my existing FreeNAS install from my old server. Rebooted with the drives connected and I still get "initializing". I tried booting with the two SAS cables connected and removed my 8 drives, and now it boots. But obviously, that's no help as I can't run a file server without any data drives.

I assume that I need to make a change to the SAS controller, but I went in there and didn't see much to change.

I am using 8x8TB WD Reds.
Not sure what to do next.

Thanks

 

[Chris Moore]

I looked at the auction for the SAS card and noticed that the vendor specifically listed an older firmware. That probably just needs to be updated.

EDIT: It is a great card, but the firmware being old will certainly cause problems. This guide, even though it is for crossflashing, which you do not need, should also contain the information you need to be able to update the firmware:

Detailed newcomers' guide to crossflashing LSI 9211/9300/9311 HBA and variants
https://www.ixsystems.com/community...shing-lsi-9211-9300-9311-hba-and-variants.54/

 

[darowley]

LOL, that guide is NOT for newcomers.

I can't seem to get the usb to work / boot to dos with out errors.

My HP H220 / 9205-8i is currently installed in a Dell XPS 8920, no drives or backplane attached

When making the usb, I used Rufus portable, then copied the files from "USB" (I assume he mean the ZIP file called "MINIMAL_USB_FOR_9211_v2.zip") then it asks if I want to replace files!!! First I tried NOT replacing the files. Upon booting (BIOS only, UEFI is not an option), I get the C: prompt, but before the C: prompt, there are many errors, one being "Incorrect MS-DOS version". Upon trying to run the sas2flsh -listall command, I get a "PAL" error. The directions say that this is because I need to boot via EFI (I assume this is UEFI, which is not an option at boot).

Second I tried pasting and "replacing" the files. That gave more errors and no C: prompt.

I then downloaded the firmware from broadcom and noticed that there was an x64 sas2flash so I tried using windows. I was able to run the sas2flash -listall command and see my adapter, but I couldn't erase the flash "using this platform".... bummer. So it's back to the USB route.

When I run rufus:
Boot Selection: FreeDOS
Partition scheme: MBR (only option)
Target system: "BIOS (or UEFI-CSM)" (only option).
File system: FAT32 / Cluster 16k

Am I making / preparing this USB incorrectly?

I've now spent at least 2.5 hours on this, possibly more.

 

[Chris Moore]

I assume this is UEFI

If you have UEFI system board, you must use the EFI version of the flash utility, because the DOS version will not work on a UEFI system.

If you only need to update the firmware, not crossflash, you can use the sas2flash utility that is built-in to the FreeNAS operating system.

 

[darowley]

Well, I can bring up the shell, but I don't know what to do from there. I have a usb stick with my bin file, but I don't know how to find it using the shell.
This is way over my head.

 

[darowley]

Also:

I have this card, to flash, a 9211 in IR mode that I need to change to IT, and anther 9211 that might need updating at some point. I assume that I will also buy more in the future, so I would like to understand how to do this outside of FreeNAS but i'm also looking forward to trying it via freenas but i'm not sure where to start.

I assume that I need to still clear / wipe the HBA using megarec then run sas2flash. Or is there a tutorial that exists?
I will admit that while I understand the idea of what i'm trying to accomplish, the specific steps to do such are very foreign to me.

I'm at a loss of why that instruction page is full of so much info, but the part about making the flash drive doesn't seem to work...

 

[sretalla]

I realized that while the cards were capable of 2Gb/s or 4Gb/s, SMB only transfers using one of the lines, maxing out at 1Gb/s. (Please Correct me if this is not true).

LACP can allow a server to push out or receive the maximum of all cards in the LACP group, but only if you have enough clients to drive that.

Due to the need to re-assemble packets in the correct order, you can only have a single TCP connection terminating on one of the NICs in an LACP group, so a single connection/stream/copy or whatever can never be faster than a single NIC. You can, as suggested by @Ericloewe, get some kind of improvement on that with SMB's experimental features, but I'd be surprised if that will translate into real-world results without some kind of lab-type conditions.