SOLVED Dual SFF-8088 & Dual SFF-8087 (PCIe)

twf85

Dabbler
Joined
Apr 18, 2019
Messages
14
I'd like to use up some free space in an nearby ATX case to add spares to my pools. I suppose I only need one 8088 to do the trick (that would at least give me one spare per planned pool), but I'd still like to know if a PCIe card exists at a reasonable price that has x2 8088 & x2 8087.

So far, the only card I've found also includes some RAID architecture that would be wasted in a FreeNAS build: LSI MegaRAID 8888ELP

I also found this with more internals than I need: Chenbro CK22803

And this, which is exactly half of what I'm looking for: LSI LSI00303

Am I searching for a unicorn or misunderstanding something fundamental here? I'm leaning towards the Chenbro, but I'm struggling with the look of the thing. Looks like it's ten years old.

Edit: Realized after posting that the Chenbro is only an Expander.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
but I'd still like to know if a PCIe card exists at a reasonable price that has x2 8088 & x2 8087.
No. Your least cost option is two cards. One for the external ports and one for the internal ports. Any four port card is going to be extra cost and there are no four port cards that I am aware of that split the connection two external and two internal although you can take two internal ports and cable them to an external bracket.
Am I searching for a unicorn or misunderstanding something fundamental here?
I think you are working on a misunderstanding. Please give more explanation of what you have and what it is you are trying to accomplish.
that would at least give me one spare per planned pool
For example, why are you planning for spares and why are you planning for more than one pool. It sounds like the design is questionable.
 

CraigD

Patron
Joined
Mar 8, 2016
Messages
343
Are you trying to add a JBOD?

If you are, keep it simple

You need a SAS Controller (I use a M1015) and a 8087 to 8088 pass thru (If you controller has no 8088 port)

If you don't have a spare 8087 port you simply add a $15 expander

And this is what you need to make a JBOD on the cheap:

I used:

1x 8087 cable plugged into a 8087-8088 pass thru in my sever
1x 8088 cable
1x Case (I have hot swap bays in a full tower)
1x PSU
1x PSU Jumper
1x Cheap Expander
1x Molex powered riser (Yes a PCIe mining riser card)
many 8087-4 SATA cables

Just make sure the riser doesn't short (I put a thin bit of wood under it)

If your think the riser is unreliable, buy a spare

In my country Rack mount cases are too expensive

Have Fun
View attachment 28131
 

twf85

Dabbler
Joined
Apr 18, 2019
Messages
14
No. Your least cost option is two cards. One for the external ports and one for the internal ports. Any four port card is going to be extra cost and there are no four port cards that I am aware of that split the connection two external and two internal although you can take two internal ports and cable them to an external bracket.


I figured as much, but I wanted to be sure. More often than not, if I can't find something in ~30 minutes, it usually means I'm using the wrong search terms or I'm looking for something that doesn't exist.


I think you are working on a misunderstanding. Please give more explanation of what you have and what it is you are trying to accomplish.

For example, why are you planning for spares and why are you planning for more than one pool. It sounds like the design is questionable.


Little backstory... The primary purpose of this FreeNAS build is to act as a media server. It replaces a Windows10 JBOD system, and to make the transition on the cheap, I bought 4x10TB disks to start with. My plan had been to leapfrog the disks from the W10 system into the FN build after their data had been transfer over, thereby expanding the FN build as I went along.

After inserting the 4 initial disks into the FN build, I realized my first folly. I started looking into FN a few years ago, and became more serious about a build right around the time the tweet came out about expanding vdevs (sometime in 2017, IIRC). Classic look before you leap situation. I assumed that functionality was ready, which was a penalizing error in judgement.

At around this same time, I also noticed the warning message that read that expanding a pool meant using only similar vdevs. Second misconception, even bigger mistake. But, I was running out of space in the W10 system and I was anxious to get going with the FN build.

So, instead of waiting another month or two to save up for more 10TB drives and RAM, I just decided to breakup my initial plan of one large pool into 4 separate pools. I bought an additional set of 10TB disks (bringing my total to 6) to complete my first pool, and then started transferring the data over.

I now have 3 pools with 6 drives each (1x6x10TB and 2x6x4TB), and although I've strayed from my initial plan, 3 storage locations is much easier to manage than 11.

In the W10 system, there is now a Storage Space I setup with 3x4TB disks. I use that to record over-the-air HDTV, which is then processed with MCEBuddy. This system also downloads content off of streaming sites (e.g. Netflix and Amazon) with PlayOn.

I haven't been happy with the transfer speeds from one system to the other, so I might start thinking about moving those disks into the FN build. I just don't know yet if the DVR will function properly when writing over a Samba connection.

This new pool would be in addition to housing spares in the W10 system, so I'm looking at needing to add ~ 8 disks or so via some SAS method. I'm fully aware I've taken the "duct tape" equivalent route with all of this, but that's part of the fun for me ;)


Are you trying to add a JBOD?

If you are, keep it simple

You need a SAS Controller (I use a M1015) and a 8087 to 8088 pass thru (If you controller has no 8088 port)

If you don't have a spare 8087 port you simply add a $15 expander


I have one of these and was planning to add another: LSI SAS9211-8i

I'm intrigued by the Expander you referenced. First I've learned of such thing, and I love the price. Perhaps I don't need another LSI card after all...

....

After doing a little bit of research, I have more questions I'll need to answer before moving forward with that solution.

Would you happen to have any pictures of how both of your towers are wired (both inside and out, and to one another)?
 

CraigD

Patron
Joined
Mar 8, 2016
Messages
343
I think this Resource will be helpful Don't be afraid to be SAS-sy

Make sure you SAS card it in IT mode

I don't have any internal photos, I just let FreeNAS do its thing, until it tells me it is sick

An expander is just like a power strip, you can plug more drives, all you do is plug a cable from you SAS card into it (in your case a 8087 cable)

I use this expander it requires power (I use a mining riser in my JBOD)

This expander can be molex powered and is preferred but pricey, If you use this a 8087 to 8088 pass thru will be needed as well as a 8087 cable in the JBOD

Why have 3 pools?

I would use s single pool with a 6-8 wide single vdev, then add vdevs as needed (try this in a VM first)

I doubt you DVR can tell the difference between a local drive and samba

Getting all your media onto your NAS is huge: It is in one place, It is in one place, It is in one place. If a drive dies no data is lost your data online and usable. Local machines often only need SSDs. You can use time machine, window backup, or other program to periodically backup all you devices, restoring after a loss is trivial

I hear you can automate tasks like PLEX, Sonarr, Tautulli, transmission, they work together and will change you life

Have Fun
 

twf85

Dabbler
Joined
Apr 18, 2019
Messages
14
Make sure you SAS card it in IT mode


I'll have to look into that. Is that something you do with a jumper pin? Edit: I will have to flash a different firmware onto the card, I believe.


I don't have any internal photos, I just let FreeNAS do its thing, until it tells me it is sick


That's too bad, but I think I've figured out how it all works now. I ordered one of the HP units you linked to on eBay, and it should be here by Friday.


Why have 3 pools?


Necessity. I mistakenly bought larger capacity drives thinking that I could mix vdevs together so long as the drives within a single vdev were of the same capacity. Kind of a moot point at this juncture, as I already have data in each of the pools.


I doubt you DVR can tell the difference between a local drive and samba


True enough, but my concern is write latency over a Gbe connection. What I'll probably do is change the buffer location to a local SSD, which should resolve any latency issues.


I hear you can automate tasks like PLEX, Sonarr, Tautulli, transmission, they work together and will change you life


I use FileBot, which works great, but I haven't delved into the CLI features for automation. Wouldn't do much good, in my estimation, as there are an awful lot of episodes with air dates instead of "S##E##" formatting. That's likely something to do with NextPVR (or the EPG source, WebGrab -> Tv.Com), but I haven't been brave enough to look for an alternative to it, as the recordings themselves are usually pretty good.
 
Last edited:

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
I mistakenly bought larger capacity drives thinking that I could mix vdevs together so long as the drives within a single vdev were of the same capacity. Kind of a moot point at this juncture, as I already have data in each of the pools.
You can have multiple vdevs in a single pool. Each vdev is limited to the size of the smallest disk, but you can have almost any mix of disks. This is the reason I asked for more information. It is too late to change now, because you would need to move all the data out of the system, delete the three pools and form a new pool with three vdevs. Then move the data back onto the single pool. I have a system at work with ten vdevs. Please review these resources:

Slideshow explaining VDev, zpool, ZIL and L2ARC
https://www.ixsystems.com/community...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/

Terminology and Abbreviations Primer
https://www.ixsystems.com/community/threads/terminology-and-abbreviations-primer.28174/

Why not to use RAID-5 or RAIDz1
https://www.zdnet.com/article/why-raid-5-stops-working-in-2009/

and it might help:

Confused about that LSI card? Join the crowd ...
https://www.ixsystems.com/community/threads/confused-about-that-lsi-card-join-the-crowd.11901/

Then this might also help:

GitHub repository for FreeNAS scripts, including disk burnin
https://www.ixsystems.com/community...for-freenas-scripts-including-disk-burnin.28/
-
Original @Bidule0hm - Scripts to report SMART, ZPool and UPS status, HDD/CPU T°, HDD identification and backup the config
https://www.ixsystems.com/community/threads/scripts-to-report-smart-zpool-and-ups-status-hdd-cpu-t°-hdd-identification-and-backup-the-config.27365/
 

twf85

Dabbler
Joined
Apr 18, 2019
Messages
14
You can have multiple vdevs in a single pool. Each vdev is limited to the size of the smallest disk, but you can have almost any mix of disks. This is the reason I asked for more information. It is too late to change now, because you would need to move all the data out of the system, delete the three pools and form a new pool with three vdevs. Then move the data back onto the single pool.


Oh man, that means my initial assumption was correct. The warning message is awful scary for a first timer...

"Extending the pool adds new vdevs in a stripe with the existing vdevs. It is important to only use new vdevs of the same size and type as those already in the pool. This operation cannot be reversed. Continue?"

I took that to mean either drive capacity or vdev capacity, not drive count. I guess I'm not the only one... But I should've been a little more patient and careful.

Is there anything preventing me from adding a new vdev to the initial pool? That should (theoretically) allow me to migrate/integrate the existing data/disks from my smaller pools into the initial pool, if I am understanding all of this correctly. I don't fancy the extra wear on the drives, but I would prefer it if I only had to keep track of one location (to cut down on duplicates).

The steps would look something like:
  • "pool0" = single vdev with 6x10TB
  • "pool1" = single vdev with 6x4TB
  • "pool2" = single vdev with 6x4TB

  1. Extend "pool0" with new 6x4TB vdev.
  2. Transfer contents of "pool1" (6x4TB vdev) to "pool0".
  3. Disconnect "pool1" and Extend "pool0" with the recently disconnected disks into a new 6x4TB vdev.
  4. Repeat 2 and 3 for "pool2".
Total disks in system would go from 18 to 24.


EDIT:

I'm marking the thread "SOLVED". I don't want to add hot spares anymore, but will use a passthrough to give me cold spares once I have filled up my primary case.
 
Last edited:

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
  1. Extend "pool0" with new 6x4TB vdev.
  2. Transfer contents of "pool1" (6x4TB vdev) to "pool0".
  3. Disconnect "pool1" and Extend "pool0" with the recently disconnected disks into a new 6x4TB vdev.
  4. Repeat 2 and 3 for "pool2".
Total disks in system would go from 18 to 24.


EDIT:

I'm marking the thread "SOLVED". I don't want to add hot spares any more, but will use a passthrough to give me cold spares once I have filled up my primary case.
Sorry I didn't reply sooner. This does look to be a workable process. If you have any other questions, please ask. You can expand your pool extensively. I don't recall if I said it previously in this thread, but I have FreeNAS systems at work with more than 60 drives attached by external SAS connections.
 
Top