Expanding FreeNAS build to include mirrored SSD boot devices and a mirrored ZIL for zpool

[Mr Splat]

Hi guys,

I am in the middle of contemplating expanding my current FreeNAS build owing to getting the ominous "CRITICAL: The boot volume state is ONLINE: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected." from my bootable USB sticks. This happened after powering my NAS up after 6 months (I recently returned from a long trip out to Asia)

From reading around these forums on numerous occasions I know that common knowledge now dictates that I should really be using at least a single SSD for the freenas operating system, I also thought that if I'm going to go through the trouble of doing this (I'll explain the "trouble" part of this soon as I know it's not hard to replace the USB sticks with an SSD and then to restore my latest config) that I might well be best served to also add a mirrored ZIL for my current 5x3tb (RAIDZ2) zpool, I have no need to add an L2ARC pool to my build as I don't have anywhere near enough RAM for a start, nor do I believe that I will be getting anywhere near 64 gigs of the stuff even if I do decide to expand (Thank you very much Cyberjock for your useful powerpoint presentation "explaining VDev, zpool, ZIL and L2ARC for noobs!").

However this is where I run into "trouble"... My current build uses a Supermicro X10SLH-F-O, and 5 of its six sata-III ports are used for my raidz2 pool, the sixth is used for the system dataset (I had a spare 32GB SSD lying around when I built my box). Therefore I went looking for a way to expand the number of sata ports available to me.

Common consensus suggests that an "IBM m1015 ServeRaid" card is the piece of hardware I'm looking for as it provides a decent amount of extra sata ports when paired with SFF-8087 cables whilst also being relatively inexpensive. I would then have to cross-flash it to IT mode to allow me to use it for my purposes.

Now, Amazon and Ebay seem to be the places to go for these things and I found the following:

https://www.amazon.co.uk/Lenovo-Thi...&ie=UTF8&qid=1483572282&sr=1-2&keywords=M1015

The above doesn't come with a picture, so I went looking around for some documentation to match up with it and I found this: http://www.lenovo.com/images/produc...ts/serveraid_m1015_sas_sata_controller_ds.pdf which I believe to be the card I am looking at above. The problem is this card seems far, far, FAR too cheap, considering these are being shipped from China on ebay for ~£50-60. However the card is being sold directly by Amazon, not a 3rd party seller with a dodgy rating. So I came here to ask:

1. Is it surprising to see these cards going so cheaply?
2. If so, what could be wrong with this lenovo ThinkServer IBM M1015 that should be putting me off buying it?

I'm no hardware guru and all I can really tell you about Lenovo is they're known for being fairly solid Laptop manufacturers but they're also gaining a somewhat decent reputation in other areas.

If my suspicions are correct about the above, I suppose I will have to go looking at the ones like this: http://www.ebay.co.uk/itm/IBM-Serve...771411?hash=item4895cb42d3:g:ngcAAOSwEzxYPb8U

For the SFF-8087 cables I have made the assumption that these: https://www.amazon.co.uk/gp/product...act_title_3?ie=UTF8&psc=1&smid=A1NKH53T1DU028 are more than good enough for what I need.

Finally, I was also thinking that these SSD's would be suitable for a mirrored ZIL and mirrored boot devices: https://www.amazon.co.uk/gp/product...act_title_1?ie=UTF8&psc=1&smid=A3P5ROKL5A1OLE plus an extra to replace to ageing system dataset SSD I have because I am thinking of getting my build to act as a Domain Controller in the near future.

The above SSD's look suspiciously like either direct competitors to Intel's S3700's that Ericloewe recommended ("You'll definitely want something with backup capacitors. Something like the Intel S3700") here: https://forums.freenas.org/index.php?threads/ssd-recommendation-for-slog-device.24343/ or otherwise I would even be suspicious that they're rebranded... which is currently reinforcing my belief that they are good zil candidates.

that Ericloewe recommended ("You'll definitely want something with backup capacitors. Something like the Intel S3700")

So, now that you know the full story :D could anyone enlighten me on the lenovo ThinkServer IBM M1015? If it's not raised anyone else's suspicions could you also please advise whether I should leave my zpool plugged directly into my motherboard and use the spare sata port for one of the bootable SSD's whilst plugging the other SSD's such as the mirrored ZIL, second boot ssd and the new system dataset hanging off the M1015?

Any help provided is really appreciated!

 

[Spearfoot]

@Mr Splat, I've deduced, perhaps incorrectly, that you reside in the United Kingdom. If so, greetings from one of your cousins here across the pond!

I would be leery of purchasing an IBM M1015 on eBay from a Chinese source. I've purchased an M1015 on eBay for $85 here in the USA, which I believe is roughly 69 quid.

That said... are you certain you need a SLOG device in your system? These typically aren't necessary unless you're serving up iSCSI or NFS storage to a VMware ESXi server.

Not sure I understand your use of an SSD strictly for the 'system dataset'. I believe you could simply boot FreeNAS from an SSD installed on your 6th SATA port and not have to install an HBA at all.

 

[Mr Splat]

Hi guys,

Just a quick update here:

Firstly, I stuck an order in for the lenovo IBM M1015 and corresponding SFF-8087 cables, at the time I wrote my original post the card was priced at £23 (I have stuck evidence to the bottom of this post) so I thought it was worth a punt, however since then Amazon have ran out of stock, more are due in but the price has jumped up to £53 . I may have got exceptionally lucky, I will wait and see.

I should also add I am considering adding an extra one of those Transcend SSD3700's to my NAS as a dedicated SLOG for the ZIL I want to add to my current pool, I do have a use case for a ZIL I am not adding it for the sake of adding it, I currently have a number of NFS shares being used by a small number of users as well as a number of raspberry pi's writing to it, I also have plans to tinker with it and possibly have more things using these shares. Needless to say, I have just shut everything down safely for the time being whilst I get new boot devices sorted out.

Finally I feel I should apologise for this thread's misleading title, It really should represent the fact this is down to a lack of sata ports available to me and that I was (or possibly still am) a bit lost in terms of the M1015 card that I found.

 

[Mr Splat]

Hi Spearfoot,

I am indeed from the UK!

Yeah, just from reading around I thought I may need a ZIL attached to my storage pool in particular because the house I'm in at the moment has the rare power outage which I think is probably one rare outage too many in FreeNAS-land, I do have an UPS for these!. I was thinking a couple of cheap 32GB SSD's would be good for the job. As you have probably surmised I have indeed just got a FreeNAS box built for a typical small home tinkering lab and I have considered whether I could ditch the system dataset SSD for an SSD boot device, but as I said I am thinking of messing about with a domain controller, and all the samba settings would be stored on the system dataset. The usb sticks failing in my current build and the many posts on these forums saying to avoid usb boot devices like the plague have made me think I should really try to keep an SSD for my system dataset whilst also having at least one as my boot device.

I will admit when I read back on some of this it probably reads like massive overkill for what I'm doing, but then again the SSD's themselves are pretty cheap so I am just leaning towards being a bit over-zealous rather than not having enough and regretting it later.

I take it when you say "These typically aren't necessary unless you're serving up iSCSI or NFS storage to a VMware ESXi server" you're imagining the use-case being in a small to medium sized business with a VM stack writing to the FreeNAS box a few thousand times per second?

Thanks for the response!

 

[Mr Splat]

I have also realised I may not be totally understanding the relationship between ZIL and SLOG, I'm going to go have another quick read. I thought that you needed a dedicated SLOG to get rid of the performance impact associated with having a ZIL stuck to a storage pool (i.e. they were two separate things), however I'm getting the feeling they're actually practically the same thing the SLOG is the ZIL that has in fact been separated from the pool it is caching for which is what removes the ZIL's performance impact from the pool's spinning disks...

 

[Mr Splat]

Hi @Spearfoot,

It does indeed look I misunderstood what I've been reading about the ZIL, I don't think I have enough on my network at the moment to actually have a noticeable performance impact on my build as it stands, furthermore I'm fairly certain I can add a dedicated ZIL as and when required if I find the tinkering I'm doing does indeed lead to a performance impact being noticed.

From what I can see, I can have redundant boot devices, but strictly speaking I won't see a great deal of benefit from them in a home environment, at best if I lose a lone SSD boot device it shouldn't be a problem for me to add a new one in its place and then upload my latest configs which I keep backed up regularly as it is.

However, I still feel I would prefer to have a dedicated system dataset available for a domain controller. in particular I see no harm in having the M1015 available to allow myself to do this, particularly when it seems I've been able to pick one up on the cheap.

I should add that of my current pool, I've eaten about 6.5/9TB so I could leave myself some room to expand by moving all my HDD's to the M1015 and leaving boot devices/system datasets/slogs on the motherboard.

Anywho's I think I'm going to have some fun couple of day's tinkering.

I realise I have probably answered my all of my own questions but you got me reading around zils much more carefully Spearfoot, cheers!

 

[melloa]

For the SFF-8087 cables I have made the assumption that these: https://www.amazon.co.uk/gp/product...act_title_3?ie=UTF8&psc=1&smid=A1NKH53T1DU028 are more than good enough for what I need.

I suppose you know those don't have a latch.

Not sure if those are better: https://www.amazon.co.uk/dp/B01HZFUNVS?psc=1, but are a little cheaper.

 

[Mr Splat]

Hi @melloa,

Yeah when I went to buy those cables I did think "Hmm, latches would just be nice to have" and lo and behold I found the exact same ones you pointed out! Being cheaper was a little bonus too.

Cheers!