SSD Only build, where are we at?

[jgreco]

Yes, they actually are Ultra II's. I got them on the same black friday sale. In Canada they were a lot more than $200, but still very well priced.

Using the 480GB Intel 535 as an example, it was released Q2 2015 at $229. Black Friday, it was $149 but normally around $179. Today it's easily found for $139 everyday.

We've been seeing a free fall in SSD prices over the last ~~ two years and it doesn't really look like it'll slow anytime soon.

Mine are mounted internally, so no blinky lights at all.

That has nothing to do with anything. There aren't any lights on the SSD. The DAS signal is on pin 11 of the power connector. That can be connected to an LED on the backplane, but could also be wired to a case light if you're ambitious.

I thought I had found some kind of write endurance specs at some point, but I'm not totally sure.

I had some vague recollection of that too, maybe, but if it was, it was something sort of pathetic. Or I might be thinking of one of the other cheap brands.

 

[fracai]

Feel free to consider it FUD... but I've seen quite a few system (both iXsystems and custom built) and I can tell you that there are reasons to avoid some models, firmwares, etc. I'm just not going to get into the details because it's very complicated.

Isn't this the place to get complicated and dive in to the details?

 

[jgreco]

Happened across this a little bit ago:

http://itknowledgeexchange.techtarget.com/storage-soup/netapp-no-ssds-have-worn-out/

Now, granted, NetApp's buying the better grades of MLC flash SSD, but it is intriguing that they're just not seeing wearout as a big issue. That seems to combine pleasantly with what's being seen on the lower end of the SSD continuum,

http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead

 

[jgreco]

Isn't this the place to get complicated and dive in to the details?

It's hard to dive into FUD.

 

[titan_rw]

That has nothing to do with anything. There aren't any lights on the SSD. The DAS signal is on pin 11 of the power connector. That can be connected to an LED on the backplane, but could also be wired to a case light if you're ambitious. :)

I realize that. Mine are not connected to a backplane or anything. A power supply connector plugs directly into the drive, and the sata cable goes directly to the MB. I had assumed that in most cases this pin is wired to a light on the drive tray / caddy that is visible from the outside.

I had some vague recollection of that too, maybe, but if it was, it was something sort of pathetic. Or I might be thinking of one of the other cheap brands.

I did find this:

http://forums.whirlpool.net.au/archive/2340493

Drive is rated for only 500 PE cycles. They tested a 120 gig version to 2000 PE cycles before any issues. Even on the 120 gig version that's well over 200 TB written.

I checked min / avg / max PE cycles on my drives. They're all 0 / 0 / 2.

Granted I've only been using them for about 4 months now. But I'm pretty sure they'll be fine.

 

[jgreco]

Yeah, but if it's actually only rated for 500 cycles, then that's only about 60TBW. I'm sure that's still just dandy for a FreeNAS user who isn't actively thrashing on the pool with torrents or whatever, of course.

That does scale reasonably well for the larger units, so I'm guessing there's nothing significant to worry about if you take some SSD's and want a nice general purpose fileserver that has a mix of long term and short term storage.

 

[diskdiddler]

People are already doing this for jails. I don't really see the difference.

MASSIVE difference from dumping your jails on an SSD which may not even be in a RAID configuration and actually your entire array of data. FreeNAS (or perhaps, any NAS) does all kinds of random writes, for whatever reasons (windows drive mappings, last modified updates, who knows?) endlessly, just always little tiny writes, often.
I personally like the idea that the NAS itself is 'smart enough to outsmart crappy disks' if Cyberjock clasess SSDs as crappy disks due to potential reliability issues, fine - but I would like the NAS to outsmart that and as long as I replace disks fast enough, with enough redundancy, I'm safe (highly prefer 3 year warranty disks, of course)

I'm *REALLY* dying for someone to buy those 4TB Crucial $500 SSDs when they are released and do a nice big, 6/8 SSD pure SSD array. Just to hear the results.
Since I gave my NAS a life injection of what I suspect is 12 to 30 months, I'm really, really hoping my next NAS is pure SSD, even if it's 'cheap crappy storage level' SSD and not high performance SSD.

 

[jgreco]

MASSIVE difference from dumping your jails on an SSD which may not even be in a RAID configuration and actually your entire array of data.

Okay, sure, dumping your jails on an SSD without redundancy is definitely a massive difference, but that's more a "massive difference" in the direction of "that's a stupid thing to do because you're at higher risk of losing data" type thing, and even there we're not hearing of problems with people doing that.

An "entire array of data" is basically not saying anything; the difference between 24 drives and 2 drives is just a factor of 12x.

FreeNAS (or perhaps, any NAS) does all kinds of random writes, for whatever reasons (windows drive mappings, last modified updates, who knows?) endlessly, just always little tiny writes, often.

You should be shutting those off, even on an HDD array. Especially the atime updates. That's just crap grade metadata.

I personally like the idea that the NAS itself is 'smart enough to outsmart crappy disks' if Cyberjock clasess SSDs as crappy disks due to potential reliability issues, fine - but I would like the NAS to outsmart that and as long as I replace disks fast enough, with enough redundancy, I'm safe (highly prefer 3 year warranty disks, of course)

I have no idea what that's all about. We use crap grade HDD with ZFS all the time. What exactly do you think desktop, green, and NAS grade hard drives are? Hint: definitely not top of the line. The whole point of RAID is "Redundant Array of Inexpensive Disks" (and yes I know that people later changed that word, manufacturers HATED the implication!). My SSD pool is based on 850 Evo, and my hesitancy to expand it has NOTHING to do with fears about ZFS and SSD, but rather just a matter of whether or not it represents the direction we need for storage here.

I'm *REALLY* dying for someone to buy those 4TB Crucial $500 SSDs when they are released and do a nice big, 6/8 SSD pure SSD array. Just to hear the results.
Since I gave my NAS a life injection of what I suspect is 12 to 30 months, I'm really, really hoping my next NAS is pure SSD, even if it's 'cheap crappy storage level' SSD and not high performance SSD.

Did Crucial announce something like that? I know they'd been pushing the BX200 as a lower cost SSD, hoping to cut into the HDD marketplace, but the price point is nowhere near what it needs to be yet. 960GB for ~$200 is still 4x the cost of 2TB for $100 (2.5" Samy M9T).

 

[diskdiddler]

I was actually incorrect, it's a Mushkin SSD. They've got a 2TB now but the 4TB was announced at CES @ $500 (which seems crazy to me but hey, more power to them. Surely it's TLC, not MLC?) I can't find it listed anywhere yet tho.

Let's pretend I get a windfall exceeding $100k (I'm very budget conscious) I'd definitely be willing to drop several thousand on 6, 8 or even 12 of those $500 drives. I'm really, REALLY tired of my NAS being noisy and hot, especially in summer. I imagine only 2 low speed decent quality fans would keep the whole thing cool if I upgraded to SSDs and the performance would be lovely.

I'm dying for this to be viable soon.
As for the rest of your posts on this, I completely agree with you, although I thought I saw someone else agree with Cyberjock at some point. It does feel illogical and FUD though, to my knowledge a disk is a disk. The controller / system should take care of the rest. (he could be right, albeit paranoid as heck... perhaps 1 or 2 SSDs are poorly designed and wouldn't like ZFS at all?)

 

[jgreco]

I was actually incorrect, it's a Mushkin SSD. They've got a 2TB now but the 4TB was announced at CES @ $500 (which seems crazy to me but hey, more power to them. Surely it's TLC, not MLC?) I can't find it listed anywhere yet tho.

Ah, that thing. Well, we'll see. Its best role in life might be that it presses prices down more. Look at what's happened to the Intel 535 in the last year. Introduced at $229 in 2Q15, down to $179 last fall ($149 Black Friday), now usually around $139 retail as we enter 2Q16. It's amazing.

Let's pretend I get a windfall exceeding $100k (I'm very budget conscious) I'd definitely be willing to drop several thousand on 6, 8 or even 12 of those $500 drives. I'm really, REALLY tired of my NAS being noisy and hot, especially in summer. I imagine only 2 low speed decent quality fans would keep the whole thing cool if I upgraded to SSDs and the performance would be lovely.

The FreeNAS Anthem (to the tune of Huey Lewis & The News' awesome "I Want A New Drug")
"I want a new NAS,
one that won't cook my disks,
one that won't make me cover my ears,
or make me feel my data's at risk.

I want a new NAS,
one that serves it from flash,
one that won't make my wallet too light,
or take up power immense.

One that won't make me nervous,
wondering what to do..
One that makes me feel like I feel when I'm with you..
When I'm alone with you!"

I'm dying for this to be viable soon.
As for the rest of your posts on this, I completely agree with you, although I thought I saw someone else agree with Cyberjock at some point. It does feel illogical and FUD though, to my knowledge a disk is a disk. The controller / system should take care of the rest. (he could be right, albeit paranoid as heck... perhaps 1 or 2 SSDs are poorly designed and wouldn't like ZFS at all?)

Of course. There are disks like that too.

It's very much like buying a car. Just because we've had stinkers like the Ford Pinto doesn't make every car bad, doesn't even make every inexpensive car bad.

If we never make any progress because of the fear of "what if," then we never make any progress. We've always said that you should back up your data.

 

[David E]

That has nothing to do with anything. There aren't any lights on the SSD. The DAS signal is on pin 11 of the power connector. That can be connected to an LED on the backplane, but could also be wired to a case light if you're ambitious. :)

Following up on this thread, a couple questions:

• Is DAS as defined by pin 11 of the power connector the standard pin that is used to blink lights on bays in Supermicro, etc enclosures? Is there a really cheap enclosure that could be bought and used to test drives for this capability?
• Is there a list anywhere of what SSDs support DAS? This seems like it will be increasingly useful to folks.

 

[jgreco]

Following up on this thread, a couple questions:
Is DAS as defined by pin 11 of the power connector the standard pin that is used to blink lights on bays in Supermicro, etc enclosures?

Yes. Sorta. Kinda. In the older days, some backplanes actually allowed for discrete signals (i.e. 3Ware 9550 to the AIC RMC4E) as an option.

Is there a really cheap enclosure that could be bought and used to test drives for this capability?

Perhaps. I'm not actually sure what the state of the art is, here, but it seems to have headed towards pin 11, so what I'd probably try is any "4x2.5-to-5.25" bay adapter that has LED's. Icy Dock makes what's reportedly supposed to be some really nice stuff, and if and when I ever actually rebuild my desktop workstation, that's probably the direction I'm going.

http://www.icydock.com/goods_cat.php?id=120

Is there a list anywhere of what SSDs support DAS? This seems like it will be increasingly useful to folks.

Yeah, well, I can tell you the 850 Pro is "locked on", the 850 Evo seems to have a normal DAS, the Intel 535 does a slowish blink, and a lot of the older SSD's do nothing at all.

 

[joeschmuck]

You know, I didn't know about the pin 11 on the power connector being the blinking activity light, it makes more sense to me now. I thought it was a bit more complicated than that, like the drive bay would sense data flow and thus lite the light.

In my main computer I have a 3.5" 5 disk drive bay with individual power on/off controls and it makes things very nice. Originally purchased to house a few removable 3.5" hard drives, I have migrated to three SSDs and two empty bays. Two SSDs are for normal operation whilst one drive is for doing risky things so I turn off the first two and turn on the third and boot off that drive, it keeps my first two drives very safe. Even though I really like VMWare Workstation, you just can't be absolute in isolation so this was my solution, plus I have a second LAN network to isolate that aspect as well. I removed the fan (80mm) from the 5 bay drive which was making too much noise and I want a silent machine. This does not impact the heat generated by the SSD's because the case pulls air in across the drives and these don't really produce much heat in the first place, plus they are in 3.5" drive bays so the drives are not stacked on each other creating a lot of heat.

I'll admit, I do like the Icydock Tough Armor MB998SP-B (I'm drooling). I still am considering a full SSD FreeNAS pool system but not until I need to replace my hard drives, and then SSD's may still be out of my financial reach for the capacity I need. I hope my drives last another 3 years, then SSD's should be obtainable.

 

[jgreco]

Actually pin 11 is a dual purpose pin. It also controls automatic spinup.

In the old days, there were a few (more) schemes for drive activity, including monitoring the data flow, or totally out-of-band signalling (as mentioned above), I2C, and we still use SGPIO. SGPIO is probably the alternative to look into if you've got a fussy device like the 850 Pro, but honestly I'd rather just find devices that "work right" with 11 because I hate magic "make it work in a subset of cases" workarounds.

 

[David E]

Perhaps. I'm not actually sure what the state of the art is, here, but it seems to have headed towards pin 11, so what I'd probably try is any "4x2.5-to-5.25" bay adapter that has LED's. Icy Dock makes what's reportedly supposed to be some really nice stuff, and if and when I ever actually rebuild my desktop workstation, that's probably the direction I'm going.

http://www.icydock.com/goods_cat.php?id=120

Good pointer, those look nice, I'll have to snag one for testing.

Yeah, well, I can tell you the 850 Pro is "locked on", the 850 Evo seems to have a normal DAS, the Intel 535 does a slowish blink, and a lot of the older SSD's do nothing at all.

Fabulous to hear the 850 Evo works, that drive plus the Mushkin Reactor's 1TB and forthcoming 2TB drives seem to be at the right performance/size/price location on their respective curves. Any idea on the Reactor? If not, I may pick up one to find out.

 

[jgreco]

No, but feel free to send me one and I'll, um, test it extensively....? Heh.

 

[David E]

Another question.. how to create 3 way encrypted mirrors if the UI doesn't support 3 way mirrors?

 

[jgreco]

Probably "very carefully." I don't think I'd risk it.

 

[cyberjock]

Another question.. how to create 3 way encrypted mirrors if the UI doesn't support 3 way mirrors?

You should be able to do 3 way encrypted mirrors. I can't try it right now, but I'm 99% sure you can do it. Just drag the vdev to make it 3 disks wide, then choose mirrors.

 

[David E]

You should be able to do 3 way encrypted mirrors. I can't try it right now, but I'm 99% sure you can do it. Just drag the vdev to make it 3 disks wide, then choose mirrors.

No dice unfortunately

https://bugs.freenas.org/issues/14599