SOLVED 9.3 USB vs SSD vs SATA DOM?

[diedrichg]

As I have gleaned from reading several posts in this section, 9.3 has several new features that will require larger and faster boot drives.

Would anyone like to weigh in on the advantages/disadvantages of sticking with a 4GB+ USB in a standard home setup and then the advantages/disadvantages of switching to a SSD vs SATA DOM, please.

Edit: Should my next upgrade for my SUPERMICRO X10SLM-F-O be a SUPERMICRO SSD-DM016-PHI SATA DOM 16GB, Read: 285 MB/s, Write: 75 MB/s?

 

[BigDave]

I agree with diedrichg, it's now time for you heavy hitters to weigh in. Jordan made clear in a recent post
that usb sticks are no longer used in IX system builds. I say lets have this on the table so we can all know
the choices we have for the coming release of 9.3.

 

[Norleif]

USB advantage:
- Leaves more SATA ports available for user data drives. (RaidZ2 instead of RaidZ1, L2ARC, ZIL... It's not hard to find a good use for spare SATA ports...)
- Cheaper then SATA SSDs
USB Disadvantage:
- Slow! *

* Any operation writing to, or reading from the OS file system (OS install, upgrading and booting too name a few) is going to be noticeably slower on USB but the extra SATA port could help increase the performance of the file system serving user data. This is, after all, the core task of a NAS.

In my first FreeNAS I have used up all the SATA ports for data drives, so running the OS from USB is the only option.
In my second FreeNAS I have 3 spare SATA ports and I only considered running the OS from SSD the amount of time it took to hit the "Buy" button on one extra data drive for that money.

 

[Mlovelace]

With the amount of writes expected to the boot device in 9.3 I'm going to go with a mirrored pair of sata doms. I can't see using a USB thumb drive for a boot device going forward.

 

[diedrichg]

Thank you everyone. Those are all valid points to consider if upgrading to 9.3.

 

[mjws00]

I kinda like having a dozen bootable usb sticks around for testing and messing with things. I'll be the guinea pig and try to wear one out.

That said, for a one and done scenario. Or even something not easily accessible. Sata dom seems like a nice touch. Unfortunately there seems to be no local supply, demand, or selection.

I have lots of small old ssd's that can be repurposed as well. So really couldn't care less. I do LOVE that we might be able to utilize a little more space. Even if it's just for logs and an additional driver, or utility.

 

[jkh]

We have always recommended SATA DOMs (we use these) over USB sticks, even with 9.2.1.x, and will continue to do so. An SSD would also be an acceptable boot device, but it's really a waste of a perfectly good SSD. USB sticks are OK only so long as they are high quality ones (which disqualifies about 95% of the ones I've seen). I wouldn't use one of these, in other words.

 

[titan_rw]

I have one of my NAS's on 9.3. I have 4 old gen1 32gb (jmicron based) ssd's from like 2008. I'll be using two in each freenas. The one that is running 9.3 is running off of a mirrored pair of them right now. Not the speediest drives around, but faster and more reliable than usb flash drives. And much more room than the 4 gig usb drive I had it booting off of before.

 

[JanJessen]

Hi Jkh

Where do you guys buy the SATA DOMs you use ? I'm from Denmark and i can't seem to find them anywhere.

Regards
Jan

 

[pjc]

With the amount of writes expected to the boot device in 9.3 I'm going to go with a mirrored pair of sata doms.

How do you fit the DOMs (especially 2) without blocking adjacent SATA ports? At least on my MB, the ports are arranged in a 2x3 grid, so maybe there are some short extension cables?

(See the X9 picture at http://supermicro.com/products/nfo/SATADOM.cfm to get an idea of the issue.)

 

[Fraoch]

I wouldn't use one of these, in other words.

But, but...it's Hello Kitty!! Think how much cuter and prettier the inside of your server will be!:) You could redo your case to look like this! http://media-cache-ec0.pinimg.com/736x/ec/a9/40/eca94061543be5ccbb33506a70c9cf41.jpg

In regards to SATA DOMs, their availability is shockingly poor. There's just nothing available. I see that I can buy a grand total of TWO here, a 16 GB one and a 64 GB one, both Supermicro branded. The 16 GB one is priced at about the level of a 64 GB consumer SSD, and the 64 GB one is priced at the high end of a 128 GB consumer SSD...almost at 256 GB levels. It may be a waste of an SSD, but it's actually more economical to get a 64 GB SSD. You can still find 20 - 40 GB SSDs at even better prices.

My old 64 GB Crucial M4 is working very well for FreeNAS and should be great for 9.3. I wasn't doing anything with it before so I might as well put it to use. It's not the fastest SSD and I've since moved on to much faster 240/256 GB SSDs on the desktop, but it's always been reliable and should be way, way more reliable than a USB key, also much faster. I'm thinking of adding a second 64 GB SSD mirror for 9.3. I presume the normal mirroring rules apply - same size or the size of both is limited to the smallest size in the array? I assume speeds don't have to be perfectly matched?

 

[fracai]

With the switch to booting from ZFS, why not support putting the OS on your pool?

 

[diedrichg]

Well, I pulled the trigger very late last night and purchased the Supermicro 16GB. By the time I had buyer's remorse and tried to cancel the order, it was too late! I was thinking the same thing as Fraoch about a cheap 64GB SSD but ultimately I am stuck with an expensive non-volatile, very small piece of memory. The upside: it's SLC with a very long expected lifespan.

 

[mjws00]

Well, I pulled the trigger very late last night and purchased the Supermicro 16GB. By the time I had buyer's remorse and tried to cancel the order, it was too late! I was thinking the same thing as Fraoch about a cheap 64GB SSD but ultimately I am stuck with an expensive non-volatile, very small piece of memory. The upside: it's SLC with a very long expected lifespan.

You forgot slow ;)

 

[diedrichg]

Well, you made me feel like a dummy so I RMA'd it. You are right though, even though the Read is respectable on the Supermicro, the Write is not.

 

[mjws00]

I was just jealous or bitter that I can't just grab one off the shelf. :) I'd use em if I was selling Mini's or TrueNAS, the wink was to denote I was being a smart @ss. :)

 

[Mlovelace]

If I had spare bays in my server I'd consider going with mirrored SSDs but as it is I'd rather put data disks in those. So, sata doms are a reasonable alternative and better performance/endurance than USB thumb drives.

 

[Ericloewe]

With the switch to booting from ZFS, why not support putting the OS on your pool?

Because it's a bad idea to mix the OS and data.

 

[L]

With the switch to booting from ZFS, why not support putting the OS on your pool?

One of the nice things about openzfs in general is that you can export import pools from different distros/releases/architectures.

 

[pjc]

So what is the activity profile of the new boot volume? What kind of data is it writing? At what frequency? How critical is it -- should it be mirrored? Or can it easily be nuked/replaced with minimal downtime?