Is 24 drives in RaidZ3 with Toshiba N300s safe?

[ezekiel.incorrigible]

You can adjust the size of the "buffer" that ZFS will allow to be fully of dirty (uncommitted to pool) data, but unless that's on a stable storage device (eg: NAND flash) you also have to accept the risk of losing that much data in case of a crash.

There's also the issue of how quickly you're going to "fill" and "empty" that buffer. Assuming an infinitely large "buffer", if you have a 10Gbps connection and shuttle in a 10GB file, that will take roughly 10 seconds. But you'll only be able to "empty" it at the pool's write speed, whether that's 800MB/s as proposed above, or 400MB/s if things get full and fragmented over time. If the writes are spaced out far enough to let the buffer empty, cool. Nothing bad happens. But keep leaning on it too hard, and you'll fall back to disk speed - and you'll also be impacting the reads while it's doing the writes.

Writes will be relatively infrequent so the buffer being able to flush itself shouldn't be an issue - I would also be happy to risk losing the 'dirty' data in the case of a powercut with sync=never - but if there is a powercut and the system is halfway through an asynchronous write of dirty data, straight from RAM, is there an elevated chance of losing my whole pool versus if I had left sync=default?

 

[danb35]

all of the suggestions in this thread ignore the advice that RAIDZ2 vdevs "should have the total number of drives equal to 2n + 2. (ie 4, 6, 10, etc)"

That's because that advice is long since outdated, now that the use of compression is standard.

 

[HoneyBadger]

Writes will be relatively infrequent so the buffer being able to flush itself shouldn't be an issue - I would also be happy to risk losing the 'dirty' data in the case of a powercut with sync=never - but if there is a powercut and the system is halfway through an asynchronous write of dirty data, straight from RAM, is there an elevated chance of losing my whole pool versus if I had left sync=default?

Provided that you set sync=disabled on the dataset, and not the pool itself, there shouldn't be any additional risk. That said, unless your remote application is requesting a sync write in the first place, there's no need to make the change at all.

Assuming a situation where you write from the remote system, the buffer is "half empty" and then there's a crash, the files themselves will be corrupted (because they won't be fully written) and you'll have to delete the fragments and re-copy from the source. The pool itself will be fine though, because any transaction that was in progress will be considered "incomplete" and discarded.

As far as a power loss, your UPS should be able to ride it out long enough to commit the contents of RAM to storage, and then shut down after a few minutes to avoid any additional data being stored.

You are planning on a UPS, right? ;)

 

[ezekiel.incorrigible]

OK, this has all been very useful. re. a UPS - I'm certainly going to make the case for one and see how much money my boss feels like spending - he's got tight purse strings though so he may not be convincible.

 

[HoneyBadger]

Find the "nice to haves" in the budget and remove them until you can afford a UPS, IMO that's a "must have" for any business-oriented solution.