two pools to reduce fragmentation on cold storage?

[seldo]

I have today a mirror of 2x2TB (1x WD Red CMR + 1x ST IronWolf CMR).
I have a 6 TB WD Red CMR backup drive connected to USB.
My machine is an HP microserver Gen8, which I picked due to its size (it fits perfectly where I cannot put anything else).

Here's my working pool:

Code:

bob: Lightroom DBs# zpool list -v MAIN_POOL
NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
MAIN_POOL     1.81T  1.54T   282G         -    49%    84%  1.00x  ONLINE  -
  mirror      1.81T  1.54T   282G         -    49%    84%
    ada0.nop      -      -      -         -      -      -
    ada1.nop      -      -      -         -      -      -

Sizes below are consolidated as if I had enough space on the NAS.
Today it is spread across the NAS and two drives):


I have two kind of usage:
- cold storage
sorted pictures: ±300 photo location directories
I do not access them often, and rarely update them either
If the files are there it means I am done with them

• original files & edit DBs (currently 1.1 TB):
  • tens to thousands of original photos files (RAW ~35 MB/file)
  • Lightroom catalog (up to 1 GB)
    .lrcat extension, which seems to contain an SQlite DB
  • Lightroom preview data directory (up to a 5/10 GB)
    .lrdata (contain hundred of files)
• there are exported files:
  • JPGs sorted in directories
• Aperture/iPhoto (230 GB + 50 GB):
  • For these two, I might move them out of the NAS and keep them on an disk I connect by USB
  • An old Aperture DB (many many small files including links)
    • Associated photo files in a separate directory
  • iPhoto DBs (many small files including links)

- WIP directory (900 GB if my count is right... but there might be duplicates)

• it's an area where I dump all my unsorted RAW until I am done sorting and editing
• the files there are usually moving between my computer and the NAS
• I copy them locally, cull and transfer back to the NAS
  • only after I export the files I move everything to the location described above
• I believe these movements create a lot of fragmentation

Recap on sizes:

• cold
  • LR: 1.1 TB
  • Export: 20 GB
  • Aperture/iPhoto (230 GB + 50 GB)
• WIP: 900 GB

Since I'm moving systems, I am also taking time to rethink my pools (and backup strategy...):

• create two pools instead of one:
  • a pool for the cold storage
    • should I pick 6 (or 8?) TB for the first mirror?
  • a pool for the WIP
    • WIP can stay in a 2 TB mirror

Splitting would give me the flexibility to increase the cold storage pool as it grows and also reduce the fragmentation caused by WIP.
I am still undecided if I should do a 3-way mirrors in the cold storage pool (at least of the original pool).
Also, I read that ZFS is better performing with lower occupancy.
However, what if my cold storage has little fragmentation: could I fill it more (like 70/80% before increasing the pool?)
That would decide me if I should go with 6 TB or 8 TB for the first mirror (that's a ~100 euros difference between 2x6 TB and 2x 8TB)

For the time being, I'd still backup to my 6TB USD drive until I figure out how to grow out of my 4-bay system.
I'd for now most likely backup both pools on the same backup drive.

I'm also thinking how to grow: new enclosure or complete new system?

 

[Chris Moore]

84%

Too Full

I believe these movements create a lot of fragmentation

Yes

Also, I read that ZFS is better performing with lower occupancy.

Yes. You would want to keep the pool less than 50% to get better performance. Performance is particularly bad with the pool over 80%.

I'm also thinking how to grow: new enclosure or complete new system?

I would suggest a single pool with more, larger capacity, drives and a system that can accommodate those additional drives. I use RAIDz-2 to make the system more resistant to drive failure.

Why not to use RAID-5 or RAIDz1
https://www.zdnet.com/article/why-raid-5-stops-working-in-2009/

 

[Chris Moore]

cold storage

Sorry. Strike what I said before about a single pool. I wasn't thinking cold storage. Yes, two pools makes good sense. The cold storage pool would maintain very low fragmentation as you would basically write to it one time and the data would never or very infrequently change.
Just be wary of the new SMR drives that are being sold as they don't work well with ZFS, particularly if you need to replace a failed drive.

 

[seldo]

Too Full

Yes. You would want to keep the pool less than 50% to get better performance. Performance is particularly bad with the pool over 80%.

I was going for 2x6TB for the coldstorage, but I changed that for 2x8TB.
This will give me a few years before I reach 50%

Why not to use RAID-5 or RAIDz1
https://www.zdnet.com/article/why-raid-5-stops-working-in-2009/

I've read your message above with the two pools. Glad to hear two pools make good sense!

About the coldstorage pool:
Using other kind of RAID, and with my wish of having two pools, I am 4-drive enclosure limited at the moment (+ backup as USB...)
I've read a bit on the different kind of RAID and the RAID10 seems easier for me to handle (2/3 disks/vdev, simple & fast resilver).
It can also start with two drives (+ backup) which fits the space requirements I have.
I guess I will bring back this idea the next time I need to grow the NAS storage.

Just be wary of the new SMR drives that are being sold as they don't work well with ZFS, particularly if you need to replace a failed drive.

I’ve read about this. I checked my current drives and they all seem to be CMR and not SMR.
I’m also being careful selecting the new ones.