Saving multiple copies of every data

[Heracles]

Hi,

My actual specs are provided at the end of this message.

About 8 months ago, I bought all my drives at once, so they will probably all fail at once To avoid that, I bought a new one, 3TB instead of 1. I will keep replacing my disks progressively, so they will not fail all together. At the end, I will have a pool 3 times bigger.

FreeNAS is the backend for my Nextcloud service, running from a separate server. I also have a Plex jail running. Because of that, all my multimedia stuff, files, agendas, calendars and more are in my FreeNAS. With my pool of 1 TB disks, I already have plenty of space. I am using 19% of it. I do plan to add a lot of data, but really the pool with 1TB drives was enough. So once with 3 TB drives, I will have way more space than needed.

I thought about using that extra space to increase the redundancy for my data by having 2 copies of everything. For that, my understanding is that ZFS can do it by itself in the background. Because my entire life is now in that server, I am looking for the maximum security. I am also working on an RSync copy of the "active" data to an offsite file server.

Questions are:
1-Can I change this setting on an existing pool and FreeNAS will start duplicating everything or is it an option available only when creating a new pool ? (how to change that setting ?)

2-I have 5 generations of snapshots (15 min -3D, 1H - 3W, 1D - 9W, 1W - 18M, 4W - 5Y). Together, they total about 950 snapshots covering 5 years. Will the copies=2 also duplicate what is currently saved in these snapshots ?

3-Other than using twice as much disk space, I expect copies=2 will also slow down write access. Any other performance effect I should consider ? (concerns about RAM?)

4-Is it an option that can be reverted ?

5-Any advice against using that option or some extra care I should take if I am about to use it ?

6-Anything special for the Plex jail ?

7-Should I wish to use ISCSI to provide storage to my ESXi 6.7 server, how this option will affect the setup ?

Thanks for your input,

Heracles

+-+-+-+-+-+-+-+
Server: Dell T-110

OS:

Code:

root@atlas:~ # uname -v
FreeBSD 11.1-STABLE #0 r321665+e0c4ca60dfc(freenas/11.1-stable): Wed May 30 14:18:20 EDT 2018	 root@nemesis.tn.ixsystems.com:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64

CPU:

Code:

root@freenas:~ # sysctl hw.ncpu
hw.ncpu: 4
root@freenas:~ # sysctl hw.model
hw.model: Intel(R) Core(TM) i3-2100 CPU @ 3.10GHz

RAM: 8 Gig (2x 4G)

Code:

root@freenas:~ # dmidecode -t memory

# dmidecode 3.1
Scanning /dev/mem for entry point. SMBIOS 2.7 present.

Handle 0x0026, DMI type 16, 23 bytes Physical Memory Array

Location: System Board Or Motherboard Use: System Memory
Error Correction Type: Multi-bit ECC Maximum Capacity: 32 GB

Error Information Handle: No Error Number Of Devices: 4

Handle 0x0028, DMI type 17, 34 bytes Memory Device

Array Handle: 0x0026
Error Information Handle: No Error
Total Width: 128 bits
Data Width: 64 bits
Size: 4096 MB
Form Factor: DIMM
Set: None
Locator: DIMM A1
Bank Locator: BANK 1
Type: DDR3
Type Detail: Synchronous Unbuffered (Unregistered) Speed: 1333 MT/s
Manufacturer: 80AD000080AD
Serial Number: 1C6605E9
Asset Tag: 011133
Part Number: HMT351U7BFR8A-H9
Rank: Unknown
Configured Clock Speed: 1333 MT/s

Disks : 6 of them

Code:

root@freenas:~ # smartctl -i /dev/ada0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build) Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===

Model Family:
Device Model:
Serial Number:
LU WWN Device Id: 5 000c50 0a5b783eb Firmware Version: SC60

errors: No known data errors

Seagate IronWolf ST1000VN002-2EY102 Z9C482NA

User Capacity:
Sector Sizes:
Rotation Rate:
Form Factor:
Device is:
ATA Version is: ATA8-ACS T13/1699-D revision 4 SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Fri Feb 2 09:54:42 2018 EST

SMART support is: Available - device has SMART capability. SMART support is: Enabled

Zpool:

Code:

root@freenas:~ # zpool status

pool: freenas-boot

state: ONLINE
scan: scrub repaired 0 in 0 days 00:01:08 with 0 errors on Wed Jan 31 03:46:08 2018

config:
NAME STATE READ WRITE CKSUM freenas-boot ONLINE 0 0 0

da0p2 ONLINE 0 0 0 errors: No known data errors

pool: vCloud state: ONLINE

scan: resilvered 171G in 0 days 00:57:13 with 0 errors on Fri Feb 2 10:24:42 2018 config:

1,000,204,886,016 bytes [1.00 TB] 512 bytes logical, 4096 bytes physical

5900 rpm

3.5 inches
In smartctl database [for details use: -P show]

NAME vCloud

raidz2-0
gptid/b6e68312-07ae-11e8-8de1-bc305be5d3f0 ONLINE gptid/16489e9d-07be-11e8-9281-bc305be5d3f0 ONLINE gptid/082cf4ad-07ea-11e8-b8b5-bc305be5d3f0 ONLINE gptid/13975389-0825-11e8-8302-bc305be5d3f0 ONLINE gptid/da2d7724-07ca-11e8-909c-bc305be5d3f0 ONLINE gptid/48ecf0c5-07d9-11e8-85bc-bc305be5d3f0 ONLINE

STATE ONLINE ONLINE

READ WRITE CKSUM 0 0 0

0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

UPS:
APC Back-UPS RS 1500

 

[kdragon75]

I am looking for the maximum security.

Build a second NAS and ship it out of state. Use replication.
As for using the extra space, You could move to RAIDz3. You can lose any 3 disks and still not lose data. The whole zfs copies = 2 thing is really not going to help you at all. With RAIDz2 you already have 3 "copies" of your data. By copies, I mean you have the original and two ways to calculate any missing or damaged bits. This unfortunately means migrating your data off of the pool, destroying the pool, and recreating it as a RAIDz3 pool. To estimate the usable space for RAIDz3 its N-3 or in your case (6*3TB)-(3*6TB)=9TB

 

[Heracles]

Hi,

Thanks for your input.

Unfortunately, to destroy and re-create the pool is not an option for now. I do wish to keep improving the robustness of my setup with a complete backup FreeNAS server but for now, I have to do with what I have. (long term plan is for that T-110 to become the off-site back-up ; my actual T-330 to turn in to the main FreeNAS and a new server for ESXi).

As for copies=2, I thought that it could help against logical corruption. I agree that it would not help against the lost of a physical hard drive; only RaidZ-3 would do that.

Any other idea / info about multiple copies of the data ?

Thanks again,

 

[kdragon75]

1-Can I change this setting on an existing pool and FreeNAS will start duplicating everything or is it an option available only when creating a new pool ? (how to change that setting ?)

If you can set it on the fly, no it would not affect the current data.

2-I have 5 generations of snapshots (15 min -3D, 1H - 3W, 1D - 9W, 1W - 18M, 4W - 5Y). Together, they total about 950 snapshots covering 5 years. Will the copies=2 also duplicate what is currently saved in these snapshots ?

See No. 1

3-Other than using twice as much disk space, I expect copies=2 will also slow down write access. Any other performance effect I should consider ? (concerns about RAM?)

As you noted, the write speed will be effective cut in half (sorta kinda not really). As for RAM, it would have no measurable effect.

4-Is it an option that can be reverted ?

See No. 1 If it can be set on the fly, the new data would not be duplicated but old data would be unless read and rewriten.

5-Any advice against using that option or some extra care I should take if I am about to use it ?

Dont.

6-Anything special for the Plex jail ?

Nope.

7-Should I wish to use ISCSI to provide storage to my ESXi 6.7 server, how this option will affect the setup ?

Assuming your are using sync=always (and you should if you care about your VMs) you will have PAINFULLY slow writes unless you have a SLOG and then it will only be annoyingly slow due to the fact that you only have one vdev and copies=2.