I recently upgraded from 16GB non-ecc to 8GB ecc (also swapped MB). I'm *slightly* concerned about only having 8GB of ram now, but in the last few days the server has been running I haven't noticed any problems. I am not running any jailed apps, just CIFS and iSCSI mounts for 3 clients.
My current hard drive configuration is 8 x 3TB drives in a volume consisting of 2 RaidZ2 sets (4 drives each).
My initial goal was to buy 1 stick of ECC ram and verify the board supported it and then buy a 2nd identical stick but the egg went out of stock in the week it took the first stick to get here. My new goal is to buy a 16gb set and go up to 24GB of ram, but i want to know if this should be an urgent issue or if it can wait a few months.
Are there any horror stories of ZFS failure/corruption on 8GB for a ZFS pool with 24TB of raw space? I know the usual warnings of 4GB system and data going, but is this something to worry about with 8GB?
I guess I would also like to know if anyone has had *ANY* problems with a similar setup in RE to the hardware used on my system:
The hardware used:
FreeNAS 9.2.1.5 64bit (upgraded from 9.1 something)
Asus Sabertooth 990FX (rev1)
ECC enabled in the bios
Using 4 of the 8 onboard sata, 2 are useless because of pseudo raid and i wanted save the other 2 MB ports for ssds in the future
Almost everything onboard is disabled: nic/sound/raid ports
AMD 1045T
1 stick of 8GB Kingston ECC unbuffered ram (KVR16LE11/8I)
2 x Syba SD-SA2PEX-2IR (flashed in IT mode - where the rest of the HD's sit)
1 x LSI 1064e chipset card (flashed in IT mode, for 2TB or less drives, i have some 2TB drives and was thinking of using these as a backup device)
Server class pci-e intel 82571EB dual port 1gb nic, but only using 1 of the ports.
PCICONF:
Side question: is this the best way to verify ECC is working: dmidecode --type 16, it returns "Error Correction Type: Multi-bit ECC", memtest has been less than useful for displaying ecc info.
The server is on a UPS: Back-UPS NS 1250 (newer black usb lcd). it is configured to gracefully shutdown the server at 37% battery.
The HD's are arranged so that 2 of the drives in each RaidZ2 set are going to the MB, and then the other two drives are split between the 2 Syba cards
ex: raidz2-0 2xMB, 1xsyba-port1-pci:slot1, 1xsyba-port1-pci:slot2 ditto for RaidZ2-1 but port 2 for the other card.
I hope this ensures that in the event of a complete failure of any SATA controller the zfs data will live on.
ZFS Layout, paranoia in effect:
My current hard drive configuration is 8 x 3TB drives in a volume consisting of 2 RaidZ2 sets (4 drives each).
My initial goal was to buy 1 stick of ECC ram and verify the board supported it and then buy a 2nd identical stick but the egg went out of stock in the week it took the first stick to get here. My new goal is to buy a 16gb set and go up to 24GB of ram, but i want to know if this should be an urgent issue or if it can wait a few months.
Are there any horror stories of ZFS failure/corruption on 8GB for a ZFS pool with 24TB of raw space? I know the usual warnings of 4GB system and data going, but is this something to worry about with 8GB?
I guess I would also like to know if anyone has had *ANY* problems with a similar setup in RE to the hardware used on my system:
Code:
Mem: 185M Active, 128M Inact, 6147M Wired, 183M Buf, 1418M Free ARC: 5611M Total, 1340M MFU, 4233M MRU, 784K Anon, 28M Header, 9978K Other Swap: 16G Total, 16G Free
The hardware used:
FreeNAS 9.2.1.5 64bit (upgraded from 9.1 something)
Asus Sabertooth 990FX (rev1)
ECC enabled in the bios
Using 4 of the 8 onboard sata, 2 are useless because of pseudo raid and i wanted save the other 2 MB ports for ssds in the future
Almost everything onboard is disabled: nic/sound/raid ports
AMD 1045T
1 stick of 8GB Kingston ECC unbuffered ram (KVR16LE11/8I)
2 x Syba SD-SA2PEX-2IR (flashed in IT mode - where the rest of the HD's sit)
1 x LSI 1064e chipset card (flashed in IT mode, for 2TB or less drives, i have some 2TB drives and was thinking of using these as a backup device)
Server class pci-e intel 82571EB dual port 1gb nic, but only using 1 of the ports.
PCICONF:
Code:
siis0@pci0:1:0:0: class=0x010600 card=0x31321095 chip=0x31321095 rev=0x01 hdr=0x00 vendor = 'Silicon Image, Inc.' device = 'SiI 3132 Serial ATA Raid II Controller' class = mass storage subclass = SATA mpt0@pci0:2:0:0: class=0x010000 card=0x30901000 chip=0x00561000 rev=0x08 hdr=0x00 vendor = 'LSI Logic / Symbios Logic' device = 'SAS1064ET PCI-Express Fusion-MPT SAS' class = mass storage subclass = SCSI em0@pci0:3:0:0: class=0x020000 card=0x7044103c chip=0x105e8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller' class = network subclass = ethernet em1@pci0:3:0:1: class=0x020000 card=0x7044103c chip=0x105e8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller' class = network subclass = ethernet siis1@pci0:5:0:0: class=0x010600 card=0x31321095 chip=0x31321095 rev=0x01 hdr=0x00 vendor = 'Silicon Image, Inc.' device = 'SiI 3132 Serial ATA Raid II Controller' class = mass storage subclass = SATA
Side question: is this the best way to verify ECC is working: dmidecode --type 16, it returns "Error Correction Type: Multi-bit ECC", memtest has been less than useful for displaying ecc info.
The server is on a UPS: Back-UPS NS 1250 (newer black usb lcd). it is configured to gracefully shutdown the server at 37% battery.
The HD's are arranged so that 2 of the drives in each RaidZ2 set are going to the MB, and then the other two drives are split between the 2 Syba cards
ex: raidz2-0 2xMB, 1xsyba-port1-pci:slot1, 1xsyba-port1-pci:slot2 ditto for RaidZ2-1 but port 2 for the other card.
I hope this ensures that in the event of a complete failure of any SATA controller the zfs data will live on.
ZFS Layout, paranoia in effect:
Code:
NAME STATE READ WRITE CKSUM %hidden% ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 gptid/RandomRandomRandomRandomRandomRandom.eli ONLINE 0 0 0 gptid/RandomRandomRandomRandomRandomRandom.eli ONLINE 0 0 0 gptid/RandomRandomRandomRandomRandomRandom.eli ONLINE 0 0 0 gptid/RandomRandomRandomRandomRandomRandom.eli ONLINE 0 0 0 raidz2-1 ONLINE 0 0 0 gptid/RandomRandomRandomRandomRandomRandom.eli ONLINE 0 0 0 gptid/RandomRandomRandomRandomRandomRandom.eli ONLINE 0 0 0 gptid/RandomRandomRandomRandomRandomRandom.eli ONLINE 0 0 0 gptid/RandomRandomRandomRandomRandomRandom.eli ONLINE 0 0 0