RAM Requirements

[nhabbott]

I currently have two TrueNAS Core systems each with a pool of 130TB of storage running in 2 RAIDZ2 arrays. Each system has 64GB of DDR4 ECC RAM. I am seeing occasional stability issues (lock ups) with a VM that has the entire 130TB mounted as a VirtIO drive. The VM is running an IP surveillance software called DW Spectrum. Is this amount of RAM sufficient for this amount of storage?

Edit: The VM is running on a separate pool of a 250GB Mirror with two Samsung 870 EVO drives.

 

[winnielinnie]

RAIDZ2 arrays

with a VM that has the entire 130TB mounted as a VirtIO drive

Perhaps due to running a VM on a RAIDZ2 pool?

The path to success for block storage

It seems like I haven't written a sticky for awhile, but just in the last week I've had to cover this topic several times. ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle...

www.truenas.com

 

[nhabbott]

Perhaps due to running a VM on a RAIDZ2 pool?

The path to success for block storage

It seems like I haven't written a sticky for awhile, but just in the last week I've had to cover this topic several times. ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle...

www.truenas.com

I'm sorry, I failed to mention that the VM is running on a separate pool of a 250GB Mirror with two Samsung 850 EVO drives.

 

[Arwen]

Some questions;

• How full are those 130TB pools?
• What is the layout of of the pools?
• What is the make & model of the drives?
• How does that VM access the 130TB worth of data, Samba, NFS, iSCSI?

ZFS pools should be kept less than 80% full. At some point, I think 95% full, ZFS goes into extreme space saving mode, slowing things down.

 

[nhabbott]

Some questions;

• How full are those 130TB pools?
• What is the layout of of the pools?
• What is the make & model of the drives?
• How does that VM access the 130TB worth of data, Samba, NFS, iSCSI?

ZFS pools should be kept less than 80% full. At some point, I think 95% full, ZFS goes into extreme space saving mode, slowing things down.

• The pool stays at 78% used
• RAIDZ2 (7 drives) + RAIDZ2 (7 drives)
• The drives are Western Digital 16TB Purple Pro (WD181PURP)
• The VM accesses the pool by a VirtIO device attached to it. Inside the VM it is seen as one 130TB VirtIO disk

 

[Patrick M. Hausen]

The VM accesses the pool by a VirtIO device attached to it

What do you mean by "device attached to a pool"?

 

[nhabbott]

What do you mean by "device attached to a pool"?

Here are screenshots because I think I'm using the wrong terminology.

 

[Patrick M. Hausen]

Ah ... so you created a Zvol on the pool and use that as a block device for a virtual disk?

 

[nhabbott]

Ah ... so you created a Zvol on the pool and use that as a block device for a virtual disk?

Correct. Sorry for the confusion, I knew I was missing something from my explanation.

 

[Patrick M. Hausen]

How large are these Zvols?

 

[NugentS]

Bet they are nearly the same size as the pool.

 

[nhabbott]

How large are these Zvols?

The VM shows a total of 132 TB and TrueNAS shows 119.15 TiB being used with 33.34 TiB available.

 

[NugentS]

Thats inside the zvol - what about the zvol itself?
How much space is left in the pool?

 

[nhabbott]

Thats inside the zvol - what about the zvol itself?
How much space is left in the pool?

The zvol is 120 TiB while the pool is 152 TiB. The remaining 32 TiB stays free to keep the total pool size less than 80% used.

 

[NugentS]

OK - well thats good. And completely blows my ideas out the window