SOLVED Extreme high RAM usage in freenNAS

[Craigk19]

I noticed when creating the windows 10 VM my ram went from using 4Gbs to 6Gbs not a big deal. Then when i installed plex on the VM it jumped to 8gbs used still no need for concern. But when I did a test of moving about 10GBs of anime over to the VM from my local machine the RAM went up to 14GBs so I thought it's writing from RAM. so i let the transfer finish got 90 mbs transfer speeds which i was expecting. Using 1gb NIC's on both esxi and freenas, the RAM never came back down it stayed at 14Gbs. I had to reboot the freeNAS box for memory usage to go back to 4Gbs. i did wait roughly 30 minutes before i rebooted to see if it would come down and it did not move. Any help would be greatly appreciated!!

My setup currently is using the freenas as a datastore for my esxi box I'm doing this using iSCSI. Yesterday took me half a day to get that working correctly. My nas is a total of 10TBs usable in raidz-2. I made the iSCSI 5TB. My over all plan was to use 4TBs of that as a Plex server on windows 10 as a VM on esxi. When building my box i knew this was the route i was attempting to take. I found that for iSCSI i should use a SLOG. My issue is i was completely unsure how to add a SLOG and was using a 250GB Samsung evo 840 for this but I'm not sure i set it up right. When I created my pool for raidz-2 i just used the log option and added the ssd as the device for that was that correct?

I'm new to freenas and I bought all my hardware on recommendations from here. My box is a single Xeon e3-1230 on a supermicro X9SCM-F with 16 GBs of unbuffered ECC Ram using a HBA 9207-8i in IT mode powered by a seasonic 750w gold focus PSU.

TDLR- New to freeNAS, eSXI 6.5 using iSCSI for datastore from FreeNAS. Not sure SLOG is setup correctly and seeing high ram on freenas when using VM on ESXi.

 

[garm]

ZFS will use all available RAM for ARC by default in FreeNAS. No need to manage RAM, it’s handled by ZFS. ARC will be relinquished if memory is requested by another service.

 

[Craigk19]

ZFS will use all available RAM for ARC by default in FreeNAS. No need to manage RAM, it’s handled by ZFS. ARC will be relinquished if memory is requested by another service.

So i should expect it to always use over 12Gbs?

 

[jgreco]

So i should expect it to always use over 12Gbs?

RAM isn't measured in gigabits per second, and your other wildly varying notation makes your first post difficult to read and parse.

https://www.ixsystems.com/community/threads/terminology-and-abbreviations-primer.28174/

If you meant that it should always be more than 12GBytes, that's also not really correct. It'll use 12GBytes if there is 12GBytes worth of ARC-eligible content. Otherwise it'll use less.

An 840 EVO is not an appropriate SLOG device. Drop the SLOG unless you can explain why you need it. An appropriate SSD requires proper power loss protection circuitry, otherwise you're just kidding yourself and are better off without it.

iSCSI and RAIDZ2 are not particularly compatible. This is a bad design choice.

https://www.ixsystems.com/community...d-why-we-use-mirrors-for-block-storage.44068/

Put your Plex data on an NFS share and just completely forget about iSCSI. You don't have the RAM for iSCSI, and you're making it work much harder than necessary.

https://www.ixsystems.com/community...res-more-resources-for-the-same-result.28178/

We typically don't recommend iSCSI with less than 64GB RAM, but you can do alright at 32GB sometimes.

 

[Craigk19]

An 840 EVO is not an appropriate SLOG device. Drop the SLOG unless you can explain why you need it. An appropriate SSD requires proper power loss protection circuitry, otherwise, you're just kidding yourself and are better off without it.

iSCSI and RAIDZ2 are not particularly compatible. This is a bad design choice.

https://www.ixsystems.com/community...d-why-we-use-mirrors-for-block-storage.44068/

How can I remove the SSD from my pool since it was set up with the original setting up of the pool? If not possible I'm not opposed to wiping and starting from scratch.

After reading that link I should Make Mirrors for the DataStores correct? I thought this was a bad idea because if one of the mirrors drives die then that will cause my entire pool to go down and has no redundancy.

Put your Plex data on an NFS share and just completely forget about iSCSI. You don't have the RAM for iSCSI, and you're making it work much harder than necessary.

https://www.ixsystems.com/community...res-more-resources-for-the-same-result.28178/

We typically don't recommend iSCSI with less than 64GB RAM, but you can do all right at 32GB sometimes.

Okay, I'll get rid of iSCSI and try to set up and NFS share for my Datastore for ESXi? This is where I got slightly lost when trying to think through the process earlier yesterday. Do I put the plex media on an SMB share or do I put it on the VM directly it's self?

I do apologize for the incorrect terminology I'm just trying to learn. No idea if I'm doing this correctly or even going down the right path.

 

[rvassar]

We typically don't recommend iSCSI with less than 64GB RAM, but you can do all right at 32GB sometimes.

I run two small iSCSI target's with 16Gb, but it's tight. No room for VM's, two jails, Plex & MySQL, and my RAM is almost always ~15Gb in use.

 

[joeschmuck]

How can I remove the SSD from my pool since it was set up with the original setting up of the pool? If not possible I'm not opposed to wiping and starting from scratch.

You need to read the User Guide for FreeNAS. Do not expect the forum to just provide you answers if you are not willing to at least read the User Guide or do a search for removing an SLOG. Not trying to be mean but we do expect forum members to at least make an attempt to figure things out before asking for help.

Do I put the plex media on an SMB share or do I put it on the VM directly it's self?

I have no idea what you are asking, atleast by the way you phrased the question. If using Plex the media just needs to be linked within Plex, that would be a folder within your pool, but it doesn't need to be a share, however generally most people would make it a share so they could add/remove media via some remote computer but that is separate from Plex operation.

 

[Craigk19]

Slog has been removed, still not sure why that was such a bad idea, the information on the VMs was not crucial to me. If the UPS I have on the boxes isn't enough, and I lost the data, it would not be a massive loss to me I had that fully in mind when I added it as a SLOG. I was able to find documentation and remove the SLOG none the less. After reading some of the linked material above it looks like NFS would be the better way to go. That being said could someone explain to me why I should use mirrors for the ESXi datastore? I was under the impression that if I used mirrors and one died then that would cause that vdev to die and would cause me to lose everything. This is why I went with Raidz2 in the first place. again I'm new to this and it's for a homelab so I can learn on it, this will only affect me and my household. I thought the route I was taking was the correct one. This is why I asked.

My goal was to create dedicated storage(FreeNas) use it for storage for my homelab stuff(ESXi) and place my Plex media and family photos as well as our pc backups on. I have no road map of what is right and wrong I've attempted to research best practices but there is lots of contradicting stuff for all routes.

 

[jgreco]

Already linked the "why we use mirrors for block storage" sticky.

What's the point of adding the SLOG? It's like riding around in a car with airbags where you don't bother to buckle up. If you don't have a proper SLOG device, then the best thing to do is to stop pretending your SSD can be a good SLOG and just do without. Bonus: things work much faster.

If you use a mirror and a mirror vdev dies, yes, your pool is toast. Similarly, if you use a RAIDZ and a RAIDZ vdev dies, your pool is toast. If what you're seeking is retaining redundancy in the face of a physical device failure, go to three-way (or greater) mirrors.

 

[rvassar]

Slog has been removed, still not sure why that was such a bad idea, the information on the VMs was not crucial to me.

Because the SLOG you chose didn't have power failure protection. Which doesn't mean an actual physical power failure... Something as simple as a kernel crash and a hardware reset could abort a transaction and corrupt your ZFS pool.

 

[myoung]

Slog has been removed, still not sure why that was such a bad idea, the information on the VMs was not crucial to me. If the UPS I have on the boxes isn't enough, and I lost the data, it would not be a massive loss to me I had that fully in mind when I added it as a SLOG. I was able to find documentation and remove the SLOG none the less. After reading some of the linked material above it looks like NFS would be the better way to go. That being said could someone explain to me why I should use mirrors for the ESXi datastore? I was under the impression that if I used mirrors and one died then that would cause that vdev to die and would cause me to lose everything. This is why I went with Raidz2 in the first place. again I'm new to this and it's for a homelab so I can learn on it, this will only affect me and my household. I thought the route I was taking was the correct one. This is why I asked.

My goal was to create dedicated storage(FreeNas) use it for storage for my homelab stuff(ESXi) and place my Plex media and family photos as well as our pc backups on. I have no road map of what is right and wrong I've attempted to research best practices but there is lots of contradicting stuff for all routes.

The idea is to remove SLOG and turn off sync writes. All your writes will be async and 100% max speed. Your SSD would not protect you from unexpected power loss which is the only point of a ZIL/SLOG, so why even have one.

 

[Craigk19]

The idea is to remove SLOG and turn off sync writes. All your writes will be async and 100% max speed. Your SSD would not protect you from unexpected power loss which is the only point of a ZIL/SLOG, so why even have one.

Thank you very much for that simplified answer. Now I fully understand why it was not a smart move and know going forward the why it was not the appropriate approach.