steven6282
Dabbler
- Joined
- Jul 22, 2014
- Messages
- 39
Ok, so if you saw my other thread recently, you know I'm working on a new FreeNAS build. While doing some more research before buying my hardware I came across some statements that eluded to a ZFS based NFS share for an ESXi datastore would have poor performance due to the way ZFS works.
It was suggested that the solution to this is ZILs and L2ARCs. I had previously decided that I did not need these based on Cyberjock's newbie guide, however now I'm questioning that decision. I do intend to be using an NFS share for my ESXi's data store.
Now I've seen Cyberjock on more than one occasion say that you shouldn't use L2Arc until you have 64GB of ram, but after a few hours of searching I have been unable to find any reasoning behind this. I could bump my build up to 32 GB of ram, but I don't think it's even possible for me to go to 64 GB of ECC currently with the board I am planning to use (A1SAM-2550F, http://www.supermicro.com/products/motherboard/Atom/X10/A1SAM-2550F.cfm) The board requires unbuffered memory, and I have not seen any ECC 16 GB strips available as unbuffered yet. Changing the board isn't something I really want to do right now. This box needs to stay as low powered as I can make it. If you must know the reason, I like our environment so I do what stuff I can to help be clean, thus my ESXi and NAS boxes + networking equipment (the stuff that runs 24 x 7) all run primarily off a small solar array in my backyard. It of course has line power backup, but I'd prefer to stay within the confines of my solar array's capacity.
So anyway, I'm trying to figure out at what point is performance bad for VMs stored on an NFS data store from FreeNas? Is it something that is only seen once you get a bunch of VMs running, or if I'm just running a small number of 4 to 5 VMs the majority of the time, will I see the problems? Sometimes I may be running as many as 10, but there are almost always at least 4 running. Are the problems more related to write speed thus needing a small ZIL, or read speed?
Also, I've seen cyberjock say in this thread: http://forums.freenas.org/index.php?threads/at-what-point-does-l2arc-make-sense.17373/
Where is this information derived from? I haven't been able to find anything concrete on this, but I have seen two other websites earlier while researching the problem estimate it closer to 2GB of ARC to index 100GB of L2ARC. I of course can't find those sites again atm cause I've literally went through 50 to 100 different pages this morning trying to find more information lol. But if I run across them again, I'll come back and add the links to them. I'm trying to figure that out because I think that is his primary basis for recommending 64GB before doing L2ARC.
So the question really boils down to if I even need to L2Arc with 4 to 5 VMs the majority of the time, and if so can I get away with a 60GB L2Arc with 32GB of system ram without any seriously adverse side effects? And also, if I need to set up a small ZIL for this situation.
Cyberjock, I know you are likely to reply to this based on previous encounters so I'm going to pre-reply to you :). I ask that if you do reply, please have links for sources for the information you provide. You definitely appear to be knowledgeable when it comes to FreeNAS / ZFS stuff, I'll grant you that, but I can't simply take one person at their word to be definitely correct. As to your common "you can search for it", I've been searching for it all morning and unable to find it lol. As for your other common "you can do it that way if you want but don't cry to me when you have problems", yeah... don't worry I wont cry to you, but I'd really rather have the facts up front before I decide how I'm going to do it. :)
Thanks in advance for any help!
It was suggested that the solution to this is ZILs and L2ARCs. I had previously decided that I did not need these based on Cyberjock's newbie guide, however now I'm questioning that decision. I do intend to be using an NFS share for my ESXi's data store.
Now I've seen Cyberjock on more than one occasion say that you shouldn't use L2Arc until you have 64GB of ram, but after a few hours of searching I have been unable to find any reasoning behind this. I could bump my build up to 32 GB of ram, but I don't think it's even possible for me to go to 64 GB of ECC currently with the board I am planning to use (A1SAM-2550F, http://www.supermicro.com/products/motherboard/Atom/X10/A1SAM-2550F.cfm) The board requires unbuffered memory, and I have not seen any ECC 16 GB strips available as unbuffered yet. Changing the board isn't something I really want to do right now. This box needs to stay as low powered as I can make it. If you must know the reason, I like our environment so I do what stuff I can to help be clean, thus my ESXi and NAS boxes + networking equipment (the stuff that runs 24 x 7) all run primarily off a small solar array in my backyard. It of course has line power backup, but I'd prefer to stay within the confines of my solar array's capacity.
So anyway, I'm trying to figure out at what point is performance bad for VMs stored on an NFS data store from FreeNas? Is it something that is only seen once you get a bunch of VMs running, or if I'm just running a small number of 4 to 5 VMs the majority of the time, will I see the problems? Sometimes I may be running as many as 10, but there are almost always at least 4 running. Are the problems more related to write speed thus needing a small ZIL, or read speed?
Also, I've seen cyberjock say in this thread: http://forums.freenas.org/index.php?threads/at-what-point-does-l2arc-make-sense.17373/
...so you don't want a 60GB L2ARC because that will take up about 12GB of your ARC...
Where is this information derived from? I haven't been able to find anything concrete on this, but I have seen two other websites earlier while researching the problem estimate it closer to 2GB of ARC to index 100GB of L2ARC. I of course can't find those sites again atm cause I've literally went through 50 to 100 different pages this morning trying to find more information lol. But if I run across them again, I'll come back and add the links to them. I'm trying to figure that out because I think that is his primary basis for recommending 64GB before doing L2ARC.
So the question really boils down to if I even need to L2Arc with 4 to 5 VMs the majority of the time, and if so can I get away with a 60GB L2Arc with 32GB of system ram without any seriously adverse side effects? And also, if I need to set up a small ZIL for this situation.
Cyberjock, I know you are likely to reply to this based on previous encounters so I'm going to pre-reply to you :). I ask that if you do reply, please have links for sources for the information you provide. You definitely appear to be knowledgeable when it comes to FreeNAS / ZFS stuff, I'll grant you that, but I can't simply take one person at their word to be definitely correct. As to your common "you can search for it", I've been searching for it all morning and unable to find it lol. As for your other common "you can do it that way if you want but don't cry to me when you have problems", yeah... don't worry I wont cry to you, but I'd really rather have the facts up front before I decide how I'm going to do it. :)
Thanks in advance for any help!
Last edited: