Help me understand vfs.zfs.write_limit_max and vfs.zfs.write_limit_shift please...

[sy5tem]

Hello

im running frenas 8.3.0 for iscsi (for vmware) and cifs. using 6 wd red drives.

now my system seem's to go catatonic after big writes... im thinking it relate to ticket # 1531? .. so

im reading like crazy around the forums and ticket... and most everything is new to me .. im no nix guru ...

now i want to set my tunable and sysctl to a value so that my system will stay alive.. and not freeze like it does now

here is what i know...

my zpool can writes @ 300MB / sec.
so changing sysctl and tunable like so will help?

Sysctl
vfs.zfs.write_limit_shift="5"
vfs.zfs.write_limit_max="262144" in kb or vfs.zfs.write_limit_max="256M" i want 256MB

ZFS tunables
vfs.zfs.txg.synctime="200"
vfs.zfs.txg.timeout="1"


are these value ok for
HP XW8200
8GB RAM
2x quad core
6x 2tb wd red in raidz2


please help me out . !!


ALSO: do i need to play with network buffer for LACP quad port gb ?
net.inet.tcp.sendbuf_max="2097152"
kern.ipc.maxsockbuf 2="2097152"
net.inet.tcp.recvbuf_max="2097152"


thank you for you help!!!!

 

[cyberjock]

AFAIK there's only one person on the forums that really has the expertise to help you.. and he stopped helping because there is no magic bullet. Literally you have to do alot of reading, understand how your hardware and your loading, how it all works together. Then after you've spent all that time(probably weeks would be my best guess just to have a remote clue what you are doing) you have alot of experimenting and testing to do to figure out how to get it working how you want.

 

[sy5tem]

AFAIK there's only one person on the forums that really has the expertise to help you.. and he stopped helping because there is no magic bullet. Literally you have to do alot of reading, understand how your hardware and your loading, how it all works together. Then after you've spent all that time(probably weeks would be my best guess just to have a remote clue what you are doing) you have alot of experimenting and testing to do to figure out how to get it working how you want.

understand that part...

but i still can't understand if those vfs.zfs.write_limit_max= settings anre in bytes , kb , or mb ? for 256MB what should i put?
vfs.zfs.write_limit_max="256M" MB
or
vfs.zfs.write_limit_max="262144' KB
or
vfs.zfs.write_limit_max="268435456 Bytes'


for me right now all i want is the system to stop going into a catatonic states! then i will tweak if needed. ...

 

[cyberjock]

Why not remove the tweaks then? If I were trying to figure out these tweaks I'd be putting them into google, clicking every one of the top 10 search results and reading every single one top to bottom. Only way you're going to figure out this stuff is to read and learn.

 

[sy5tem]

i have no tweak done.

my system is going into catatonic states after a while on heavy writes.

i need to fix this.

i goggled . found thousand of pages but i can't find anywhere if the number after the write limit is in bytes, kb or mb ....

i don't understand why these forums exist if we are constantly toad to google ... ?

 

[cyberjock]

Your issue really isn't with FreeNAS at all. FreeNAS really is just a pretty GUI that runs commands based on what you input in the GUI.


Your issues really have to do with FreeBSD and ZFS. ZFS in particular is very specialized. If you really know the ins and outs you can make good money tweaking ZFS for businesses. I'm not sure what the best forum is to read about ZFS(especially since Sun's forums are gone now that ZFS is "owned" by Oracle) but I'd start with the FreeBSD forums. Definitely read their forum rules though, because they are ruthless with people that use bad punctuation regularly. For instance, I saw people get flamed for saying 'freebsd' instead of 'FreeBSD'.

 

[cyberjock]

Also, your system isn't meeting the recommended system requirements for RAM. Considering you have 6x2TB drives in a RAIDZ2 and following the manual you should have at least 14GB of RAM. My home server is having issues because it has 12GB with a 20TB zpool. Just waiting for 3 8GB sticks to arrive in the mail(Monday) then I'll be back in business.

I'd definitely start with RAM before jumping to tweaking. With that little RAM I wouldn't expect you'd ever tweak it to actually work well.

 

[sy5tem]

Also, your system isn't meeting the recommended system requirements for RAM. Considering you have 6x2TB drives in a RAIDZ2 and following the manual you should have at least 14GB of RAM. My home server is having issues because it has 12GB with a 20TB zpool. Just waiting for 3 8GB sticks to arrive in the mail(Monday) then I'll be back in business.

I'd definitely start with RAM before jumping to tweaking. With that little RAM I wouldn't expect you'd ever tweak it to actually work well.

yes im trying to get my hands on 8x2 GB PC2-3200 (DDR2 400 MHz) ECC Registered but its so expensive .. it need to be single bank too :(

you think adding L2ARC for now would help?

 

[cyberjock]

Not really. The manual says you should always max out your RAM before adding an L2ARC. An L2ARC isn't a dumb cache like the Windows caching or the cache that comes on most RAID controllers. It only caches certain things and only at a certain rate. Just like with the ZIL, a L2ARC is only useful for certain types of loading. If your loading doesn't meet those types it won't help at all. Not to mention that using an L2ARC will steal RAM from the ARC. If you don't have enough RAM you can actually lower performance even more.

 

[Quadgnim]

Sy5tem,

You ever get things resolved? I'm looking to make similar config changes and wondering where you ended up? I have 4x2TB drives (16 TB), configured as 2 pairs of raid 1 (mirrored). I have an AMD triple core processor with 8 gig ram. I think my ram ratios are ok. I'm also considering changing it to 1 pool or raid 10, but regardless, I'm getting the hangs everyone has been talking about just doing a simple DD from a VM.

furthermore, I read that limit_max is a calculated field based on limit_shift, so maybe you don't have to worry about kb or mb, just tweak limit_shift. I'm told the default is 3 which is a 4 GB max, then for each +1 limit_shift drops limit_max in half. Therefore 5 would be 1GB, 6=512, 7=256 (what you want) and 8 = 128, etc...

 

[cout]

I have the same issue. The machine freezes after a few weeks of uptime, not to mention that it took me months to narrow it down and I'm pretty sure it's zfs since so many people had the same problem. The issue that threw me off was that listing other drives on the unit were fine, while the machine was catatonic, but this particular raidz2 would just hang on an ls. The whole machine is catatonic however, starting another shell from a remote machine would just hang, no password prompt, nothing. I thought it were the raid cards in the beginning and blaming it on different things. I have 8gb of ram on this unit with a 6 tb raidz2 that is used on a daily basis. Some other iscsi targets, I think, freeze the system at night when some 2008 servers back themselves up.
I'll add another 8gb of ram to it but it sounds like a "throw more money at it" government way of solving problems though, not an intelligent way out. I will also look into tweaking it because I'm so in love with zfs and freenas. I will also try the nas4free on a spare unit with the same specs but I doubt it that it's the OS itself. We'll see.... In the meantime if someone can help us with ideas on the zfs options would be great. I'll also look online for other people's experiences.