Tuning Suggestions.

[kspare]

I'm working on turning our 10gb cards and L2Arc, looking to see what others have done.

I have two systems, both with 64gb ram, Intel 750 NVE SSD (400GB) for L2Arc, 24 7200rpm 2TB drives running mirrored Vdevs.

Using HBA with matching firmware.

One server has a chelsios 320 10GB nic, the other has an Intel X520 10GB Nic.

Any suggestions on tuning the 10gb nics and the l2arc.

I've played around with the default tuning options and i've played around with some options i've found in research. From what i've read for L2arc you need to set the max read and burst to be around 90% of the write speed of your ssd, which in my case max write is 900MB/s so i've set it to 800MB/s.

I'm not getting really great write speeds though, it's kind of odd, my disks only show writing at 10MB/s but read speeds were working great!

Just looking to see what others are doing as I work on tuning today.

 

[wreedps]

Subscribing, Looking to build a similar setup.

 

[kspare]

here is what I have right now. see the attached.

One thing to note, i've noticed that if you change the tuning on the l2arc it works better if you rebuild the whole storage...not sure why. so be prepared to migrate all your data off.

with this config my disks are hitting 150MB/s and my l2arc has hit as high as 800MB/s, I've migrated my file servers and terminal servers back on to see how things work and it's quite perky. i'm still planning to upgrade my ram to 128gb as i'm only at 64 right now, so so far so good, performance is great and I can see the L2arc actually doing things now.

 

[cyberjock]

You most certainly don't want to set values to 90% of throughput. You want the values to be relatively low so you avoid high turnover of the L2ARC if your system is undersized. Generally I've heard anywhere from 5-10% is good, and some people have said you want it to trickle fill so values like 50MB/sec are about the highest you should ever do.

All that you will do is force fill the L2ARC with data very quickly, then you'll be fighting the L2ARC to allow you to evacuate the old useless data with more important data.

Also, if you set your values to 90% of throughput, that means you're literally leaving yourself with 10% of the throughput for reads. Does that make sense? Of course not. :P

Yes.. I realize I'm a bit late to the show.

 

[kspare]

What would you recommend for my tuning to be then? I'm not having any performance issues, infact it's working really well! My L2Arc hits are the highest they've ever been.

 

[cyberjock]

No clue. Tuning isn't something someone can just willy-nilly do over a forum. :P

That's why its called the "zfs evil tuning guide".