Lowest Frag contest [Phase I closed, tests underway]

Status
Not open for further replies.

tc60045

Dabbler
Joined
Nov 22, 2016
Messages
25
Phase I is closed. Contestants BiduleOhm, SweetAndLow and DarkWarrior, please PM me with the commands you want me to run. I will not disclose to any contestant what another is doing for their entry. I will, however, advise each if they are about to duplicate a less-than-stellar approach (I've had a few now...).

Once Bidule has his ready, I will execute his first and report results to all (but not approach), then SweetAndLow, then DarkWarrior. Winner will be announced, prizes awarded. Then we can still run a few more tests more publicly. Any objections, speak now. Otherwise, begin the PMs and let's light this candle!
 

darkwarrior

Patron
Joined
Mar 29, 2015
Messages
336
Hi there,
I'm definitely interested in the results, but I'm far away from being knowledgeable enough to actually submit any worthwhile sync suggestion.
I will be watching from the spectators bench :cool:
 
Joined
Feb 2, 2016
Messages
574
While tests are ongoing - thank you, @tc60045 - can someone explain to me in a vague, rule-of-thumb way when we should care about fragmentation?

We are running between 17% and 48% fragmented (below) and our performance isn't far off the baseline testing we did upon installation.

Code:
NAME		SIZE	ALLOC	FREE	FRAG	CAP	USE
V-D2600		32.5T	16.9T	15.6T	33%	51%	Security Video
V-Live1000	7.25T	5.97T	1.28T	32%	82%	Replication Target
V-Live600	3.78T	482G	3.31T	17%	12%	Replication Target
V-SSD		1.73T	338G	1.40T	48%	19%	XenServer VMs
V-Super		10.9T	4.21T	6.66T	20%	38%	Office File Server

Is our low overall usage (pools generally less than 50% full) what keeps us from noticing any fragmentation-related performance issues? Well, that, and our most heavily fragmented pool living on SSD?

Cheers,
Matt
 

tc60045

Dabbler
Joined
Nov 22, 2016
Messages
25
While tests are ongoing - thank you, @tc60045 - can someone explain to me in a vague, rule-of-thumb way when we should care about fragmentation?

We are running between 17% and 48% fragmented (below) and our performance isn't far off the baseline testing we did upon installation.

Matt, I think that performance is the critical lens for you or me -- if you are getting great performance, why worry?

I'm getting all the performance I need from my existing pool. Thus, I don't care about it.

Yet, it seems weird to me that a ZFS Send / Receive to a new, identically-configured pool (nearly identical underlying drives) would end up with free space more fragmented than in the source. If you look up in the thread, you'll see that during my last send / receive, free space fragmentation increased as the pool filled up. I'll admit that I don't like that. In contrast, the source VDEV is now at 90% full and has 23% free space fragmentation. I'd just like to find a way to make my new (encrypted) VDEV start where my current VDEV stands.

Hence the tests. Some people line up their books on bookshelves in alpha order, others by color, others by descending height. OCD? Sure. Some people will do whatever it takes to lower free space fragmentation because it is somehow important to them. I think it is worthwhile to run a few tests because heck: I've got the tools and a little bit of free time, so why not? [edited to remove an implication that free space fragmentation may in some way be correlated with performance; I've no intention of making that claim]

TC
 
Last edited:

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
I think assuming higher % free space fragmentation == lower pool performance may not be valid. The calculation is not that simple. I wish I could find the video where one of the devs is discussing metaslab data structures and proposing changes to take pool size into account, and other interesting tidbits, but my Google-fu is not strong enough.
 

tc60045

Dabbler
Joined
Nov 22, 2016
Messages
25
I think assuming higher % free space fragmentation == lower pool performance may not be valid. The calculation is not that simple. I wish I could find the video where one of the devs is discussing metaslab data structures and proposing changes to take pool size into account, and other interesting tidbits, but my Google-fu is not strong enough.

Robert, to be clear, I never said that the two were correlated. I will change my driving analogy immediately, as I want to avoid implying it in any way. I stated a preference of avoiding free space fragmentation unnecessarily, which I have represented consistently.

Now, if we want to test performance at different levels of free space fragmentation -- write a test and I'll run it!! I will have two identical pools: one 23% free space fragmented and very full, the other at 49% free space fragmentation and identically full. What shall you have me test?
 

tc60045

Dabbler
Joined
Nov 22, 2016
Messages
25
I've written a number of python-and-redis scripts that I think are really helpful for ZFS file wrangling. I will publish them to GitHub and discuss that in a separate thread soon, but for my contest entrants -- and for others watching here -- I thought this was pretty interesting.

My "media" dataset has been bugging me. It has the same recordsize of 128K as all my datasets, but I have allowed a mix of file types and sizes to gather there, and I think I could stand to divide and conquer. In particular, all the media "config" files are there, with all their attendant "junk" -- things like Plex .jpeg files and scene maps and so on.

Have a look at the picture. I chose the file size bins to look for things like 512B versus 4K. Happy to re-size bins if looking at this differently would be (more) interesting. Also, with a little bit more work, I will soon have a list of all files in descending size for each dataset. In case that would be helpful.
Picture 2016-12-01 at 9.54.22 PM.jpg
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
While tests are ongoing - thank you, @tc60045 - can someone explain to me in a vague, rule-of-thumb way when we should care about fragmentation?
Large write operations. If TXGs end up spread around, performance will suffer.
 

Bidule0hm

Server Electronics Sorcerer
Joined
Aug 5, 2013
Messages
3,710
Phase I is closed. Contestants BiduleOhm, SweetAndLow and DarkWarrior, please PM me with the commands you want me to run. I will not disclose to any contestant what another is doing for their entry. I will, however, advise each if they are about to duplicate a less-than-stellar approach (I've had a few now...).

Once Bidule has his ready, I will execute his first and report results to all (but not approach), then SweetAndLow, then DarkWarrior. Winner will be announced, prizes awarded. Then we can still run a few more tests more publicly. Any objections, speak now. Otherwise, begin the PMs and let's light this candle!

It's ok for me, I'll PM you the commands ASAP.
 

tc60045

Dabbler
Joined
Nov 22, 2016
Messages
25
@tc60045, I'm not attacking you, and yours is not the only other voice in this thread.

Nothing, thanks ;)
Robert, apologies -- I know you are not attacking me. It is out of respect to the vast experience you and others have here in this world of ZFS that I wanted to make very clear that this total neophyte has no business and no interest in making claims given a data set of 1 pool and 6 months of experience. No way.

What you say on that matter -- or jgreco or others -- will carry much more water. I'm just trying to stay in the "helpful" box. Thx! TC
 
Status
Not open for further replies.
Top