Sanity check of my server, scheduled tasks etc

[paradoxiom]

Scrubs are turned off right now because I noticed I had about 3,000 snapshots so I think I configured them incorrectly and was creating too many. What should I change if anything on this?

Thanks in advance.

 

[Chris Moore]

You have received no responses before mine because nobody wanted to hunt through this immense graphic to try and figure out what you are talking about.
I almost gave up on it myself.
Ask the question fist, not after the graphics. Then post individual graphics, with text captions, not a single massive graphic.

For example:

Then you can have some more text to tell more about what your question is, possibly even following up with more details and more images.

In essence, your whole question is about snapshots, so it baffles me why you put all the other information in that graphic.

Scrubs are turned off right now because I noticed I had about 3,000 snapshots so I think I configured them incorrectly and was creating too many. What should I change if anything on this?

The answer is, if you are taking snapshots every hour, you shouldn't keep them for 2 weeks. There are a lot of hours in two weeks.
The snapshot schedule I use is to keep snapshots every 15 minutes for an hour; then hourly snapshots for 24 hours; then daily snapshots for 31 days; then monthly snapshots for 12 months. It still ends up being a lot of snapshots, but snapshots only take up the amount of space equivalent to how much has changed since the last snapshot.
I don't understand why you turned off scrubs? The scrub is looking for corruptions in the data. The number of snapshots could increase the amount of data, but is otherwise not related to snapshots.

 

[Chris Moore]

What should I change if anything on this?

More specific questions would be easier to answer.

 

[paradoxiom]

I don't understand why you turned off scrubs? The scrub is looking for corruptions in the data. The number of snapshots could increase the amount of data, but is otherwise not related to snapshots.

Thanks, but to address this first, that was a mistake in typing - I meant I turned off snapshots not scrubs.

Also to be more specific, essentially do my scheduled scrubs/snapshots/smart tests look ok, no conflicts, good frequency of each, etc?

I think my server probably needs more RAM with the amount of data I have, it runs Plex fine with a few users but I remember the rule like 1GB ram per TB of data or something, which i'm far beyond.

My Gen8 only has two slots - both are filled with 8gb sticks - should I shell out for a 16gig stick (or two?) or look at upgrading my entire server in some way? To be honest the Gen8 is just super easy and any other non-microserver looks pretty daunting. Not that i'm a stranger to building systems or anything, I watercool my own built PCs and have done a lot of customization of the Gen8 internally (had about 3 of them at this point). So is there a pre-built server like the Gen8 with more RAM slots I should go for, or would I be better off buying a case with lots of internal drive space and a server board with more RAM slots etc? I had a look at them a while back and they are pretty expensive, wheras a bare-bones Gen8 is like 150quid.

Thanks again Chris.

 

[Chris Moore]

essentially do my scheduled scrubs/snapshots/smart tests look ok, no conflicts, good frequency of each, etc?

One thing at a time.

The smart tests. You have the long test at hour 3 and the short test at hour 4. This means that on the 8th day of the month, the long test will still be running when the system tries to start the short test. The simple solution is to run the short test first. I have my short tests run at midnight everyday. The short test only takes between 2 and 6 minutes, depending on drive size and it doesn't hurt a thing to run it every day. I run my long tests to start at 7 (on my home system) because nobody will be there and trying to use it. Running a long test does slow the drives down because it is actually doing a surface scan. I also configure that for every day of the month on the "Day of month" but then set it to only happen on Monday in the "Day of week", so we are not inconvenienced by it happening on a weekend. Running that once a week shouldn't hurt anything.

I am curious as to why you are keeping a snapshot every hour from 9 AM to 6 PM. Is this a business where you have users making frequent modifications to files?
On systems at work where I have users making modifications and I want to be able to roll back if one of them should make a horrible mistake, I keep a snapshot every 15 minutes and keep 4, then a snapshot every hour and keep 24, then a snapshot every day and keep 31, then a snapshot every month and keep 12. So, I end up with a quantity of snapshots, but I don't understand your schedule in the context of a home system. On my home system I take a single snapshot per day, as a safeguard against accidental deletion because there are only a couple people using it and we will pretty much know when we make a mistake and need to go back to yesterday's version.

I would not have all the pools scrub on the same day. The scrub takes some CPU resources to run and with them all running at once, they will all run slower. Since you have four pools, you should pick four separate days. I have my scrubs set to run once every 3 weeks, by setting the threshold days to 20, selecting every day for "Day of month" but I pick the day it is allowed to run by unchecking the days I don't want it to run.

For the amount of storage and the number of pools, I would say you need more memory. You are consuming 38TB of storage, with some free space still out there, and the general rule is 1GB of memory for 1TB of data. That is not a hard guideline, but the way you are using your system makes it look to me that more memory is called for.

My Gen8 only has two slots - both are filled with 8gb sticks - should I shell out for a 16gig stick (or two?)

You would need to check the documentation for your server, but I don't thing the Xeon E3 V2 processors can access 16GB memory modules. I suppose it is possible that HP did some fumy business in the firmware to allow for a 16GB memory module to work, but I don't expect that to be true.

To be honest the Gen8 is just super easy and any other non-microserver looks pretty daunting. Not that i'm a stranger to building systems or anything, I watercool my own built PCs and have done a lot of customization of the Gen8 internally (had about 3 of them at this point). So is there a pre-built server like the Gen8 with more RAM slots I should go for, or would I be better off buying a case with lots of internal drive space and a server board with more RAM slots etc? I had a look at them a while back and they are pretty expensive, wheras a bare-bones Gen8 is like 150quid.

Since you live on Saturn,

I am not sure what would be available at a reasonable shipping price. I was able to obtain a used server from eBay for about $700 that allows me to have up to 512GB of RAM though it only has 64GB right now, and it has space for 24 hard drives. Pickings are a little slim on eBay right now, but I expect more things to show up later in the year.
That might not help you any. If you have a local system similar to eBay, you might want to investigate used server gear that incorporates the same generation processor you are using. You might be able to score a 'bare bones' model that has no CPU or RAM and sells for less because of it. Then you could transplant your CPU and RAM into the upgraded chassis / different system board.