Can I mix Hard Drives

Status
Not open for further replies.

9C1 Newbee

Patron
Joined
Oct 9, 2012
Messages
485
I think usage is a factor. I can't tell you exactly how or why it needs it. But I value my data. I am not a FreeNAS ninja. So I maxed out my system with 32GB of RAM just to be on the safe side. So far, no problems, other than junk drives.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
I don't understand that because it would make more sense it the RAM requirments were contingent on mow many user clients were logging onto the NAS at one time requesting data. In other words, I have heard of the 1GB per TB rule, but if you had an 8TB array and were using it for a home NAS streaming media with one user at a time and on a rare occasion maybe two, why wouldn't 4GB work without those issues. And to further the scenario, if you had the same at home NAS, 1-2 users, but it was a 16TB NAS, why would you need 16GB or RAM?

You are partially correct in that more clients would make more RAM more useful, but it also has to do with pool size and usage patterns. ZFS performance tends to increase as RAM increases, basically because of the increase in read and write cache.

Our 30TB (12 4TB in RAIDZ3 with a spare) filer here is running under ESXi on a 128GB platform, so experimenting a bit I noticed that write performance when it had 32GB of RAM was about 3X what it was if I gave it 6GB. No users. Just writing data at full speed locally.

So the problem with complex systems is that they are complex. We develop rules of thumb like the 1GB/1TB things as a one-size-kinda-fits-most but there are still modifiers and qualifiers.

We say min 8GB RAM these days. for example, because less has been observed to sometimes result in unmountable pools in crash/powerfail scenarios.

We say 1GB/1TB because that seems to result in good average use case performance.

We also tend to double (or more!) the RAM size for VM storage usage largely because ZFS is a CoW filesystem and therefore 512-byte sector-sized writes in the middle of a file wreak a bit of havoc. Other things like L2ARC are further modifiers.

But basically if you know what you are doing and don't mind low performance the only rule I would consider "mandatory" is the 8GB min. Even that can be mitigated if you understand it.

We develop these guidelines for the benefit of newcomers.

But 100 users attached and doing nothing is a lot less stressy on a NAS than a single user using an iSCSI extent for video editing work. So the number of users is less relevant than what the NAS is actually doing. So we mostly consider use case, actual activity, and pool size as important factors. The number of users isn't generally too useful in considering RAM sizing.
 

Richman

Patron
Joined
Dec 12, 2013
Messages
233
I think usage is a factor. I can't tell you exactly how or why it needs it. But I value my data. I am not a FreeNAS ninja. So I maxed out my system with 32GB of RAM just to be on the safe side. So far, no problems, other than junk drives.

32GB for ....."Just me and the girlfriend streaming Gilmore Girls" I get it, your just one of those people with deep pockets. Any spare change you can throw my way. I am living under technology highway north bridge overpass in a cardboard server packing create.
_____________________________________
So I guess it is safe to say if I started out with a 3-4TB pool until I can buy some 2-3TB disks that 4GB would work on a NAS for home without these issues yo mentioned or can some of them still be an issue. I may eventually set up a web server with attached storage but that isn't for a while yet.

We say min 8GB RAM these days. for example, because less has been observed to sometimes result in unmountable pools in crash/powerfail scenarios.
I didn't realize that ZFS had any crash/powerfail protection and thought that without a UPS you were up a creek without a paddle, almost like unplugging a drive mid write. Which reminds me that maybe my supermicro SCE-M35T may not be a great choice since it isn't locking. I could put it behind a lock panel maybe. Important since I can see home visitors saying, "Whats this"? An pressing the burgundy button to dismount my drive.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
32GB for ....."Just me and the girlfriend streaming Gilmore Girls" I get it, your just one of those people with deep pockets. Any spare change you can throw my way. I am living under technology highway north bridge overpass in a cardboard server packing create.

Hey, I have less than you do. When it comes to RAM, what jgreco said is spot on. How much you need to be happy with performance varies widely. There is no "this is what you need". You should always try to overshoot as undershooting has proven fatal for users. I can't tell you how many user's we've had that had a sudden loss of power or system crash, rebooted and their data was gone. There are no recovery tools for ZFS, so if the pool doesn't mount you lose your data forever. This isn't like Windows where you can use tools like rstudio to do data recovery.

In short, if you get stupid with RAM and try to use less than the recommended don't expect much sympathy when things go bad. I used to get upset about it. But you know what, if you don't want to listen to the stickies then don't. Be one of the few that "sticks it to the man" and gets away with it. But plenty of people here have learned the hard way and after they lose their data they go back and just blindly follow the stickies because losing data once is painful enough for you to be humbled back to reality that you might not know as much as you think you do and that you should listen to the people on the forum that are familiar with this new fangled technology.
 

Richman

Patron
Joined
Dec 12, 2013
Messages
233
I swear The hardware wiki said 4GB min just weeks ago but now says 8GB.. In a 3-4 GB array the whole array would almost fit n RAM. I just plan to try it out on hardware and then start purchasing upgraded hardware and then go past 4TB. Maybe I will just do a 3TB JBOD to back op other disks till I get the hardware.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
It's said 8GB for quite a few months.

A 4GB array will only fit in RAM if you have about 8GB or so of RAM.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
I didn't realize that ZFS had any crash/powerfail protection and thought that without a UPS you were up a creek without a paddle, almost like unplugging a drive mid write. Which reminds me that maybe my supermicro SCE-M35T may not be a great choice since it isn't locking. I could put it behind a lock panel maybe. Important since I can see home visitors saying, "Whats this"? An pressing the burgundy button to dismount my drive.

ZFS is supposed to be crash/powerfail safe but we periodically see people experience problems that appear to be related to this. The code was well-tested for this kind of thing under Solaris, but FreeBSD is rather different and it is not clear that the testing of all possible failure modes would have been as comprehensive as it would have been in the commercial OS development environment at Sun.
 

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
ZFS is supposed to be crash/powerfail safe but we periodically see people experience problems that appear to be related to this. The code was well-tested for this kind of thing under Solaris, but FreeBSD is rather different and it is not clear that the testing of all possible failure modes would have been as comprehensive as it would have been in the commercial OS development environment at Sun.

Yeah, on the couple of occassions that I've improperly powered off my NAS I've always had to redo the USB drive before it'd start working again. It just wouldn't bopot back up after cold shutdown.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,680
That may be more a function of stuff having been going on with writing to the flash and/or other problems inherently related to appliance-style designs. It is very difficult to get all bases covered properly - I speak from experience ;-) ... but quite frankly on a scale of 1...10 the need to reload a USB device is about a 2 compared to loss of pool. I'm more concerned when I hear about pool loss. In theory ZFS should not be subject to such problems.
 

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
Yeah, definitely a lot worse than losing the entire pool. But just really trying to say, if a cold kill can destroy the USB drive, who's to say it can't do it to a zpool?
 

Dusan

Guru
Joined
Jan 29, 2013
Messages
1,165
Yeah, definitely a lot worse than losing the entire pool. But just really trying to say, if a cold kill can destroy the USB drive, who's to say it can't do it to a zpool?
Apples & oranges -- It's two totally different file systems (ZFS vs. UFS) with completely different approach to redundancy and resiliency.
 

Knowltey

Patron
Joined
Jul 21, 2013
Messages
430
Ah yeah, forgot that was on UFS.
 
Status
Not open for further replies.
Top