Adding redundancy (RAID10) at a later date. Possible?

[BirdFlu]

I can't seem to find a straight answer to this. All my searching thus far has turned up results from people wanting to convert an existing RAID0 to a RAID10 while retaining the same number of disks.

What I want to do is create a RAID0 array with 2 disks, with a view to add a further 2 disks at a later date creating a RAID10. I want redundancy, NOT extra storage.

I just want to know whether this is possible, I haven't set up anything yet, haven't even built a PC to house all this. I have 2 disks ready to go, they're just expensive and I can't afford to purchase another 2 right now.

In my head it seems logical enough, I've learned that these things are rarely as easy as they sound!

Cheers!

*edit - I'm thinking in RAID terms when perhaps I should be better off thinking in RAID-Z. Now I'm confused.

I will be starting out with a "stripe" setup, with a view to doubling the disks and moving to a RAID-Z2 setup. Hopefully without having to move everything elsewhere when the time comes to add the second lot of disks.

edit 2 - Been playing with FreeNAS in vmware, it seems that the only way of doing what I want to achieve is by starting with a mirror and then extending that with another new mirror. Is this correct or can I do it as I originally proposed? I might be able to get by for a while with half the storage space, but would rather not if at all possible.

Cheers again!

 

[HoneyBadger]

Edit 2 is correct, you can't convert a stripe to a mirror.

You could start with a stripe, create a new mirror when you get two more drives, try to cram everything into the half-size mirror set (either via manual/pool compression or simply a judicious pick-and-choose of what you really need) and then destroy your stripe and use the two freed drives to extend the mirror.

 

[BirdFlu]

Weird... There was another reply that seems to have disappeared...

Thanks for the reply, I think for the purpose of keeping things simple I'll just mirror it to start with. It's doubtful I'll fill it up in the next 6 months or so.

The other reply that mysteriously is in my email inbox but not appearing here outlined how to do what I originally proposed through the command line; creating 2 new vdevs for each new disk, then using the "zpool attach" command to attach those disks.

Got my HP microserver today, going to start playing around with it soon. No doubt I'll be back on here in no time!

 

[jgreco]

Please post the output of "zpool status" to clarify what you have.

 

[gpsguy]

Did you buy a Gen 8 or Gen 7?

Since they only have 2 memory slots, I'd encourage you to upgrade straight to 16Gb ECC RAM, if you haven't bought a memory upgrade yet.

Got my HP Microserver today, going to start playing around with it soon.

 

[BirdFlu]

It's a g7, is 16gb really necessary?! I was planning on installing 4! Guy I got it off on eBay sold it with 1!

 

[gpsguy]

You talked about using some form of RAIDz. ZFS on FreeNAS, requires a minimum of 8Gb of RAM. With 4Gb, you'd be limited to using UFS. I'm using the memory below, in my N54L.

Kingston 16GB (2 x 8GB) 240-Pin DDR3 SDRAM DDR3 1333
ECC Unbuffered Server Memory Model KVR1333D3E9SK2/16G

If you want to start with just 8Gb, buy one of these: KVR1333D3E9S/8G It'll give you the flexibility to upgrade to 16Gb later on, without loosing an investment in RAM.

 

[jgreco]

It's a g7, is 16gb really necessary?! I was planning on installing 4! Guy I got it off on eBay sold it with 1!

What you really need to do is to actually read the Hardware Recommendations and notice that 8GB is the minimum required for stable operation with ZFS. This isn't 2005 anymore. 4, 6, these have been pool-losing numbers for people. 16 is not strictly necessary but you'd be smart to buy a single 8GB module rather than two 4GB modules, so that you could later upgrade to 16 without having to throw away the pair-of-fours.

 

[HoneyBadger]

The reason that the Microservers can get away with such low amounts of RAM installed from the factory is simple - cost cutting. You can get away with installing a Linux distro or WHS, formatting your disks as ext4 and sharing via Samba, and it will be stable if unimpressive.

As posted by the others, get a single 8GB stick so you have the other slot free for upgrades.

 

[jgreco]

It isn't cost cutting. It is simply providing a product that is useful to many people. For most of the intended uses of the Microserver, 1GB is completely adequate. It is intended as a low cost entry level server that is functional.

Unfortunately, while FreeNAS will work with such a processor, it will not work with 1GB of RAM. And the 250GB HDD that is included with many of the Microservers is also not too useful for the average FreeNAS installation. So rip out what isn't needed and fix it. :)

 

[BirdFlu]

What you really need to do is to actually read the Hardware Recommendations and notice that 8GB is the minimum required for stable operation with ZFS. This isn't 2005 anymore. 4, 6, these have been pool-losing numbers for people. 16 is not strictly necessary but you'd be smart to buy a single 8GB module rather than two 4GB modules, so that you could later upgrade to 16 without having to throw away the pair-of-fours.

Thankyou so much for your valuable insight! As a matter of fact I have read the hardware recommendations and even more astoundingly I am well aware of the current year! For some reason I got it into my head that 4GB was the minimum recommended RAM for ZFS, not 8. Simple brain fart.

I will start with a single 8GB module as suggested. Still seems like a lot for a home server, I have 16GB in my MacBook and consider this to be merited for the work I do on it ... but for a box that just serves media files on my home network? Jeez!

 

[jgreco]

ZFS is intended as an enterprise grade file system for serving many terabytes of data.

It was designed from the ground up to assume that compute resources, both CPU and memory, were readily available, which is typically the case on the big Sun servers that it was designed for. ZFS has a ton of great features that conventional filesystems do not have.

Speaking as someone who learned UNIX on machines with approx ~ 256KB (yes, KB) of system memory, I actually expect that a system with 32MB of RAM ought to be plenty for a fileserver when using a traditional filesystem. However, as a software designer, I also understand that if you design something for lots of resources, it might simply not work (right, or at all) when given less than what it was designed for.

 

[BirdFlu]

I'm a software engineer myself and it seems like a very sloppy design decision that such a fundamental part of a system (the filesystem) is inoperable if 1 of 2 4GB DIMMs fails! Even if it is intended for enterprise scale servers!

Thanks for the info anyway, 8GB is the way forward. If I want to double it to 16GB in the future then I always can.

 

[jgreco]

Yes, it's a sloppy design if you know your software requires that and you fail to order the appropriate hardware.

Quite frankly, I had to give up on labeling software as "too bloated" years ago so I won't play that game. In the case of ZFS, I've been inside it and I understand what it is doing and why. From the perspective of someone who spent a few decades working with some of the most challenging and intensive I/O environments, I am very keenly aware of the limits of the legacy filesystem designs - while simultaneously appreciating how far they managed to scale (albeit with lots of help). Those things were designed to work on systems with a megabyte of RAM. At a certain point, it is a healthy thing to reassess your preconceived notions. Sun did that, and went big. They wanted a filesystem that would be reasonable for the NEXT twenty five years (or more!). They reasonably guessed that resources would grow, and now that we're eight years into its lifecycle, we can see that they didn't miss the mark.

Like it or not, if I ever want to serve files off of 4MB of RAM again, I'll have to pull a Sun 3/50 out of the closet and hook it up. FreeBSD hasn't run on such a small machine since, what, maybe 2.x? So be aware: if you're running ZFS, you're going to need resources. If you don't want to provide them, Openfiler's over thataway.

 

[david kennedy]

Yes, it's a sloppy design if you know your software requires that and you fail to order the appropriate hardware.

Quite frankly, I had to give up on labeling software as "too bloated" years ago so I won't play that game. In the case of ZFS, I've been inside it and I understand what it is doing and why. From the perspective of someone who spent a few decades working with some of the most challenging and intensive I/O environments, I am very keenly aware of the limits of the legacy filesystem designs - while simultaneously appreciating how far they managed to scale (albeit with lots of help). Those things were designed to work on systems with a megabyte of RAM. At a certain point, it is a healthy thing to reassess your preconceived notions. Sun did that, and went big. They wanted a filesystem that would be reasonable for the NEXT twenty five years (or more!). They reasonably guessed that resources would grow, and now that we're eight years into its lifecycle, we can see that they didn't miss the mark.

Like it or not, if I ever want to serve files off of 4MB of RAM again, I'll have to pull a Sun 3/50 out of the closet and hook it up. FreeBSD hasn't run on such a small machine since, what, maybe 2.x? So be aware: if you're running ZFS, you're going to need resources. If you don't want to provide them, Openfiler's over thataway.

It is interesting to see people come here and say they want ZFS and all of its benefits, but then balk at the memory needs?
What are we talking about, $100 for 8GB of ram these days?

As you point out, there are non-ZFS alternatives like OpenFiler with lower memory requirements.

 

[BirdFlu]

I can see I've wandered into somewhat of a circlejerk here...

Having never used ZFS, I think users could be forgiven for thinking 'woah, I need 8gb RAM minimum to support a file system? That seems like an awful lot for my little home server'

And yeah, $100 on top of everything else (I did decide to go with the 4 disks after all to return back to topic) is a lot.

 

[jgreco]

Yeah, well, you're really not going to get a whole lot of sympathy. It comes as an awful shock to people who are running Windows XP on 64MB RAM and an ISA sound card that their PC's won't run Windows 8.1, sure. Both numbers come as a shock to Bill ("640K is enough for anybody!") Gates. But quite frankly one of the things I've learned over the decades of working with computers is that the documentation is usually right, and therefore usually worthy of a readthrough.

And since we basically cover the topic several times a week from those who:

1) Cannot be bothered to read for themselves, or

2) Cannot be bothered to believe that someone else already explained it in simple words elsewhere in the forum and demands that it be explained in simple words personally to them, or

3) Cannot cope with the overall concept that Sun leapfrogged well ahead of the hardware growth curve when designing a next-gen product, which we're only now approaching as being a reasonable minimum, or

4) One of other various subclassifications of "but but but..." that most of us are all very tired of listening to and writing answers to.

You can label it a circlejerk if you want. But I'm not paid to be here. I'm a user, just like you. Feel free to sit here and field the questions for a few years. Be sure to suffer through a whole bunch of n00b tears when people who REALLY though that NO WAY IN HELL did that "8GB" mean that they really needed 8GB, and then something went wrong and shredded their pool. We don't know why. The users that suffer those catastrophes are invariably too new to UNIX to do any deep exploration of the issue. Several of us who are more experienced, or do this professionally, or whatever, noted observationally from the experiences of other forum members that the problem disappeared at 8GB. Again, we don't know why. I'm not going to risk pools trying to figure it out. So instead of saying "thanks for the warning" you label it a circlejerk.

Well, at least we know you like to make an impression when you enter the room.

 

[BirdFlu]

I will start with a single 8GB module as suggested.

Thanks for the info anyway, 8GB is the way forward. If I want to double it to 16GB in the future then I always can.

6 and 3 replies back respectively ... But anyway...

I posted a fair enough question and when it came to light I had it wrong I changed my view. So what if I responded to new information with a perfectly valid air of puzzlement? If it's such a common topic of debate why not make a sticky on it?

When I see terms thrown around like 'noob tears' referring to new users' teething problems it certainly doesn't make me want to participate in this community, which is a shame since community is such an important element of projects like FreeNAS.

I shall perhaps visit /r/freenas for support in the future if I am to expect this sort of response here.

 

[jgreco]

"Why not make a sticky?"

Because it's already in the manual, and the hardware sticky, and about a thousand forum posts.

In the meantime, since you never answered the original request for information and are appearing to be a troll, your opportunity to get a solution to your problem from me has vanished.