About to expand my zpool

[mjban]

Hi everybody.
First I want to thank this great forum and people that helped me build my first NAS.
I have the current setup:

ASRock E3C226D2I (Dual LAN - ECC Support)
Intel Pentium G3220
Crucial 8GB DDR3 ECC unbuffered (I will buy another 8GB in the next two months)
2 x 3TB WD Green (changed Intellipark feature to 5 minutes)
1 x 4TB WD Red
1 x 2TB WD Black (I know it's not the best for NAS)

While setting up the ZFS volume, the only option was Mirror, so I have a ZFS Mirrored, only 6Tb usable.

I have another 2TB drive, a Seagate Barracuda and a WD Red 2Tb on it's way since I'm planning to expand the size of the zpool, My only question will be: by adding an extra 4TB mirrored (2Tb usable), I will still have redundancy in my configuration correct?

Also (out of context). I've been having issues configuring Rsync over SSH as PULL from an Ubuntu setup, I have done all possible configurations and I can also SSH login to my FreeNAS passwordles from my Ubuntu computer, but when I a create the Rsync Task I keep receiving the "You must a user with the SSH public key in the home dir" message and of course I have the key assigned to the root user, I have double checked the key like 10 times. Any suggestions?

Thanks!

 

[gpsguy]

Before you think about adding storage, please tell us more about your current setup. If you have 6Tb usable, I'd be surprised if you have a mirror now.

Post the output of: zpool status in code tags.

Also read cyberjock's guide, to learn about vdev's, zpools, etc.

 

[mjban]

Yes, I was kind of surprise too but the only options in the volume manager were stripe, log, cache and mirror.

This is the output:

Code:

  pool: NAS
state: ONLINE
  scan: none requested
config:
 
    NAME                                            STATE    READ WRITE CKSUM
    NAS                                            ONLINE      0    0    0
      gptid/4c02c85e-4287-11e3-958b-bc5ff4e79b13    ONLINE      0    0    0
      gptid/4cf68977-4287-11e3-958b-bc5ff4e79b13    ONLINE      0    0    0
      mirror-2                                      ONLINE      0    0    0
        gptid/4dc60fa7-4287-11e3-958b-bc5ff4e79b13  ONLINE      0    0    0
        gptid/4e7b632c-4287-11e3-958b-bc5ff4e79b13  ONLINE      0    0    0
 
errors: No known data errors

I have read the cyberjock's guide, very useful thought. But actually I haven't seen the type of configuration I want to add which is 4Tb mirrored to a 12Tb mirrored configuration.

 

[titan_rw]

You have very little redundancy in the pool.

You have a 3 way stripe. One member is a single disk. Another member is another single disk. And the last member is a mirror of two disks.

If either of the first two disks fail, you lose the pool.

Not exactly the best configuration.

 

[gpsguy]

OK, so as I suspected you actually have a stripe, with one mirrored vdev.

My guess is that the 2-3Tb drives are mirrored, so you could lose one of them and have the pool survive. But, should either of the other 2 disks fail, you'll lose everything.

To fix the situation, you'll need to backup your data and start over. Since you have drives on the way, I'd look at starting over with a new configuration. I've got to hit the hay now, but hopefully others will chime in tonight.

 

[mjban]

OK, I thought I had good redundancy. I appreciate your inputs, I have about 3Tb already transferred to the pool :(
So, the extra drive will arrive by wednesday, in the meantime I guess I will have to fix my configuration and start over as gpsguy mentioned, what would be the recommended configuration with the additional drive:

1 x 4Tb WD Red
1 x 2Tb WD Red
2 x 3Tb WD Green
1 x 2Tb WD Black
1 x 2Tb Seagate Barracuda
Total: 16Tb

 

[cyberjock]

It's just as gpsguy said. You do need to redo your configuration. If either of the 2 single disk vdevs fails you will lose all of your data. You won't get a "get out of jail free" card on that one.

You should backup your data and rebuild your pool from scratch properly. ;)

If you build it and want someone to verify its a safe configuration post the output of zpool status here or in IRC and someone can tell you if its good.

 

[cyberjock]

You posted just before I did. Mixing drives doesn't work all that well. Just like with hardware RAID you'll be limited to the smallest disk per vdev. The remaining space will be unavailable. :(

 

[mjban]

Thank you cyberjock. I guess the best option will be to have two or three different zpools, the only issue is that I don't have another 4Tb drive

 

[cyberjock]

You could mix the different sized disks in different vdevs. That would allow you to have one large pool and not sacrifice disk space from mis-sized disks.

 

[mjban]

My BIG mistake:
Proceeding with my backup, I just found out that the HDD I thought was 2Tb is instead 500Gb! I don't know how this happened, but I have the 500Gb HDD in my zpool, the 2Tb HDD is not assigned to any zpool :(. Of course I think I will have to discard the 500Gb drive (use it for backup), the final configuration should be:

1 x 4Tb WD Red
1 x 2Tb WD Red
2 x 3Tb WD Green
1 x 2Tb Seagate Barracuda Green

 

[gpsguy]

Get another 4Tb drive.

Also, document your build, so you know exactly what you have. Label the end of the drive with the serial number. Should you need to replace a drive, having it visible might prevent you from accidentally removing the wrong one.

 

[mjban]

I can grab another 4tTb Drive but I won't be able to buy the ECC memory to expand to 16Gb at least until next year. Will that affect my performance? since I will end up with 6Tb more than expected when I started the build.
ZFS Mirror is still the best configuration for performance?

 

[gpsguy]

Certainly it could effect your performance.

You say you've read cyberjock's guide, but "haven't seen the type of configuration I want to add which is 4Tb mirrored to a 12Tb mirrored configuration." First of all, I don't know what you're trying to say/do. And, with your mix of drives, you don't have enough to create a 12Tb mirror. Perhaps, that's why haven't see the configuration you're looking for - it doesn't exist.

Starting with your final configuration drives, here are some choices (assuming you want some redundancy/fault tolerance).

1 x 4Tb WD Red
1 x 2Tb WD Red
2 x 3Tb WD Green
1 x 2Tb Seagate Barracuda Green

1) Create 2 mirrored vdevs with 2x2 and 2x3 [keep the 4Tb drive as a spare]
a) Create 2 zpools, one ~ 1.8Tb, the other ~ 2.7 Tb
b) Create a single zpool using the 2 vdev's above. Usable space would be ~4.5 Tb

2) Buy another 4Tb drive. Mirroring it, would yield about 3.6 Tb of usable space. Look at options 1a and 1b above for placement ideas.

3) Create a RAIDz2 with the drives above and an additional 4Tb drive. Usable space would be ~7.2 Tb. But, you'd only need 2 parity drives to protect the whole pool. If, over time, you were able to replace the 4 smaller drives with 4Tb ones, you'd be able to automatically expand your pool to ~ 14.4 Tb, without having to rebuild your array from scratch again. Unfortunately, you won't get the added space until all your drives have been upgraded to larger ones.

 

[mjban]

The problem was that I unfortunately did not write down the drive's serial numbers, I followed the device number and mismatch the final configuration. :(

Now, I have two drives coming in tomorrow: WD Red 2Tb and WD Red 4Tb.

gpsguy, I think your third option will be great based on that I have another 4Tb drive on it's way. My only concern will be performance, I have read that in a 1Gbps Ethernet environment, you won't see difference in performance with RAIDz2 or Mirrored. My current motherboard supports Dual-LAN but my router doesn't :( so I can't take advantage of this. Probably in the future I will upgrade my router so I really want to be sure that I select the best option based on performance.

I was thinking that option 2 will good too, I will end with three zpools but better performance.

 

[mjban]

With all 6 drives, I'm unable to create a RAIDz2, the only option its Mirror.

 

[warri]

You're running into this problem: http://forums.freenas.org/threads/raidz-on-different-sized-hdds.15161/
FreeNAS 9.x doesn't allow the creation of "non-optimal" pools via the GUI. You have to do it via CLI or downgrade to 8.3.2.

 

[mjban]

I see! :)
8.3.2 does not recognize my ethernet adapter, so I'm stuck with 9.X. Any CLI guide for FreeNAS?

 

[warri]

Look at the following thread, especially post 2 and 3: Create zfs mirror by adding a drive?
They attach a drive, but creating a pool from scratch consists basically of the same steps.

To set up the partitions, repeat for every drive (roughly, see linked post): gpart create, gpart add swap, gpart add zfs.
Then get the gptids with gpart list (the adaXp2's) and create a pool with zpool create <poolname> raidz2 <list gptids> (instead of zpool attach as described in the other thread).

You'll get an error about mount point not existing, but don't worry about that. Last but not least you need to export the pool with zpool export <poolname>. After that you should be able to auto-import via the GUI.

EDIT:
Fixed the URL, thanks.

 

[gpsguy]

warri, there was a typo in the URL you provided.

i was just looking at the same thread.