Expanding storage

[sdspieg]

I currently have 2x3TB in zfs-mirrored, and am running out of space

Code:

root@freenas ~]# zpool status                                                  
  pool: HomeNAS                                                                 
 state: ONLINE                                                                  
  scan: none requested                                                          
config:                                                                         
                                                                                
        NAME                                            STATE     READ WRITE CKSUM                                                                              
        HomeNAS                                         ONLINE       0     0     0                                                                              
          mirror-0                                      ONLINE       0     0     0                                                                              
            gptid/7644d7d8-f9c9-11e1-b403-3085a943b0e3  ONLINE       0     0     0                                                                              
            gptid/76dd3a3d-f9c9-11e1-b403-3085a943b0e3  ONLINE       0     0     0          

Can I add a second vdev to this zpool with 2x2TB and would that then give me a total 5TB to play with (redundantly)? Also, could I add 3x2TB in RAID-Z for an additional 1TB of available space (i.e. a total of 6 then)? And do I do that by just adding another volume with the same name. i.e.

Code:

$ zpool add HomeNAS raidz /dev/da3 /dev/da4 /dev/da5

? Did I get this right? If not, can somebody please set me straight? Thanks much!

-Stephan

 

[paleoN]

Can I add a second vdev to this zpool with 2x2TB and would that then give me a total 5TB to play with (redundantly)?

Yes. What's more you can upgrade the 2x2TB vdev by replacing the drives with 2x3TB in the future for 6TB raw. Keep in mind when you add a vdev ZFS doesn't restripe existing data across the vdevs.

Also, could I add 3x2TB in RAID-Z for an additional 1TB of available space (i.e. a total of 6 then)? And do I do that by just adding another volume with the same name.

You could, but it's not recommended to mix different vdev types. More importantly I wouldn't run 3TB drives in raidz1 anyway. You could always add 4 3TB drives and have a 6x3TB raidz2 pool for 12TB raw.

 

[sdspieg]

Thanks for that warning. But so what would you recommend? The main reason I ran out of space is the fact that I finally copied an older picture collection from an NTFS-drive to the NAS. So it's now on the first vdev. Would it help if I deleted that folder first, THEN created the second vdev, and then recopied that folder to the new expanded zpool? Would that 'balance' the load more equally? Or would it not matter?

 

[sdspieg]

Ok - so here is where I stand now:

Code:

[root@freenas ~]# dmesg | grep -e ada                                           
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0                                     
ada0: <ST3000DM001-9YN166 CC4B> ATA-8 SATA 3.x device                           
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)                    
ada0: Command Queueing enabled                                                  
ada0: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)                 
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0                                     
ada1: <ST3000DM001-1CH166 CC24> ATA-8 SATA 3.x device                           
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)                    
ada1: Command Queueing enabled                                                  
ada1: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)                 
ada2 at ahcich2 bus 0 scbus2 target 0 lun 0                                     
ada2: <ST3000DM001-9YN166 CC4B> ATA-8 SATA 3.x device                           
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)                    
ada2: Command Queueing enabled                                                  
ada2: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C)                 
ada3 at ahcich3 bus 0 scbus3 target 0 lun 0                                     
ada3: <ST3000DM001-1CH166 CC24> ATA-8 SATA 3.x device                           
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)                    
ada3: Command Queueing enabled                                                  
ada3: 2861588MB (5860533168 512 byte sectors: 16H 63S/T 16383C) 

Code:

[root@freenas ~]# camcontrol devlist                                            
<ST3000DM001-9YN166 CC4B>          at scbus0 target 0 lun 0 (ada0,pass0)        
<ST3000DM001-1CH166 CC24>          at scbus1 target 0 lun 0 (ada1,pass1)        
<ST3000DM001-9YN166 CC4B>          at scbus2 target 0 lun 0 (ada2,pass2)        
<ST3000DM001-1CH166 CC24>          at scbus3 target 0 lun 0 (ada3,pass3)        
<HitachiG ST 0000>                 at scbus6 target 0 lun 0 (pass4,da0)         
< USB Flash Memory 1.00>           at scbus7 target 0 lun 0 (pass5,da1)

Code:

[root@freenas ~]# zpool status                                                  
  pool: HomeNAS                                                                 
 state: ONLINE                                                                  
  scan: none requested                                                          
config:                                                                         
                                                                                
        NAME                                            STATE     READ WRITE CKS
UM                                                                              
        HomeNAS                                         ONLINE       0     0    
 0                                                                              
          mirror-0                                      ONLINE       0     0    
 0                                                                              
            gptid/7644d7d8-f9c9-11e1-b403-3085a943b0e3  ONLINE       0     0    
 0                                                                              
            gptid/76dd3a3d-f9c9-11e1-b403-3085a943b0e3  ONLINE       0     0 

I'm still looking for a command line command so see which ada's are actually in the first vdev now. Can anybody help me with that? I obviously don't want to make any mistakes here.

And once I have identified the 2 new drives that I want to add to the zpool, can somebody please confirm that this is what I should do (in order to add a second vdev with the same 'mirror' group type as the first vdev to my HomeNAS zpool): "$ zpool add HomeNAS mirror /dev/ada3 /dev/ada4"? And is that all then?

Thanks much! I am totally new to this and I AM trying to do my homework, but there are still very few noobie-proof instructions out there for what I am trying to do here...

 

[paleoN]

But so what would you recommend?

What I wrote above. Get 4x3TB drives to make a 6x3TB raidz2 pool. With all six drives in the system you can copy all existing data from the current pool to the new one and maintain redundancy. You just have to play around with it a little.

The main reason I ran out of space is the fact that I finally copied an older picture collection from an NTFS-drive to the NAS. So it's now on the first vdev. Would it help if I deleted that folder first, THEN created the second vdev, and then recopied that folder to the new expanded zpool?

Yes, and I would unless you go with the above suggestion and then it isn't needed. I would actually change this slightly and create a new pool with the new drives, copy over the data (minus the picture collection), destroy the old pool and extend the new pool with the old drives. Optionally, copy the moved data in the new pool to a new location in the pool to have it 'balance' across the vdevs. This step is somewhat unnecessary and depending on the amount of data may not be very helpful. Finally, copy the picture collection back.

 

[paleoN]

I'm still looking for a command line command so see which ada's are actually in the first vdev now. Can anybody help me with that?

Code:

glabel status

And once I have identified the 2 new drives that I want to add to the zpool, can somebody please confirm that this is what I should do (in order to add a second vdev with the same 'mirror' group type as the first vdev to my HomeNAS zpool): "$ zpool add HomeNAS mirror /dev/ada3 /dev/ada4"?

Justed posted after you did above. With 2 new drives I would create a new pool the the new drives via the gui, copy the data (minus picture collection) from the old pool to the new pool, destroy the old pool via the CLI, and then extend the new pool with the old drives. This is cleaner as you won't have 'overfilled' the new pool. Your zpool command is correct and should look like the following, assuming the new pool is tank and the old drives are ada0 & ada1:

Code:

zpool add tank mirror /dev/ada0p2 /dev/ada1p2

 

[sdspieg]

Much appreciated! Sounds like a plan. But just for my comprehension (and that of other people with a similar predicament) - why is your solution better than deleting the picture collection of the old one and then copying it to the new one? Does zfs not 'like' that?

 

[sdspieg]

And btw - here's what the command you gave me shows

Code:

[root@freenas ~]# glabel status                                                 
                                      Name  Status  Components                  
                              ntfs/Hitachi     N/A  da0s1                       
                             ufs/FreeNASs3     N/A  da1s3                       
                             ufs/FreeNASs4     N/A  da1s4                       
gptid/7644d7d8-f9c9-11e1-b403-3085a943b0e3     N/A  ada0p2                      
gptid/76c8f525-f9c9-11e1-b403-3085a943b0e3     N/A  ada2p1                      
gptid/76dd3a3d-f9c9-11e1-b403-3085a943b0e3     N/A  ada2p2                      
                            ufs/FreeNASs1a     N/A  da1s1a                      
                    ufsid/508970881f659d1d     N/A  da1s2a                      
                            ufs/FreeNASs2a     N/A  da1s2a 

Does this mean I did NOT use zfs but ufs? And would you then recommend doing it differently for the new one, i.e. creating a new pool, copying everything from the old to the new (minus the pics), deleting the old one and then also configure it differently (with zfs mirror)? Or is the ufs-stuff just the usb-stick from which I boot FreeNAS?

 

[paleoN]

But just for my comprehension (and that of other people with a similar predicament) - why is your solution better than deleting the picture collection of the old one and then copying it to the new one? Does zfs not 'like' that?

I'm just somewhat paranoid about filling a pool too much. Even after deleting the picture collection there can be some fragmentation, albeit minor, left over. My method avoids any of that.

Does this mean I did NOT use zfs but ufs?

Your zpool command show you used ZFS. ;) Yes, da1 is your FreeNAS install. You also have a single NTFS partition on da0 and your zpool consists of ada0 & ada2.

What drives do you have available and how many drive bays? If you have an additional 2TB drive or the 2x1TB drives there is a better way to move everything to striped mirrors while still being redundant. This way also doesn't matter when the picture collection is copied.

 

[sdspieg]

I'm just somewhat paranoid about filling a pool too much. Even after deleting the picture collection there can be some fragmentation, albeit minor, left over. My method avoids any of that.

I see. So that's what I will do then tonight.

Your zpool command show you used ZFS. ;) Yes, da1 is your FreeNAS install. You also have a single NTFS partition on da0 and your zpool consists of ada0 & ada2.

I had forgotten about that other one. I'm not using it though. I'll have to look into mounting it and checking out what's on there.

What drives do you have available and how many drive bays? If you have an additional 2TB drive or the 2x1TB drives there is a better way to move everything to striped mirrors while still being redundant. This way also doesn't matter when the picture collection is copied.

Well right now I have the old 2x3TB and the new 2x3TB. And that should really be more than enough for quite a while. Until 4k becomes big :) But thanks for all your help - you're an ace!

 

[sdspieg]

Hmmm. I did a cp -R HomeNAS/* tank, but that didn't work very well: it ran for hours, and at the end, the newly created pool 'tank' is now at 100%, even though the original one was only 71% after deletion of the picture collection. What did I do wrong, and how can I fix it?

 

[sdspieg]

Hmmm. I did a cp -R HomeNAS/* tank, but that didn't work very well: it ran for hours, and at the end, the newly created pool 'tank' is now at 100%, even though the original one was only 71% after deletion of the picture collection. What did I do wrong, and how can I fix it?

Still trying to follow the suggestion given here and so I started again. Deleted the new pool, recreated it again and then typed in the following command: cp -iprv HomeNAS/Movies/* tank/Movies (from the /mnt dir). I'll do that for the TVShows dir as well, and will then just reinstall the jail again.

I guess the -R parameter meant that all symlinks were followed, explaining why the size was so much bigger (jail vs outside jail). I hope this time will be the right time...