SOLVED Pool size unchanged after adding larger hard drive

[burnthelies]

Hi guys, my experience with FreeNAS is pretty limited. I built it myself being a technical guy but used the FreeNAS documentation and maybe some guidance from walkthroughs etc. to set it up initially.

Of course this has created its own set of problems in that when i built it i used 4 x 6TB WD Red hard drives expecting to be able to add additional drives when needed. Of course i now understand that i should have created the pool differently to make that scenario much easier, but live and learn.

That brings me to my current dilemma, my girlfriend and i do a lot of freelance photography/videography work and over the past 2 years we've filled our NAS so it is currently at 82% capacity.

I added an additional 10TB drive and ran the resilver process before disconnecting the chosen 6TB drive.

All ran perfectly but once the system was up and running again the size of the pool has not increased. I've read a few different posts on here and found that the 'autoexpand' function was set to 'off' by default but after many hours working in the CLI i just don't have the knowledge and can't find it anywhere on these forums or elsewhere to solve the problem.

Here are the results of zpool status etc.

Code:

0                                                                           
            gptid/51868abd-40d8-11e6-8547-408d5cd6231f  ONLINE       0     0   
 0                                                                             
            gptid/52268263-40d8-11e6-8547-408d5cd6231f  ONLINE       0     0   
 0                                                                             
            gptid/52cdaa8e-40d8-11e6-8547-408d5cd6231f  ONLINE       0     0   
 0                                                                             
            gptid/e8ff310a-3260-11e9-abc9-408d5cd6231f  ONLINE       0     0   
 0                                                                             
                                                                                
errors: No known data errors                                                   
                                                                                
  pool: freenas-boot                                                           
 state: ONLINE                                                                 
  scan: scrub repaired 0 in 0 days 00:00:12 with 0 errors on Sun Feb 17 03:45:12
 2019                                                                           
config:                                                                         
                                                                                
        NAME        STATE     READ WRITE CKSUM                                 
        freenas-boot  ONLINE       0     0     0                               
          da0p2     ONLINE       0     0     0                                 
                                                                                
errors: No known data errors

NAME           SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT                                                                               
burnstore     21.8T  17.9T  3.87T         -    33%    82%  1.00x  ONLINE  /mnt 
freenas-boot    14G   842M  13.2G         -      -     5%  1.00x  ONLINE  -     

NAME       PROPERTY    VALUE   SOURCE                                           
burnstore  autoexpand  on      local

Any help would be greatly appreciated, so i can stop pulling out what remaining hair i have.

 

[HoneyBadger]

You seem to be missing some output in your zpool status command - specifically how your pool is configured.

The short answer is that the smallest drive in a vdev determines the sizing. If this is a pool with a single RAIDZ vdev, then you have to replace the other three drives before the vdev will expand.

Is attaching additional drives an option with your current motherboard/HBA/case combination?

 

[Jailer]

I added an additional 10TB drive and ran the resilver process before disconnecting the chosen 6TB drive.

How exactly did you do this?

 

[HoneyBadger]

How exactly did you do this?

By following the directions in the doc, I assume:

https://www.ixsystems.com/documentation/freenas/11.2/storage.html#replacing-disks-to-grow-a-pool

 

[burnthelies]

Thanks for the replies, HoneyBadger is correct re: how i added the 10TB and resilvered.

The motherboard i have has 6 x SATA connectors, but by the sounds of it my worst fears may have been realised. Again, HoneyBadger, your assumptions are correct - this is a single RAIDZ vdev comprising 4 drives.

I've read a whole heap about best practices that i didn't find when i initially built the system, seems i could have done it better.

What are my best options here then?

Sounds like i almost need to go and buy 3 x 10TB drives and just replace the remaining 3 x 6TB ones. I had planned to build a couple of new workstations in a month or two, so the 6TB drives won't go to waste.

Thoughts?

 

[danb35]

Sounds like i almost need to go and buy 3 x 10TB drives and just replace the remaining 3 x 6TB ones.

That's pretty much it. But if you're using RAIDZ (rather than RAIDZ2) with even the 6 TB disks, you're taking a pretty big risk with your data.

 

[Stux]

Pretty much. Do that an it should expand. If not we can then help

 

[Stux]

There then is the issue of raidz1 and 10TB drives being a bit risky

 

[burnthelies]

What makes the 10TB drives risky with raidz1 in particular?

 

[danb35]

Anything larger than about 1 TB is considered fairly risky in RAID5/RAIDZ1 due to the risk of a read error on resilver resulting in data corruption.

 

[burnthelies]

Ok thanks for that additional info danb35, i'll keep that in mind while i decide my plan of attack / discuss with the missus

 

[danb35]

It's worth pointing out that, unlike with RAID5, this probably wouldn't result in complete loss of the pool, but rather isolated corruption, with ZFS able to tell you what's affected. Metadata corruption would be the biggest concern, but ZFS stores at least two (and as many as six) copies of all metadata, so a URE on resilver shouldn't result in, for example, inability to recover the uberblock. Of course, if a second disk outright fails (opinions vary on the likelihood of this--there's a contingent who believe that the resilver places some sort of abnormal or excessive stress on the remaining disks, significantly increasing the likelihood of another disk failing. I haven't seen any evidence that a resilver does any more to a disk than a scrub, which you should be doing regularly in any event, so I don't buy it), you're toast.

The best $/TB value right now is a shucked WD 8 TB external EasyStore (from Best Buy, when on sale for $139) or Elements (from Amazon, $150). Easy to shuck, gives you a WD Red disk, and again you can't beat the value. But getting a pool layout that will provide better protection for your data is going to mean building a new pool and migrating data over.