zpool unavail, cant import 6TB Data loss

[ghost reaper]

hi i did post before had no reply how to try fix problem

i install fresh freenas 9.3 on same usb stick for different pc hardware try do import and it fails
zpool: RAID1 & type: RAIDz1 & missing disk 3TB in pool

going to runs this commands hope it will help you
# camcontrol devlist
# gpart show
# zpool import
# zpool import -f RAID1

Code:

[root@freenas ~]# camcontrol devlist                                           
<HL-DT-ST DVDRAM GSA-H42N RL00>    at scbus0 target 0 lun 0 (cd0,pass0)        
<WDC WD20EARS-00MVWB0 51.0AB51>    at scbus1 target 0 lun 0 (ada0,pass1)       
<ST3000DM001-9YN166 CC9F>          at scbus1 target 1 lun 0 (ada1,pass2)       
<ST2000DM001-1E6164 SC48>          at scbus2 target 0 lun 0 (ada2,pass3)       
<ST2000DM001-1E6164 SC48>          at scbus2 target 1 lun 0 (ada3,pass4)       
<SanDisk Cruzer Switch 1.26>       at scbus4 target 0 lun 0 (da0,pass5)

Code:

[root@freenas ~]# gpart show                                                   
=>      34  15633341  da0  GPT  (7.5G)                                         
        34      1024    1  bios-boot  (512k)                                   
      1058         6       - free -  (3.0k)                                    
      1064  15632304    2  freebsd-zfs  (7.5G)                                 
  15633368         7       - free -  (3.5k)   
                                
                                                                               
=>        34  3907026988  ada0  GPT  (1.8T)                                    
          34          94        - free -  (47k)                                
         128     4194304     1  freebsd-swap  (2.0G)                           
     4194432  3902832584     2  freebsd-zfs  (1.8T)                            
  3907027016           6        - free -  (3.0k)
                              
                                                                               
=>        34  5860530988  ada1  GPT  (2.7T)                                    
          34        2014        - free -  (1M)                                 
        2048  5860528968     1  freebsd-zfs  (2.7T)                            
  5860531016           6        - free -  (3.0k)
                              
                                                                               
=>        34  3907029101  ada2  GPT  (1.8T)                                    
          34          94        - free -  (47k)                                
         128     4194304     1  freebsd-swap  (2.0G)                           
     4194432  3902834696     2  freebsd-zfs  (1.8T)                            
  3907029128           7        - free -  (3.5k)
                              
                                                                               
=>        34  3907029101  ada3  GPT  (1.8T)                                    
          34          94        - free -  (47k)                                
         128     4194304     1  freebsd-swap  (2.0G)                           
     4194432  3902834696     2  freebsd-zfs  (1.8T)                            
  3907029128           7        - free -  (3.5k) 

Code:

[root@freenas ~]# zpool import                                                 
   pool: RAID1                                                                 
     id: 12678803231248864854                                                  
  state: UNAVAIL                                                               
status: The pool was last accessed by another system.                         
action: The pool cannot be imported due to damaged devices or data.           
   see: http://illumos.org/msg/ZFS-8000-EY                                     
config:                                                                       
                                                                               
        RAID1                                           UNAVAIL  missing device
          gptid/61cabbaa-5930-11e4-840c-00241d870195    ONLINE                 
          mirror-2                                      ONLINE                 
            gptid/03041df4-7224-11e4-9488-00241d870195  ONLINE                 
            gptid/4799431b-d8ee-11e3-844c-00241d870195  ONLINE

Code:

[root@freenas ~]# zpool import -f RAID1                                        
cannot import 'RAID1': one or more devices is currently unavailable  

oh yer i inserted live gpart disc and over disc partions have label: RAID1 but 3tb dose not

 

[danb35]

Your pool is not RAIDZ1. It's a single disk, striped with a two-disk mirror, striped with God only knows what else--and the "what else" seems to be missing. Further than that, I don't know--other than to say that the exact build of FreeNAS you're using, and detailed hardware specs, might be helpful in trying to figure out what's going on.

 

[ghost reaper]

old system was freenas 9.2.1 on this hardware intel core 2 duo 3.0ghz & 4G ram & 4 sata max (still got pc, not OS)
finally got real server 10bay 2 x amd quad core with 16ECC ram http://www.tyan.com/Motherboards_S2912-E_S2912WG2NR-E
it is freenas 9.3-release just download it 2 weeks ago, all i did select raidz and it did rest and just looking at gpart 3tb missing swap part overs got it and first part seems be diffrent if that matters and do remember running bad sector program on 3tb before import now im thinking BAD idea

hope here back from you

 

[SweetAndLow]

Did you remove a drive of any thing? Did you ever have 4 disks or just the 3?

 

[Bidule0hm]

"remember running bad sector program on 3tb before import" Well, I guess he has wiped his drive.

 

[danb35]

RAID 1 is not the same as RAIDZ1. RAID 1 is mirroring, while RAIDZ1 is similar (but not identical) to RAID5 of the "traditional" RAID levels. This array has never been any kind of RAIDZ. If I had to guess (and I do, since you don't seem to know what you did to your pool), I'd say you initially created a two-disk mirrored pool (which you called RAID1), and at some point later added a third disk, mistakenly believing that FreeNAS would automagically convert the two-disk mirror into a three-disk RAIDZ1 (in actually, this stripes the new disk with the existing mirror, making that disk critical--when it fails, you lose all your data). I have no idea what you did with the 3 TB disk--that partition layout doesn't look anything like anything FreeNAS would have done. Nonetheless, it seems you somehow striped that disk with the rest of your pool as well. Now that the contents of that disk are hosed, your entire pool is hosed.

Your old hardware was inadequate for use with ZFS, and it's possible that a RAM shortage might have led to pool corruption, but I doubt it. Even your new hardware, though, isn't what's recommend--AMD processors and FreeBSD are known to not play well together.

 

[ghost reaper]

well i had this set up on pc for a while with 4 drives at start hole time no adding and pc change, sata chip giving issue with seagate drives
i did do check on 3tb but no problem, no changes made, put it in new server, can't mount 3tb GPT error, someone on freenas try gpart to fix table
that why partition looks like that, scrubbing been fine.

SO got any ideas at all, i just lost everything, can't see why can't mount it if it mirror & mirror and stripe together, mirror should still run?

be online for next 3hours

 

[Robert Trevellyan]

can't see why can't mount it if it mirror & mirror and stripe together, mirror should still run?

No, when you stripe two vdevs together, which is what you did, every vdev must remain functional to avoid data loss. If you plan to continue using FreeNAS, you need to read this guide before trying again.

 

[ghost reaper]

OK thanks can't fix it and I was planing go to raid6, give me reason to do it now

 

[danb35]

I hope you weren't planning to do RAID6 with FreeNAS, as FreeNAS doesn't directly support RAID6 and the use of hardware RAID controllers is a Very Bad Idea. Perhaps you meant RAIDZ2? If so, please say so--words mean things, and when they're used incorrectly they make communication even more difficult than it already is. Take a look at https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/ for some of the common issues with terminology.

Assuming you meant RAIDZ2, your reason to do it would have to be your own. Presumably it would be because you wanted some redundancy in your pool, and were willing to sacrifice some capacity in order to obtain it.

 

[david kennedy]

I hope you weren't planning to do RAID6 with FreeNAS, as FreeNAS doesn't directly support RAID6 and the use of hardware RAID controllers is a Very Bad Idea. Perhaps you meant RAIDZ2? If so, please say so--words mean things, and when they're used incorrectly they make communication even more difficult than it already is.

Posts like this are really odd. You have a group of people with a lot of knowledge in a specific area being critical of others that dont "speak the same language".

Let me ask a question. When you go to the doctors do you use the correct medical terms to describe the issue? Why not, as it would avoid communication issues and words mean things and all that.

You can spend hours studying medical texts to understands the correct meaning of something like " anterior" instead of trying to use "front".

Your doctor gets what you mean, why do we need new users to study these " use this terminology only" links?

 

[SweetAndLow]

If they said zfs 2 disk redundancy instead of raidz2 we would understand and it would fit in with your metaphor. But saying raid6 is just completely wrong.

 

[david kennedy]

If they said zfs 2 disk redundancy instead of raidz2 we would understand and it would fit in with your metaphor. But saying raid6 is just completely wrong.

Playing devils advocate this is the definition of raid6:

RAID 6 extends RAID 5 by adding another parityblock; thus, it uses block-level striping with twoparity blocks distributed across all member disks.

And raidz2

Double Parity RAID-Z (raidz2)
A redundant RAID-Z configuration can now have either single- or double-parity, which means that one or two device failures can be sustained respectively, without any data loss. You can specify the raidz2 keyword for a double-parity RAID-Z configuration. Or, you can specify the raidz or raidz1 keyword for a single-parity RAID-Z configuration.


They seem pretty close and maybe you can see where the confusion comes from?

 

[danb35]

@david kennedy, the basic problem is that people do dumb stuff like building a RAID 5 array on a hardware RAID controller, and then using ZFS on it. They then get cranky when they lose their data. The OP in this thread has already made some pretty poor decisions regarding pool layout, and has also shown a lack of understanding of what he's done in that regard. You'd suggest that I just assume he must have meant RAIDZ2 when he said RAID6. While I agree that's likely what he meant, I believe that both the general history of posters here, and the specific history of this poster, makes it worth clarifying what he actually meant.

To your other points--your definition of RAID 6 doesn't fit, because it's built on RAID 5. RAIDZ1 is not RAID 5.

Your example of speaking with the doctor doesn't really fit either. In your example, I use a layman's term rather than a technical term. @ghost reaper used a technical term in a way that's probably, but far from certainly, incorrect. It's as though I went to the doctor complaining of a fractured humerus, and pointing to my leg--but even in that case, it'd be pretty obvious I was using the term incorrectly. Here, I can't be confident that's what's happening.

 

[jgreco]

Guys, feel free to refer posters to https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/ rather than trying to infer what they might mean. This thread is poster child for exactly why I wrote that sticky.

Original poster, please refer to https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/ and try to use the ZFS terminology if you mean a ZFS thing. Picking a somewhat similar RAID concept and hoping we understand what you mean leads to communication failure. At the VERY LEAST you are expected to say something like "the ZFS equivalent of RAID6" if you cannot figure out what the appropriate term is ... not just call it RAID6. We're happy to help you learn, and try to advise on your problem, but when A is B and C might be D and E isn't F, the confusion resulting from all that is not productive or helpful.

Your friendly terminology police,
Officer Grinchy

 

[Robert Trevellyan]

Your example of speaking with the doctor doesn't really fit either.

And your doctor is getting paid to try to figure out what the heck you mean ...

 

[cyberjock]

Well...

- You didn't use recommended hardware
- You didn't meet the minimum specs
- You added at least a single disk to the pool despite *numerous* locations that warn against doing such a thing
- You seem to be having difficulty with our terminology

You shouldn't be too surprised, and you'll find little sympathy from anyone with experience in zpool recovery. Why? Because we warn people every day not to do exactly what you did.

Sorry, but I'm not biting on this one.

 

[jgreco]

Playing devils advocate this is the definition of raid6:

RAID 6 extends RAID 5 by adding another parityblock; thus, it uses block-level striping with twoparity blocks distributed across all member disks.

And raidz2

Double Parity RAID-Z (raidz2)
A redundant RAID-Z configuration can now have either single- or double-parity, which means that one or two device failures can be sustained respectively, without any data loss. You can specify the raidz2 keyword for a double-parity RAID-Z configuration. Or, you can specify the raidz or raidz1 keyword for a single-parity RAID-Z configuration.


They seem pretty close and maybe you can see where the confusion comes from?

Of course we see where the confusion comes from. Now reverse the picture and imagine what I'm thinking when I see someone come in and say they've got a RAID5 or RAID6, and their pool's failed, and I waste time trying to figure out what their issue is, and it turns out they've ACTUALLY built a ZFS pool on a hardware RAID controller with a RAID5 volume, and I've just wasted a half hour on someone whose situation is untenable and irrecoverable.

Terminology matters. RAID6 is not RAIDZ2, and the difference is important.

 

[HoneyBadger]

Of course we see where the confusion comes from. Now reverse the picture and imagine what I'm thinking when I see someone come in and say they've got a RAID5 or RAID6, and their pool's failed, and I waste time trying to figure out what their issue is, and it turns out they've ACTUALLY built a ZFS pool on a hardware RAID controller with a RAID5 volume, and I've just wasted a half hour on someone whose situation is untenable and irrecoverable.

Terminology matters. RAID6 is not RAIDZ2, and the difference is important.

It absolutely matters, and I'll bail out of a "ZFS on hardware RAID" thread like a rat from a sinking ship, but it generally doesn't take very long to ascertain if they're just misspeaking or if They Dun Goofed.

@david kennedy The problem with expecting leniency on the terminology is that you're asking a bunch of engineers to lay off the jargon. ;)

 

[jgreco]

A bunch of engineers who've been repeatedly bitten by leniency over the years, at that.