ZFS Vol Errors After Reboot

[imcintyr]

Hi, Noob here. A friend of mine turned me on to FreeNAS and I am very excited about it. I spent the better part of the weekend researching this issue but am at my wits end.

Basically I can create UFS volumes that are stable (meaning they survive a reboot) but I cannot get a stable ZFS anything (mirror, raidz 1 or 2) to stay past a reboot. I can create the volume and then a dataset but after a reboot I see "getting used/available/total space (Error)" next to my volume. The drives are no longer available to create other volumes as well. I must export the volume (erase disks option checked) and recreate only to lose it again.

I have tried several remedies that I have seen posted around the forum. For example the commands below:

mount -uw /
zfs set mountpoint=/mnt yourpool
zpool export yourpool
zpool import yourpool
mount -ur /

They didn't help. Seems the zpool doesn't exist.

I have dd'd one disc that I believe to be bad. (This disc is very naughty and will be replaced but the other 5 seem to be OK and still unable to hold a ZFS volume.)

Below are the threads I have read from this site and there were many others out there in the wild, wild internet too. Still I am not seeing a possible solution.

Error getting used space (Error)
Disque_1 /mnt/Disque_1 Error getting usError getting used space (Error)
Freenas 8 Issues - Space Error
Erro creating a new ZFS volume?
FAQ - Those same questions that keep getting Asked & Answered about FreeNAS 8
FAQ - What hardware should I Buy-Get-Use, or is recommended?



Software

FreeNAS 8.0.3 on a 4GB USB stick. (How this would matter I don't know but it is inside the case connected to what would be the front connecting pins. I just didn't want it sticking out the back.)


Hardware (Nothing fancy)

1 x AMD Athlon 64 X2 4600+
1 x MSI K9N Platinum motherboard
2 x Crucial 4GB (2 x 2GB) 240-Pin DDR2 SDRAM (PC2 5300) Dual Channel CT2KIT25664AA667
6 x Seagate Barracuda Green ST2000DL003 2TB 5900 RPM SATA 6.0Gb/s 3.5" Internal -Bare

 

[ProtoSD]

Hi Imcintyr,

Thanks for posting great detailed info about what you've tried and your hardware etc., it's rare and appreciated.

Usually that error indicates some mismatch between what FreeNAS has detected previously and stored in the database.

Have you tried creating a volume without the "naughty" disk? / What happens?

What happens if you create a ZFS volume with a single disk?

Maybe you could create a ZFS pool/volume, then from the command line do a gpart show both before & after rebooting and post the results.

Also do these 2 commands from the command line & post the results before & after a reboot:

sqlite3 /data/freenas-v1.db "select * from storage_disk;"

sqlite3 /data/freenas-v1.db "select * from storage_diskgroup;"

 

[survive]

Hi imcintyr,

You might also want to try using the "dd" command to write a couple of megs of zeros to the beginning & end of each drive. Sometimes there is existing partition information on the disks that for one reason or another FreeNAS can't deal with. You can do the same thing with Darik's Boot & Nuke from dban.org.

Do you see the controller or disks listed on the console when you boot up? You can use the "dmesg" command to display the boot information.

-Will

 

[imcintyr]

Thanks for the help. I am going to try these tonight after a stop to get a new drive. I'd like to eliminate that as a variable. I did try to make a vol using ada2 (bad disc). Results below. The 'dd' command will not work in this guy either.

Creating UFS vol using ada2:
"unable to gpt format the disc ada2:

Creating ZFS vol using ada2 (force 4k)
"unable to gpt format the disc ada2:

More results tonight. Again thanks for your help!

 

[ProtoSD]

Just for clarity, when you check the box to erase the disk, it does wipe the proper places at the beginning & end of each disk. If you're unable to get dd to work, that may be part of the problem as well as the error "unable to gpt format the disc". Doing the gpart show before & after will let you see which discs were actually formatted properly.

 

[imcintyr]

Replaced the bad drive and created several zvols with no luck. Here is the before shot:


[root@freenas] ~# sqlite3 /data/freenas-v1.db "select * from storage_disk;"
Disabled|Always On|{uuid}2dee8515-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||1|ada0
Disabled|Always On|{uuid}2e741ec5-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||2|ada1
Disabled|Always On|{uuid}2ef08da6-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||3|ada2
Disabled|Always On|{uuid}2f6e8531-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||4|ada3
Disabled|Always On|{uuid}2ff56fa5-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||5|ada4
Disabled|Always On|{uuid}307001d1-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||6|ada5

[root@freenas] ~# sqlite3 /data/freenas-v1.db "select * from storage_diskgroup;"
1|orthancraidz2|raidz2|1


After:

[root@freenas] ~# sqlite3 /data/freenas-v1.db "select * from storage_disk;"
Disabled|Always On|{uuid}2dee8515-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||1|ada0
Disabled|Always On|{uuid}2e741ec5-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||2|ada1
Disabled|Always On|{uuid}2ef08da6-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||3|ada2
Disabled|Always On|{uuid}2f6e8531-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||4|ada3
Disabled|Always On|{uuid}2ff56fa5-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||5|ada4
Disabled|Always On|{uuid}307001d1-578f-11e1-8b23-0016178e327b|1|Disabled|Auto|Member of orthanc raidz2|1||6|ada5

[root@freenas] ~# sqlite3 /data/freenas-v1.db "select * from storage_diskgroup;"
1|orthancraidz2|raidz2|1

More tomorrow...

 

[imcintyr]

What happens if you create a ZFS volume with a single disk?

Maybe you could create a ZFS pool/volume, then from the command line do a gpart show both before & after rebooting and post the results.

1) Single disk ZFS Vols suffer the same fate.



2) Here is the Gpart show for each disk before/after:

Before:

[root@freenas] ~# gpart show
=> 63 7896987 da0 MBR (3.8G)
63 1930257 1 freebsd [active] (943M)
1930320 63 - free - (32K)
1930383 1930257 2 freebsd (943M)
3860640 3024 3 freebsd (1.5M)
3863664 41328 4 freebsd (20M)
3904992 3992058 - free - (1.9G)

=> 0 1930257 da0s1 BSD (943M)
0 16 - free - (8.0K)
16 1930241 1 !0 (943M)

=> 34 3907029101 ada0 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada1 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada2 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada3 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada4 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

****************************************

After:

[root@freenas] ~# gpart show
=> 63 7896987 da0 MBR (3.8G)
63 1930257 1 freebsd [active] (943M)
1930320 63 - free - (32K)
1930383 1930257 2 freebsd (943M)
3860640 3024 3 freebsd (1.5M)
3863664 41328 4 freebsd (20M)
3904992 3992058 - free - (1.9G)

=> 0 1930257 da0s1 BSD (943M)
0 16 - free - (8.0K)
16 1930241 1 !0 (943M)

=> 34 3907029101 ada0 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada1 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada2 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada3 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada4 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

 

[ProtoSD]

Can you verify how many disks you're using for your raidz2?

It looks like 5, but the database thinks there should be 6 and ada5 is not there before or after in the "gpart show", but it's in the database.

If you can verify the number of disks you're using, I can help you adjust the database and see if that helps.

 

[imcintyr]

Can you verify how many disks you're using for your raidz2?

It looks like 5, but the database thinks there should be 6 and ada5 is not there before or after in the "gpart show", but it's in the database.

If you can verify the number of disks you're using, I can help you adjust the database and see if that helps.

Right, one tiny little thing I forgot to mention. I replaced ada2 with a new drive which solved the bad drive issue. At that point I ran the db code you asked for. Then later on that evening... I needed to have a CD rom on that box so I used one of the drives connections reducing the number of drives to 5. Then I ran the gparts and sent them off. Probably not the best plan on my part. If it is necessary, I can plug the drive back in tonight and run them again. Let me know about that. Sorry for the confusion.

Ian

 

[ProtoSD]

Yup, that explains that. That's why I only use 5 disks in my pool, I left my 6th port open for situations like you just mentioned. Unfortunately FreeNAS doesn't deal with that very well when you swap stuff around and it expects to find something else.

You can try plugging it back in and see what happens and post the "gpart show" again.

Just to confirm again, after you do that are you still getting the "Error getting used space" message?

 

[imcintyr]

You can try plugging it back in and see what happens and post the "gpart show" again.

Just to confirm again, after you do that are you still getting the "Error getting used space" message?

(The post below boarders on TMI)

The error message persists. I have rebuilt the RAIDZ2 with all 6 drives now. The previous was exported with the destroy data box checked. Your initial two requests are posted. Will, I attached the dmesg in the last section of this post. At the bottom of the dmesg is an error about drive geometry.

At this point I am really beginning to wonder if I need a new motherboard or drives or whatnot. I'm good going to update hardware but I just don't want to spin out again.




Pre Reboot: All 6 drives are in a zraid2. GUI shows that the vol is healthy at 7.1 TiB.


[root@freenas] ~# sqlite3 /data/freenas-v1.db "select * from storage_disk;"
Disabled|Always On|{uuid}a22a5e51-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||1|ada0
Disabled|Always On|{uuid}a2b87a48-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||2|ada1
Disabled|Always On|{uuid}a33cd74d-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||3|ada2
Disabled|Always On|{uuid}a3cde9b0-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||4|ada3
Disabled|Always On|{uuid}a45884f8-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||5|ada4
Disabled|Always On|{uuid}a4e9021d-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||6|ada5


[root@freenas] ~# sqlite3 /data/freenas-v1.db "select * from storage_diskgroup;"
1|zraid2testraidz2|raidz2|1


[root@freenas] ~# gpart show
=> 63 7896987 da0 MBR (3.8G)
63 1930257 1 freebsd [active] (943M)
1930320 63 - free - (32K)
1930383 1930257 2 freebsd (943M)
3860640 3024 3 freebsd (1.5M)
3863664 41328 4 freebsd (20M)
3904992 3992058 - free - (1.9G)

=> 0 1930257 da0s1 BSD (943M)
0 16 - free - (8.0K)
16 1930241 1 !0 (943M)

=> 34 3907029101 ada0 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada1 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada2 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada3 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada4 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada5 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)




******************************************************************************************************************************

After: errors abound getting total and available space.


[root@freenas] ~# sqlite3 /data/freenas-v1.db "select * from storage_disk;"
Disabled|Always On|{uuid}a22a5e51-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||1|ada0
Disabled|Always On|{uuid}a2b87a48-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||2|ada1
Disabled|Always On|{uuid}a33cd74d-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||3|ada2
Disabled|Always On|{uuid}a3cde9b0-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||4|ada3
Disabled|Always On|{uuid}a45884f8-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||5|ada4
Disabled|Always On|{uuid}a4e9021d-58ce-11e1-aa34-0016178e327b|1|Disabled|Auto|Member of zraid2test raidz2|1||6|ada5


[root@freenas] ~# sqlite3 /data/freenas-v1.db "select * from storage_diskgroup;"
1|zraid2testraidz2|raidz2|1


[root@freenas] ~# gpart show
=> 63 7896987 da0 MBR (3.8G)
63 1930257 1 freebsd [active] (943M)
1930320 63 - free - (32K)
1930383 1930257 2 freebsd (943M)
3860640 3024 3 freebsd (1.5M)
3863664 41328 4 freebsd (20M)
3904992 3992058 - free - (1.9G)

=> 0 1930257 da0s1 BSD (943M)
0 16 - free - (8.0K)
16 1930241 1 !0 (943M)

=> 34 3907029101 ada0 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada1 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada2 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada3 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada4 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)

=> 34 3907029101 ada5 GPT (1.8T)
34 94 - free - (47K)
128 4194304 1 freebsd-swap (2.0G)
4194432 3902834703 2 freebsd-zfs (1.8T)


*************************************************************************************************************************

[root@freenas] ~# dmesg
Copyright (c) 1992-2011 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.2-RELEASE-p6 #0: Wed Jan 18 13:52:11 PST 2012
root@build.ixsystems.com:/build/home/gcooper/e2e-bld.Pp7kV0qa/obj.amd64/build/home/gcooper/e2e-bld.Pp7kV0qa/FreeBSD/src/sys/FREENAS.amd64 amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4600+ (2400.01-MHz K8-class CPU)
Origin = "AuthenticAMD" Id = 0x40fb2 Family = f Model = 4b Stepping = 2
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x2001<SSE3,CX16>
AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!>
AMD Features2=0x1f<LAHF,CMP,SVM,ExtAPIC,CR8>
real memory = 8589934592 (8192 MB)
avail memory = 8245153792 (7863 MB)
ACPI APIC Table: <MSIISM OEMAPIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <MSIISM OEMRSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of fec00000, 1000 (3) failed
acpi0: reservation of fee00000, 1000 (3) failed
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, cff00000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x2008-0x200b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <memory, RAM> at device 0.0 (no driver attached)
isab0: <PCI-ISA bridge> port 0x2f00-0x2f7f at device 1.0 on pci0
isa0: <ISA bus> on isab0
pci0: <serial bus, SMBus> at device 1.1 (no driver attached)
ohci0: <nVidia nForce MCP55 USB Controller> mem 0xfcffb000-0xfcffbfff irq 22 at device 2.0 on pci0
ohci0: [ITHREAD]
usbus0: <nVidia nForce MCP55 USB Controller> on ohci0
ehci0: <NVIDIA nForce MCP55 USB 2.0 controller> mem 0xfcffac00-0xfcffacff irq 23 at device 2.1 on pci0
ehci0: [ITHREAD]
usbus1: EHCI version 1.0
usbus1: <NVIDIA nForce MCP55 USB 2.0 controller> on ehci0
atapci0: <nVidia nForce MCP55 SATA300 controller> port 0xb480-0xb487,0xb400-0xb403,0xb080-0xb087,0xb000-0xb003,0xac00-0xac0f mem 0xfcff9000-0xfcff9fff irq 21 at device 5.0 on pci0
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
atapci1: <nVidia nForce MCP55 SATA300 controller> port 0xa880-0xa887,0xa800-0xa803,0xa480-0xa487,0xa400-0xa403,0xa080-0xa08f mem 0xfcff8000-0xfcff8fff irq 22 at device 5.1 on pci0
atapci1: [ITHREAD]
ata4: <ATA channel 0> on atapci1
ata4: [ITHREAD]
ata5: <ATA channel 1> on atapci1
ata5: [ITHREAD]
atapci2: <nVidia nForce MCP55 SATA300 controller> port 0xa000-0xa007,0x9c00-0x9c03,0x9880-0x9887,0x9800-0x9803,0x9480-0x948f mem 0xfcff7000-0xfcff7fff irq 23 at device 5.2 on pci0
atapci2: [ITHREAD]
ata6: <ATA channel 0> on atapci2
ata6: [ITHREAD]
ata7: <ATA channel 1> on atapci2
ata7: [ITHREAD]
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xc000-0xc0ff mem 0xfd000000-0xfdffffff,0xfeaff000-0xfeafffff irq 16 at device 0.0 on pci1
pci0: <multimedia, HDA> at device 6.1 (no driver attached)
nfe0: <NVIDIA nForce MCP55 Networking Adapter> port 0x9400-0x9407 mem 0xfcff6000-0xfcff6fff,0xfcffa800-0xfcffa8ff,0xfcffa400-0xfcffa40f irq 20 at device 8.0 on pci0
miibus0: <MII bus> on nfe0
ciphy0: <Vitesse VSC8601 10/100/1000TX PHY> PHY 0 on miibus0
ciphy0: 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
nfe0: Ethernet address: 00:16:17:8e:32:7b
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe1: <NVIDIA nForce MCP55 Networking Adapter> port 0x9080-0x9087 mem 0xfcff5000-0xfcff5fff,0xfcffa000-0xfcffa0ff,0xfcff4c00-0xfcff4c0f irq 20 at device 9.0 on pci0
miibus1: <MII bus> on nfe1
ciphy1: <Vitesse VSC8601 10/100/1000TX PHY> PHY 1 on miibus1
ciphy1: 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
nfe1: Ethernet address: 00:16:17:78:e8:66
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
pcib2: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 12.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 13.0 on pci0
pci4: <ACPI PCI bus> on pcib4
ahci0: <Marvell 88SE912x AHCI SATA controller> port 0xec00-0xec07,0xe880-0xe883,0xe800-0xe807,0xe480-0xe483,0xe400-0xe40f mem 0xfebff800-0xfebfffff irq 17 at device 0.0 on pci4
ahci0: [ITHREAD]
ahci0: AHCI v1.20 with 8 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich0: [ITHREAD]
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich1: [ITHREAD]
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich2: [ITHREAD]
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich3: [ITHREAD]
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich4: [ITHREAD]
ahcich5: <AHCI channel> at channel 5 on ahci0
ahcich5: [ITHREAD]
ahcich6: <AHCI channel> at channel 6 on ahci0
ahcich6: [ITHREAD]
ahcich7: <AHCI channel> at channel 7 on ahci0
ahcich7: [ITHREAD]
atapci3: <Marvell 88SE912x UDMA133 controller> port 0xe080-0xe087,0xe000-0xe003,0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd80f mem 0xfebff400-0xfebff40f irq 18 at device 0.1 on pci4
pcib5: <ACPI PCI-PCI bridge> at device 14.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 15.0 on pci0
pci6: <ACPI PCI bus> on pcib6
amdtemp0: <AMD K8 Thermal Sensors> on hostb3
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed00fff on acpi0
Timecounter "HPET" frequency 25000000 Hz quality 900
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
powernow0: <PowerNow! K8> on cpu0
device_attach: powernow0 attach returned 6
powernow1: <PowerNow! K8> on cpu1
device_attach: powernow1 attach returned 6
fuse4bsd: version 0.3.9-pre1, FUSE ABI 7.8
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <nVidia> at usbus0
uhub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <nVidia> at usbus1
uhub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub0: 10 ports with 10 removable, self powered
uhub1: 10 ports with 10 removable, self powered
ugen0.2: <DELL> at usbus0
ukbd0: <DELL DELL USB Keyboard, class 0/0, rev 1.10/1.05, addr 2> on usbus0
kbd2 at ukbd0
ugen1.2: <USB 2.0> at usbus1
umass0: <USB 2.0 USB Flash Drive, class 0/0, rev 2.00/1.00, addr 2> on usbus1
ada0 at ata2 bus 0 scbus0 target 0 lun 0
ada0: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1 at ata3 bus 0 scbus1 target 0 lun 0
ada1: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
da0 at umass-sim0 bus 0 scbus14 target 0 lun 0
da0: <USB 2.0 USB Flash Drive 0.00> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 3856MB (7897088 512 byte sectors: 255H 63S/T 491C)
ada2 at ata4 bus 0 scbus2 target 0 lun 0
ada2: <ST2000DL003-9VT166 CC3C> ATA-8 SATA 3.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3 at ata5 bus 0 scbus3 target 0 lun 0
ada3: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada4 at ata6 bus 0 scbus4 target 0 lun 0
ada4: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada4: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada5 at ata7 bus 0 scbus5 target 0 lun 0
ada5: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
pass6 at ahcich7 bus 0 scbus13 target 0 lun 0
pass6: <Marvell 91xx Config 1.01> Removable Processor SCSI-0 device
pass6: 150.000MB/s transfers (SATA 1.x, UDMA4, ATAPI 12bytes, PIO 8192bytes)
SMP: AP CPU #1 Launched!
GEOM: da0s1: geometry does not match label (16h,63s != 255h,63s).
Trying to mount root from ufs:/dev/ufs/FreeNASs1a
ZFS filesystem version 4
ZFS storage pool version 15

 

[ProtoSD]

ahci0: <Marvell 88SE912x AHCI SATA controller> port 0xec00-0xec07,0xe880-0xe883,0xe800-0xe807,0xe480-0xe483,0xe400-0xe40f mem 0xfebff800-0xfebfffff irq 17 at device 0.0 on pci4
ahci0: [ITHREAD]
ahci0: AHCI v1.20 with 8 6Gbps ports, Port Multiplier not supported

Imcintry,

The database stuff all looks fine, I'm a little stumped about why you're getting the "Error getting used space" message.

The only thing that looks a little suspicious from dmesg is "Port Multiplier not supported" message.

The error about "GEOM: da0s1: geometry does not match label" is completely normal "Noise" and nothing to worry about.

Don't run out and replace your hardware. My guess is it has to do with the "Port multiplier" not being supported. Maybe there's someone else that has the same hardware that confirm that controller works. The drives all appear to be online and have the partition info intact, it's just FreeNAS has some issue that I haven't seen before.

What do you see if you do a "zpool status -v" ?

What about a "df -h" ?

 

[imcintyr]

Here are the commands


[root@freenas] ~# zpool status -v
no pools available
[root@freenas] ~# df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/ufs/FreeNASs1a 927M 375M 477M 44% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/md0 4.6M 1.8M 2.3M 44% /etc
/dev/md1 824K 1.5K 757K 0% /mnt
/dev/md2 149M 8.3M 129M 6% /var
/dev/ufs/FreeNASs4 20M 559K 18M 3% /data

There were 8 SATA ports on that last DMESG. MB has 6 and I had a 2-port SATA controller card installed as well. (Incidently, the reason I didn't use one of the controller card connections for the previously mentioned CDROM is that I didn't have enough MOLEX to SATA power adaptors so I had to steal power from a drive.) For what it's worth, the SATA controller was only unboxed and installed last night. In other words, the problem has persisted outside of it's introdction.

Thanks for sticking with me on this. Below is the new DMESG w/o the 2-port sata controller. After I removed the sata controller I made another zvol and rebooted. Same problem.


[root@freenas] ~# dmesg
Copyright (c) 1992-2011 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.2-RELEASE-p6 #0: Wed Jan 18 13:52:11 PST 2012
root@build.ixsystems.com:/build/home/gcooper/e2e-bld.Pp7kV0qa/obj.amd64/build/home/gcooper/e2e-bld.Pp7kV0qa/FreeBSD/src/sys/FREENAS.amd64 amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 4600+ (2400.02-MHz K8-class CPU)
Origin = "AuthenticAMD" Id = 0x40fb2 Family = f Model = 4b Stepping = 2
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x2001<SSE3,CX16>
AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!>
AMD Features2=0x1f<LAHF,CMP,SVM,ExtAPIC,CR8>
real memory = 8589934592 (8192 MB)
avail memory = 8245153792 (7863 MB)
ACPI APIC Table: <MSIISM OEMAPIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
cryptosoft0: <software crypto> on motherboard
acpi0: <MSIISM OEMRSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of fec00000, 1000 (3) failed
acpi0: reservation of fee00000, 1000 (3) failed
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, cff00000 (3) failed
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x2008-0x200b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <memory, RAM> at device 0.0 (no driver attached)
isab0: <PCI-ISA bridge> port 0x2f00-0x2f7f at device 1.0 on pci0
isa0: <ISA bus> on isab0
pci0: <serial bus, SMBus> at device 1.1 (no driver attached)
ohci0: <nVidia nForce MCP55 USB Controller> mem 0xfcffb000-0xfcffbfff irq 22 at device 2.0 on pci0
ohci0: [ITHREAD]
usbus0: <nVidia nForce MCP55 USB Controller> on ohci0
ehci0: <NVIDIA nForce MCP55 USB 2.0 controller> mem 0xfcffac00-0xfcffacff irq 23 at device 2.1 on pci0
ehci0: [ITHREAD]
usbus1: EHCI version 1.0
usbus1: <NVIDIA nForce MCP55 USB 2.0 controller> on ehci0
atapci0: <nVidia nForce MCP55 SATA300 controller> port 0xd480-0xd487,0xd400-0xd403,0xd080-0xd087,0xd000-0xd003,0xcc00-0xcc0f mem 0xfcff9000-0xfcff9fff irq 21 at device 5.0 on pci0
atapci0: [ITHREAD]
ata2: <ATA channel 0> on atapci0
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci0
ata3: [ITHREAD]
atapci1: <nVidia nForce MCP55 SATA300 controller> port 0xc880-0xc887,0xc800-0xc803,0xc480-0xc487,0xc400-0xc403,0xc080-0xc08f mem 0xfcff8000-0xfcff8fff irq 22 at device 5.1 on pci0
atapci1: [ITHREAD]
ata4: <ATA channel 0> on atapci1
ata4: [ITHREAD]
ata5: <ATA channel 1> on atapci1
ata5: [ITHREAD]
atapci2: <nVidia nForce MCP55 SATA300 controller> port 0xc000-0xc007,0xbc00-0xbc03,0xb880-0xb887,0xb800-0xb803,0xb480-0xb48f mem 0xfcff7000-0xfcff7fff irq 23 at device 5.2 on pci0
atapci2: [ITHREAD]
ata6: <ATA channel 0> on atapci2
ata6: [ITHREAD]
ata7: <ATA channel 1> on atapci2
ata7: [ITHREAD]
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xe000-0xe0ff mem 0xfd000000-0xfdffffff,0xfebff000-0xfebfffff irq 16 at device 0.0 on pci1
pci0: <multimedia, HDA> at device 6.1 (no driver attached)
nfe0: <NVIDIA nForce MCP55 Networking Adapter> port 0xb400-0xb407 mem 0xfcff6000-0xfcff6fff,0xfcffa800-0xfcffa8ff,0xfcffa400-0xfcffa40f irq 20 at device 8.0 on pci0
miibus0: <MII bus> on nfe0
ciphy0: <Vitesse VSC8601 10/100/1000TX PHY> PHY 0 on miibus0
ciphy0: 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
nfe0: Ethernet address: 00:16:17:8e:32:7b
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe0: [FILTER]
nfe1: <NVIDIA nForce MCP55 Networking Adapter> port 0xb080-0xb087 mem 0xfcff5000-0xfcff5fff,0xfcffa000-0xfcffa0ff,0xfcff4c00-0xfcff4c0f irq 20 at device 9.0 on pci0
miibus1: <MII bus> on nfe1
ciphy1: <Vitesse VSC8601 10/100/1000TX PHY> PHY 1 on miibus1
ciphy1: 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
nfe1: Ethernet address: 00:16:17:78:e8:66
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
nfe1: [FILTER]
pcib2: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 12.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pcib4: <ACPI PCI-PCI bridge> at device 13.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> at device 14.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 15.0 on pci0
pci6: <ACPI PCI bus> on pcib6
amdtemp0: <AMD K8 Thermal Sensors> on hostb3
acpi_button0: <Power Button> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed00fff on acpi0
Timecounter "HPET" frequency 25000000 Hz quality 900
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
powernow0: <PowerNow! K8> on cpu0
device_attach: powernow0 attach returned 6
powernow1: <PowerNow! K8> on cpu1
device_attach: powernow1 attach returned 6
fuse4bsd: version 0.3.9-pre1, FUSE ABI 7.8
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <nVidia> at usbus0
uhub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <nVidia> at usbus1
uhub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub0: 10 ports with 10 removable, self powered
uhub1: 10 ports with 10 removable, self powered
ugen0.2: <DELL> at usbus0
ukbd0: <DELL DELL USB Keyboard, class 0/0, rev 1.10/1.05, addr 2> on usbus0
kbd2 at ukbd0
ugen1.2: <USB 2.0> at usbus1
umass0: <USB 2.0 USB Flash Drive, class 0/0, rev 2.00/1.00, addr 2> on usbus1
ada0 at ata2 bus 0 scbus0 target 0 lun 0
ada0: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1 at ata3 bus 0 scbus1 target 0 lun 0
ada1: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
da0: <USB 2.0 USB Flash Drive 0.00> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 3856MB (7897088 512 byte sectors: 255H 63S/T 491C)

ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2 at ata4 bus 0 scbus2 target 0 lun 0
ada2: <ST2000DL003-9VT166 CC3C> ATA-8 SATA 3.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3 at ata5 bus 0 scbus3 target 0 lun 0
ada3: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada4 at ata6 bus 0 scbus4 target 0 lun 0
ada4: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada4: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada5 at ata7 bus 0 scbus5 target 0 lun 0
ada5: <ST2000DL003-9VT166 CC32> ATA-8 SATA 3.x device
ada5: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
SMP: AP CPU #1 Launched!
GEOM: da0s1: geometry does not match label (16h,63s != 255h,63s).
Trying to mount root from ufs:/dev/ufs/FreeNASs1a
ZFS filesystem version 4
ZFS storage pool version 15

 

[ProtoSD]

This is just really bizarre, WTF is wrong / why isn't it working!??

Ok, don't recreate any pools from the GUI for now. You've already demonstrated that your pool is being created, it's just not being imported or mounted after a reboot.

Try this:

mount -uw /

zpool import (If the import fails, post the output, maybe this will give a clue why it's not getting mounted after a reboot)

zpool status -v

 

[imcintyr]

Here what my friend has replied with. Doesn't bode well. For the life of me I cannot think of a reason the MB would restrict the pool but my searches to find others having problems with the hdds is fairly clean of issues. That points me at the MB.

On a side note, is the earlier issue with the port multiplier an indication I need to get a different one if I want it to work?

[root@freenas] ~# mount -uw /
[root@freenas] ~# zpool import
pool: zraid2_02161027
id: 16683045551647643542
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://www.sun.com/msg/ZFS-8000-3C
config:

zraid2_02161027 UNAVAIL insufficient replicas
raidz2 UNAVAIL insufficient replicas
ada0p2 UNAVAIL cannot open
ada1p2 UNAVAIL cannot open
ada2p2 UNAVAIL cannot open
ada3p2 UNAVAIL cannot open
ada4p2 UNAVAIL cannot open
ada5p2 UNAVAIL cannot open
[root@freenas] ~# zpool status -v
no pools available
[root@freenas] ~#

 

[ProtoSD]

That is just really bizarre, I'm stumped. The disks are detected by FreeBSD, they have valid partitions, they work with UFS, there is nothing significantly different about ZFS that I can see that would cause this.

On a side note, is the earlier issue with the port multiplier an indication I need to get a different one if I want it to work?

My guess on that is that this version of FreeBSD that FreeNAS is using doesn't have a driver for that port multiplier. There is a lot of discussion on using SAS controllers flashed to IT mode if you poke around the forums here. They are reasonably cheap and *might* be the answer to your problem. What happens with other OS's, you could try FreeBSD 9 or PCBSD or even FreeNAS 7 has a recent release, they all have ZFS and it might be interesting to install one of them and see what happens. If it were me that's what I'd do, my gut says it should be working.

There's even my Unofficial build of FreeNAS 8.0.3-p1 that is built on FreeBSD 9.0 Release that would be worth giving a try. There's a link in my signature below if you want to give that a shot. It's really for experimenting at this point, so don't dive in too deep, but see if your volume survives a reboot.

 

[imcintyr]

Dénouement -- no reply necessary this is for your information.

I replaced the mobo (MSI K9N Platinum) and the naughty disc. The new MoBo I got is this one http://www.asus.com/Motherboards/AMD_CPU_on_Board/E35M1M_PRO/. I mistakenly ordered the "M PRO" instead of the "I Fusion" so I had to add a startac sata card which worked well. All is working just fine now. Easy setup. 7.1TiB. Happy.


pool: orthanc
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
orthanc ONLINE 0 0 0
raidz2 ONLINE 0 0 0
ada0p2 ONLINE 0 0 0
ada1p2 ONLINE 0 0 0
ada2p2 ONLINE 0 0 0
ada3p2 ONLINE 0 0 0
ada4p2 ONLINE 0 0 0
ada5p2 ONLINE 0 0 0
spares
ada6p2 UNAVAIL cannot open

As you can see I still have a bad drive but it's the spare and will be replaced soon. What is the deal with these manufacturers?! (I know, different forum altogether right?)

Thanks again for all your help!

Ian