First build with questions!

[SkyMonkey]

I actually saw that thread but didn't dig into it and didn't have any immediately obvious ideas. Doing so, I do notice one thing, and I think you should look at it. Your hardware lists a HighPoint SATA add-on. Now I'm pretty cynical and I'll take either end as being likely problematic for most problems, but I noticed on several reviews there that there was some nasty stuff in the one star review section for that card...

So could you please map out your drives and see if maybe you yanked a drive off the HP and the other drive was what dropped off. Also what driver does that attach with? It is possible that despite being labeled "hot-swap" in mfr specs that the FreeBSD driver is only partially baked.

Good thought, but I was careful to not pull one of the drives off the HP controller as my first test. That was going to be my followup test, if I could prove hotswap on the motherboard ports worked as expected. Just double checked, and I did pull one that was on the motherboard ports during my test (ada5), and the disks that dropped (ada3 and ada4) are both on the motherboard as well. For some reason, ada0/1 are what the controller disks are listed as.

For the record, here's my dmesg output:

Code:

[root@freenas] ~# dmesg
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.3-RELEASE-p7 #1 r249203M: Sat Apr  6 09:28:27 PDT 2013
    root@build.ixsystems.com:/tank/home/jpaetzel/fn8.3/freenas/os-base/amd64/tank/home/jpaetzel/fn8.3/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Celeron(R) CPU G555 @ 2.70GHz (2693.90-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206a7  Family = 6  Model = 2a  Stepping = 7
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x59ae3bf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,POPCNT,TSCDLT,XSAVE>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  TSC: P-state invariant
real memory  = 17179869184 (16384 MB)
avail memory = 16445571072 (15683 MB)
ACPI APIC Table: <INTEL  S1200KPR>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  2
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
hpt27xx: RocketRAID 27xx controller driver v1.0 (Apr  6 2013 09:28:15)
cryptosoft0: <software crypto> on motherboard
aesni0: No AESNI support.
acpi0: <INTEL S1200KPR> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
ahci0: <Marvell 88SE912x AHCI SATA controller> port 0xe090-0xe097,0xe080-0xe083,0xe070-0xe077,0xe060-0xe063,0xe050-0xe05f mem 0xfe521000-0xfe5217ff irq 16 at device 0.0 on pci1
ahci0: [ITHREAD]
ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich0: [ITHREAD]
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich1: [ITHREAD]
atapci0: <Marvell 88SE912x UDMA133 controller> port 0xe040-0xe047,0xe030-0xe033,0xe020-0xe027,0xe010-0xe013,0xe000-0xe00f mem 0xfe520000-0xfe52000f irq 17 at device 0.1 on pci1
vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xfe000000-0xfe3fffff,0xe0000000-0xefffffff irq 16 at device 2.0 on pci0
pci0: <simple comms> at device 22.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection 7.3.2> port 0xf080-0xf09f mem 0xfe600000-0xfe61ffff,0xfe624000-0xfe624fff irq 20 at device 25.0 on pci0
em0: Using an MSI interrupt
em0: [FILTER]
em0: Ethernet address: 70:54:d2:16:93:cc
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfe623000-0xfe6233ff irq 16 at device 26.0 on pci0
ehci0: [ITHREAD]
usbus0: EHCI version 1.0
usbus0: <EHCI (generic) USB 2.0 controller> on ehci0
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 16 at device 28.1 on pci0
pci3: <ACPI PCI bus> on pcib3
em1: <Intel(R) PRO/1000 Network Connection 7.3.2> port 0xd000-0xd01f mem 0xfe400000-0xfe41ffff,0xfe420000-0xfe423fff irq 17 at device 0.0 on pci3
em1: Using MSIX interrupts with 3 vectors
em1: [ITHREAD]
em1: [ITHREAD]
em1: [ITHREAD]
em1: Ethernet address: 70:54:d2:16:93:cd
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xfe622000-0xfe6223ff irq 23 at device 29.0 on pci0
ehci1: [ITHREAD]
usbus1: EHCI version 1.0
usbus1: <EHCI (generic) USB 2.0 controller> on ehci1
pcib4: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci4: <ACPI PCI bus> on pcib4
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci1: <Intel Cougar Point AHCI SATA controller> port 0xf0d0-0xf0d7,0xf0c0-0xf0c3,0xf0b0-0xf0b7,0xf0a0-0xf0a3,0xf060-0xf07f mem 0xfe621000-0xfe6217ff irq 19 at device 31.2 on pci0
ahci1: [ITHREAD]
ahci1: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich2: <AHCI channel> at channel 0 on ahci1
ahcich2: [ITHREAD]
ahcich3: <AHCI channel> at channel 1 on ahci1
ahcich3: [ITHREAD]
ahcich4: <AHCI channel> at channel 2 on ahci1
ahcich4: [ITHREAD]
ahcich5: <AHCI channel> at channel 3 on ahci1
ahcich5: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
orm0: <ISA Option ROMs> at iomem 0xce800-0xd17ff,0xd1800-0xd27ff,0xd2800-0xd37ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: cannot reserve I/O port range
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
fuse4bsd: version 0.3.9-pre1, FUSE ABI 7.8
Timecounters tick every 1.000 msec
hpt27xx: no controller detected.
GEOM_RAID5: Module loaded, version 1.1.20110927.40 (rev 00ce00e5abb4)
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
ugen0.2: <vendor 0x8087> at usbus0
uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus0
ugen1.2: <vendor 0x8087> at usbus1
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus1
uhub2: 6 ports with 6 removable, self powered
uhub3: 8 ports with 8 removable, self powered
ugen0.3: <MUSHKIN> at usbus0
umass0: <MUSHKIN MKNUFDMH8GB, class 0/0, rev 2.00/1.00, addr 3> on usbus0
ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <WDC WD20EFRX-68AX9N0 80.00A80> ATA-9 SATA 3.x device
ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
ada1: <WDC WD20EFRX-68AX9N0 80.00A80> ATA-9 SATA 3.x device
ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada2 at ahcich2 bus 0 scbus2 target 0 lun 0
ada2: <WDC WD20EFRX-68AX9N0 80.00A80> ATA-9 SATA 3.x device
ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada3 at ahcich3 bus 0 scbus3 target 0 lun 0
ada3: <WDC WD20EFRX-68AX9N0 80.00A80> ATA-9 SATA 3.x device
ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada4 at ahcich4 bus 0 scbus4 target 0 lun 0
ada4: <WDC WD20EFRX-68AX9N0 80.00A80> ATA-9 SATA 3.x device
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
ada5 at ahcich5 bus 0 scbus5 target 0 lun 0
ada5: <WDC WD20EFRX-68AX9N0 80.00A80> ATA-9 SATA 3.x device
ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada5: Command Queueing enabled
ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C)
da0 at umass-sim0 bus 0 scbus6 target 0 lun 0
da0: <MUSHKIN MKNUFDMH8GB PMAP> Removable Direct Access SCSI-0 device SMP: AP CPU #1 Launched!
da0: 40.000MB/s transfers
 
da0: 7640MB (15646720 512 byte sectors: 255H 63S/T 973C)
GEOM: da0s1: geometry does not match label (16h,63s != 255h,63s).
GEOM: da0s2: geometry does not match label (16h,63s != 255h,63s).
Trying to mount root from ufs:/dev/ufs/FreeNASs2a
ZFS filesystem version 5
ZFS storage pool version 28
GEOM_ELI: Device ada0p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada1p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada2p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada3p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada4p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada5p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
[root@freenas] ~#

 

[jgreco]

So yeah, I think I'd have guessed at problems given that the HighPoint is a Marvell chipset, but I have no idea about the Cougar Point PCH ... they work fine here and I'm pretty sure we've tested a variety of nasty things on them, though it's probably been a while. So I'm not coming up with any useful ideas.

Is it possible you managed to cause a cabling event of some sort while trying your test?

 

[SkyMonkey]

I don't believe so, I went in and wiggled/checked all the cables while the system was up after that and couldn't reproduce. Just writing off hot swapping for the moment...

 

[cyberjock]

My biggest issue with hot-swap is the possibility for human error. It's easy to identify and detach the failed disk, but what if you don't have that one disk labeled correctly and you pulled a disk and realized the serial number is wrong. Now you've lost 2 disks worth of redundancy(or lost your pool if you have a RAIDZ1). I prefer to do disk replacements with the server off. If I pulled the wrong disk its no big deal, just push it back in and find the right disk. Much less risk involved with doing any kind of maintenance with the server off.

 

[Zefy]

So just an update here. I got the system built and I've been very slowly learning my way around the system over the last week or so. I've moved stuff on and off of it just to see how well it was working (or if it was working). Turns out it works! Haven't really had any major hiccups so I'm fairly pleased.

A few notes of things that I did. I got pretty much exactly what I listed prior, (intel server mobo, 16gb ECC memory, and the g860 processor.) except I went with only 3 WD 3TB red drives. So I am only running raidz1... My reasoning behind this is my data is not extremely important. Anything vital I can back up elsewhere and in the event of a failure, I asked myself, "what is my data worth?" If someone had my data and asked for $160 for it (price of a WD red), I would not buy it...

So onwards from that... I want to get some clarification about some stuff before I start piling all my data onto this thing.

I was reading that a scrub should be performed on regular basis. I was reading the manual and it says that scrubs are only done on sectors of the drive that actually have data on them. I may be misunderstanding this part, as I thought the point was to detect bad sectors and not write to them. But if there is no data on the bad sector, it wouldn't run the scrub on it...? Just for fun, I clicked the scrub volume button and it prompted me saying that it's going to use up a lot of resources... Cool, there is nothing on it... But then it didn't really do anything after that. I'm assuming that this is because there is no data on the server...?

So this is where the SMART tests come in, ya? I didn't see a button that just let me run a SMART test like there is for the scrubs... Is there? I tried setting one up that would trigger shortly however when the clock rolled around it didn't do anything... Not sure what I'm missing... SMART tests are enabled and I have them set up but it doesn't seem to do anything. The scrubs I was planning on running twice a month. But I'm not sure about the interval for the SMART tests. The way the manual is written it sounds like it should be run much more regularly. What is a good interval for short tests? I'm also not totally clear on what they do... From the manual: "A ZFS scrub only checks used disk space. To check unused disk space, schedule a S.M.A.R.T. Test Type of Long Self-Test to run once or twice a month." So the short tests should be run on what kind of basis? Should I run a long test prior to putting data on the server?

Next thing. So my mobo has two gigabit ports on it. I was reading in another noob thread that it doesn't really increase speed unless I have lots of traffic on my server. Well, it's just me... so probably not a big deal but I would like to have both connected just for the fun of it. I spent 8 dollars on cat6 cables! I better use them! I know it automatically sets up one of them (as followed by the quick start guide in the manual) but what about the other one? I see that there is traffic on it using the reporting tab in the GUI so I'm assuming it's working?

Finally (for now), I tried dumping a large file (9gb) onto the server to see how ran and then pulled it off again. Tried on both my laptop and desktop computer. The desktop saw a peak of 75mb/s write speeds and a read of 80mb/s. This tapered up and down at the start and end of the transfers so on average it was probably something like 60mb/s write and 65mb/s read. My laptop (connected through a cat5 cable) saw about half that speed. I want to try a cat6 to see if it makes a difference... Haven't tried wireless yet with the laptop. These tests were done with just the little data window in exporer... Nothing fancy. I want to do some real benchmarking soon but need to research it more. Do these seem like reasonable numbers so far though?

Thanks everyone!

 

[cyberjock]

So when you tried to run a scrub, then thought nothing happened, do you even know how to check if anything happened? I'm not talking HDD LEDs here...

Same question for running a SMART test. You said when a SMART should have run it didn't trigger anything. Are you sure? Do you know how to check?

LACP requires your network cards and network switch to support it(hint: 99% of "home" network switches and router's don't support LACP). It also requires you to know how to properly setup LACP in FreeNAS(many people don't do it right for whatever reason). My advice.. give up on LACP and keep your $8 for something more useful.

As for when to run short and long tests, that depends on personal preferences. Each does specific things and you should research them, see what some of the tests do and what some of the SMART wizards do, then chose something accordingly.

Your speeds seem fairly normal, but your laptop can probably do better if you figure out why its so slow.

Your questions really aren't geared towards FreeNAS nor with your build, so I'm not sure why you are responding to your own thread which has nothing to do with your most recent post. If you can't stay on-topic with your current thread you should have made a new one...

 

[Zefy]

So when you tried to run a scrub, then thought nothing happened, do you even know how to check if anything happened? I'm not talking HDD LEDs here...

Same question for running a SMART test. You said when a SMART should have run it didn't trigger anything. Are you sure? Do you know how to check?

LACP requires your network cards and network switch to support it(hint: 99% of "home" network switches and router's don't support LACP). It also requires you to know how to properly setup LACP in FreeNAS(many people don't do it right for whatever reason). My advice.. give up on LACP and keep your $8 for something more useful.

As for when to run short and long tests, that depends on personal preferences. Each does specific things and you should research them, see what some of the tests do and what some of the SMART wizards do, then chose something accordingly.

Your speeds seem fairly normal, but your laptop can probably do better if you figure out why its so slow.

Your questions really aren't geared towards FreeNAS nor with your build, so I'm not sure why you are responding to your own thread which has nothing to do with your most recent post. If you can't stay on-topic with your current thread you should have made a new one...

Seriously...? How am I not on topic? The topic is QUESTION: "first nas build with questions!"

This is still my first nas build... And they're questions... Still on topic... What was NOT on topic was that discussion you had about hotswapping... I didn't really care but since you brought up being off topic... Second, I posted in this thread partly for an update. I've already had two people PM me about my build as this configuration seems to be a popular one. The other part was asking noob questions in my noob build thread in the noob forum.

Anyways... No I do not know how to check the status of SMART tests or scrubs. There is no meantion of that in the manual here: http://doc.freenas.org/index.php/ZFS_Scrubs or http://doc.freenas.org/index.php/S.M.A.R.T._Tests ... That pointed me towards this: http://smartmontools.sourceforge.net/man/smartctl.8.html which was basically gibberish to me. Maybe I missed something in that text fort of a document but I didn't see a description of what they are except for this:

short - [ATA] runs SMART Short Self Test (usually under ten minutes). This command can be given during normal system operation (unless run in captive mode - see the '-C' option below). This is a test in a different category than the immediate or automatic offline tests. The "Self" tests check the electrical and mechanical performance as well as the read performance of the disk. Their results are reported in the Self Test Error Log, readable with the '-l selftest' option. Note that on some disks the progress of the self-test can be monitored by watching this log during the self-test; with other disks use the '-c' option to monitor progress.
short - [SCSI] runs the "Background short" self-test.

And

smartctl -t long -d areca,4 /dev/sg2
Start a long self-test on the fourth SATA disk connected to an Areca RAID controller addressed by /dev/sg2.

Do I enter these in Shell or...?

And I'm still unsure of how to actually check to see if it is running. I'm assuming a scrub (being processor and memory intensive) would show an increase in load on the system but I am not seeing any of that?

This thread ( http://forums.freenas.org/threads/scrub-and-smart-tasks-not-starting-as-planned.11391/ ) shed quite a bit of light on the subject so I will be playing around with it when I get home.

Thanks for the heads up about LACP. I won't bother since I have a cheap switch and router which probably doesn't support it.

 

[thelambentonion]

You can manually scrub and initiate SMART tests from the shell; regardless of how well FreeNAS implements its WebGUI, I'd recommend getting comfortable with the CLI as you'll probably have to dive into it a bit.

Oracle has some good documentation on ZFS, which you can read here:

http://docs.oracle.com/cd/E19082-01/817-2271/gbbwa/

As for SMART, the snippet you've quoted actually shows you how to view if a test has been run on one particular drive:

Code:

smartctl -l selftest /dev/ada0

This would list all tests run on drive ada0.

 

[cyberjock]

No, your questions aren't geared towards your BUILD. You have it built, now you have more questions. Start a new thread. But whatever.. I'll stop answering since you are getting your panties in a bind over something stupid.