~11MB/s transfer with ftp? ideas?

Status
Not open for further replies.

papageorgi

Explorer
Joined
Jul 16, 2013
Messages
51
hello all, i was hoping for some in sight. i've got a pretty awesome server but it's seemingly running very slowly maxing out at 11MB/s transfer.

observations: (all speed is over sftp)
-typical transfer from ssd gigabit lenovo laptop to mac mini over lan with sftp yielded ~50MB/s
-expanding 1st array with 4tb hdds array r/w is stably maxing usb2 data source ~25MB/s
-pool degraded with 5x1tb raidz1 + 5x4tb raidz1 (i.e. raid50) still ~25MB/s
-new pool, swapped 1tb hdds for 4tb models, 10 exact hdds in raidz2 ~11.5MB/s (scrubs 183-281M/s)
next steps:
-try fresh image of freenas 9.1.0rc2 x64
-zil & l2arc on usb/ssds/pcie? home use would be great to have +50MB/s over gigabit lan to usb3
goal = all data in once redundant device, low $/energy/noise, media streamer & backup/share server
hardware:
-cat5e wired, gigabit lan ports / switch / & router rj-45 are 50ft, same speed on 6ft lengths or direct connection
-pcie: lsi 9211-8i HBA, nic (dual port non aggregated) supermicro AOC-SG-i2 NIC
-AMD PHENOM II X6 1045T (cpu load is rarely touches 25%)
-FreeNAS (newest) on 16gb lexar usb 2.0 flash drive
-16gb (4x4GB) ddr2-5300 (passed 8hrs of memtest)
-430w antec earth watts psu, mother board asus m4a78-e bios v.2603
-case old lian-li mid-atx, two icy dock 5in3 MB975SP-B
-raidz2 [10 x 4tb Seagate 4TB NAS ST4000VN000]
-nas is backed up via usb2 external hard drives using sftp, changing soon to usb3
zfs config:
-autotune enabled, lz4 compression (other settings no effect)
-dedupe off, atime off, hdd sleep 30 min, acoustic level: med, power level: 127=intermidiate w/standby
-smart (long & short) passes every 7 days, auto scrub every 10 days
-snapshot every 15 min. held for 6 weeks
-i previously had an issue retrieving data so i disabled the ashift so i undid that first.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
If its at 25% that sounds like you might be limited to a single threaded application. Also, 11MB/sec sounds suspiciously close to the theoretical limit for 100Mb LAN.


I'd check the cable and your link speed. Something tells me you are 100Mb sometimes and not knowing it. If you have a cable that's flaky you should just replace it. Flaky hardware should just be discard and replaced. It's really not worth the hassle later when you are having unexplained issues because of hardware that intermittently works properly.

Edit: Didn't do the math in my head.. 25% is probably not a single threaded app, but that CPU may have trouble doing all the work. I'd look at top and see if any threads are hitting CPU usage that is at 7-8%. If they are, that's an indicator of a thread that's CPU-bound.
 

papageorgi

Explorer
Joined
Jul 16, 2013
Messages
51
hello again cyberjock, thanks for responding. that's what's so strange, i'm using all the same hardware that i was using in my previous freenas settings with the exact hardware. the only difference was just a moment after using two raidz1 that were mirror to going to raidz2 i saw this. it's all gigabit lan, but i know i would be limited by the usb2 connection the hard drive is using to transfer data. i've for a year have been seeing 22-30MB/s using FileZilla on my linux laptop. and my cpu is almost always idle.
Code:
[smurfy@nas] /mnt/stor/temp# sysctl -a | grep tcp
net.inet.tcp.rfc1323: 1
net.inet.tcp.mssdflt: 536
net.inet.tcp.keepidle: 7200000
net.inet.tcp.keepintvl: 75000
net.inet.tcp.sendspace: 32768
net.inet.tcp.recvspace: 65536
net.inet.tcp.keepinit: 75000
net.inet.tcp.delacktime: 100
net.inet.tcp.v6mssdflt: 1220
net.inet.tcp.cc.available: newreno
net.inet.tcp.cc.algorithm: newreno
net.inet.tcp.hostcache.purge: 0
net.inet.tcp.hostcache.prune: 300
net.inet.tcp.hostcache.expire: 3600
net.inet.tcp.hostcache.count: 3
net.inet.tcp.hostcache.bucketlimit: 30
net.inet.tcp.hostcache.hashsize: 512
net.inet.tcp.hostcache.cachelimit: 15360
net.inet.tcp.recvbuf_max: 2097152
net.inet.tcp.recvbuf_inc: 16384
net.inet.tcp.recvbuf_auto: 1
net.inet.tcp.insecure_rst: 0
net.inet.tcp.ecn.maxretries: 1
net.inet.tcp.ecn.enable: 0
net.inet.tcp.abc_l_var: 2
net.inet.tcp.rfc3465: 1
net.inet.tcp.rfc3390: 1
net.inet.tcp.rfc3042: 1
net.inet.tcp.drop_synfin: 0
net.inet.tcp.delayed_ack: 0
net.inet.tcp.blackhole: 0
net.inet.tcp.log_in_vain: 0
net.inet.tcp.sendbuf_max: 2097152
net.inet.tcp.sendbuf_inc: 8192
net.inet.tcp.sendbuf_auto: 1
net.inet.tcp.tso: 1
net.inet.tcp.local_slowstart_flightsize: 4
net.inet.tcp.slowstart_flightsize: 1
net.inet.tcp.path_mtu_discovery: 1
net.inet.tcp.reass.overflows: 0
net.inet.tcp.reass.cursegments: 0
net.inet.tcp.reass.maxsegments: 0
net.inet.tcp.sack.globalholes: 0
net.inet.tcp.sack.globalmaxholes: 65536
net.inet.tcp.sack.maxholes: 128
net.inet.tcp.sack.enable: 1
net.inet.tcp.soreceive_stream: 0
net.inet.tcp.isn_reseed_interval: 0
net.inet.tcp.icmp_may_rst: 1
net.inet.tcp.pcbcount: 36
net.inet.tcp.do_tcpdrain: 1
net.inet.tcp.tcbhashsize: 512
net.inet.tcp.log_debug: 0
net.inet.tcp.minmss: 216
net.inet.tcp.syncache.rst_on_sock_fail: 1
net.inet.tcp.syncache.rexmtlimit: 3
net.inet.tcp.syncache.hashsize: 512
net.inet.tcp.syncache.count: 0
net.inet.tcp.syncache.cachelimit: 15360
net.inet.tcp.syncache.bucketlimit: 30
net.inet.tcp.syncookies_only: 0
net.inet.tcp.syncookies: 1
net.inet.tcp.timer_race: 0
net.inet.tcp.per_cpu_timers: 0
net.inet.tcp.rexmit_drop_options: 1
net.inet.tcp.keepcnt: 8
net.inet.tcp.finwait2_timeout: 60000
net.inet.tcp.fast_finwait2_recycle: 0
net.inet.tcp.always_keepalive: 1
net.inet.tcp.rexmit_slop: 200
net.inet.tcp.rexmit_min: 30
net.inet.tcp.msl: 30000
net.inet.tcp.nolocaltimewait: 0
net.inet.tcp.maxtcptw: 27767
[smurfy@nas] /mnt/stor/temp# ifconfig -a
ale0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=c319a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MCAST,WOL_MAGIC,VLAN_HWTSO,LINKSTATE>
ether 00:24:8c:45:00:18
inet 192.168.0.10 netmask 0xffffff00 broadcast 192.168.0.255
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa
inet 127.0.0.1 netmask 0xff000000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[smurfy@nas] /mnt/stor/temp# netstat -i
Name    Mtu Network      Address              Ipkts Ierrs Idrop    Opkts Oerrs  Coll
ale0  1500 <Link#1>      00:24:8c:45:00:18 28210979    0    0 31091579    0    0
ale0  1500 192.168.0.0  192.168.0.10      28212048    -    - 31091661    -    -
usbus    0 <Link#2>                              0    0    0        0    0    0
usbus    0 <Link#3>                              0    0    0        0    0    0
usbus    0 <Link#4>                              0    0    0        0    0    0
usbus    0 <Link#5>                              0    0    0        0    0    0
usbus    0 <Link#6>                              0    0    0        0    0    0
usbus    0 <Link#7>                              0    0    0        0    0    0
usbus    0 <Link#8>                              0    0    0        0    0    0
ipfw0 65536 <Link#9>                              0    0    0        0    0    0
lo0  16384 <Link#10>                          45894    0    0    45894    0    0
lo0  16384 localhost    ::1                      4    -    -        4    -    -
lo0  16384 fe80::1%lo0  fe80::1                  0    -    -        0    -    -
lo0  16384 your-net      localhost            45890    -    -    45890    -    -
[smurfy@nas] /mnt/stor/temp# dmesg
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-STABLE #0 r+1a51513: Thu Jul 25 09:39:42 PDT 2013
    root@build.ixsystems.com:/tank/home/jkh/src/freenas/os-base/amd64/tank/home/jkh/src/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64
gcc version 4.2.1 20070831 patched [FreeBSD]
CPU: AMD Phenom(tm) II X6 1045T Processor (2709.20-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x100fa0  Family = 0x10  Model = 0xa  Stepping = 0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x802009<SSE3,MON,CX16,POPCNT>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x37ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT>
  TSC: P-state invariant, performance statistics
real memory  = 17179869184 (16384 MB)
avail memory = 16509214720 (15744 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <041311 APIC1015>
FreeBSD/SMP: Multiprocessor System Detected: 6 CPUs
FreeBSD/SMP: 1 package(s) x 6 core(s)
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1
cpu2 (AP): APIC ID:  2
cpu3 (AP): APIC ID:  3
cpu4 (AP): APIC ID:  4
cpu5 (AP): APIC ID:  5
WARNING: VIMAGE (virtualized network stack) is a highly experimental feature.
ACPI Warning: Optional field Pm2ControlBlock has zero address or length: 0x0000000000000000/0x1 (20110527/tbfadt-586)
ioapic0 <Version 2.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
aesni0: No AESNI support.
cryptosoft0: <software crypto> on motherboard
acpi0: <041311 XSDT1015> on motherboard
acpi0: Power Button (fixed)
acpi0: reservation of fee00000, 1000 (3) failed
acpi0: reservation of ffb80000, 80000 (3) failed
acpi0: reservation of fec10000, 20 (3) failed
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, dff00000 (3) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xc000-0xc0ff mem 0xf0000000-0xf7ffffff,0xfbde0000-0xfbdeffff,0xfbc00000-0xfbcfffff irq 18 at device 5.0 on pci1
pci1: <multimedia, HDA> at device 5.1 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> irq 19 at device 3.0 on pci0
pci2: <ACPI PCI bus> on pcib2
mps0: <LSI SAS2008> port 0xd000-0xd0ff mem 0xfbefc000-0xfbefffff,0xfbe80000-0xfbebffff irq 19 at device 0.0 on pci2
mps0: Firmware: 07.00.00.00, Driver: 14.00.00.01-fbsd
mps0: IOCCapabilities: 185c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,IR>
pcib3: <ACPI PCI-PCI bridge> irq 18 at device 6.0 on pci0
pci3: <ACPI PCI bus> on pcib3
ale0: <Atheros AR8121/AR8113/AR8114 PCIe Ethernet> port 0xec00-0xec7f mem 0xfbfc0000-0xfbffffff irq 18 at device 0.0 on pci3
ale0: 960 Tx FIFO, 1024 Rx FIFO
ale0: Using 1 MSI messages.
ale0: 4GB boundary crossed, switching to 32bit DMA addressing mode.
miibus0: <MII bus> on ale0
atphy0: <Atheros F1 10/100/1000 PHY> PHY 0 on miibus0
atphy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow
ale0: Ethernet address: 00:24:8c:45:00:18
ahci0: <ATI IXP700 AHCI SATA controller> port 0xb000-0xb007,0xa000-0xa003,0x9000-0x9007,0x8000-0x8003,0x7000-0x700f mem 0xfbbffc00-0xfbbfffff irq 22 at device 17.0 on pci0
ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
ohci0: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfbbfd000-0xfbbfdfff irq 16 at device 18.0 on pci0
usbus0 on ohci0
ohci1: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfbbfe000-0xfbbfefff irq 16 at device 18.1 on pci0
usbus1 on ohci1
ehci0: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfbbff800-0xfbbff8ff irq 17 at device 18.2 on pci0
ehci0: AMD SB600/700 quirk applied
usbus2: EHCI version 1.0
usbus2 on ehci0
ohci2: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfbbfb000-0xfbbfbfff irq 18 at device 19.0 on pci0
usbus3 on ohci2
ohci3: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfbbfc000-0xfbbfcfff irq 18 at device 19.1 on pci0
usbus4 on ohci3
ehci1: <AMD SB7x0/SB8x0/SB9x0 USB 2.0 controller> mem 0xfbbff400-0xfbbff4ff irq 19 at device 19.2 on pci0
ehci1: AMD SB600/700 quirk applied
usbus5: EHCI version 1.0
usbus5 on ehci1
pci0: <serial bus, SMBus> at device 20.0 (no driver attached)
atapci0: <ATI IXP700/800 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xff00-0xff0f at device 20.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
isab0: <PCI-ISA bridge> at device 20.3 on pci0
isa0: <ISA bus> on isab0
pcib4: <ACPI PCI-PCI bridge> at device 20.4 on pci0
pci4: <ACPI PCI bus> on pcib4
ohci4: <AMD SB7x0/SB8x0/SB9x0 USB controller> mem 0xfbbfa000-0xfbbfafff irq 18 at device 20.5 on pci0
usbus6 on ohci4
amdtemp0: <AMD CPU On-Die Thermal Sensors> on hostb4
acpi_button0: <Power Button> on acpi0
orm0: <ISA Option ROM> at iomem 0xd4800-0xd57ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: cannot reserve I/O port range
acpi_throttle0: <ACPI CPU Throttling> on cpu0
hwpstate0: <Cool`n'Quiet 2.0> on cpu0
Timecounters tick every 1.000 msec
ipfw2 (+ipv6) initialized, divert enabled, nat enabled, default to accept, logging disabled
DUMMYNET 0xfffffe0003611540 with IPv6 initialized (100409)
load_dn_sched dn_sched PRIO loaded
load_dn_sched dn_sched QFQ loaded
load_dn_sched dn_sched RR loaded
load_dn_sched dn_sched WF2Q+ loaded
load_dn_sched dn_sched FIFO loaded
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 480Mbps High Speed USB v2.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 12Mbps Full Speed USB v1.0
usbus5: 480Mbps High Speed USB v2.0
usbus6: 12Mbps Full Speed USB v1.0
ugen0.1: <ATI> at usbus0
uhub0: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <ATI> at usbus1
uhub1: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <ATI> at usbus2
uhub2: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus2
ugen3.1: <ATI> at usbus3
uhub3: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <ATI> at usbus4
uhub4: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus4
ugen5.1: <ATI> at usbus5
uhub5: <ATI EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus5
ugen6.1: <ATI> at usbus6
uhub6: <ATI OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
(aprobe0:ahcich0:0:15:0): NOP. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe0:ahcich0:0:15:0): CAM status: Command timeout
(aprobe0:ahcich0:0:15:0): Error 5, Retries exhausted
(aprobe1:ahcich1:0:15:0): NOP. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe1:ahcich1:0:15:0): CAM status: Command timeout
(aprobe1:ahcich1:0:15:0): Error 5, Retries exhausted
(aprobe2:ahcich2:0:15:0): NOP. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe2:ahcich2:0:15:0): CAM status: Command timeout
(aprobe2:ahcich2:0:15:0): Error 5, Retries exhausted
(aprobe3:ahcich3:0:15:0): NOP. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe3:ahcich3:0:15:0): CAM status: Command timeout
(aprobe3:ahcich3:0:15:0): Error 5, Retries exhausted
(aprobe4:ahcich4:0:15:0): NOP. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe4:ahcich4:0:15:0): CAM status: Command timeout
(aprobe4:ahcich4:0:15:0): Error 5, Retries exhausted
uhub6: 2 ports with 2 removable, self powered
uhub0: 3 ports with 3 removable, self powered
uhub1: 3 ports with 3 removable, self powered
uhub3: 3 ports with 3 removable, self powered
uhub4: 3 ports with 3 removable, self powered
uhub2: 6 ports with 6 removable, self powered
uhub5: 6 ports with 6 removable, self powered
ugen2.2: <Lexar> at usbus2
umass0: <Lexar JumpDrive, class 0/0, rev 2.00/11.00, addr 2> on usbus2
ugen2.3: <Apple, Inc.> at usbus2
uhub7: <Apple, Inc. Keyboard Hub, class 9/0, rev 2.00/96.15, addr 3> on usbus2
uhub7: 3 ports with 2 removable, bus powered
ugen2.4: <Apple, Inc> at usbus2
ukbd0: <Apple, Inc Apple Keyboard, class 0/0, rev 2.00/0.69, addr 4> on usbus2
kbd2 at ukbd0
uhid0: <Apple, Inc Apple Keyboard, class 0/0, rev 2.00/0.69, addr 4> on usbus2
da0 at mps0 bus 0 scbus0 target 2 lun 0
da0: <ATA ST4000VN000-1H41 SC42> Fixed Direct Access SCSI-5 device
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C)
da1 at mps0 bus 0 scbus0 target 3 lun 0
da1: <ATA ST4000VN000-1H41 SC42> Fixed Direct Access SCSI-5 device
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C)
da2 at mps0 bus 0 scbus0 target 4 lun 0
da2: <ATA ST4000VN000-1H41 SC42> Fixed Direct Access SCSI-5 device
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C)
da3 at mps0 bus 0 scbus0 target 5 lun 0
da3: <ATA ST4000VN000-1H41 SC42> Fixed Direct Access SCSI-5 device
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C)
da4 at mps0 bus 0 scbus0 target 6 lun 0
da4: <ATA ST4000VN000-1H41 SC42> Fixed Direct Access SCSI-5 device
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 3815447MB (7814037168 512 byte sectors: 255H 63S/T 486401C)
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: <ST4000VN000-1H4168 SC42> ATA-9 SATA 3.x device
ada0: 300.000MB/s transfers (SATA 2.x, da5 at umass-sim0 bus 0 scbus8 target 0 lun 0
da5: <Lexar JumpDrive 1100> Removable Direct Access SCSI-0 device
da5: 40.000MB/s transfers
da5: 15296MB (31326208 512 byte sectors: 255H 63S/T 1949C)
da5: quirks=0x2<NO_6_BYTE>
UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at ahcich1 bus 0 scbus2 target 0 lun 0
ada1: <ST4000VN000-1H4168 SC42> ATA-9 SATA 3.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad6
ada2 at ahcich2 bus 0 scbus3 target 0 lun 0
ada2: <ST4000VN000-1H4168 SC42> ATA-9 SATA 3.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada2: Previously was known as ad8
ada3 at ahcich3 bus 0 scbus4 target 0 lun 0
ada3: <ST4000VN000-1H4168 SC42> ATA-9 SATA 3.x device
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada3: Previously was known as ad10
ada4 at ahcich4 bus 0 scbus5 target 0 lun 0
ada4: <ST4000VN000-1H4168 SC42> ATA-9 SATA 3.x device
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 3815447MB (7814037168 512 byte sectors: 16H 63S/T 16383C)
ada4: Previously was known as ad12
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #4 Launched!
Timecounter "TSC-low" frequency 1354599490 Hz quality 800
Trying to mount root from ufs:/dev/ufs/FreeNASs2a [ro]...
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
GEOM_RAID5: Module loaded, version 1.1.20110927.40 (rev 00ce00e5abb4)
GEOM_ELI: Device da0p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device da1p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device da2p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device da4p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device da3p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada0p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada1p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada2p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada3p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
GEOM_ELI: Device ada4p1.eli created.
GEOM_ELI: Encryption: AES-XTS 256
GEOM_ELI:    Crypto: software
[smurfy@nas] /mnt/stor/temp# uname -v
FreeBSD 9.1-STABLE #0 r+1a51513: Thu Jul 25 09:39:42 PDT 2013    root@build.ixsystems.com:/tank/home/jkh/src/freenas/os-base/amd64/tank/home/jkh/src/freenas/FreeBSD/src/sys/FREENAS.amd64
[smurfy@nas] /mnt/stor/temp# arc_summary.py |more
System Memory:
 
        1.02%  160.89  MiB Active,    0.88%  139.29  MiB Inact
        53.85%  8.33    GiB Wired,      0.01%  1.14    MiB Cache
        44.25%  6.84    GiB Free,      0.00%  672.00  KiB Gap
 
        Real Installed:                        16.00  GiB
        Real Available:                99.68%  15.95  GiB
        Real Managed:                  96.97%  15.47  GiB
 
        Logical Total:                          16.00  GiB
        Logical Used:                  56.37%  9.02    GiB
        Logical Free:                  43.63%  6.98    GiB
 
Kernel Memory:                                  8.02    GiB
        Data:                          99.72%  8.00    GiB
        Text:                          0.28%  22.71  MiB
 
Kernel Memory Map:                              10.66  GiB
        Size:                          74.60%  7.95    GiB
        Free:                          25.40%  2.71    GiB
                                                                Page:  1
------------------------------------------------------------------------
 
ARC Summary: (HEALTHY)
        Storage pool Version:                  5000
        Filesystem Version:                    5
        Memory Throttle Count:                  0
 
ARC Misc:
        Deleted:                                240.21k
        Recycle Misses:                        10.03k
        Mutex Misses:                          3
        Evict Skips:                            3
 
ARC Size:                              80.53%  7.94    GiB
        Target Size: (Adaptive)        80.53%  7.94    GiB
        Min Size (Hard Limit):          12.50%  1.23    GiB
        Max Size (High Water):          8:1    9.86    GiB
 
ARC Size Breakdown:
        Recently Used Cache Size:      97.75%  7.76    GiB
        Frequently Used Cache Size:    2.25%  183.39  MiB
 
ARC Hash Breakdown:
        Elements Max:                          99.58k
        Elements Current:              98.43%  98.02k
        Collisions:                            91.21k
        Chain Max:                              5
        Chains:                                15.06k
                                                                Page:  2
------------------------------------------------------------------------
 
ARC Efficiency:                                95.42k
        Cache Hit Ratio:                97.39%  92.92k
        Cache Miss Ratio:              2.61%  2.49k
        Actual Hit Ratio:              96.47%  92.05k
 
        Data Demand Efficiency:        99.99%  30.21k
 
        CACHE HITS BY CACHE LIST:
          Most Recently Used:          24.00%  22.30k
          Most Frequently Used:        75.06%  69.75k
          Most Recently Used Ghost:    1.12%  1.04k
          Most Frequently Used Ghost:  0.00%  0
 
        CACHE HITS BY DATA TYPE:
          Demand Data:                  32.50%  30.20k
          Prefetch Data:                0.00%  0
          Demand Metadata:              65.61%  60.97k
          Prefetch Metadata:            1.89%  1.75k
 
        CACHE MISSES BY DATA TYPE:
          Demand Data:                  0.08%  2
          Prefetch Data:                0.00%  0
          Demand Metadata:              45.77%  1.14k
          Prefetch Metadata:            54.15%  1.35k
                                                                Page:  3
------------------------------------------------------------------------
 
                                                                Page:  4
------------------------------------------------------------------------
 
File-Level Prefetch: (HEALTHY)
 
DMU Efficiency:                                6.32m
        Hit Ratio:                      80.31%  5.08m
        Miss Ratio:                    19.69%  1.25m
 
        Colinear:                              1.25m
          Hit Ratio:                    0.00%  0
          Miss Ratio:                  100.00% 1.25m
 
        Stride:                                5.08m
          Hit Ratio:                    100.00% 5.08m
          Miss Ratio:                  0.00%  4
 
DMU Misc:
        Reclaim:                                1.25m
          Successes:                    0.06%  689
          Failures:                    99.94%  1.24m
 
        Streams:                                365
          +Resets:                      0.00%  0
          -Resets:                      100.00% 365
          Bogus:                                0
                                                                Page:  5
------------------------------------------------------------------------
 
                                                                Page:  6
------------------------------------------------------------------------
 
ZFS Tunable (sysctl):
        kern.maxusers                          384
        vm.kmem_size                            11766556364
        vm.kmem_size_scale                      1
        vm.kmem_size_min                        0
        vm.kmem_size_max                        14708195456
        vfs.zfs.l2c_only_size                  0
        vfs.zfs.mfu_ghost_data_lsize            267585024
        vfs.zfs.mfu_ghost_metadata_lsize        779776
        vfs.zfs.mfu_ghost_size                  268364800
        vfs.zfs.mfu_data_lsize                  115081216
        vfs.zfs.mfu_metadata_lsize              24576512
        vfs.zfs.mfu_size                        139840000
        vfs.zfs.mru_ghost_data_lsize            59113472
        vfs.zfs.mru_ghost_metadata_lsize        152130560
        vfs.zfs.mru_ghost_size                  211244032
        vfs.zfs.mru_data_lsize                  8269841408
        vfs.zfs.mru_metadata_lsize              5559296
        vfs.zfs.mru_size                        8316626944
        vfs.zfs.anon_data_lsize                0
        vfs.zfs.anon_metadata_lsize            0
        vfs.zfs.anon_size                      19187712
        vfs.zfs.l2arc_norw                      1
        vfs.zfs.l2arc_feed_again                1
        vfs.zfs.l2arc_noprefetch                1
        vfs.zfs.l2arc_feed_min_ms              200
        vfs.zfs.l2arc_feed_secs                1
        vfs.zfs.l2arc_headroom                  2
        vfs.zfs.l2arc_write_boost              8388608
        vfs.zfs.l2arc_write_max                8388608
        vfs.zfs.arc_meta_limit                  2647475181
        vfs.zfs.arc_meta_used                  124269576
        vfs.zfs.arc_min                        1323737590
        vfs.zfs.arc_max                        10589900727
        vfs.zfs.dedup.prefetch                  1
        vfs.zfs.mdcomp_disable                  0
        vfs.zfs.nopwrite_enabled                1
        vfs.zfs.write_limit_override            0
        vfs.zfs.write_limit_inflated            51377037312
        vfs.zfs.write_limit_max                2140709888
        vfs.zfs.write_limit_min                33554432
        vfs.zfs.write_limit_shift              3
        vfs.zfs.no_write_throttle              0
        vfs.zfs.zfetch.array_rd_sz              1048576
        vfs.zfs.zfetch.block_cap                256
        vfs.zfs.zfetch.min_sec_reap            2
        vfs.zfs.zfetch.max_streams              8
        vfs.zfs.prefetch_disable                0
        vfs.zfs.no_scrub_prefetch              0
        vfs.zfs.no_scrub_io                    0
        vfs.zfs.resilver_min_time_ms            3000
        vfs.zfs.free_min_time_ms                1000
        vfs.zfs.scan_min_time_ms                1000
        vfs.zfs.scan_idle                      50
        vfs.zfs.scrub_delay                    4
        vfs.zfs.resilver_delay                  2
        vfs.zfs.top_maxinflight                32
        vfs.zfs.write_to_degraded              0
        vfs.zfs.mg_alloc_failures              9
        vfs.zfs.check_hostid                    1
        vfs.zfs.deadman_enabled                1
        vfs.zfs.deadman_synctime                1000
        vfs.zfs.recover                        0
        vfs.zfs.txg.synctime_ms                1000
        vfs.zfs.txg.timeout                    5
        vfs.zfs.vdev.cache.bshift              16
        vfs.zfs.vdev.cache.size                0
        vfs.zfs.vdev.cache.max                  16384
        vfs.zfs.vdev.trim_on_init              1
        vfs.zfs.vdev.write_gap_limit            4096
        vfs.zfs.vdev.read_gap_limit            32768
        vfs.zfs.vdev.aggregation_limit          131072
        vfs.zfs.vdev.ramp_rate                  2
        vfs.zfs.vdev.time_shift                29
        vfs.zfs.vdev.min_pending                4
        vfs.zfs.vdev.max_pending                10
        vfs.zfs.vdev.larger_ashift_minimal      1
        vfs.zfs.vdev.larger_ashift_disable      0
        vfs.zfs.vdev.bio_delete_disable        0
        vfs.zfs.vdev.bio_flush_disable          0
        vfs.zfs.vdev.trim_max_pending          64
        vfs.zfs.vdev.trim_max_bytes            2147483648
        vfs.zfs.cache_flush_disable            0
        vfs.zfs.zil_replay_disable              0
        vfs.zfs.sync_pass_rewrite              2
        vfs.zfs.sync_pass_dont_compress        5
        vfs.zfs.sync_pass_deferred_free        2
        vfs.zfs.zio.use_uma                    0
        vfs.zfs.snapshot_list_prefetch          0
        vfs.zfs.version.ioctl                  3
        vfs.zfs.version.zpl                    5
        vfs.zfs.version.spa                    5000
        vfs.zfs.version.acl                    1
        vfs.zfs.debug                          0
        vfs.zfs.super_owner                    0
        vfs.zfs.trim.enabled                    1
        vfs.zfs.trim.max_interval              1
        vfs.zfs.trim.timeout                    30
        vfs.zfs.trim.txg_delay                  32
                                                                Page:  7
------------------------------------------------------------------------
 

Attachments

  • top.png
    top.png
    242 KB · Views: 445

papageorgi

Explorer
Joined
Jul 16, 2013
Messages
51
hmmm maybe it was autotune needing more time? looks like it just bumped up to my normal speeds, maxing usb2 and awaiting my usb3 hard drive dock.
 

Attachments

  • backtonormal?.png
    backtonormal?.png
    27.1 KB · Views: 343

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
Odd. I don't have any recommendations to solve your problem, but I see a big red flag.

Using USB2 drives aside from temporary storage has proven to be a major problem for many users in the forum. Trying to run scrubs on zpools doesn't go well(if at all), there is no ability to monitor your drives for SMART errors(which is one of the main methods to identify a disk that may be starting to fail), the extra USB cables increase the chance of accidentally being unplugged, and sometimes even a single bad sector can cause the drive to "disconnect" from the system.

You are kind of playing with fire by trying to use external disks for a file server. When a USB drive starts having problems you will find that you have very little tools to identify the actual problem let alone fix it. You'll be in a situation where you know something is very wrong, but there is no way to prove a diagnosis.
 

papageorgi

Explorer
Joined
Jul 16, 2013
Messages
51
oh i'm sorry i wasn't clear. the usb drives are not in or part of the server anymore than just being a backup of the server is all. i've used the laptop to backup the server through ftp.
it seems it's now very much faster somehow. i think the auto tune kicked in suddenly, which is strange since it's had many days and also several restarts since...

my setup = freenas server with 10 4tb hdds -> gigbit lan -> ssd lenovo laptop with linux -> connecting via filezilla's sftp -> usb2 external hard drives to backup data

i do have a usb3/esata/firewire 800 drive coming from newertechnology to help partially remedy this and hopefully will see the transfer jump to at least 60MB/s. i know this isn't ideal i just can't afford the hardware to build another nas.
 

cyberjock

Inactive Account
Joined
Mar 25, 2012
Messages
19,526
The autotune changes only "apply" on bootup. Weird nonetheless.
 
Status
Not open for further replies.
Top