Hi,
Sorry, apologies, that was a mistake - it's not a N54L - it's actually a HP MicroServer Gen8 (the model after the N54L) - I also have a N54L...lol.
And this was a Mac Mini with a SSD, not a Macbook Pro.
The Gen8 has 10 Gb of RAM in it (I know ZFS doesn't deal well with constrained memory situations).
Code:
[victorhooi@freenas] /mnt/datastore% dmesg | grep memory
real memory = 10737418240 (10240 MB)
avail memory = 10126843904 (9657 MB)
pci2: <memory> at device 0.0 (no driver attached)
The drives are all 3TB each.
The switch is a HP PS1810-8G switch (the one they sell with the Gen8 server).
@gpsguy - I think you found the issue - silly, I should have checked this myself.
I think the issue may be the USB 3.0 external drive - I did a copy from the Mac's SSD to the NAS - and I'm getting between 70-90 MB/s:
Code:
capacity operations bandwidth
pool alloc free read write read write
-------------------------------------- ----- ----- ----- ----- ----- -----
datastore 405G 10.5T 3 761 13.6K 71.2M
raidz1 405G 10.5T 3 761 13.6K 71.2M
gptid/36d5bb2e-86cf-11e3-8a5c-10604b92dc14 - - 0 292 4.00K 20.7M
gptid/37e9e6af-86cf-11e3-8a5c-10604b92dc14 - - 1 291 4.80K 20.6M
gptid/388b045d-86cf-11e3-8a5c-10604b92dc14 - - 0 373 1.60K 24.3M
gptid/39059520-86cf-11e3-8a5c-10604b92dc14 - - 0 336 3.20K 24.3M
-------------------------------------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool alloc free read write read write
-------------------------------------- ----- ----- ----- ----- ----- -----
datastore 405G 10.5T 4 952 20.0K 90.5M
raidz1 405G 10.5T 4 952 20.0K 90.5M
gptid/36d5bb2e-86cf-11e3-8a5c-10604b92dc14 - - 0 423 4.00K 31.3M
gptid/37e9e6af-86cf-11e3-8a5c-10604b92dc14 - - 1 392 5.60K 31.2M
gptid/388b045d-86cf-11e3-8a5c-10604b92dc14 - - 0 420 3.20K 31.3M
gptid/39059520-86cf-11e3-8a5c-10604b92dc14 - - 1 380 7.20K 31.3M
-------------------------------------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool alloc free read write read write
-------------------------------------- ----- ----- ----- ----- ----- -----
datastore 406G 10.5T 4 924 16.8K 90.4M
raidz1 406G 10.5T 4 924 16.8K 90.4M
gptid/36d5bb2e-86cf-11e3-8a5c-10604b92dc14 - - 0 416 4.00K 31.3M
gptid/37e9e6af-86cf-11e3-8a5c-10604b92dc14 - - 0 370 3.20K 31.2M
gptid/388b045d-86cf-11e3-8a5c-10604b92dc14 - - 1 413 6.40K 31.3M
gptid/39059520-86cf-11e3-8a5c-10604b92dc14 - - 0 389 3.20K 31.3M
-------------------------------------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool alloc free read write read write
-------------------------------------- ----- ----- ----- ----- ----- -----
datastore 406G 10.5T 5 981 21.6K 90.7M
raidz1 406G 10.5T 5 981 21.6K 90.7M
gptid/36d5bb2e-86cf-11e3-8a5c-10604b92dc14 - - 0 434 4.00K 31.4M
gptid/37e9e6af-86cf-11e3-8a5c-10604b92dc14 - - 1 397 7.20K 31.3M
gptid/388b045d-86cf-11e3-8a5c-10604b92dc14 - - 0 431 3.20K 31.4M
gptid/39059520-86cf-11e3-8a5c-10604b92dc14 - - 1 396 7.20K 31.4M
-------------------------------------- ----- ----- ----- ----- ----- -----
capacity operations bandwidth
pool alloc free read write read write
-------------------------------------- ----- ----- ----- ----- ----- -----
datastore 407G 10.5T 3 604 13.6K 57.6M
raidz1 407G 10.5T 3 604 13.6K 57.6M
gptid/36d5bb2e-86cf-11e3-8a5c-10604b92dc14 - - 1 310 7.18K 19.9M
gptid/37e9e6af-86cf-11e3-8a5c-10604b92dc14 - - 0 295 0 19.6M
gptid/388b045d-86cf-11e3-8a5c-10604b92dc14 - - 1 306 6.39K 19.9M
gptid/39059520-86cf-11e3-8a5c-10604b92dc14 - - 0 282 0 19.9M
-------------------------------------- ----- ----- ----- ----- ----- -----
But copying from the USB 3.0 external drive to the Mac's SSD - I get between 10-20 MB/s:
Code:
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.0.shadowIndexGroups
19,580,706 2% 13.50MB/s 0:00:01 (xfr#37, ir-chk=1175/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.0.shadowIndexHead
19,584,802 2% 13.38MB/s 0:00:01 (xfr#38, ir-chk=1174/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.directoryStoreFile
21,681,954 2% 13.70MB/s 0:00:01 (xfr#39, ir-chk=1173/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.directoryStoreFile.shadow
22,861,602 3% 13.56MB/s 0:00:01 (xfr#40, ir-chk=1172/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.indexArrays
60,521,954 7% 15.52MB/s 0:00:03 (xfr#41, ir-chk=1171/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.indexCompactDirectory
67,832,458 8% 15.66MB/s 0:00:04 (xfr#42, ir-chk=1170/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.indexDirectory
68,797,750 9% 15.68MB/s 0:00:04 (xfr#43, ir-chk=1169/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.indexGroups
68,998,442 9% 15.67MB/s 0:00:04 (xfr#44, ir-chk=1168/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.indexHead
69,002,538 9% 15.65MB/s 0:00:04 (xfr#45, ir-chk=1167/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.indexIds
71,009,458 9% 15.19MB/s 0:00:04 (xfr#46, ir-chk=1166/1220)
datastore_backup/.Spotlight-V100/Store-V2/903BCB91-0446-436A-8B71-3B866206B68D/live.1.indexPositions
245,091,051 32% 18.02MB/s 0:00:12 (xfr#47, ir-chk=1165/1220)
But I'm guessing the 70-90 MB/s seems more in line with what we should be seeing on the NAS, with this sort of hardware? Or is there hope of pushing through more?
So the theoretical limit of Gigabit is 125 MB/s - how close could I be aiming to get close to that?
Cheers,
Victor