Need help with results of my HDDs tests. Bad HDDs ?

[marcouan]

Hello All,

New to the forum and TrueNas.
Great community hope u can help.
I have a used Dell Poweredge R330 with E-3 1280v5 16GB of RAM
PERC H730 Adapter in hba mode and 7 x 1TB sata 3.5inc "Entreprise" HDDs 7200rpm.

Now i know is better to start with new HDDs with larger capacity and leave room for SSDs instead.
Maxing on memory is also a must i understand.

But for now i just added a normal 1TB 7200 HDD.
Before i create a pool i wanted to bencmark my hdds to verify that all perform the same
Please see my results below.
I expected from the enterprise HDDs to perform at least the same with the new Desktop HDD?
What is wrong here?
Thanks for ur replies and Marry Christmass

Run status group 0 (all jobs):
root@truenas1[~]# fio --filename=/dev/da0 --direct=1 --fsync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=4k-sync-write-test
4k-sync-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.27
WRITE: bw=309KiB/s (317kB/s), 309KiB/s-309KiB/s (317kB/s-317kB/s), io=18.1MiB (19.0MB), run=60002-60002msec
root@truenas1[~]#
root@truenas1[~]# dd if=/dev/zero of=/dev/da0 bs=2048k count=1k
1024+0 records in
1024+0 records out
2147483648 bytes transferred in 48.100330 secs (44645923 bytes/sec)
root@truenas1[~]#
Disk Type:
HDD

Description:
Model:
ATA TOSHIBA MG03ACA1

Transfer Mode:
Auto

Rotation Rate (RPM):
7200

HDD Standby:
ALWAYS ON

Adv. Power Management:
DISABLED

Acoustic Level:
DISABLED

Enable S.M.A.R.T.:
true

S.M.A.R.T. extra options:
##########################
root@truenas1[~]# fio --filename=/dev/da1 --direct=1 --fsync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=4k-sync-write-test
4k-sync-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.27
Jobs: 1 (f=1): [f(1)][100.0%][w=19KiB/s][w=4 IOPS][eta 00m:00s]
WRITE: bw=309KiB/s (316kB/s), 309KiB/s-309KiB/s (316kB/s-316kB/s), io=18.1MiB (19.0MB), run=60001-60001msec
root@truenas1[~]#
root@truenas1[~]# dd if=/dev/zero of=/dev/da1 bs=2048k count=1k
1024+0 records in
1024+0 records out
2147483648 bytes transferred in 47.914339 secs (44819227 bytes/sec)
root@truenas1[~]#
Disk Type:
HDD

Description:
Model:
ATA TOSHIBA MG03ACA1

Transfer Mode:
Auto

Rotation Rate (RPM):
7200

HDD Standby:
ALWAYS ON

Adv. Power Management:
DISABLED

Acoustic Level:
DISABLED

Enable S.M.A.R.T.:
true

S.M.A.R.T. extra options:
##########################
root@truenas1[~]# fio --filename=/dev/da2 --direct=1 --fsync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=4k-sync-write-test
4k-sync-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.27
WRITE: bw=310KiB/s (317kB/s), 310KiB/s-310KiB/s (317kB/s-317kB/s), io=18.1MiB (19.0MB), run=60001-60001msec
root@truenas1[~]#
root@truenas1[~]# dd if=/dev/zero of=/dev/da2 bs=2048k count=1k
1024+0 records in
1024+0 records out
2147483648 bytes transferred in 47.922465 secs (44811628 bytes/sec)
root@truenas1[~]#

Disk Type:
HDD

Description:
Model:
ATA TOSHIBA MG03ACA1

Transfer Mode:
Auto

Rotation Rate (RPM):
7200

HDD Standby:
ALWAYS ON

Adv. Power Management:
DISABLED

Acoustic Level:
DISABLED

Enable S.M.A.R.T.:
true

S.M.A.R.T. extra options:

###########################
root@truenas1[~]# fio --filename=/dev/da3 --direct=1 --fsync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=4k-sync-write-test
4k-sync-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.27
WRITE: bw=302KiB/s (309kB/s), 302KiB/s-302KiB/s (309kB/s-309kB/s), io=17.7MiB (18.5MB), run=60005-60005msec
root@truenas1[~]#
root@truenas1[~]# dd if=/dev/zero of=/dev/da3 bs=2048k count=1k
1024+0 records in
1024+0 records out
2147483648 bytes transferred in 28.763373 secs (74660354 bytes/sec)
root@truenas1[~]#
Disk Type:
HDD

Description:
Model:
ATA HUS722T1TALA600

Transfer Mode:
Auto

Rotation Rate (RPM):
7200

HDD Standby:
ALWAYS ON

Adv. Power Management:
DISABLED

Acoustic Level:
DISABLED

Enable S.M.A.R.T.:
true

S.M.A.R.T. extra options:
###############################################
root@truenas1[~]# fio --filename=/dev/da4 --direct=1 --fsync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=4k-sync-write-test
4k-sync-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.27
WRITE: bw=301KiB/s (308kB/s), 301KiB/s-301KiB/s (308kB/s-308kB/s), io=17.6MiB (18.5MB), run=60007-60007msec
root@truenas1[~]#
root@truenas1[~]# dd if=/dev/zero of=/dev/da4 bs=2048k count=1k
1024+0 records in
1024+0 records out
2147483648 bytes transferred in 28.963225 secs (74145183 bytes/sec)
root@truenas1[~]#
Disk Type:
HDD

Description:
Model:
ATA HUS722T1TALA600

Transfer Mode:
Auto

Rotation Rate (RPM):
7200

HDD Standby:
ALWAYS ON

Adv. Power Management:
DISABLED

Acoustic Level:
DISABLED

Enable S.M.A.R.T.:
true

S.M.A.R.T. extra options:
#########################################
root@truenas1[~]# fio --filename=/dev/da5 --direct=1 --fsync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=4k-sync-write-test
4k-sync-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.27
WRITE: bw=303KiB/s (310kB/s), 303KiB/s-303KiB/s (310kB/s-310kB/s), io=17.8MiB (18.6MB), run=60001-60001msec
root@truenas1[~]#
root@truenas1[~]# dd if=/dev/zero of=/dev/da5 bs=2048k count=1k
1024+0 records in
1024+0 records out
2147483648 bytes transferred in 28.846676 secs (74444753 bytes/sec)
root@truenas1[~]#
Disk Type:
HDD

Description:
Model:
ATA HUS722T1TALA600

Transfer Mode:
Auto

Rotation Rate (RPM):
7200

HDD Standby:
ALWAYS ON

Adv. Power Management:
DISABLED

Acoustic Level:
DISABLED

Enable S.M.A.R.T.:
true

S.M.A.R.T. extra options:
#######################
root@truenas1[~]# fio --filename=/dev/da6 --direct=1 --fsync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=4k-sync-write-test
4k-sync-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.27
WRITE: bw=300KiB/s (307kB/s), 300KiB/s-300KiB/s (307kB/s-307kB/s), io=17.6MiB (18.4MB), run=60001-60001msec
root@truenas1[~]#
root@truenas1[~]# dd if=/dev/zero of=/dev/da6 bs=2048k count=1k
1024+0 records in
1024+0 records out
2147483648 bytes transferred in 28.881136 secs (74355927 bytes/sec)
root@truenas1[~]#
Disk Type:
HDD

Description:
Model:
ATA HUS722T1TALA600

Transfer Mode:
Auto

Rotation Rate (RPM):
7200

HDD Standby:
ALWAYS ON

Adv. Power Management:
DISABLED

Acoustic Level:
DISABLED

Enable S.M.A.R.T.:
true

S.M.A.R.T. extra options:
##############
root@truenas1[~]# fio --filename=/dev/da7 --direct=1 --fsync=1 --rw=randwrite --bs=4k --numjobs=1 --iodepth=1 --runtime=60 --time_based --group_reporting --name=4k-sync-write-test
4k-sync-write-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.27
WRITE: bw=606KiB/s (621kB/s), 606KiB/s-606KiB/s (621kB/s-621kB/s), io=35.5MiB (37.2MB), run=60005-60005msec
root@truenas1[~]#
root@truenas1[~]# dd if=/dev/zero of=/dev/da7 bs=2048k count=1k
1024+0 records in
1024+0 records out
2147483648 bytes transferred in 11.755133 secs (182684762 bytes/sec)
root@truenas1[~]#
Disk Type:
HDD

Description:
Model:
ATA ST1000DM010-2EP1

Transfer Mode:
Auto

Rotation Rate (RPM):
7200

HDD Standby:
ALWAYS ON

Adv. Power Management:
DISABLED

Acoustic Level:
DISABLED

Enable S.M.A.R.T.:
true

S.M.A.R.T. extra options:

 

[marcouan]

Do u know if the performance of an hdd drops after some years?

 

[jgreco]

PERC H730 Adapter

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

There is no such thing as

hba mode

Dell's marketing people may want to think so, but painting a semi truck to look like a pickup does not actually make it a pickup truck.

Additional reading:

Dell PERC H730 Working with SAS, not SATA

I had a working 11.2 Freenas instance, installed on a Dell T430. This has a PERC H730 in HBA mode, and passes through 8 SAS disks which were in a RAIDZ2. I bought new sata drives, and stuck them in. Same controller, and TrueNAS 12 does not see them at all. I have factory reset, re-enabled HBA...

www.truenas.com

I expected from the enterprise HDDs to perform at least the same with the new Desktop HDD?

And as for this, a 1TB 7.2KRPM HDD probably designed at least a decade ago is likely to seek faster but transfer more slowly than a modern 5900RPM capacity-oriented HDD. Both results will be skewed by your use of an unsuitable controller.

 

[marcouan]

Hello Jgreco

Thanks for your reply and useful info.
Wish I had this info before spending money on adaptec asr71605.

I had the impression that h730 is a good choice. At least that came with the server and the 7 old drives.

So LSI hba is the only choice?
Looks like the project will be put on hold until I can get the right hardware

Thanks again for the info

 

[jgreco]

Yeah, sorry, the 71605 is a trash-grade RAID controller. There was a day long ago when Adaptec was king of SCSI, but that's not the case today.

So LSI hba is the only choice?

ANYTHING that reliably talks to the drives, can manage immense levels of I/O, provides direct access to the drive, and has solid driver support is absolutely 100% fine.

However, in practice, the actual controllers eligible to be put on that list boil down to:

Intel AHCI SATA controllers
Some add-on AHCI controllers (which can be hard to guarantee authenticity)
Intel PCH SCU SATA/SAS controllers
LSI mpr/mps based HBA's

Of those, only the LSI HBA's are readily available and cheap to acquire, if you don't already have some reasonable solution on your mainboard.