FreeNAS Mini - best uses for extra storage devices?

[joeschmuck]

If you have V1.03 you might be ok.

I'm curious on what your version is. I hope someone didn't sell you a lemon.

 

[Constantin]

.... especially since ASROCK is no longer replacing them under warranty.

As for the question further up re: 7,200 RPM vs. 5,400 RPM drives, that's unlikely to make much of a difference re: transfer, seek, etc. in a Mini. If you're going replacement drive shopping, a much bigger factor is helium vs. non-helium, CMR vs. SMR, etc. Farmerpling2 has a nice resource page re: drives that I'd peruse, as well as the resource created by Yorick re: the evils of SMR drives in a ZFS application.

TL; DNR; Helium CMR drives are the way to go in constricted cases like the Mini XL Ablecom T80. The upper drives in particular got toasted nicely due to absent air flow. The Mini drives may not be as marginal since the rear fan is closer to all the drives in that case but I'd still try to get something that puts out little heat to prolong HDD life.

Also beware that repeat offenders like WD are now marketing "5,400 RPM Class" drives, which is marketing-speak for 7,200 RPM drives that have been artificially throttled to give you the worst of all worlds, i.e. the heat, shorter life, and power consumption of a 7,200 RPM drive with the performance of a slower 5,400 RPM drive.

 

[joeschmuck]

Also beware that repeat offenders like WD are now marketing "5,400 RPM Class" drives, which is marketing-speak for 7,200 RPM drives that have been artificially throttled to give you the worst of all worlds, i.e. the heat, shorter life, and power consumption of a 7,200 RPM drive with the performance of a slower 5,400 RPM drive.

Great point, and this kind of marketing drives me nuts.

Overall the FreeNAS Mini has a small case (I have not seen one in person) so cooling the drives should be of concern to you, this is where a slower RPM drive generally comes in better, but if your 7200 RPM drives are doing the job, there is no need to change anything until they fail or you just need more storage.

 

[dbsoundman]

Sorry I didn't get back to you sooner, been a crazy week.

Drive ada0 (Serial Number: WD-WCC4E5NPVP0E) looks fine. The key things I look at are ID's 5, 193, 194, 196, 197, 198, 199, 200, and the SMART Self-test log. Notice that only a single Extended test has been conducted. I personally recommend a daily Short test for all drives and a weekly Extended/Long test for all drives. Also becasue you bought a used system I looked up the warranty for that drive, it expired Feb 2019. Here is the URL to look that up for your other drives.

Go ahead and post the output of any other drive you want a second opinion on.

I have one more to look at as code 200 reported a '1' and I see what seems to be an error logged toward the end of the output:

Code:

truenas% sudo smartctl -a /dev/ada6
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.2-RELEASE-p3 amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Red
Device Model:     WDC WD40EFRX-68WT0N0
Serial Number:    WD-WCC4E0XTNTVR
LU WWN Device Id: 5 0014ee 20cefa6f4
Firmware Version: 82.00A82
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Thu Feb 25 20:54:50 2021 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      ( 113) The previous self-test completed having
                                        the read element of the test failed.
Total time to complete Offline
data collection:                (51420) seconds.
Offline data collection
capabilities:                    (0x7b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 514) minutes.
Conveyance self-test routine
recommended polling time:        (   5) minutes.
SCT capabilities:              (0x703d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       15
  3 Spin_Up_Time            0x0027   196   176   021    Pre-fail  Always       -       7200
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       78
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   046   046   000    Old_age   Always       -       40029
 10 Spin_Retry_Count        0x0032   100   253   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   253   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       78
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       65
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       863
194 Temperature_Celsius     0x0022   121   112   000    Old_age   Always       -       31
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       1

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed: read failure       10%     40009         3468362128

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

 

[Redcoat]

The thing that really confuses me here is how do I tell when this test was run?

The previous extended test was run 16 "drive-powered-hours" before you issued the "sudo smartctl -a /dev/ada0" command - look at the "lifetime" and "power on hours" values.

Also I noticed the drives are 5400 RPM, which is not ideal.

? Why so "not ideal"? Many of us here are happy to enjoy the low heat production of 5400rpm drives in our servers, As an ex-FreeNAS Mini owner who suffered heat stress issues in the tight Ablecom chassis with 4 off 4TB WD EFRX reds (5400rpm) and the overall low-air flow potential with the single case fan and the passively-cooled CPU, I would not advocate changing to 7200rpm drives.

 

[Redcoat]

I would not advocate changing to 7200rpm drives.

And I make that comment even though I added a fan to pull air over the CPU heatsink, put in a larger rear case fan and modified the lower front of the chassis to accept two small fans to feed air to the CPU fan. And I ran a/c almost year-round in Cleveland, OH, to keep my office cool.

 

[joeschmuck]

# 1 Extended offline Completed: read failure 10% 40009 3468362128

Red Flag, your drive has failed a read test. This might be related to ID 200 error but ID 200 errors are a hit and miss in my book.

Run a SMART Short test, see if it passes (it might), then run a long test again. If it fails either test then your drive will likely start incrementing the sector error counts once some data is written to that LBA area of the hard drive. If this is the case, plan to replace it soon. Remember, you must also be able to pass a SMART Long/Extended test for the drive to be considered good.

 

[HoneyBadger]

Wow, maybe I don't want a ZIL, the latency on this thing is almost as bad as the "terrible" benchmark set forth in this thread; his topped out at 70000 usec/IO, mine got up to 63568.2 usec/IO!

Note that this NAS is the FreeNAS Mini, so it's probably around 5 years old or so, these could be the best SSDs available at the time.

Code:

truenas% sudo diskinfo -wS /dev/ada4
Password:
/dev/ada4
    512             # sectorsize
    64023257088     # mediasize in bytes (60G)
    125045424       # mediasize in sectors
    4096            # stripesize
    0               # stripeoffset
    124053          # Cylinders according to firmware.
    16              # Heads according to firmware.
    63              # Sectors according to firmware.
    SanDisk SD6SB1M064G1022I    # Disk descr.
    151929400303    # Disk ident.
    Yes             # TRIM/UNMAP support
    0               # Rotation rate in RPM
    Not_Zoned       # Zone Mode

Synchronous random writes:
     0.5 kbytes:    834.0 usec/IO =      0.6 Mbytes/s
       1 kbytes:    834.9 usec/IO =      1.2 Mbytes/s
       2 kbytes:    836.6 usec/IO =      2.3 Mbytes/s
       4 kbytes:    704.8 usec/IO =      5.5 Mbytes/s
       8 kbytes:    700.9 usec/IO =     11.1 Mbytes/s
      16 kbytes:    780.2 usec/IO =     20.0 Mbytes/s
      32 kbytes:    924.8 usec/IO =     33.8 Mbytes/s
      64 kbytes:   1769.6 usec/IO =     35.3 Mbytes/s
     128 kbytes:   8857.1 usec/IO =     14.1 Mbytes/s
     256 kbytes:  12305.2 usec/IO =     20.3 Mbytes/s
     512 kbytes:  12988.5 usec/IO =     38.5 Mbytes/s
    1024 kbytes:  16346.3 usec/IO =     61.2 Mbytes/s
    2048 kbytes:  23025.6 usec/IO =     86.9 Mbytes/s
    4096 kbytes:  36680.5 usec/IO =    109.0 Mbytes/s
    8192 kbytes:  63568.2 usec/IO =    125.8 Mbytes/s

A tangential note here on interpreting the results of the diskinfo benchmark - the I/O size (leftmost column) matters here quite a bit when determining if a particular drive will be "fast" or "slow" for your purposes. Simply comparing "biggest number" to "biggest number" won't cut it, because different types of workloads write different sizes of records.

Users looking for random-write performance (such as with virtualization or databases) are often most concerned with the small 4/8/16K sizes the hypervisor/DB engine sends writes in, whereas someone looking for general bulk NAS traffic might be able to leverage the 64K/128K speeds.

For example; while your X110 kicks out 125MB/s at the high end, while an old Intel 320 80GB of mine (of older vintage) only gets to around 92MB/s. But lower the block size to 4KB, and the Intel drive churns out 26.5MB/s - over 5x the speed of the Sandisk. The edge drops to roughly 4-4.5x by the time we get to 8KB or 16KB blocks, but if the workload you added the SLOG for is using those smaller sizes, it will be a decisive victory in favour of the "slower" Intel drive.

@joeschmuck is absolutely correct though in that most (almost all) users don't need an SLOG, as they don't have a synchronous-write workload (such as remote VMs or databases) so this is a moot point, I just wanted to help you and anyone else reading along with their understanding of the results.

 

[dbsoundman]

Agreed, which is why I went with the MiniXl way back when. If you take off the side, you should be able to see the motherboard version iteration next to the PCIE 3x8 slot. If you have V1.03 you might be ok. That what I got from iXSystems as a replacement and the CPU is married to the board...

Good news, I got in there with my phone camera and found the motherboard revision is indeed 1.03, so I've got that going for me! Now I just have to go hard drive shopping a little earlier than anticipated.

 

[dbsoundman]

.... especially since ASROCK is no longer replacing them under warranty.

As for the question further up re: 7,200 RPM vs. 5,400 RPM drives, that's unlikely to make much of a difference re: transfer, seek, etc. in a Mini. If you're going replacement drive shopping, a much bigger factor is helium vs. non-helium, CMR vs. SMR, etc. Farmerpling2 has a nice resource page re: drives that I'd peruse, as well as the resource created by Yorick re: the evils of SMR drives in a ZFS application.

TL; DNR; Helium CMR drives are the way to go in constricted cases like the Mini XL Ablecom T80. The upper drives in particular got toasted nicely due to absent air flow. The Mini drives may not be as marginal since the rear fan is closer to all the drives in that case but I'd still try to get something that puts out little heat to prolong HDD life.

Also beware that repeat offenders like WD are now marketing "5,400 RPM Class" drives, which is marketing-speak for 7,200 RPM drives that have been artificially throttled to give you the worst of all worlds, i.e. the heat, shorter life, and power consumption of a 7,200 RPM drive with the performance of a slower 5,400 RPM drive.

Any recommendations on a Helium CMR drive that actually runs at 5400 RPM natively? WD Red Pro is the only thing I'm aware of at the moment but it's "5400 class" as you mentioned. I know this thread is way off topic at the moment but might as well keep the conversation going!

 

[Constantin]

You might be able to find them used but IIRC all He drives being made now are 7,200 RPM. He drives are usually high capacity and destined for data centers where operators appreciate the high data density combined with low heat and power consumption. That lowers the cooling bill.