(un)Planned upgrade

[Bhoot]

EPILOGUE

A bit of the background. My SYSTEM SPECS are available in my signature. It’s been nearly 3 years from original build and about 10-12 months from my last upgrade/addition of the noctua fans.

Obviously the digital media is growing at an exponential rate and my family now uses my FreeNAS box as a dump bin. I’m not complaining at all but each 4K video does hurt the storage available. When I constructed the server in 2015 I assumed it wouldn’t fill past 80% in the first 4-5 years.
Fast forward 2018 I’m running close to 75% and I’m aware of not crossing 80% of the storage and don't intend to.

The last scrub took 95 hours and at times with lesser data stored when the scrub repaired disks I've seen that figure cross 120 hours.

Code:

zpool status
  pool: bhoot
 state: ONLINE
  scan: scrub repaired 0 in 94h55m with 0 errors on Mon Feb  5 02:55:31 2018
config:

		NAME											STATE	 READ WRITE CKSUM
		bhoot										   ONLINE	   0	 0	 0
		  raidz2-0									  ONLINE	   0	 0	 0
			gptid/5663b940-bdde-11e5-9e00-f07959376c84  ONLINE	   0	 0	 0
			gptid/cd427285-e4d8-11e4-b39d-f07959376c84  ONLINE	   0	 0	 0
			gptid/ec0f7827-2d2c-11e6-b1de-f07959376c84  ONLINE	   0	 0	 0
			gptid/ce06b19f-e4d8-11e4-b39d-f07959376c84  ONLINE	   0	 0	 0
			gptid/ce69a75d-e4d8-11e4-b39d-f07959376c84  ONLINE	   0	 0	 0
			gptid/b1f3389f-5382-11e6-885d-f07959376c84  ONLINE	   0	 0	 0
			gptid/f3b91656-f1d6-11e7-be68-f07959376c84  ONLINE	   0	 0	 0
			gptid/cf91d6e8-e4d8-11e4-b39d-f07959376c84  ONLINE	   0	 0	 0

errors: No known data errors

  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0h1m with 0 errors on Tue Jan 23 03:46:16 2018
config:

		NAME											STATE	 READ WRITE CKSUM
		freenas-boot									ONLINE	   0	 0	 0
		  mirror-0									  ONLINE	   0	 0	 0
			gptid/40460acb-cf27-11e5-b12b-f07959376c84  ONLINE	   0	 0	 0
			da1p2									   ONLINE	   0	 0	 0

errors: No known data errors

Code:

zpool list
NAME		   SIZE  ALLOC   FREE  EXPANDSZ   FRAG	CAP  DEDUP  HEALTH  ALTROOT
bhoot		   29T  22.0T  6.98T		 -	39%	75%  1.00x  ONLINE  /mnt
freenas-boot  14.2G  1.05G  13.2G		 -	  -	 7%  1.00x  ONLINE  -

Now that I've given most the information everyone would need I am stuck at a fork and need help in making a decision.
I live in India in a Coastal Tropical city. My winter time temperatures are a funny 15C (60F) and summers scorch to about 50C (120F) and then there is humidity. Being on a hot coast I've never seen it drop lower than 50%. I live in a condo and the room my server is put in is exposed to the elements. Current (random) stats at (near) idle at 7 am on Feb 07th.

Code:

CPU 0: 49 C
CPU 1: 50 C
CPU 2: 49 C
CPU 3: 50 C
CPU 4: 54 C
CPU 5: 50 C

ada0 WD-WCC4E3CDR5SC: 34 C
ada1 WD-WCC4E4JL9NDZ: 37 C
ada2 WD-WCC7K2VDVPKT: 33 C
ada3 WD-WCC4E4KC79HK: 34 C
ada4 WD-WCC4E0ESU744: 34 C
ada5 WD-WCC4E2VSE6NP: 35 C
ada6 WD-WCC4E1FSUL4N: 36 C
ada7 WD-WCC4E4JL90L8: 35 C

CHAPTER
Over the period of 3 years of warranty my hard disks have toasted while scrubbing/general use and replacement has not been much of a pain. In short I have a firm belief in WD and FreeNAS. I won't lie but I have considered switching to other options provided by enclosure case manufacturers to switch to their pre-made hardware and 'enjoy' the ease of storing data, but the lovely system of FreeNAS along with the most active community pulls me back. I've also thought about server racks but honestly I realized they will just run hotter and noisier, and being in a condo I really don't have a place to put it. Points to be noted

My system is going to run hot.
I can't afford 24x7 aircon.
I don't mind RMA to and fro from WD.
I do have a cold spare all the time (unless a disk is in RMA).
It's a raidz2 system.
Can't think of expanding to a 4u (24 bay case)

Sorry for getting a bit carried away but I wanted everyone to know what I have to consider.Now coming to the problem at hand, planned replacement of the hard drives is obviously is the name of the game (resilver each of the 8 drives weekly/fortnightly/monthly).
I like WD red and I think the community also approves of it as being one of the best at business (except for a batch with 8 sec Head unloading timer).
So my current choices:
WD red/pro 8tb and 10tb
Don't really hate it but not a fan of Seagate (but)
Ironwolf /pro 8tb and 10tb
Not sure about HGST (prices/services in India) but I'm sure they will be giving the WD level of support.
He8/He10/He12.

All the above mentioned HDD except for WD Red run at 7200 rpm. The pro's and He carry 2 years additional warranty for a minor price bump.
I don't want to be running around for RMA drops every month. In the last 34 months of WD reds I've had 6 hard disks failing on me(mostly SMART or with Unreadable (pending) sector), out of which 2 were RMA'ed drives and I'm willing to accept these numbers, but anything more might be frustrating.

Code:

+------+---------------+----+-----+-----+-----+-------+-------+--------+------+------+------+-------+----+
|Device|Serial		 |Temp|Power|Start|Spin |ReAlloc|Current|Offline |UDMA  |Seek  |High  |Command|Last|
|	  |			   |	|On   |Stop |Retry|Sectors|Pending|Uncorrec|CRC   |Errors|Fly   |Timeout|Test|
|	  |			   |	|Hours|Count|Count|	   |Sectors|Sectors |Errors|	  |Writes|Count  |Age |
+------+---------------+----+-----+-----+-----+-------+-------+--------+------+------+------+-------+----+
|ada0  |WD-WCC4E3CDR5SC| 34 |13109|   50|	0|	  0|	  0|	   0|	 0|   N/A|   N/A|	N/A|   0|
|ada1  |WD-WCC4E4JL9NDZ| 36 |22939|  217|	0|	  0|	  0|	   0|	 0|   N/A|   N/A|	N/A|   0|
|ada2  |WD-WCC7K2VDVPKT| 33 |  744|	2|	0|	  0|	  0|	   0|	 0|   N/A|   N/A|	N/A|   0|
|ada3  |WD-WCC4E4KC79HK| 34 |22939|  218|	0|	  0|	  0|	   0|	 0|   N/A|   N/A|	N/A|   0|
|ada4  |WD-WCC4E0ESU744| 34 |16788|   74|	0|	  0|	  0|	   0|	 0|   N/A|   N/A|	N/A|   0|
|ada5  |WD-WCC4E2VSE6NP| 35 |22939|  211|	0|	  0|	  0|	   0|	 0|   N/A|   N/A|	N/A|   0|
|ada6  |WD-WCC4E1FSUL4N| 35 |14277|   56|	0|	  0|	  0|	   0|	 0|   N/A|   N/A|	N/A|   0|
|ada7  |WD-WCC4E4JL90L8| 35 |22759|  212|	0|	  0|	  0|	   0|	 0|   N/A|   N/A|	N/A|   0|
+------+---------------+----+-----+-----+-----+-------+-------+--------+------+------+------+-------+----+

I just wanted to know if sticking to the slower spinning WD red will be a good idea to reduce the hard disk heating or not care a lot about heating woes, and go with the thought if the drives fail just replace them. How much heat (temperature) increase can I expect from the 7200rpm disks vs WD reds? Will the scrub times reduce with faster drives?

Personal experience/reviews/articles about each of the product as a whole?

Sorry for the long post but I wanted to give all my thoughts and facts at hand to give every reader everything he/she might require.

Thanks a lot in advance.

 

[Chris Moore]

In your situation, moving to the WD Red 8TB (not the Red Pro) is probably the best choice. The Red Pro gives a 5 year warranty, but before then you will probably be looking to move to a larger drive. The Red Pro also generates significantly more heat.
The Helium (He) filled drives are supposed to run cooler, but I have not had any of them and can't say how they compare.

Can't you setup a small AC just for a small room where the server is? I feel that the heat is causing most of these drive failures.

 

[Bhoot]

-snip-

Thanks for a quick reply. I do hope (praying to God really) within the next few years a few of my family members will get their own cloud devices and take the load off the server.
Also another thing I would like to point out about WD Red (and pro) above 8tb have helium filling. The technical reason I remember for that is to increase the density of platters.

 

[Chris Moore]

The technical reason I remember for that is to increase the density of platters.

Also true, but there have been posts that indicated they ran cooler. I have not used them myself.

At work, I have servers that run the Red Pro drives and other servers running the regular WD Red. The Red Pro is rated to handle a higher running temp, so it might be worth the extra cost to get the potential durability. It is a choice you will have to make.

Here is a resource on drive price -

Disk Price/Performance Analysis Buying Information 27-Jan-2018:
https://forums.freenas.org/index.ph...e-performance-analysis-buying-information.62/

Unfortunately, those are US prices.

 

[tvsjr]

It wouldn't surprise me if, at some point, WD flags you for RMAing too many drives. The harsh reality is that, if you can't get the temperature under control, you're going to kill drives. If the area reaches 50C, even if you have massive airflow, the drives will always be 50C or over. Which will kill them quickly. Doesn't matter if you run FreeNAS on your own server or any other box out there... if the incoming air temp is 50C, the drives will remain at 50C. The only way around this would be SSD, which isn't cheap.

If you can't provide the right cooling, perhaps it's time to consider some sort of cloud storage (assuming you have connectivity to make that possible).

 

[Chris Moore]

and summers scorch to about 50C (120F)

You really must get that heat under control with some Air Conditioning in the summer. If you could keep the inlet air on the server down to 35C, it would make a huge difference.

 

[Bhoot]

Also true, -snip-

Thanks. The link is really what I kind of needed to get started on my requirements vs price I could expect.

-snip-

and

second comment -snip-

Thank you for the feedback and the fair warning. Yes the temperatures do touch 50C outside but for those times I do have air conditioners. The thing is that my disks have never gone beyond 45C during scrubs (with/without AC) and that does mean waking up in the middle of the night to switch on the aircon for 15 mins. During other periods I do follow Cyberjock who believes 39C is the time you need to cool or shut down the system and TBH, I am pretty strict on that.. I have windows treated with reflective films and installed thick black out curtains (My home theater shares the same room as the FreeNAS). Most of my reporting is therefore done between 1200-1500 to catch the drives at the hottest temperatures of the day to maintain a trend analysis and apply corrective actions (AC or Shutdown). I quite frankly am thinking about arduino based solution for the aircon :D (temperature sensor in the hard disk rack) but well that's for another forum. It's just that the electricity in the metro I live in is quite expensive and the 24x7 will never be possible. I am thinking of asking (bribing :p) my local ISP to take my box and keep it in his server room for a certain rent, but again that looks more difficult.

EDIT
I just ran SmartCtl for all disks and the max lifetime temps (I hope that's what it is) are 43 on 1 disk and 42 for most of the others. 1 disk was even found with 39.
Also if you see my FreeNAS drives do have 200+ start/stop cycles and none of those are coz of power outage.

 

[Bhoot]

In other NEWS I did read a little something about WD Gold and I am really confused at what exactly they are.
1) From their home page (this) they state
"Enterprise-class storage to rely on"
which means they are not enterprise but are like enterprise
2) under the warranty section of WD they are listed under Desktop drives which make me think they are WD Blue Pro.
3) Apparently they are WD gold are meant for datacenters and Red are meant for NAS???!!! Again I don't understand the difference between the 2, Writes and Rewrites are better on Gold and they also have TLER and "Vibration Protection". A long conversation with WD Pre-Sales team was 30 minutes of me educating someone over the phone of how a NAS and Datacenter essentially mean the same thing or well maybe for FreeNAS users, we would fall more on the side of "DataCenter" than true "NAS".
4) The prices of the Red Pro and Gold are near identical and except for the "theoretical values" (ure and mtbf) it is essentially the same hard disk repacked and recolored.

This is from a Reddit post
The differences for 10TB drives are (Red Pro specs, Gold specs):

• Slight bump in performance. Gold can do 249MB/s, Red Pro can do 240MB/s.
• Gold has an MTBF of 2.5 million hours, Red Pro only 1 million.
• Gold has an annual failure rate spec, Red Pro does not.
• Red Pro has a workload rate of 300TB/yr, Gold's specs are defined for a workload of 219TB/yr but the drives are designed for 550TB/yr.
• Gold is more power hungry. 7.1/6.7W for sequential read/write vs. 5.7W on the Red Pro. 5W idle vs. 2.8W on the Red Pro.
• Gold has a higher operating shock rating of 70G vs. 30G read write, 65G read only on the Red Pro.
• Gold is slightly louder on seeks at 36dBA vs. 29dBA
• Gold weighs 10g more than a Red Pro.

TL;DR: Gold is specced to be more reliable but consumes more power.

 

[Chris Moore]

In servers at work that were delivered to us with Red or Red Pro drives, we replace failures with the Gold drives and they are very reliable. None of them have failed since we started the practice.

Sent from my SAMSUNG-SGH-I537 using Tapatalk