Hardware setup and raid solution

[fudi87]

Hey
Recently got myself a new setup, new mainboard with used xeon cpu's, and now i want to move over to trunenas.
And btw Truenas is completely new for me.


My Setup:
Mainboard new: Supermicro X10DRL-i
Cpu's used: two Intel Xeon 2680 V3 with Noctua NH-U9s coolers.
Memory 8x16 DDR4 ECC
LSI 9361-8i running in Jbod ( Can't find any IT mode on this cards.. )
PSU: Seasonic Prime GX 750
Drives 5x3tb Seageate's desktop drives.. , and 3 HGST NAS 3 tb drives
Boot drive: Intel SSDSC2KW256G8 Sata
Rack chassies 4U with 10 slots for harddrives.


I Installed truenas and builded a ZFS 1 pool with 8x3tb drives on the LSI controller, for now to test it out.
Running like a charm,copied 15 TB over, and have been running for couple weeks now.
The Setup doesn't need 24/7 uptime, i got 2 other offline backups of the data, but 1 of the old systems i don't trust anymore
and the plan is to get rid of it.

My plan is to buy new 18TB WD Ultrastar DC HC550 drives, and here is the mainly question for the setup:
The plan is to set up5x 18tb drives on the LSI contoller in a ZFS 1 setup then after a while when the wallet accepts it buy 5 more 18 TB DC HC550 drives and make another pool but now running on the internal sata slots on the mainboard, and run that on ZFS 1 too. then sync data between the pools.
have any other more clever suggestions?

​

 

[Arwen]

Your first task is to learn TrueNAS and ZFS, both usage and terminology. The fact that you used "ZFS 1" pool thrice indicates your knowledge of ZFS is, shall we say?, beginner.

However, other than the LSI card which I can't comment on without researching it, the hardware looks quite decent. You can mix and match SATA and SAS ports just fine, (assuming none are from hardware RAID controllers).

We have lots of forum sticky posts and Resource articles on many subjects. Resources are available on the top of this web page. Here are some suggested reading materials:

Introduction to ZFS

This is a short introduction to ZFS. It is really only intended to convey the bare minimum knowledge needed to start diving into ZFS and is in no way meant to cut Michael W. Lucas' and Allan Jude's book income. It is a bit of a spiritual...

www.truenas.com

Terminology and Abbreviations Primer

We realize that new users have a lot to learn when they come to FreeNAS. There's a certain amount of confusion added to discussions when users pick random/approximate terms to describe things. I've spent a lot of time quietly trying to...

www.truenas.com

Hard Drive Burn-in Testing

@jgreco did a nice system build/test/burn-in guide here, but I (and many others) found the details a bit lacking in the hard drive section. He mentions S.M.A.R.T. tests, but doesn't go over how to run them, or how to view the results, etc. and...

www.truenas.com

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

Many root cause analysis for TrueNAS failures with data loss, is lack of understanding either TrueNAS or ZFS. For example;

• Not setting up scrubs
• Not setting up regular SMART tests, (which don't work on RAID controllers, but do work on HBAs)
• Not setting up E-Mail alerts
• Using SMR disks
• Using pretty old non-server class hardware

The last is something not as clear as some people think. Server motherboards tend to be made to last 10 years or more. But desktop or gaming boards, not so much. Yet, people want to recycle a 5 year old desktop or gaming board and expect it to work great as a server. (It might, but then again, it might not with spectacular failures.)

 

[fudi87]

Hi!

Thanks for the reply!
Yes indeed in Truenas/ZFS world i am a noob, but better start the long journey late than never?

RAIDZ1 i assume is the correct formulation?

I was Thinking about running two pools in RAIDZ1 and sync the data between them.
But i understand that RAIDZ2 is recommended and there is greatly higher risk with just one parity drive.
But still a bit unsure, as long i have the data in two pools, and have cold storage on the side.

As the LSI card appears to work, i am not 100% sure its in the right mode, i can see information about the drives ( serial number,model etc)and run smart tests, but is there a other way to control this? anyway i will go deeper in the documentation.

At the last of your post i agree 100% that servergrade hardware is the way, and for me it was important that motherboard and a quality PSU had to be completely new as they are mostly the first to "go home". ( Drives dies all the time)


But anyway thank you for the links, i will take a peak on all you provided, thanks!
Some time i will pass?

Knut

 

[Arwen]

RAID-Z1 is a bit of a judgement call. For a relatively new user, it might be better not to use RAID-Z1. But, if you understand the risks, and mitigate them, (good backups, ability to re-create the data, understand a full restore can take weeks...), then RAID-Z1 can be used.

ZFS does 2 things different, which help mitigate RAID-Z1 verses RAID-5, for disks that don't totally fail:
- Replace a disk in place, meaning the failing disk is left in and the replacement disk is added to the server, then replaced
- If a 2nd disk has some bad blocks, only the affected files are impacted, and thus only those need to be restores.

Last, ZFS ALWAYS duplicates metadata by default, (like directory data), in addition to any pool level redundancy. So a complete failure of one disk in a RAID-Z1, and partial loss of another, is unlikely to cause an entire directory path's loss. Critical metadata has even more copies.