Thank you so much for taking that much time, I will repay this in answering your comments as good as possible. Just one little thing to myself: I work in IT now for over 15 years, run my own homelab since the beginning and am quite qware about some querks... But I made my journey from selfbuild Windows Servers for years and jumped the QNAP Train some time ago. But I'm annoyed beyond a lot of things about how QNAP handles alot of stuff and am looking for a replacement for my whole setup. After a long journey in reading about different new ways of managing my storage AND now finaly the possibility to move all my stuff to a rack where noise and space is nearly no concern I'm ready to beginn a new chapter and give something new a try. And after fiddling around with OMV, Synology DSM (on custom hw, cause the synologys have sooo weak hardware, windows Server (again) I finally fell in love with how truenas handles everything and am quite sure that I really want to invest time and money in revamping my whole setup. And for that I want to beginn with the least critical bit of hardware and thats the offline backup System.
Also to understand my situation more, here is an overview of everything I run atm and what is already planned:
Main NAS: Qnap 873 with 8x8TB Ironwolf RAID 6 - Acutally loaded with ~19TB of stuff, where I have ~12TB of really important data (such as photos, documents etc. I'm a part time photographer and drone pilot and really need those backed up properbly)
Online Offsite Backup: Qnap 431 with 4x4TB WD Green RAID0 - It just runs... I had to switch from resilio to RTRR due to lack of memory, which is kind of a pain in the ass, as the buddy of mine, where it is located sometimes messes around with his router, so I plan to replace that real soon, also cause it gets a little full. Just replicates the "Important stuff"
Offline Onsite Backup: Qnap 431 with 4x4TB Seagate Desktop RAID0 - That is the one that I want to replace first and what is the thread all about. It boots up every day at 2 PM, pulls the important stuff and shuts down afterwards. That is just to safe some power and protect me against any crypto virus and stuff.
Cloud Backup: I have all my photos on amazon and everything else, that is important on onedrive for business, except videos which get more and more and I'm not sure atm. how to handle those...
So basically I'm fine, running even RAID0 on my backups, as I only had one drive dying over the last years and it was easy to replace.... I had to resync the whole machine, but that was the price I was happily willing to pay for saving some money on drives.
What I have generally planned in 2023:
Build the new offline Backup Server (thats what the post is about) and backup the whole NAS instead of just the important bits. Its supposed to handle the whole data thats on it right now and also have enought free space to load some crap off if I need, lets say a second space for something temporary, like redoing a friends PC etc. I also try to figure out, how I want to sync everything and if I can shut it down when not needed and how truenas handles being only runnign a couple of hours a day. And yes, I know, that a lot of NAS drives are meant to run rather than start, but I dont want to have 12 drives spinning 24/7 when I only need them 30 minutes a day. And if i let them spindown, there is really now reason in my opinion to nut just shutdown the entire system.... I think I will stick with running it once a day and then lets say on sunday keep it up for the whole day for scrubs etc.
Replace the Online Backup: As soon as the offline Server is done, I have a Microserver Gen8 sitting next to me with 12 gb ECC RAM and a G1610 CPU. I wait for 4x6TB SAS drives in and a HBA I have ordered for it to replace the old junk... I will put it in RAIDZ1 at first, but am also pretty confident in running it as a stripe, if it gets too full. I don't have any performance concerns for that one, cause it will only get its data via my 40mbit upload, so that should be fine...
PC Backup on demand: I currently backup all my OS disks from my Desktop and all our notebooks with Veeam and it just works. But: I have together 8TB of SSD storage in my desktop and, as there is nothing really important on it, as everything important is also on the QNAP or stuff like steam games etc. I want to be able to backup everything, so that I don't have to redownload everything and so on. For that I have a Bluechip Server with a 1245v5 and 16GB ram sitting here, together with 5x3TB drives, that I still have laying around and will be just a stripe (or Z1, still deciding) and will be booted up with WoL once a week, get synced and then remote shutdown. I also want to use that one as a playing device for trueNAS as I don't want to touch my productive machines with new releases etc.
And when I'm done with all that, I plan on getting a new 12 (or16) bay server end of next year to replace the QNAP. I plan on reusing the 8TB Ironwolfs for backups and will then see, If I really get going with truenas for my production too or go a different path.... But thats something for next summer...
So maybe some of my questions and answers below will be in better context now, as I (at least) think I know pretty much, what I'm getting into with but am always open to suggestions
Welcome!
Read the following resource to gain a basic understanding of how ZFS works.
I did that before the Post, but thanks again for the links, its handy to have them all in one thread for further reference :-D
Oh the pain. Make sure they are all CMR. Also do note if there are differences in RPM. Make sure to properly test your drives, especially the used ones. Read the following resources.
I read about it and will check soon if there are any SMR drives in my collection and will see.... I really don't think that SMR will be that much of a hassle in my particular case but want to avoid it if possible. Maybe, if I have more than two SMRs I will grab another pair of used 4TB SAS drives from ebay, I only paid 50€ for the last two I ordered, so that is still to be decided.
With such a number of drives you want a proper PSU, please read the following resource.
I've seen about 1,000 threads like this one where people decide that they can power a dozen hard drives off a 360 watt supply. DO NOT DO THIS. I've seen another 1,000 threads where people decide to buy the cheapest power supply that they can...
www.truenas.com
Yeah I know... My chassis has a 560watts PSU that came with the case... I will use it for now and monitor the power draw with an external monitor and then maybe replace it. Also all my stuff is backed with an online UPS, so at least I know that the power it gets ist "good" :-D
Since you plan on using it for backup (with very few simultaneous users, I think that a single vdev with all 12 disks in RAIDZ3 is your best option (altough you would be at the limits of a vdev max suggested width).
Please read the following resource in order to predict your pool performance.
Thats what I figured. I mean, atm I only want to sync it with the main NAS via 1gb LAN, so that should not be a problem at all. I mainly thought about doing two RAIDZ1 vdevs for other stuff. If I could grow the vdevs (like in QNAP) i would not even think about not using RAIDZ2 and just start with less disks and expand later. Or even go with RAIDZ3 (having in mind how old some of the drives already are and be done with it...
For LACP you can read this other resource.
Yeah I know... I mean my QNAP runs on LACP, so that if my backup runs the same time as someone wants to use plex there is enough bandwith but I don't know if I don't just stick on not using it for the moment... It was just a thought that maybe later, its worth enabling it. I also think about upgrading to 10g but that would cost quite a lot as I would need a switch and a couple of cards... So I will see in the future.
What do you consider reasonable?
If you have read the resource about pool layout I linked you should already have the answer: main performance benefit is the increased IOPS (that your use case doesn't need) at the cost of significant risk. Consider RAIDZ1 dead alongside RAID5.
It doesn't matter, a proper HBA in IT mode will act like an extension of your motherboard, virtually increasing the SATA ports on it. ZFS will recognize each device.
Yeah, thats why I asked in the first place... I don't mind the risk (I run it with RAID0 atm), but was hoping to sicnificantly reduce srubbing and resilvering times with two smaller pools. I really don't want to use more than two disks for parity all together as I want to be able to use the thing quite a little time without it getting full...
Then ZFS (and as such TrueNAS) might not be the right solution for you: ZFS focuses on data integrity, nothing comes before it. You can see it as "all data or no data".
Yeah I know and thats what brought me here in the first place as I really want that for my main NAS, as soon as I am ready to replace the QNAP and am familiar with truenas enough to make the full switch. As stated above, thats also the reason I want to start with the "unimportant" machines, rather then fully switch right now
Regarding rebalancing I am not sure: iirc yes, the new data should be written in the emptier vdev, but there should also be a few community made scripts available.
Yes, you can switch the drives the way you wrote (offline, resilver, repeat). There are a few tricks that can be used (like replacing more than 1 disk simultaneously).
It is reccomended to have symmetrical vdevs.
Have fun reading and forgive me for any spelling issue. It was tough writing all this from a smartphone.
Does anyone else have more insights here? Do I get any benefits in having two smaller RAIDZ1 vdev rather than one big RAIDZ2 vdev other than IO performance?
