Why I build my own NAS even though I have a DS718+

[mltam]

Hi. I just ordered the ingredients to build my first NAS. I'll be using the Ryzen 5 pro 2400GE.

For me, the main reason to build my own NAS and not a ready made one is that I'm not ready to give the keys to my data to a company. I had a house fire a few years ago. My Synology NAS was destroyed, though the disks survived enough to be readable after cleaning. I had to buy another Synology to read my data. That is not acceptable! I also hate how I can't just stick these old disks on a USB drive to read them, and equivalently, I can't take out a disk from the NAS and put it in on USB to read it. That's my biggest gripe. I also don't like that Synology controls what runs on my NAS. I don't need indexing, but it is virtually impossible to disable. It is the main activity my disks do. And, there are a few packages that I can't uninstall. I also can't upgrade easily the number of drives in my NAS.

Though, I have to say that without that one big annoyance, and the few small ones, I'd be really happy with my Synology NAS. I have the DS718+, and like it.

But, finally I'll be able to compare it to my own build. (If I succeed). I do care about energy use, and I am worried about having another 50-100W on 24/7. I should check the energy use of my Synology, though.

BTW: in the winter, when I heat the house, there is almost no additional energy use, since the NAS just heats the house instead of my heater. I just pay an additional cost of heating with electricity instead of gas. If I had a heat pump, I'd loose a bit more. In the summer, though, I pay double, since I use the electricity, and then use additional energy to cool the house back down.

 

[Arwen]

A few cautions of TrueNAS.

1. While you can get access to the ZFS data disks on Linux or FreeBSD, you need all the disks from the "pool" to have any hope of reading data. (Their are slight exceptions, see below.)
2. Next, even though Linux & FreeBSD have ZFS, if your pool has all the latest features enabled, you might find an older Linux simply can't access the ZFS pool because it is using an earlier release of ZFS. Thus, lots of us here in the Forums run our data pools without all the latest features enabled.
3. TrueNAS, Core or SCALE, is not designed to be energy efficient. It can be if you are careful. Many of the forums users don't have the knowledge to make the most energy efficient NAS.
4. It is critical you don't use a hardware RAID controller in any RAID mode. Some require similar models in case of failure, which is still a type of vendor lockin. Plus, ZFS does not work well with HW RAID controllers. So simply HBAs are highly recommended.
5. Last, ZFS does not like SMR, (Shingled Magnetic Recording) disks. In some cases they fail to work properly, (some WD Red models). In most cases they are so slow at disk replacement, it puts the rest of your data at risk.

You can simplify #1 by using single vDev Mirrored pools. This allows you to, in worst case, put only 1 of the Mirrored disks on a Linux or FreeBSD computer and get access to the degraded ZFS pool.

I do understand your desired to "own" your data and not be tied up. Just understand TrueNAS, Core or SCALE, takes a learning curve.

 

[mltam]

Thank you!
1. I thought that if one of the disks fails, you can still get all the data if you have the rest of the disks. Oh, I think I understand. You mean that I can't just take one of the disks and plug it into another computer to get the data. Yes. I understand that. Though I have been thinking all day long about a "RAID" system that would distribute files between drives, not bits or bytes. Hmmm.. a vDev Mirrored pool. Maybe. I guess if each file is in 2 drives, then I always loose 50% of my capacity.
BTW: I never used raid on the sylogic, just single disks. But, even a single disk is put in a volume, which then can't be read from the usb drive!
2. That I understand. I hope it doesn't happen too often... (and I'd just need to install the right distribution, or boot from the right CD/DVD/usb, not buy another expensive computer.)
3. Yes. I remember years ago when I tried to track every little noise coming from the disk on my laptop to try to make it as efficient as possible...
4. Thanks, no, I don't have that.
5. Yesterday I looked :) luckily, my WD Red drives are so old that they are till CMR. Even though they are 4GB and 6GB!

 

[mltam]

Oh, I know! Divide each disk into 4 parts. On each disk, 3 parts are fully readable, and one is just a checksum for 3 parts on 3 other disks. This way, from each single drive you can read 3/4 of the files with no problem. The last 1/4 which is a checksum can help recover all files even if you only have 3 disks of 4. Probably too complicated to implement. So, 4 disks lose 25% of space, and each single disk also loses 25%.

 

[mltam]

I think I'm reinventing the wheel here. This is exactly how raid5 works
See this quora answer with a nice image.
It doesn't easily preserve files, though. I wonder why. Maybe it has to do with separation of layers? The disk layer doesn't know what a file is?
It would have been nice. You could also make a system where as long as the disks aren't totally full you can actually get more than 3/4 of the files on each disk. Though that would involve much more disk access.

 

[Whattteva]

For me, the main reason to build my own NAS and not a ready made one is that I'm not ready to give the keys to my data to a company. I had a house fire a few years ago. My Synology NAS was destroyed, though the disks survived enough to be readable after cleaning. I had to buy another Synology to read my data. That is not acceptable!

I mean, is that really so bad? The situation is kinda' similar in TrueNAS. It runs ZFS. Not many things will read ZFS. Linux distros only very recently even decide to include support for it in the default installation. Also, even among OS's with ZFS support, the version matters because older versions cannot read newer pool versions. So.... it isn't really as easy or straight-forward as you may think.

I also hate how I can't just stick these old disks on a USB drive to read them, and equivalently, I can't take out a disk from the NAS and put it in on USB to read it. That's my biggest gripe.

I'm going to again, burst your bubble here. ZFS does not really play nice with USB. It takes a certain special kind of sauce to run it successfully and reliably or else, you may end up like this guy or this guy. And again, ZFS isn't exactly an "accessible" filesystem that you can just plug it into another PC and expect it to just seamlessly mount and read.

But, finally I'll be able to compare it to my own build. (If I succeed). I do care about energy use, and I am worried about having another 50-100W on 24/7. I should check the energy use of my Synology, though.

A "proper" TrueNAS build will likely not be as energy efficient because it's designed for enterprise use. The RAM requirements alone mean you likely will be running more RAM sticks than other solutions since ZFS likes to suck RAM like nobody's business.

Definitely not saying it's impossible, but you need to know that TrueNAS is no magic bullet and if you're expecting it to be smooth-sailing plug-n-play Synology style, you may be in for a rude awakening.

 

[mltam]

I mean, is that really so bad? The situation is kinda' similar in TrueNAS. It runs ZFS. Not many things will read ZFS. Linux distros only very recently even decide to include support for it in the default installation. Also, even among OS's with ZFS support, the version matters because older versions cannot read newer pool versions. So.... it isn't really as easy or straight-forward as you may think.

Actually, I realized I'm totally wrong. I thought synology had some secret part of the file system that couldn't be accessed from other OSes. But that, I think is wrong. Here the procedure is described. I remember looking for this once, because I did want to get my data back. Maybe I didn't look deep enough.

I'm going to again, burst your bubble here. ZFS does not really play nice with USB. It takes a certain special kind of sauce to run it successfully and reliably or else, you may end up like this guy or this guy. And again, ZFS isn't exactly an "accessible" filesystem that you can just plug it into another PC and expect it to just seamlessly mount and read.

Hmmm. If I have a pool with a single vDev and the vDev has a single disk. Now I connect that disk via USB. It won't automount? Or at least mount easily? That's sad. Should be solvable, though...

A "proper" TrueNAS build will likely not be as energy efficient because it's designed for enterprise use. The RAM requirements alone mean you likely will be running more RAM sticks than other solutions since ZFS likes to suck RAM like nobody's business.

Definitely not saying it's impossible, but you need to know that TrueNAS is no magic bullet and if you're expecting it to be smooth-sailing plug-n-play Synology style, you may be in for a rude awakening.

I wasn't clear. I don't build a NAS in order to beat the power requirements of the DS718, or even a bigger Synology system. I'm pretty sure I'll never manage that. I was just saying that I'm worried that the power requirement of my system will be a lot. Or maybe even unacceptable...

 

[Arwen]

...
Hmmm. If I have a pool with a single vDev and the vDev has a single disk. Now I connect that disk via USB. It won't automount? Or at least mount easily? That's sad. Should be solvable, though...
...

Yes, you can access a single vDev pool, which has a single disk from an USB enclosure.

Our comment is that it won't be reliable in the long run. Some people think it should be reliable. Their is what people want and think should be, and then their is reality. Based on TrueNAS forum community experience, permanently attached USB storage with ZFS is not reliable, period.

Their are various reasons why USB is less reliable than SATA / SAS ports:

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often in the last year or 2. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data...

www.truenas.com

All that said, if you GOAL is to extract the data off of a disk, (or several disks), in an USB enclosure, that CAN work. It can go off the rails when things like a scrub or re-silver need to run.

 

[Whattteva]

Actually, I realized I'm totally wrong. I thought synology had some secret part of the file system that couldn't be accessed from other OSes. But that, I think is wrong. Here the procedure is described. I remember looking for this once, because I did want to get my data back. Maybe I didn't look deep enough.

As far as I'm aware (I don't own a Synology), Synology uses either BTRFS or EXT4 file system. Both of which are widely available in Linux, and as a result, far more accessible than ZFS is. EXT4, in particular, is mountable by practically every Linux Distro out there. Hence, if you want portability, Synology is, by far, more portable.

Hmmm. If I have a pool with a single vDev and the vDev has a single disk. Now I connect that disk via USB. It won't automount? Or at least mount easily? That's sad. Should be solvable, though...

That's not what I mean. Yes it will mount with USB, but it's not as simple as say, an EXT4 or NTFS or FAT32 file system. It requires another OS that supports ZFS that also happens to support that particular version of yours. Moreover, ZFS is often built with multiple disks in a vdev for redundance, this means that it will refuse to mount if you only have 1 disk if your layout is anything other than a simple 1-vdev mirror.

 

[WI_Hedgehog]

If you use TrueNAS and ZFS properly, it does what you want, namely run on any reasonable replacement hardware and your data is again accessible within an hour or so. Follow the standard install suggestions and keep an off-site copy of your TrueNAS bootable USB install stick with your 3-2-1 data backup and you're golden.

Simple 3-2-1 backup strategy

Hi everyone! I'm new to this community but I spent 2 months reading the forum before today's registration. I experienced bit rot recently on my photos and I realized that my backup strategy was not enough to preserve my data integrity, this is why I switched to zfs. My freenas setup (used only...

www.truenas.com

How I Learned to Stop Worrying and Love RAIDZ

The popularity of OpenZFS has spawned a great community of users, sysadmins, architects and developers, contributing a wealth of advice, tips…

www.delphix.com

 

[loriot]

I started with TrueNAS (Scale) just a few weeks back and actually have a very similar motivation like @mltam :

• sold my DS 918+ on Ebay and bought a Terramaster 2 Bay NAS. I replaced the TerraMaster OS with TrueNAS .
• Getting hands on my data on a Synology via USB was one of my major concerns after 2 disk failures and for me fiddeling with the volume groups was the painful part, not the filesystem ...
• I also realized that TrueNAS (Scale) is not so straight forward either, especially the need to handle kubernetes & HELM instead of docker scares me a lot. And managing the pools and datasets on another Debian based Linux via USB, will for sure be no fun .
  However i am sure, Arwen is right. I'll need to be patient and there will be a learning curve.
• btw. energy consumption was also a concern for me and here the TM does a great job and consumes only ~20W, while Syno takes ~35W

As my Terramaster has just two 3,5 bays and 2 slots for NVMe, i used NVMe for system stuff and opted for a Sata SSD to hold my Nextcloud data and have my "old" WD RED 6TB holding media = music, photos and videos. Not sure if it is still CMR or SMR. I will check that.
I have to admit i also do not understand @Arwen point on single vDev Mirrored pools. I am not mirroring anything on TrueNAS just right now, because i also have a 10 years old Synology. This old Syno is powered off most of the time and i start it only to rsync my data from the TrueNAS to the old Syno. That works quite well and i plan to automate that to have a reliable backup.

However, i have a very similar issue / problem and search in the forum didn't help me so far. So i hope someone from this thread is still reading and can give me a hint
In case i want to swap some disk or in case of some need to recover data, what would be the procedure to take my disk out of the TrueNAS and mount it on my Debian based Laptop via USB, as an example?
Is there any How-To, where i can get such an info?

 

[Whattteva]

However, i have a very similar issue / problem and search in the forum didn't help me so far. So i hope someone from this thread is still reading and can give me a hint
In case i want to swap some disk or in case of some need to recover data, what would be the procedure to take my disk out of the TrueNAS and mount it on my Debian based Laptop via USB, as an example?
Is there any How-To, where i can get such an info?

I've never setup ZFS with Debian, but should be supported under Bullseye, but NOT enabled by default. This makes it tricky to setup on your own and since you're asking this question I'd suggest you to use a distro that's easier to setup or comes with ZFS support by default. I think the Ubuntu-based ones do.

Keep in mind, this will also only work if your pool runs the latest version of ZFS. ZFS pools created by FreeBSD 12 and earlier (earlier versions of TrueNAS CORE) are most likely not supported due to lack of support for newer feature flags.

 

[Etorix]

II also realized that TrueNAS (Scale) is not so straight forward either, especially the need to handle kubernetes & HELM instead of docker scares me a lot. And managing the pools and datasets on another Debian based Linux via USB, will for sure be no fun .

OpenMediaVault is Linux-based, if that's your thing (de gustatis coloribusque non disputandum) and comes with options for ZFS and for installing Docker. No idea how good it is at managing either, much less the combination.

Alternatively, one can set up a Linux VM on TrueNAS (including CORE) to run regular Docker in there.

Different learning curves, and different kinds of fun.

In case i want to swap some disk or in case of some need to recover data, what would be the procedure to take my disk out of the TrueNAS and mount it on my Debian based Laptop via USB, as an example?

Just that: Plug the disk(s) into any ZFS-enabled system. At least for read access; write access will require support for all the feature flags from TrueNAS—which likely means TrueNAS itself.
For recovery, you'd go to Klennet ZFS Recovery… on Windows (yuck). $399 license, but the free evaluation version lets you know beforehand what it can recover. The ZFS paradigm is that ZFS tries hard not to get corrupt, hence the strict hardware requirements. If corruption occurs anyway, there's no "zfsck". The motto is: "Destroy the pool and get that tape backup…"

 

[loriot]

Thanks for the quick replies.
I have to admit, i misunderstood what ZFS is about. I did believe it is a file system just like ext4, btrfs, ntfs and so on. However it seems to be much more. So i started reading the FreeBSD manual for ZFS just right now.

About OpenMediaVault. I gave it a try as well but was so much under-impressed, that i immediately wiped the disk and started the TrueNAS Scale journey.
Not sure, about the referral to Windows and the 399$ license. I guess i will try OpenZFS, it seems to be open source and hopefully can be used for free.

 

[kiriak]

Home user here, previous Synology user

Now I have
- a small TrueNAS Core as the ultimate (for me at least ) data keeper that runs for a couple of days every week.
- an old mini PC based OpenMediaVault for a few services I use or plan to use (like Syncthing, Piwigo, Nextcloud maybe). This was bought used and with an SSD it has a consumption about 6 W and runs 24/7.

 

[Whattteva]

For recovery, you'd go to Klennet ZFS Recovery… on Windows (yuck). $399 license, but the free evaluation version lets you know beforehand what it can recover.

For me, the yuck part isn't Windows. It's that big $399 licensing cost... now if someone else would cover it, that'd be a different story, haha.

 

[Whattteva]

Home user here, previous Synology user

Now I have
- a small TrueNAS Core as the ultimate (for me at least ) data keeper that runs for a couple of days every week.
- an old mini PC based OpenMediaVault for a few services I use or plan to use (like Syncthing, Piwigo, Nextcloud maybe). This was bought used and with an SSD it has a consumption about 6 W and runs 24/7.

Huh.... that's actually a good strategy. Just run a plain ol' non-redundant simple file server for daily use on a potato and use TrueNAS just for offline backups and probably save a lot in power bill (and noise for some people).

 

[Etorix]

I have to admit, i misunderstood what ZFS is about. I did believe it is a file system just like ext4, btrfs, ntfs and so on. However it seems to be much more. So i started reading the FreeBSD manual for ZFS just right now.

Basically, ZFS is a file system and volume manager merged in a single package. Think btrfs+mdadm.
Reading the manual is always a good idea…
To make it easier, what you need to have a good grasp is:

• physical (disk -> vdev -> pool) vs. logical (datasets, snapshots, clones);
• mirror vs. raidz#, benefits and drawbacks of both—since pool design has to be right from the start.

All further, and advanced, features (L2ARC, SLOG, special vdev, dedup, dRAID, etc.) are best skipped as first intention, and left to be studied later, if and when you think you might need them. In many case, the answer will be that you do NOT need them!

Not sure, about the referral to Windows and the 399$ license. I guess i will try OpenZFS, it seems to be open source and hopefully can be used for free.

You mentioned "recovery", so I pointed to the one and only recovery option I know of.
There's no recovery in ZFS itself: You get it to work right, and always maintain proper backups. If you fail to do so and end up in trouble, you either lose your data (ZFS can make this easy if mismanaged…), learn to read ZFS binary structures to rescue your data with a hex editor and raw read commands (good luck with that!) or part with a significant amount of money for a highly skilled consultant to do it and rescue your data.
I have never used Klennet, and hope I'll never will; but if I'd put myself in the situation of having trusted the one and only copy of my family archives to a borked ZFS pool, the $400 license would likely be a bargain compared to finding and hiring a ZFS consultant.

 

[loriot]

Huh.... that's actually a good strategy. Just run a plain ol' non-redundant simple file server for daily use on a potato and use TrueNAS just for offline backups and probably save a lot in power bill (and noise for some people).

Actually my strategy is a bit the opposite: I am using my old / outdated Synology rarely and only for offline backup purposes.
I did also play a lot with Raspberry Pi based NAS, Nextcloud, a new/modern Synology and now with TrueNAS as the daily driver. For me personal data is very valuable, so i am not happy using too low power devices as a primary NAS.
So I have switched to TrueNAS as my daily driver and hope i can stick to it. With the Terramaster 2 bay hardware, i am using, the energy consumption was very moderate till now, so around 20 Watt and noise level is also very low.
My main concern is the customizations of the Debian and the disabled docker usage, e.g. no docker-compose usage. For me fiddeling with k3s and Helm is an overkill and i would prefer sticking to docker. The other 2 issues are ZFS and missing Debian's apt.
What is the experience here: In case of security fixes, how quickly do they come from iX? The worst scenario for any NAS user is in my opinion to loose all data because of malware...

@Etorix
You mentioned "recovery", so I pointed to the one and only recovery option I know of.
There's no recovery in ZFS itself: You get it to work right, and always maintain proper backups.

when i refer'd to recovery i had more the case of hardware or operating system broken in mind, means:
take the harddisks out of the NAS and copy the data from the disks to some other server / NAS. So OpenZFS should be hopefully fine for that.

 

[Etorix]

when i refer'd to recovery i had more the case of hardware or operating system broken in mind, means:
take the harddisks out of the NAS and copy the data from the disks to some other server / NAS. So OpenZFS should be hopefully fine for that.

In that case you just plug in the drives and hit "Import".
If rebuilding the server, or just replacing a failed boot drive, you install anew and load the configuration file you've hopefully saved. Done.