Will it TrueNAS (Scale)? Upgraded Huawei RH2288Hv.3 - pondering how to get things working

[sretalla]

I do not understand. Is this supposed to be so complicated? Why?

I don't think it's set up to be automatic, but it's as simple as I expect it to be...

I didn't have SMB running on my SCALE machines, so I took your challenge...

I used an existing dataset.

I went and created a new user (leaving the default for Samba Authentication as on).

Assigned that user permissions to the dataset as owner under the datasets section, Permissions | Edit

Then applied those permissions recursively.

I then went to shares and created a new Windows (SMB) share.

I selected the dataset location, picked the share purpose as Multi-protocol (SMB/NFS).

Enabled the SMB service (making sure my SCALE server had a unique name in the config there, then restarted the service.

Accessed the "network" from a Windows 11 system, found the SCALE box listed, then after double clicking and entering the user credentials I created before, was able to see the share, open it and edit a file and save it...

I even went to a Windows 10 system and attempted to access it as another (non-existent on the NAS) user and was correctly denied access, then when supplying the right user, was able to access the share and edit files.

I don't see how much simpler you wanted it to be.

 

[SecCon]

See that's the 1-2-3 steps I weren't able to find. I was linked to a vid and was able to sort it out via a bit of guessing since the vid was for a previous version of Core, not Scale.

Now I just have to do and document this a couple of times so I have the steps with screenshots to follow when I go prod.

Thanks @sretalla , I am not used to this kind of granular access management. While I worked many years with similar stuff, it was greatly scripted and customized so did not really learn the mechanics behind.

 

[SecCon]

Still in the test setup for this, running a virtual windows and it seems to be very stable. Had some heating issues with a passive cooler on that server, but added more chassis fans and changed their orientation and it helped a enough...

SMB is stable, can copy on and off as I like and the mapping in Windows seems rock solid. Still need to setup a relevant backup and in this I am not sure about a few basic things but the main being if I should configure rsync on one storage machine or on both?

I just don't get that and while I have been on and off on the rsync pages looking for answers it remains the proverbial greek to me. Even this do not answer that question, a pair or not? : https://everythinglinux.org/rsync/

You must set up one machine or another of a pair to be an "rsync server" by running rsync in a daemon mode ("rsync --daemon" at the commandline) and setting up a short, easy configuration file (/etc/rsyncd.conf).​

Is that a pair or not?

To be honest I am actually pondering running a virtual Windows, map the share on Scale and map the share on the NAS and let the windows software handle it. Dunno if that would be a performance hit, but at least I would know exactly what I am doing.

 

[sretalla]

rsync can be run interactive, but is often suited to running in daemon mode (see under Services in TrueNAS).

Under the services setup, you have a second tab to setup the modules to connect to from your clients.

 

[jgreco]

I just don't get that and while I have been on and off on the rsync pages looking for answers it remains the proverbial greek to me.

In my opinion (LOOOOOONNG time rsync user) you are so much better off getting things to run via ssh FIRST and then converting your configs on each side to daemon mode. Not TrueNAS-specific advice. rsync will sing and dance and do really incredible crap for you if you know what you're doing. But it can take some work to get there. Maybe I can help shortcut you.

What you're looking to do is to configure rsync as a server on one machine and a client on the other; direction is not determined by "client" or "server". rsync can push or pull with equal deftness.

So let's say that I have a bunch of video content.

# zfs list storage1/Video
NAME USED AVAIL REFER MOUNTPOINT
storage1/Video 42.6T 47.2T 41.6T /mnt/storage1/Video

I send this across the country to a backup site hourly;

2023/03/07 15:00:02 [87280] building file list
2023/03/07 15:00:31 [87280] 188722 files to consider
2023/03/07 15:00:35 [87280] sent 7452319 bytes received 17 bytes total size 80098275804133

and yes it really does take all of 35 seconds to sync almost 200,000 files when there are no changes.

On one side, you set up an rsync task from /mnt/storage1/Video, user: root, direction: push, schedule: hourly, recursive, remote host: as needed, rsync mode: module (means connect to rsync server), remote module name: Video, set your options, and I also like to include "-H" "--progress" and "--no-inc-recursive", just so you can see how to set those weird manpage options that are available to you. On the server side, you go under Services, rsync, enable rsync, then edit, add a module with name of Video, the appropriate target path, enabled, write-only (or whatever), user/group root/wheel, hosts allow should be your client machine, and save it.

 

[SecCon]

Hmmm....

Just to be clear: I have the Scale server and I have a NAS to setup for this

Scale = Source and main fileshare, available to users
Nas = backup, mirrored, not available to users

So what you are saying is that I run a rsync server on Scale and push mirror backup to daemon on Nas?

Running rsync as a daemon on the NAS does not seem to be a viable option according to the info I have:

Rsync Server

www.asustor.com

Dammit, its gonna be a long haul before I get this

 

[jgreco]

Then you need to figure out what options are available to you with what you have.

 

[SecCon]

I do.

Needs to be a push from Scale > to > Nas solution. That's seems to be possible in rsync, trying to see what happens. Its all so werd syntax... doing tests as I type this. .... and SSH fucks it all up. Its a local server on the local network. So another showstopper to setup SSH on the intended Scale server. Thats not even working yet. Nor installed.

Tried Syncthing but was unable to sort out how to map the SMB shares , seems to require additional rights somehow... It's like it it is in a different language.
Duplicati is probably a contender, yet only heard of it, never used it, also hard to understand info..
Bacula might be over the top but still feasible solution, If I keep it on the Scale machine. At least they seem to have an interface nowadays.

Also:
Virtual Windows on Scale, map the share of Scale and map the share of NAS and let the windows software handle it. Dunno if that would be a performance hit, but at least I would know exactly what I am doing.

 

[sretalla]

Virtual Windows on Scale, map the share of Scale and map the share of NAS and let the windows software handle it. Dunno if that would be a performance hit, but at least I would know exactly what I am doing.

It's your gear and your time... if you want to do it the way you're most comfortable with, go ahead. It will certainly not be the most efficient use of resources, but those are yours to decide to allocate or not.

 

[sretalla]

Duplicati is probably a contender, yet only heard of it, never used it, also hard to understand info..

I helped someone out with an issue with installing that in a jail and still have it here somewhere. It looks quite good from a features perspective and seemed (to me at least) to be easy enough to work out. I would recommend at least install it and have a look.

 

[SecCon]

Time? Had it been something I would think as a good solution for the Scale environment it would have been done already, days, even weeks ago.
As it is I am trying to use what Scale has inbuilt but it is just raising one hurdle after another. Why do you think I left FreeNAS three years ago?

Ah, you want this excellent system with these outstanding features!
All right. Do this.
Ah, but in order to do that you must have done H before.
For H to work you need to have a working Y conf.
Did you miss that you needed W before config Y to get H to work? So sorry.
Thank you for understanding.

And on top of that instructions for each system respectively that seem to be written in the basement of MIT while the Q&A peeps are on LSD.

Sorry, feeling a bit frustrated.

 

[SecCon]

Get started with OpenSSH for Windows

Learn how to install and connect to remote machines using the OpenSSH Client and Server for Windows.

learn.microsoft.com

Working SSH. Now for the other rsync errors.

> netstat -a Results:TCP [::]:22 <server ip>:0 LISTENING

So when is rsync using port 873 anyways... never mind

Needs to be repeated. This is just a test setup between Scale and a Windows Server.
When sorted, I need to test Scale from the same testmachine with my NAS

 

[jgreco]

Why do you think I left FreeNAS three years ago?

It sounds like you want something akin to a self-driving car or perhaps an Uber, rather than owning your own vehicle. Your NAS has a bunch of great features, and so did FreeNAS three years ago, but it is technologically complicated and sophisticated. You need to have an appropriate server, and you need to have an appropriate network, and on that network you must have a reasonable IPv4 or v6 design, and services such as DNS, NTP, perhaps DHCP, AD, etc. Then you need some clients, which could be UNIX (NFS), or Apple (AFP), or Windows (SMB), and you need an environment that is appropriately configured for each of those, which often contain various subtopics within them (such as SMB and its Windows-like permissions). The clients each have their own unique administration challenges as well, which include firewalling, nameservice, timesync, etc., and then there are also the services that you can add on to your NAS system, including a variety of namespace-based jails or containers, virtual machines, etc.

And on top of that instructions for each system respectively that seem to be written in the basement of MIT while the Q&A peeps are on LSD.

Yeah, it's going to seem like that if you're not actually interested, willing, and/or able to invest the time in learning the complex topics that the NAS builds upon as a network appliance. When you call an Uber, you don't have to worry that the car has air in its tires, or gas in its tank, or that the driver has been educated on the rules of the road, or knows how to get to your destination. When you buy your own car, you have to start being aware of these things, and have to have plans for issues such as what happens when your car breaks down (roadside assistance?), or when you rear-end another driver (insurance), and a bunch of other complex topics that most people just assume that they're going to have to deal with when they buy their own car. The underlying documentation for this is relatively complicated too; one of my kids used to think he didn't need to have his driver's license on him because it could be "looked up", but the law says otherwise and if you know where to look in the statutes, you can find the text that says so and what the penalty for failure is.

No one's faulting you for wanting an Uber experience, but this is a community of (and for) vehicle owner/operators. If your expectations are that you don't need to internalize responsibility for the stuff on your server and network, yes, those of us who chase down and write about all the intricate details of car maintenance and upkeep are certainly going to seem like druggies to you. DIY NAS is not for everyone. I'm not able to determine if that applies to you, but I do want you to clearly hear this message: It MAY not be for you, and IF it isn't, no one here will think less of you for it. On the other hand, this community will spend a lot of time and effort trying to get you up to speed if that's the way you prefer to go.

 

[SecCon]

With all due respect

I drive. I don't use Cabs or Ubers. For anything. Not even when after a pub because I hardly drink any alcohol. Seriously can't remember when I took a cab last. Must have been at least 10 years ago. I do have 15 years in the automotive industry though, within Service Management and Infrastructure. There are many excellent manuals for many parts and several key systems that people have written and I have revised and sent back for a rewrite before approval.

I understand this is complex and not something you can just throw at your machines and networks without some consideration and reading up. I have a layer2 network. I have a competent Firewall and Router, that I am actually helping a bit in it's development. It is of course not without its challenges and I am not a programmer, much anyways. My hardware meets, to some degree, corporate requirements, I know it does, since that is what I buy and some I got from my workplace. Not sure about the WD NAS drives though... but it is what I can afford and happy it's recommended.

This thread is about my experience with implementing Scale. I run a test. I do trials. I document what I do and write up failures, and probable cause, as I go along, not everything here though. I recently started using OpenAI ChatGPT to get some explanations and troubleshooting information I can not find in the Scale manuals or documentation. It is a very awkward experience, and many times a better one.

Just the other day I realized that I may have ignored one item. I ignored it because I do not need it, but it seems it may be one key to make things work better, namely SSH. It is not enabled on my Scale. Yet I do not see anything in the docs that says "you should have SSH enabled", only "these are the available SSH options" - that you may or may not use. I loathe that kind of documentation. It does not tell you what to do, only what options you have.
I guess it comes down to writing something that is general enough for many to understand, yet specific enough to list all the options and details.

I also think I need to review this more than I have so far: https://www.truenas.com/docs/scale/scaletutorials/systemsettings/services/configrsyncservicescale/ What I do not get is why rsync defines port 22 as "default", that is reserved for SSH and is not by any means a reserved port for anything else. Sure it can use it, but it is the default SSH port. It says here that that rsync uses port 873: https://www.truenas.com/docs/scale/...emsettings/services/rsyncservicesscreenscale/.

So, many small things, some of them just confusing semantics.

I am staying. I have a workaround should rsync, or similar items, remain a challenge. Much prefer to get it working though. The possible limitation I see for now is the push-pull setup. I would have to setup rsync from Scale to push only. The hardware resources on my NAS may not be enough to run a daemon, but I will check with the manufacturers and see what can be done. It is 9 years old and has run a RAID6 backup for most of those years, with several disk upgrades over time. No plan to replace it.

TrueNAS (Scale) has everything most, and I, could wish for. A stable development, an active community, a successful company behind it selling it's implementations. So many solutions have not been able to set that up or keep true to the combination of free software with a selling product. I have seen many fail over the years. That is mostly why I am here.

 

[jgreco]

I drive.

So, then, my point is this:

Just the other day I realized that I may have ignored one item. I ignored it because I do not need it, but it seems it may be one key to make things work better, namely SSH. It is not enabled on my Scale. Yet I do not see anything in the docs that says "you should have SSH enabled", only "these are the available SSH options" - that you may or may not use. I loathe that kind of documentation. It does not tell you what to do, only what options you have.

When you get in your car, you need to have some implicit understanding of what your goal is. You can drive forwards or backwards to get to your destination. But you need to understand how to steer. No one is going to tell you "you should turn the steering wheel to the left", because we don't know the path needed to get to your destination. There are better options and worse options, and some options might not work for you. For example, ZFS replication won't work if you don't have ZFS on the target system. We're not here to do the big job of teaching you how to drive, even though you could probably pick it up from what's available on the forum. All the bits are here, but we may not be the best resource for that. We're not going to tell you how to drive. We might explain why most people drive forward most of the time.

I also think I need to review this more than I have so far: https://www.truenas.com/docs/scale/scaletutorials/systemsettings/services/configrsyncservicescale/ What I do not get is why rsync defines port 22 as "default",

Because most people run rsync over ssh. rsync by itself is not encrypted, and your data is not protected from prying eyes. This may be okay for some applications, in which case you can run rsync over to the rsyncd server on 873, which is very fast and does not incur the performance penalty of SSH's compression and encryption. However, most users of rsync need ssh as part of their recipe, so the defaults guide you to use SSH. You seem to be laboring under some misconception that there's a third option where rsync can run over port 22. It does not, 22 is sshd (unless you reconfigure it). When you specify the Rsync Mode as "Module", it will go directly to rsyncd over port 873. When you specify the Rsync Mode as "SSH", it will go to sshd over port 22 and then invoke the rsync binary as a subsidiary process.

I do not see anything in the docs that says "you should have SSH enabled",

Enabling SSH makes certain things easier, but other things more dangerous. SSH is a common attack vector in the Internet industry, and enabling it means that there is one more path an intruder could take to get into your system. Infrastructure devices such as hypervisors, NAS, routers, switchgear, etc., generally come with SSH disabled, requiring you to opt in and accept the risk. You shouldn't have SSH enabled unless there's a need. This is congruent with many other decisions about enabling services, such as: don't enable SMB on an NFS-only server. Etc.

 

[SecCon]

I do not get how I am not clear on how I want my overall setup to work. Want me to repeat?

ZFS replication is a no-go. Not without buying a new NAS and drives and stuff. I already knew that.

You can mitigate SSH as attack vector by defining a different port and specifying a user+folder access, something Bitvise is excellent at handling in a well defined and documented interface, which I have done in other solutions completely different from this.

Aside from that I missed that enabling "Module" would trigger the non SSH port 873 functionality. I see now that I have to find that setting in the rsync Service sub menu, after enabling the Service, and not where I was looking for it, under ... well, wherever it was.

See, you are helping. I guess you may not really intend to actually do that, but I guess I am a Workshop oriented person, I may think best when challenged and discussing something. Now give me back the damn wheel so I can do some burnouts.

 

[SecCon]

Got rsync working doing a basic file copy from the Scale test bed to my NAS.
Lots of fiddling back and forth in the docs, but managed to sort out the info in a structured 1-2-3 relevant (to me) reading sequence after copy pasting text to a new doc. Again, troubleshooting via ChatGPT helped. Creds to this site as well for additional information about ACL among other things: https://docs.xstar97thenoob.com/docs/storage/media-and-downloads/

Not my Jag, but almost​

Now for more advanced rsync conf.

 

[SecCon]

Had a timeout reconfing my FW over a few days.