How to connect disks (block devices) from network storage to TrueNAS?

[Aku]

Hi all! I'm afraid I don't know how to ask the right question, so sorry if I'm being unprofessional.

I have a dedicated server on Hetzner. I installed TrueNAS SCALE on it and it works fine. Now I want to increase the amount of storage, but I can no longer install more physical disks on the server. If I understand correctly, there are so-called network storages that, via iSCSI or something like that, give access to block devices over the network. So I need to rent another server, make network storage out of it and somehow connect its disks to the first TrueNAS server? Or are there other solutions?

By the way, I see that TrueNAS can act as an iSCSI Target. But I don't see anywhere that it could be an iSCSI Initiator. Is this even possible without interfering with the OS?

 

[Samuel Tai]

No, sorry. This will be a sure way to lose data.

 

[Aku]

No, sorry. This will be a sure way to lose data.

Ok. But how does the system scale? Or it is necessary to look towards clustering?

 

[WI_Hedgehog]

In simple (and somewhat ambiguous terms) you could perhaps buy a box that holds drives (JBOD), and put "a bunch" of drives in it. That box gets shoved into the rack within about 5 feet of your current server.

In your current server you add a Host Bus Adapter (hard drive controller card) which has external ports and using the external ports on that card connect the server to the box of drives.

In TrueNAS you add the drives and...there you go.

Does that sound possible to your network guy?

 

[Aku]

In simple (and somewhat ambiguous terms) you could perhaps buy a box that holds drives (JBOD), and put "a bunch" of drives in it. That box gets shoved into the rack within about 5 feet of your current server.

In your current server you add a Host Bus Adapter (hard drive controller card) which has external ports and using the external ports on that card connect the server to the box of drives.

In TrueNAS you add the drives and...there you go.

Does that sound possible to your network guy?

View attachment 65977

Do you mean HBA? As far as I know my provider (Hetzner) does not provide this service.

 

[Samuel Tai]

What specific SKU of Hetzner server are you leasing?

 

[Aku]

What specific SKU of Hetzner server are you leasing?

 

[Samuel Tai]

The closest I can find is the SX294 at https://www.hetzner.com/dedicated-rootserver/matrix-sx. Under Optional Features, it's possible to add a 16-port RAID controller, and a Storage Box with up to 20 TB. Expansion does appear to be possible. You should inquire with Hetzner whether they have an HBA option rather than a RAID controller, and the connectivity options for the additional storage shelves.

 

[Aku]

The closest I can find is the SX294 at https://www.hetzner.com/dedicated-rootserver/matrix-sx. Under Optional Features, it's possible to add a 16-port RAID controller, and a Storage Box with up to 20 TB. Expansion does appear to be possible. You should inquire with Hetzner whether they have an HBA option rather than a RAID controller, and the connectivity options for the additional storage shelves.

Thanks, I'll definitely check it out. But is adding disks to server only way to expand? Why do we need clustering then?

Maybe glusterfs or ceph can help me?

 

[Samuel Tai]

Thanks, I'll definitely check it out. But is adding disks to server only way to expand? Why do we need clustering then?

Maybe glusterfs or ceph can help me?

There are 2 purposes for clustering: high-availability (having the same data on multiple servers actively serving data) and big data (storing datasets too large to fit on one server). Unless you absolutely need one or the other use case, clustering is more trouble than its worth.

For the high availability case, you need to sync writes across all members of the cluster, and this isn't trivial.
For the big data case, you need some way of referencing reads and writes of the same dataset across all members of the cluster, and this is even less trivial than the HA case.

Unless your needs encompass 6x 9s data availability, or processing multi-100 PB datasets, you don't need clustering.

 

[Aku]

There are 2 purposes for clustering: high-availability (having the same data on multiple servers actively serving data) and big data (storing datasets too large to fit on one server). Unless you absolutely need one or the other use case, clustering is more trouble than its worth.

For the high availability case, you need to sync writes across all members of the cluster, and this isn't trivial.
For the big data case, you need some way of referencing reads and writes of the same dataset across all members of the cluster, and this is even less trivial than the HA case.

Unless your needs encompass 6x 9s data availability, or processing multi-100 PB datasets, you don't need clustering.

Thanks, valuable information. But do I understand correctly that clusters are unite servers themselves and do not use combination of external (network) storage? Or are external (network) storages for servers with TrueNAS generally contraindicated and not used?

 

[Samuel Tai]

TrueNAS has basically no mechanisms to attach network storage. It can attach SAS shelves. If you paid for the Enterprise version, it can also attach Fibre Channel shelves.

Any cluster node can use supported attached external storage; the caveat is all the nodes have to be identical.

 

[Aku]

TrueNAS has basically no mechanisms to attach network storage. It can attach SAS shelves. If you paid for the Enterprise version, it can also attach Fibre Channel shelves.

Any cluster node can use supported attached external storage; the caveat is all the nodes have to be identical.

Of course, I understand that one can simply add physical disks to server. But sooner or later this opportunity will end. What to do then?

I read this article https://www.truenas.com/blog/truenas-scale-clustering/ about clusters, but I don't understand what type is needed to increase final pool size.

For example: we have a TrueNAS server and X more of same, but "clean" servers, without an installed OS. What do I need to substitute on rest of servers (TrueNAS, pure Debian or another OS) and how to set them up with each other, so that in end I get not X + 1 different servers, but, as it were, a single server with a single storage. Of course, I'm not asking for a full manual, I'm trying to understand what technologies I need to use for this.

 

[sretalla]

I read this article https://www.truenas.com/blog/truenas-scale-clustering/ about clusters, but I don't understand what type is needed to increase final pool size.

TrueNAS SCALE clusters employ glusterfs to provide storage which spans cluster members.

No pool can grow beyond the bounds of the node it is created on.

Gluster volumes can be as big as you want to make them.

 

[sretalla]

For example: we have a TrueNAS server and X more of same, but "clean" servers, without an installed OS. What do I need to substitute on rest of servers (TrueNAS, pure Debian or another OS) and how to set them up with each other, so that in end I get not X + 1 different servers, but, as it were, a single server with a single storage

Perhaps I didn't answer this fully/clearly...

You would install SCALE on all of them and provision one (or more) pools on each.

Then, with TrueCommand (or API calls if you prefer) create a cluster volume, which gets pointed at each of the pools you want included in that unit (and specifying how they should behave in terms of redundancy, etc.), which can then be consumed by gluster clients or used for SMB shares.

 

[Aku]

Perhaps I didn't answer this fully/clearly...

You would install SCALE on all of them and provision one (or more) pools on each.

Then, with TrueCommand (or API calls if you prefer) create a cluster volume, which gets pointed at each of the pools you want included in that unit (and specifying how they should behave in terms of redundancy, etc.), which can then be consumed by gluster clients or used for SMB shares.

In TC, I only see cluster creation and iSCSI creation. What exactly do I need to create?

 

[sretalla]

In TC, I only see cluster creation and iSCSI creation. What exactly do I need to create?

Cluster.

Clusters Screen

Option descriptions in the TrueCommand Clusters menu.

www.truenas.com

 

[Aku]

Cluster.

Clusters Screen

Option descriptions in the TrueCommand Clusters menu.

www.truenas.com

I created a test environment on virtual machines in Hyper-V from three identical TrueNAS SCALE nodes and one Debian node with TrueCommand. But, as I understand it, I need to configure network in a special way. Went to sort it out...

 

[Samuel Tai]

Yes, each cluster node needs a dedicated cluster internal sync interface.

 

[Aku]

Yes, each cluster node needs a dedicated cluster internal sync interface.

I have dealt with network. Now there was a question about Active Directory. Do I need a separate server where I must set up a domain with Active Directory, or does TrueCommand create Active Directory in cluster itself?