Will it TrueNAS (Scale)? Upgraded Huawei RH2288Hv.3 - pondering how to get things working

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
So now I have the HBA card.

jeez... no excuses, just a bunch of reading up

I am nervous about doing all of the below, yes I still have all the files on my NAS and that is accessible, but it will take hours and days to get below done, then the file copy back which will take days, maybe a week... 11TB
  1. Kill primary storage and delete all current raids
  2. Check the drives and clean them
  3. Kill the machine
  4. Open the machine
  5. Remove raid card and cabling
  6. Insert HBA card and connect cables
  7. Check that it works.
  8. Load up the truenas core iso
  9. Install truenas core (headless) on a pair of SSD drives using ZFS stripe or equivalent... hmm, the docs are not very clear on that, but I assume I just tick the drives I want to use...?

    Ok. I need an answer to that. When I installed a FreeBSD based app for something else some time ago I could select two drives and used a ZFS Stripe on those two drives. What does TrueNAS Core do these days? Why is it not in the man pages....


  10. Network setup and such should be no biggie, pretty much default, just select the right NIC and so.
  11. Configure pools , at least two, make sure to select correct disks for each pool, one red one blue ( the drives have small coloured labels on the caddies ), red being the main storage... I see no improvements in that interface from freenas 11, still tiny, unguided, and a bit confusing.
  12. Configure smb and test
  13. Make a few tests with syncthing, seems to have an readable interface as opposed to rsync. New to me, I really hope it will work without scripts or cmds or general typing script stuff
  14. Copy back the whole shebang of 11TB data on a 1 GBit network connection and SMB speeds... about 100MBit/s..
  15. Did I mention pray...?

... procrastinating...
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
15,464
When I installed a FreeBSD based app for something else some time ago I could select two drives and used a ZFS Stripe on those two drives.
I'm sure you meant "ZFS Mirror", and that's how TrueNAS works too.
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
damn, no more excuses to procrastinate.. but much appreciated @danb35 :smile:
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
Ah, forgot
0. Server mounted on it's rails. :wink:
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
I keep posting here to keep my own little Q&A as I move relevant information to my Confluence, but if you rather have me asking different things in different topics please let me know.

I reviewed my drives yesterday and reading a bunch of ZFS documentation, not least linked by @Davvo over here I realize there are some items that may need clarification.

So this are my current RAID conf on my current controller, which will give you an idea on how I plan to do 2 items, the setup of TrueNAS Core on the mirrored 128GB SSD's drives - located in the rear of the server - and the main storage on the 4 x 6TB drives on the front. That will be a RAIDZ1 18TB pool. Yes, I should probably get another couple of those for a RADZ2 and may do so. Question in this is, can I, eventually, upgrade the RAIDZ1 to RAIDZ2 without losing data on the pool?

As you can see I have two pairs of 2TB and 4TB drives in the same enclosure. All drives are SATA HDD 7200rpm.
I do not plan to add more drives or exchange them for now, but all in all there is room for 12 LFF in the front.

Since they are not the same sizes I will use either as space for virtual machines, also mirrored, and /or additional temporary storage for whatever comes up.
 

Attachments

  • raidcontroller.png
    raidcontroller.png
    38.5 KB · Views: 75

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,112
Question in this is, can I, eventually, upgrade the RAIDZ1 to RAIDZ2 without losing data on the pool?
No. To do that you need to go through "backup-destroy-restore". Better get some extra drives NOW and create a raidz2.

As you can see I have two pairs of 2TB and 4TB drives in the same enclosure. All drives are SATA HDD 7200rpm.
I do not plan to add more drives or exchange them for now, but all in all there is room for 12 LFF in the front.

Since they are not the same sizes I will use either as space for virtual machines, also mirrored, and /or additional temporary storage for whatever comes up.
You can make two mirror vdevs and put them both in the same pool, for 6 TB of redundant storage. No issue here.
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
I know that 1 drive failure acceptance may be a bit touch-and-go, but I do have backup should 1 drive fail. I even have backup if 2 drives should fail and the RAID5, or RAIDZ1, go down. Don't feel that is a big issue.

The biggest issue is feeling safe and secure doing things right in TrueNAS. That is not the case, yet. Despite me reading a lot this last week.
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
2 additional drives would give this according to this site: https://wintelguy.com/zfs-calc.pl

ZFS / RAIDZ Capacity Calculator (beta)​


RAID type:
Mirror (two-way) RAID-Z1 (single parity) RAID-Z2 (double parity) RAID-Z3 (triple parity)
Drive capacity (GB):

Drive sector size:

512 B 4 KiB
Single drive cost:

Number of drives per RAID group:

Number of RAID groups:

ZFS record size:

512 B 1 KiB 2 KiB 4 KiB 8 KiB 16 KiB 32 KiB 64 KiB 128 KiB
Take into account:


Slop space allocation


% free space limit



RAID type:RAID-Z2 (Double parity with variable stripe width)
Number of RAID groups:1
Number of drives per RAID group:6
Total number of drives:6
Drive capacity (GB):6000
Drive capacity (TiB):5.456968
(TiB)(TB)(%)
Total raw storage capacity:32.74180936.000000100
Zpool storage capacity:32.50000035.73412899.26
Reservation for parity and padding:10.83333311.91137633.09
Zpool usable storage capacity:21.66666723.82275266.17
Slop space allocation:0.6770830.7444612.07
ZFS usable storage capacity:20.98958323.07829164.11
Minimum free space:
Practical usable storage capacity:20.98958323.07829164.11
Single drive cost:
Cost per TiB usable:
Total cost:
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
Actually, I see know it would be enough, for now, with 1 additional drive for RAIDZ2
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,112
To keep it simple, a n-wide raidz vdev of level 'z' (1,2,3) provides (roughly) (n-z)*(size) of raw storage, of which you should not use more than about 75-80% to keep ZFS happy and snappy. The size of extra ZFS metadata disappears in this "about".
So for z=2 and 6 TB,
4 drives, 12 TB raw, 9 TB usable
5 drives, 18 TB raw, 14 TB usable
6 drives, 24 TB raw, 18 TB usable
etc.

Since it is not possible to change z or n afterwards, you want to begin with a vdev that is large enough for present data and for the foreseeable future data. If your system still has free slots, it is also advisable to pick up a width which will fit well to enlarge the pool with further vdevs of the same width.
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
@Etorix
So with current 11,5TB data on projected 14TB usable (18 raw) I should actually consider going for yet additional space up to 18TB usable (24 raw). The threshold for usable data and max 80% on 14TB is 11,2TB, so I am already there even with the extra drive I just ordered for 170$. Bleh.

I can't predict the future, so the foreseeable amount is nowhere to be found, but I would guess that 18TB usable would last me a few more years, maybe 5. It has taken me about 13 years to get up to current 11,5TB.

Having said that it poses a challenge towards my backup, on RAID6, which has 14TB usable data, I think, need to check, but not running ZFS.
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
k' another 170$ spent.
 

Etorix

Wizard
Joined
Dec 30, 2020
Messages
2,112
The 80% guideline applies to ZFS only, due to its CoW nature and thus its need for sufficient free blocks to perform any operation (including releasing space…).
But, yes, running ZFS has some front costs.
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
Also verified appropriate drives

1676033730606.png



So ordered two more, I guess they will be WD60EFRX.

 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
Got the last two drives.

Checking backups and stuff. Lets see if I find the time this weekend to blow up some standard raids...

he-he-he-raccoon.gif


... and perhaps not go into OMG WHAT DID I DO - mode
 

SecCon

Contributor
Joined
Dec 16, 2017
Messages
175
Port 9 of 12 (0-11) is not detecting the drive. I need to check that cabling, and of course test with another drive.

It is dark outside, a storm is coming. Past 19hours and windy. I guess taking a rain check due to an approaching storm may be excusable.

1676657028328.png


Oh, and a new episode of ST:Picard. :wink:
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,151
Check if the lane is shared with anything else too.

Also Malmo was the only saving grace in that image; everything else is foreign.
 

c77dk

Patron
Joined
Nov 27, 2019
Messages
467
It is dark outside, a storm is coming. Past 19hours and windy. I guess taking a rain check due to an approaching storm may be excusable.

View attachment 63717
Just drove from Roskilde to Malmö in the nice weather :smile: Had to take a detour via the ferry Helsingør-Helsingborg. I love the ferry keeps sailing, even though the bridge is closing down due to wind :smile: (and to the non-locals: There's a tunnel+bridge connection between Copenhagen and Malmö)
 
Top