Truly Confused and upset

[William Bravin]

Hello all

As i mentionned I am truly confused and upset with myself

Over the las month, One of my servers had an issue. After 2 days of searching I realised that the SATA controller was shot (problem No 1). Since I had 4 disks connected this controller and 4 more connected to the motherbord, I decided to replave it with a 10 port card to put all disks on the same card.

At this point I found that i had 2 HD not healthy (problem No 2) I replace them

At this point I wanted to rebuild my dataset on my repaired server ... no problem

However when I went to the truenas site i saw that 13 was released. Because 13's new structure I saw the possibility to install Home assistant on this server. So I installed Truenas 13 . Please bear in mind that it has been 4 years since i did an installation. So I installed 13 in a vere simplistic configuration with samba, Rsync as services.

Once installed and configured i proceeded to rsync from my second server to populate my repaired server

In doind so my repaired server whilst pulling the data from my other server shut down ennexpectidly (problem No 3). I had it cleanned and has the thermal past and the fan replace.

During this tribulation I accidently deleted all my music and my movies library (problem No 4) this is why I am very upset

All my datasets are configured the same way and all have the same ACL (i am the owner of the dataset and i also added me as an additional ACL with full control

Now that all is fixed and the rsync tasks are running fine I noticed the following

On the truenas 13 and on the other server (ver 12,7) all the dataset ( say Movies) have a subset dataset called wbravin (me) with the exception of the tv shows dataset

I cannot seer the diffeerence in how i constructed or configured the dataset.

1) In addition the dataset called tv shows, when I rebuild the data (add data ) no issues. However when i need to move or delete an item i get a permission error.

2) I then go the the dataset and edit its ACL (ichange nothing and save)
3) then I go bac to my internet explorer and proceed to move and delete the items and it work Great

I then go to add more files to this dataset and then need to move, rename and delete and again i get the permission error then i go back to 1

What did i miss?

An other curiosity is that some dataset have a folder call zfs and within this i see an other dolder called snapshot. the funny thisg is that in neither of my server did i configure snapshops

Thank you in advance for your patience and help

 

[HoneyBadger]

Since I had 4 disks connected this controller and 4 more connected to the motherbord, I decided to replave it with a 10 port card to put all disks on the same card.

I'm a little concerned about this piece of hardware as I don't know of any reliable HBA that natively comes in a 10-port configuration. Do you have a make and model number?

 

[winnielinnie]

I'm a little concerned about this piece of hardware as I don't know of any reliable HBA that natively comes in a 10-port configuration.

The OP might be referring to one of those generic 10-port SATA PCI-e cards. Maybe something like this:

https://www.amazon.com/MZHOU-Profile-Bracket-Support-Converter/dp/B08L7W8QFT/

 

[William Bravin]

Hello

WOW thank you for replying so quickkly @winnielinnie Yes it is the exact unit i purchased

 

[Arwen]

That 10 port SATA card uses;

• ASM 1166 PCIe to SATA controller, 6 ports
• JMB575 5 port SATA multiplier

The JMB575 does require 1 of the ASM 1166 ports as a source, so that is why their are 10 ports and not 11.

SATA port multipliers are not a good idea for TrueNAS. Not just that 5 of the ports all share the bandwidth of 1, 6Gpbs SATA port. But their can be problems with corner cases, (aka rare occurrences in software drivers, firmware in the chips or disks). See;

Multiply your problems with SATA Port Multipliers and cheap SATA controllers

This resource was originally created by user: jgreco on the TrueNAS Community Forums Archive. Please DM this account or comment in this thread to claim it. In the last year or two, we’ve had a resurgence of users asking about SATA Port Multipliers and cheap SATA controllers. Please, do NOT use...

www.truenas.com

 

[William Bravin]

Thank you for your reponse

are you suggesting that all my issues are due to my sata controller?

if so sould i revert to connecting 4 HD to a 4 port sata card and the other drives to the motherboard would this resolve my issues?

This only resolve some of my issues.

I am still faced with the acl issues in the "tv shows" why is it that once set and working fine it then revets back to permission denied. Is this too related to the controller.

As i am not technically inclined what controller card that is compatible with truenas on an asus based server would your recommend

 

[Arwen]

The 10 port SATA card will not affect the data on the disks directly. But, regular ZFS scrubs are needed to confirm the data was not mis-written by driver software bugs or firmware bugs.

Their are plenty of choices for cheap, LSI, (or branded IBM / Dell cards that are LSI chips), that work great.

 

[William Bravin]

Thank you so much.

interesting

I do not see this happening in 12.X

I did not initiate an scrubs task on iether servers.

Is this something automatic in 13? (BTW when i rebooted the repaired server today they dissapeared) I must have done somthing stupid!

You raise in my mind an interesting point that i never considerred (because all of my music is ripped from cds or vinyl and the movies and tv shows are (for the most part) ripped from DVD or video disks and a small quantity from youtube). I will try to implement them on both servers.

From your post i gather that the key is a for the controller card is LSI functionality. I never considered this

I looked at LSI cards there is a plethora of them with version or model numbers. To me this is all foreing.

would something like this one be compatible ?

LSI Internal PCI-Express SAS/SATA HBA, 9211-8I, 8-Port 6Gb/s Raid Controller Card​

When i do change the controller card, will this cause a rebuild of the media and its related dataset?

again thank you for your patience and teachings

 

[Arwen]

Without doing the research, I don't know about that specific LSI card.

No, ZFS will not need to "rebuild" on change of disk controller.

ZFS scrubs are part of the TrueNAS options you should be using. Perhaps set one up once or twice a month. Some people even do weekly, but it's up to individuals to determine how often.

Here is where you can start your research;

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

Don't be afraid to be SAS-sy ... a primer on basic SAS and SATA

With the introduction of SAS 12Gbps, seems like "it's time" to do a braindump on SAS. Work in progress, as usual. History By the late '90's, SCSI and PATA were the dominant technologies to attach disks. Both were parallel bus multiple drop...

www.truenas.com

 

[William Bravin]

Thank you @Arwen for this information. I truly appriciate it. I do have a dl380 g7 to replace this asus based server. It will use SAS drives as does my current supermicro.
FYI I have started a scrub function on my supermicro following yout post.

Many of the LSI i find mentione that they are also raid controllers . Would this interfeer with trunas raid function?. Is there a process to stop the LSI from preforming a RAID function?

Once again many thanks Stay Cool and may the force be with you (i'm nore a Star Wars fan)

 

[Arwen]

@William Bravin - Many of the LSI cards that can have IR firmware, can be re-flashed to IT firmware.

Basically LSI makes 2 lines of SAS / SATA, (and some tri-mode that support PCIe / NVMe drives), controllers. The supposed higher end is the MegaRAID, and they do full on RAID, like RAID-5/50 and RAID-6/60, as well as RAID-1/10 or just RAID-0.

The cards we, (as ZFS users), are desiring, are the lower end, (but still high quality, just no NV-RAM, no RAID-5/6 parity engine, etc...), are the IR / IT cards / controllers. IR stands for Integrated RAID, and that firmware supports RAID-1/10 as well as RAID-0. It's meant to be used on server motherboards for the OS and or built in disks. Sun Microsystems used such on some of their SPARC server motherboards. This controller's IT firmware, (for Integrated Target), removes all the RAID functions and simply does pass through for disks.

 

[William Bravin]

Thank youn @Arwen for these teaching. Although they are waaaay above my head, I apreciate very much leaning and improving my environment

 

[William Bravin]

Hello all

I must be Jinksed

Now on this rebuilt server I get the following error

could this ber due to the disksa or the controller?

I do not know what tyhis means

Expected Finished Time:

ada0: 2022-07-21 18:24:37
ada1: 2022-07-21 18:24:37
ada2: 2022-07-21 18:24:37
ada3: 2022-07-21 18:24:37
ada4: 2022-07-21 18:24:37
ada5: 2022-07-21 18:24:37
ada6: 2022-07-21 18:24:37
ada7: 2022-07-21 18:24:37

Errors:

da3
smartctl failed for disk da3: smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RC6 amd64] (local build) Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org Read Device Identity failed: scsi error unsupported field in scsi command A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.
da4
smartctl failed for disk da4: smartctl 7.2 2021-09-14 r5236 [FreeBSD 13.1-RC6 amd64] (local build) Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org Read Device Identity failed: scsi error unsupported scsi opcode A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.

It talks about scusi and i have no scusi implemented (at least i think)

Could this be why my rsync task is hangin?

do you recommend i change those disks or the controller first?


Thank you once again

 

[HoneyBadger]

USB devices will attach themselves as daX which is interpreted as SCSI, but often require a different method of communicating their SMART data. Do you have USB device(s) attached for booting?

I do have a dl380 g7

Fair warning that the embedded SAS controller here (SmartArray P410i) is not suitable for ZFS use either. You'll still want one of the LSI HBA (or "IT mode") controllers.

 

[William Bravin]

USB devices will attach themselves as daX which is interpreted as SCSI, but often require a different method of communicating their SMART data. Do you have USB device(s) attached for booting?



Fair warning that the embedded SAS controller here (SmartArray P410i) is not suitable for ZFS use either. You'll still want one of the LSI HBA (or "IT mode") controllers.

Thank you
Yes
The only USB device i have on the rebuilt server is the truenas booting oftware. The issue here above is on an asus motherboard based server

Thank you for the warning on the dl380. I will need to sort this out before migrating

Is something like this is what you mean

Dell H200 SAS/SATA PCI-e x8 47MCV HBA LSI 9211-8i P20 IT Mode ZFS FreeNAS unRAID​

https://www.ebay.it/itm/234202819061?hash=item36879369f5:g:sUkAAOSwjkpg-t6C
again thank you

 

[Redcoat]

With reference to the Ebay ad to which you linked, three comments:

1. Yes, that's an example of the HBA device that's been suggested to you, pre-flashed with the P20 IT firmware that Arwen discussed.
2. Look at the several photographs in the ads - you will see that some units have the two 8087 connectors on the top, some on the back end of the board. Make sure that you select a unit with connector location compatible with your case layout. My personal preference would be connectors on the rear edge.
3. Many of us here have had excellent experience with products from The Art of Server who has a product in the "Oggetti sponsorizzati acquistati da altri utenti" line - trouble free products, products as described, very good after-sales technical support (if needed). Definitely worthy of your consideration in my opinion, especially given the uncertainties you have expressed.

Good luck with your project(s)!

 

[William Bravin]

Thanks to you @Redcoat and @Arwen for the teachings all of this is so far from my comfort zone. However happy to learn how to better my environment.

Hopefully this will resolve many issues i have

Again Thank you