TrueNAS scale keeps crashing while transfering file to it

[hellomynameisleo]

I am using TrueNAS Scale and when transfering files/data to the share over SMB it crashes constantly and shows up as below image. I haven't played with the settings or anything so was wondering what would cause it to happen. I run TrueNAS in a VM and the host accesses the share through SMB without a need of a network card.

I have 8GB RAM and 16 cores of CPU assign to trueNAS Scale
verison: TrueNAS-SCALE-22.12.3.2

Any Ideas what can be causing this issue?

 

[HoneyBadger]

I have 8GB RAM and 16 cores of CPU assign to trueNAS Scale

This is a rather unusual proportion of RAM and CPU cores - TrueNAS would benefit from more RAM and likely fewer processors.

Since this is a VM, can you please describe your host hardware and software, as well as how you've presented the disks to the TrueNAS VM itself?

 

[NugentS]

This is a rather unusual proportion of RAM and CPU cores - TrueNAS would benefit from more RAM and likely fewer processors.

If we had an understatement of the year award - I would nominate @HoneyBadger for it for that statement

 

[hellomynameisleo]

This is a rather unusual proportion of RAM and CPU cores - TrueNAS would benefit from more RAM and likely fewer processors.

Since this is a VM, can you please describe your host hardware and software, as well as how you've presented the disks to the TrueNAS VM itself?

The host pc is using Hyper-V manager and passes through the hard disks through to the VM. It also gets weird behavior similar to this as well when I hit 100% CPU load on the host system, it crashes the trueNAS scale inside the VM.
TrueNAS scale currently has 20TB out of 28TB space used

host system spec:
64GB RAM
24 cores
2TB SSD
Windows 10

Also I get the below where it says blocking disabled state sometimes:

 

[NugentS]

May I suggest, just to try something reduce the number of cores assigned to TN to 4 and upgrade the memory to at least 16GB
Secondly as @HoneyBadger asked - we need to see your full hardware spec please.
Make and model of all devices, HDD, SSD Motherboard, CPU.
Type of memory (ECC or non ECC)
How the disks are attached to the motherboard

Then how the VM is defined in Hyper-V

 

[hellomynameisleo]

May I suggest, just to try something reduce the number of cores assigned to TN to 4 and upgrade the memory to at least 16GB
Secondly as @HoneyBadger asked - we need to see your full hardware spec please.
Make and model of all devices, HDD, SSD Motherboard, CPU.
Type of memory (ECC or non ECC)
How the disks are attached to the motherboard

Then how the VM is defined in Hyper-V

The VM I created as normal in hyper-v it created a VHD file and I setup trueNAS scale by booting into the ISO.

I use 2 14TB hard drives both seagate ironwolves attached to sata port in the motherboard and powered using the PSU
The host system SSD is an crucial MX500 2TB
Motherboard MSI Z690-A Pro DDR4 WIFI
13900k CPU
4x16GB DDR4 RAM non-ecc
Founders 3090 GPU
PSU EVGA G2 850W

I have also changed the trueNAS scale to have 16GB RAM and only 8 cores

 

[NugentS]

I doubt it even needs 8 cores - I assume (perhaps wrongly - see below) that you are using this (the TrueNAS) for storage only.
I can see a few issues:
1. 13th Gen CPU which is not fully supported by TrueNAS yet - not sure how Hyper-V effects this
2. Still too many CPU cores for a NAS vs memory. Try 4 - might help with the crashing when CPU load goes high.
3. Passing individual disks - and not the controller

Are you using containers on the NAS? If so - why - wouldn't it be better to run the containers on their own VM? If you are not using containers then I suggest switching off kubernetes

You say no network card - I assume you mean not physical network card and that you have a virtual network card attached to a virtual switch of some kind which other VM's can see. Can you access the NAS from outside the VM Host?

 

[hellomynameisleo]

I doubt it even needs 8 cores - I assume (perhaps wrongly - see below) that you are using this (the TrueNAS) for storage only.
I can see a few issues:
1. 13th Gen CPU which is not fully supported by TrueNAS yet - not sure how Hyper-V effects this
2. Still too many CPU cores for a NAS vs memory. Try 4 - might help with the crashing when CPU load goes high.
3. Passing individual disks - and not the controller

Are you using containers on the NAS? If so - why - wouldn't it be better to run the containers on their own VM? If you are not using containers then I suggest switching off kubernetes

You say no network card - I assume you mean not physical network card and that you have a virtual network card attached to a virtual switch of some kind which other VM's can see. Can you access the NAS from outside the VM Host?

I've set to 4 cores and 8 threads, I only use truenas for storage and have no contains or vms on truenas and the network virtual card is default made by hyper-v and not accessable outside of the host that I am using for the truenas VM.

its weird because it was working fine earlier but is there logs that I can help tell what caused the issue to cause the crash or servicies shutdown like on unraid?

 

[NugentS]

SMB is by default single threaded - so I suspect that even 4c/8t is still overkill - but much more sensible however.

This community has little knowledge of TrueNAS on hyper-v - it mostly reccomends ESX - others use proxmox - but if one theme is consistent is that the disks have to be on their own controller, passed through to the TN guest. Something that you haven't said you do.

Unfortunately getting information out of you is proving difficult - I will ask again "How the VM is defined in Hyper-V"?

I also suspect that you have Kubernetes turned on - I suggest turning it off by insetting the Apps pool (Apps/Settings/Unset Pool) - which I think works - not that it will help much I think - it may however by a point reduce where issues can occur.

One of the symptoms of not following the guides when it comes to virtualisation is wierd and random loss of data. It works till it doesn't. [Which doesn't help you]

Reading your messages - its seems to indcate that you have an issue with the floppy disk - and I bet you don't have one.

Do you have recent long smart tests run on every disk and can you post a zpool status -v please in codeblocks

 

[hellomynameisleo]

SMB is by default single threaded - so I suspect that even 4c/8t is still overkill - but much more sensible however.

This community has little knowledge of TrueNAS on hyper-v - it mostly reccomends ESX - others use proxmox - but if one theme is consistent is that the disks have to be on their own controller, passed through to the TN guest. Something that you haven't said you do.

Unfortunately getting information out of you is proving difficult - I will ask again "How the VM is defined in Hyper-V"?

I also suspect that you have Kubernetes turned on - I suggest turning it off by insetting the Apps pool (Apps/Settings/Unset Pool) - which I think works - not that it will help much I think - it may however by a point reduce where issues can occur.

One of the symptoms of not following the guides when it comes to virtualisation is wierd and random loss of data. It works till it doesn't. [Which doesn't help you]

Reading your messages - its seems to indcate that you have an issue with the floppy disk - and I bet you don't have one.

Do you have recent long smart tests run on every disk and can you post a zpool status -v please in codeblocks

what do you mean by how the VM is defined in hyper-v? though it seem it hasn't been crashing since I have changed the amount of RAM to 16GB and CPU 4c/8t. :)

Code:

  pool: TrueNAS
 state: ONLINE
  scan: scrub in progress since Tue Sep 26 06:30:02 2023
        17.2T scanned at 285M/s, 16.8T issued at 278M/s, 17.9T total
        0B repaired, 94.08% done, 01:06:31 to go
config:

        NAME                                    STATE     READ WRITE CKSUM
        TrueNAS                                 ONLINE       0     0     0
          9b7e0763-77b4-4795-82ee-b34035c2ae24  ONLINE       0     0     0
          625b29f1-c414-46bf-bce0-d889c324f8eb  ONLINE       0     0     0

errors: No known data errors

  pool: boot-pool
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:00:09 with 0 errors on Sat Sep 23 03:45:10 2023
config:

        NAME        STATE     READ WRITE CKSUM
        boot-pool   ONLINE       0     0     0
          sda3      ONLINE       0     0     0