11.2 Beta2 - system lock up when deleting files

[UDO]

Hello all,
HP Microserver Gen 7 N54L, 8GB, ZFS RaidZ1 4x 4TB WD Red. Running FreeNAS since V8. Home nas, on 24/7. and was updated from v8 to 9.x sometime in 2016.
Recently upgraded from 9.10 to 11.1 Beta 1 on 21 July. (didnt realise this was a beta.. o well . lost access to existing Jails and Plugins - late night upgrades hey!!).
Upgraded to 11.2 Beta1 on 29th July - introduction of new UI.
Upgraded to 11.2 Beta2 on 4th Aug. (yesterday)
v11 has introduced massive delays in boot times.. over 5mins from power on to prompt.

I have been cleaning up and reorganising data, moving, copying and migrating data off to external systems and disks, then deleting that data from nas.
This has been working well until 11.2 Beta 2.

Removing files and folders either via smb/cifs, or direct on nas via cmd rm/ rm -R is causing complete system lock up. - the web interface renders page not found, ssh disconnects, ping unavailable and on the direct console the host is completely unresponsive. -- A complete power cycle is required.

Is there a way to identify if the data is problematic or if beta 2 has a gremlin. ?
would booting back into env 11.1/11.2b1 work ?

Pool is healthty. 70% full. scrubs are clean. temps are good, smartd is happy.
thanks in advance.

 

[Chris Moore]

Is there a way to identify if the data is problematic or if beta 2 has a gremlin. ?
would booting back into env 11.1/11.2b1 work ?

Did you update your pool?
If you have not updated your pool with the new feature flags, you can certainly boot into your old boot environment.
If you updated, chances are not so good.

 

[Chris Moore]

Is there a way to identify if the data is problematic or if beta 2 has a gremlin. ?
would booting back into env 11.1/11.2b1 work ?

PS. You might want to submit a bug report, since it is a beta, so the developers can look into it.

 

[UDO]

Hi Chris, Unfortunately i DID update the pool only this morning though.
For clarity - this issue only began from 11.2b2 update, with older 11.1 zpool version.
i then updated the pool to lastest version hoping that might correct the locking, however the issue is still there.

...looks like its "eyes forward, don't look back"

the files in question are media (tv/movies) and i can remove one a time, and sometimes a few, or a small folder. If however a season is removed, it bombs.

 

[Chris Moore]

the files in question are media (tv/movies) and i can remove one a time, and sometimes a few, or a small folder. If however a season is removed, it bombs.

That doesn't sound very good. Did you open a ticket?

 

[UDO]

Thanks Chris, appreciate the concern.
bug = 40820

 

[diskdiddler]

Hello all,
HP Microserver G8 N54L
.

FYI, the N54L series is actually Gen7 not Gen8 (that's the Intel one which can only just fit 5 HDDs in the case, the Gen7 can fit 6 if you're tricky)

 

[UDO]

o thanks for the correction.. yes it is an Gen 7.. (gen 8 was new at the time.) - ill update post if i can.
i only have 4 disks. and booting off internal usb.

 

[UDO]

Whislt we wait for the bug to be appropriately assessed....

....Is there such as thing as bad data ?
could there been something wrong with these media files thats causing this issue ?
im certaining able to play the files, and skip through.. so they appear fine.

are there other checks or tools i can run ?

 

[Chris Moore]

The content of the files should not change your ability to delete them. You also said that you tried to delete the files directly from the command prompt. I am guessing that you are connected with SSH as root?
The root user should be able to delete any files, even if it would break the system.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[UDO]

Yes, I've tried both as named user and as root.
Scenarios include: over smb, via ssh cmd, or direclty at the console. the issue remains consistent, initial success, with eventual freeze.
i've further updated the bug with system info. (uname, version, zpool, camcontrol, lspci)

other than that , its fine :)

 

[diskdiddler]

Good luck, if you can isolate a particular quantity of files, or size, I can try to replicate with a similar dataset. (N54L with 8GB)

 

[Chris Moore]

other than that , its fine :)

Works good as long as you don't want to delete anything? That is kind of funny, but not. Sorry the update broke this.

i can remove one a time, and sometimes a few, or a small folder. If however a season is removed, it bombs.

I know it is a pain, but it doesn't sound totally unmanageable...

 

[Chris Moore]

Whislt we wait for the bug to be appropriately assessed....

@Ericloewe just made this post in another thread:
https://forums.freenas.org/index.ph...ow-locks-up-on-pool-import.69028/#post-474106
Contrary to my previous beliefs, you should be able to boot into your previous boot environment and still access the pool, even with the new feature flags.
If you don't mind giving that a try, it will allow us to know if the problem deleting the files is definitively linked to the version of FreeNAS.

 

[UDO]

@ DD,
im cleaning out a particular folder. its only a few directories down.
/mnt/{pool}/{dataset}/movies/show_name/series_num/files.ext

most are .avi, .mp4, .m4v, and various .png and .jpg thrown in. i haven't noted any .mkv in this group.
Depending on folder/series numbers vary from 1 to 24 files, and what i consider small in size. sub 200Mb each.
i've been trying rm -R one folder at a time. (series_num)

Previous to Beta 2, i was successfully running rm -R ../show_name/ to purge complete shows in bulk, where file sizes are between 500Mb and 2Gb each.
which was in the same pool and dataset, /mnt/{pool}/{dataset}/tv shows/show_name/series_num/files.ext (as expected)

I have other datasets with contain photos, general data, music and backups, (and jails that no-longer work) but i dont plan on deleting those :)

@ Chris,
I've had it bomb out on a single deletion as well.
its not critical for me right now. i hope its just an odd ball case, and perhaps the next update will magically resolve it.

if the bug review goes no where and the issue persists, ill try to reinstall on new usb and import pool.

cheers.

 

[UDO]

If you don't mind giving that a try, it will allow us to know if the problem deleting the files is definitively linked to the version of FreeNAS.

WOW good thinking 99

Successfully revert to 11.2 Beta 1 via GUI and Booted.
root@mininas:~ # cat /etc/version
FreeNAS-11.2-BETA1 (c8ef82608)


Complete success - I was able to remove:

Previously Un-tried:
tv show 1 - rm season at a time
season 1 - 8 .mkv files betwene 700Mb and 1.1Gb
season 2 - 9Gb 44 files very small to 2.2Gb
season 3 - 10Gb many files

tv show 2 - one rm -R command
.avi x12 90-200Mb

tv show 3 - one rm -R command
.mkv, .mp4, .avi 200Mb - 1.2Gb file sizes - 19Gb total


Previous troublesome Files
each show was one rm -R command.
show 4 - 8 Gb
show 5 - 1.1 Gb
show 6 - 3.5 Gb


Fantastic.

 

[Chris Moore]

Fantastic.

You should add a note about this to the bug you submitted. I bet they will like to know. Something didn't go properly between BETA1 and BETA2...

 

[pjr_atl]

I am having the exact same problem on an N41 microserver with 5 disks...

going to revert to Beta 1 now

 

[Chris Moore]

I am having the exact same problem on an N41 microserver with 5 disks...

going to revert to Beta 1 now

Please file a bug report first.

 

[pjr_atl]

I will if you can tell me how to revert to Beta1 :) I had already deleted that from by boot drive :)