After updating to 13.0, it seems the zfs pool is full and can't proceed. All of my regular processes that write to disk have quotas in place to leave plenty of space, which makes me think it was maybe part of the automated upgrade process / snapshots that filled it up?
I found a past thread indicating that once a volume is 100% full, it's essentially toast: https://www.truenas.com/community/threads/unable-to-destroy-snapshot-io-error.95452/post-664778
It does mention reconstituting from recent data, but I'm not sure there's a way to do that in place. Trying to remove snapshots or truncate/remove files fails (see below). If anyone has suggestions on how to proceed it'd be very appreciated. It's a relatively large pool so destroying and restoring from backup is not very desirable (though from what I've seen so far that seems to be all that's left besides swapping out for larger disks and expanding in place).
I found a past thread indicating that once a volume is 100% full, it's essentially toast: https://www.truenas.com/community/threads/unable-to-destroy-snapshot-io-error.95452/post-664778
It does mention reconstituting from recent data, but I'm not sure there's a way to do that in place. Trying to remove snapshots or truncate/remove files fails (see below). If anyone has suggestions on how to proceed it'd be very appreciated. It's a relatively large pool so destroying and restoring from backup is not very desirable (though from what I've seen so far that seems to be all that's left besides swapping out for larger disks and expanding in place).
Code:
freenas# zfs destroy vol0/iocage/jails/unifi@ioc_update_12.1-RELEASE-p13_2022-03-20_19-09-55 cannot destroy snapshots: I/O error
Code:
freenas# df -h ... vol0 18T 18T 0B 100% /mnt/vol0
Code:
freenas# echo "" > bigolfile -bash: bigolfile: No space left on device
Code:
freenas# rm SABnzbd_nzf_726atrpp rm: SABnzbd_nzf_726atrpp: No space left on device