ZFS is full and system is now unuseable

[dovaka]

for some reason or another my zfs pool is entirely full and no matter what i do i cant seem to get the system to work well enough to purge out some data. ive tried using cifs, afp, ftp and even command line to remove files but keep running into various issues with each. im not really sure what to try next.

 

[cyberjock]

Do some searching in the forums. There is a command that you have to use to wipe out a file, but I forget what it is. Unfortunately I don't have the time to search for you right now. Sorry.

 

[gpsguy]

Look at message #3 in a thread named: Disk full - can't delete any files


Sent from my phone

 

[dovaka]

is there any way to use this command on a directory?

 

[dovaka]

actually even when I try it on a file I get this

Code:

[root@studionas ~]# echo > /mnt/Main_Storage/Movies/9.avi                   
bash: /mnt/Main_Storage/Movies/9.avi: No space left on device               
[root@studionas ~]#  

 

[mpfusion]

actually even when I try it on a file I get this

Code:

[root@studionas ~]# echo > /mnt/Main_Storage/Movies/9.avi                 
bash: /mnt/Main_Storage/Movies/9.avi: No space left on device             
[root@studionas ~]#  

This actually write a newline to the file. If your filesystem is THAT full that it doesn't even take a single byte you can try to wipe the file instead of writing a newline to it:

Code:

: > /mnt/Main_Storage/Movies/9.avi 

 

[dovaka]

same result

Code:

[root@studionas ~]# : > /mnt/Main_Storage/Movies/9.avi                       
bash: /mnt/Main_Storage/Movies/9.avi: No space left on device               
[root@studionas ~]#    

 

[solarisguy]

If you have snapshots, stop new ones, and wait until the oldest one expires

 

[solarisguy]

When you regain control, read FreeNAS documentation and sections of http://www.freebsd.org/cgi/man.cgi?query=zfs related to quota, refquota, refreservation, reservation

 

[dovaka]

i had snap shots which is what i believe caused the disk to fill up and the retention time is something ridiculous so they arent expiring anytime soon

 

[solarisguy]

Step 1. stop snapshots

 

[dovaka]

did that yesterday

 

[solarisguy]

Step 2. execute zfs list -H -o name -t snapshot

and start deleting from that list manually (start with oldest one) zfs destroy snapshot_name

 

[dovaka]

i ran that command and it appears to be doing something as all the drives are showing activity but its been there for over an hour without posting anything

 

[solarisguy]

Let's wait then....

Do you, by any chance, have thousands of snapshots?

 

[dovaka]

it took one every 15 minutes for the last 2 years or so.

 

[solarisguy]

So you are on FreeNAS 8.x? The good news would be that you do not have .system dataset

 

[dovaka]

it started as 8 but it's now 9.2.1 release

 

[dovaka]

ok it justspit out a bunch of stuff like this

Code:

Main_Storage/bil@auto-20140513.0847-10y                                   
Main_Storage/bil@auto-20140513.1044-10y                                   
Main_Storage/bil@auto-20140513.1240-10y                                   
Main_Storage/bil@auto-20140513.1436-10y                                   
Main_Storage/bil@auto-20140513.1639-10y                                   
Main_Storage/bil@auto-20140513.1846-10y                                   
Main_Storage/bil@auto-20140513.2051-10y                                   
Main_Storage/bil@auto-20140513.2255-10y                                   
Main_Storage/bil@auto-20140514.0059-10y                                   
Main_Storage/bil@auto-20140514.0306-10y                                   
Main_Storage/bil@auto-20140514.0509-10y                                   
Main_Storage/bil@auto-20140514.0706-10y                                   
[root@studionas ~]#    

 

[dovaka]

which then returns this

Code:

[root@studionas ~]# : > /mnt/Main_Storage/bil@auto-20140512.0706-10y           
bash: /mnt/Main_Storage/bil@auto-20140512.0706-10y: No space left on device   
[root@studionas ~]#