Files missing but space still consumed

[Liviu D]

Hello guys,

I'm using freenas for more than an year without any problem, but 2 days ago i've had an big problem.

i've found one of my folders on the main dataset empty, but the space ocupied reported by graphs is the same.
Unfortunately i do not have snapshots and i really need the files back.

My hardware is:
Intel SR1500 family server with 2 x Quad 3ghz, 16 Gb ddr2 fbdimm
Disk controller: LSI Logic / Symbios Logic SAS1068 PCI-X Fusion-MPT SAS
Storage Dell MD1000 with 5 x 1tb and 10 x 500gb enterprise disks.

After scrub i have :

pool: md1000
state: ONLINE
scan: scrub repaired 0 in 38h29m with 0 errors on Fri Feb 19 04:43:28 2016

config:

NAME STATE READ WRITE CKSUM
md1000 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
gptid/929e8478-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/931a8125-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/9387a433-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/940f761c-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/94a51d5b-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/9538bd77-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/95cb2563-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/965f5452-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/96e214a8-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
gptid/976e8f74-79a5-11e4-b20d-0015174f551e ONLINE 0 0 0
raidz2-1 ONLINE 0 0 0
gptid/d95f055b-143f-11e5-8281-0015174f551e ONLINE 0 0 0
gptid/c77862f8-13fe-11e5-bed5-0015174f551e ONLINE 0 0 0
gptid/2216f494-11ae-11e5-9c96-0015174f551e ONLINE 0 0 0
gptid/b5246993-ce20-11e5-95d5-0015174f7306 ONLINE 0 0 0



errors: No known data errors

Everything seems to be ok, but i cannot see my files.

Please advise.

Thanks

 

[dlavigne]

How were you accessing the files? eg over the network using a share? If so, what type of share? Is the service for that share still running? Can you connect using the share?

 

[SweetAndLow]

Did you do a reboot recently? Maybe you created a dataset over a folder and after the reboot the dataset got mounted. Try to unmount the dataset that you think your data is in

Code:

zfs unmount tank/media

 

[Liviu D]

How were you accessing the files? eg over the network using a share? If so, what type of share? Is the service for that share still running? Can you connect using the share?

I'm using only nfs. I have 4 nfs exports for 4 independent locations. The other 3 exported folders are still in productions. Anyway the problem persists on the NAS console.

 

[Liviu D]

Did you do a reboot recently? Maybe you created a dataset over a folder and after the reboot the dataset got mounted. Try to unmount the dataset that you think your data is in

Code:

zfs unmount tank/media

Yes the nas rebooted 1 day before the problem appeared because an power outage, i only have 2 datasets on other locations.

But i'm afraid i omitted something important. On the day the problem manifests, one of my applications what works with the files on has, and is written in php, reported many errors.
The error was "mkdir(); Too many links"

As i find on the zfs docs, the subfolders limit is 2 at 48, but is impossible i've reached this limit, i had maximum 3 mil folders in that location.

 

[Liviu D]

Hello,

I have an new development. After reading some posts, i want to try an detach/attach on the zfs pool.
But first i want to make an backup.
So i connected 3 new hard drives to the server sata ports, and i've make an new pool.
Now i'm trying to copy data with rsync to the backup pool. The problem is the freenas is rebooting with now error messages.

Everything looks good on dmesg, no alerts, but is rebooting.

I double checked the server hardware logs, i have no memory errors. Smart tests on all disks are ok and the LSI controller firmware reports everything ok.

So in this moment i cannot do my backup because i have mostly small files (images, html, txt) and rsync take an log time to index data.