I have a couple of cron jobs that are set up to sync my remote server's files down to my local freenas server throughout the hour. I have a total of 4 jobs that run. One job for movies, one for tv shows, and one for everything else. I also have a fourth one that syncs files up to the remote server in case theres anything that needs to go up.
My issue is on one particular folder that is a part of my dataset. I have 4 WD Red 3TBs in RAIDZ1 (yes I know RAIDZ1 is not recommended, that is not what this post is about), and I have a 750GB dataset in that pool that holds Movies and TV shows (in a Movie and TV folder, respectively). My issue is that the Movies folder always works fine, but after the LFTP script that syncs files to the TV folder runs I notice that afterwards the TV folder will throw an error on my Windows machine saying "The file handle is invalid."
I thought at first maybe one of my drives was DOA since these drives are less than a week old, so I ran SMART tests and even a scrub on the pool and both came back clean. Whats strange is that if I force-refresh the permissions on that dataset the problem goes away and I am able to browse and use files in that directory again until the next time LFTP runs.
What about LFTP could be causing this? Each sync script runs 5 parallel downloads maxed a total of 2.5Mbps each (so 10 Mbps among all 4 scripts). Could it be that the scripts are running too close together and are causing too much load to be put on the server? Not sure how that could cause the issue I am having but I am running out of ideas here.
My next test will be to disable all of my cron jobs that run those scripts and see if the problem occurs again. From there I can possibly narrow it down to a particular script if that is really the issue, but I was wondering if anyone here had ever experienced anything like this before?
My issue is on one particular folder that is a part of my dataset. I have 4 WD Red 3TBs in RAIDZ1 (yes I know RAIDZ1 is not recommended, that is not what this post is about), and I have a 750GB dataset in that pool that holds Movies and TV shows (in a Movie and TV folder, respectively). My issue is that the Movies folder always works fine, but after the LFTP script that syncs files to the TV folder runs I notice that afterwards the TV folder will throw an error on my Windows machine saying "The file handle is invalid."
I thought at first maybe one of my drives was DOA since these drives are less than a week old, so I ran SMART tests and even a scrub on the pool and both came back clean. Whats strange is that if I force-refresh the permissions on that dataset the problem goes away and I am able to browse and use files in that directory again until the next time LFTP runs.
What about LFTP could be causing this? Each sync script runs 5 parallel downloads maxed a total of 2.5Mbps each (so 10 Mbps among all 4 scripts). Could it be that the scripts are running too close together and are causing too much load to be put on the server? Not sure how that could cause the issue I am having but I am running out of ideas here.
My next test will be to disable all of my cron jobs that run those scripts and see if the problem occurs again. From there I can possibly narrow it down to a particular script if that is really the issue, but I was wondering if anyone here had ever experienced anything like this before?