Torrents -> Usenet: What should I know?

[diedrichg]

I've gone through numerous settings changes over the years as I've learned new things so I can't explain why I still have +Delete set, I just know that with my settings, it all works.

You shouldn't have to manually intervene. That's the beauty of these programs.

 

[pirateghost]

I was able to download several working movies last night. The odd behavior is that they were all properly renamed in the original folder, a copy was made of the original, but the new copy was not renamed. I guess it's not a big deal to manually rename them after the fact, just want to figure out what is causing this.

make sure SAB is not doing any processing. It should just be categorizing the download as movie/tv/etc. the only processing it should do is download/categorize/extract. Have incompletes go to one directory, completes go to another, and let them categorize properly. (/media/downloads/incompletes/movie & /media/downloads/complete/movie, or something similar). You can then let couch potato take care of the rest with renaming and copying. It sounds like you have SAB downloading and copying and then couch potato is renaming the downloaded, but not moving it because SAB already moved it? I don't know, because I have not seen your actual configs.

Here is my couch potato settings (these have worked fine for years):

 

[Robert Trevellyan]

I can't explain why I still have +Delete set

It means "delete the junk left over after extracting the good stuff", so it's the default.

 

[Robert Trevellyan]

At the risk of stating the obvious, I just want to explain the process at a high level, in case you're confused. If you already understand this, just ignore it:

1. In the media client (Sonarr, CouchPotato etc), you request a download, either manually or automatically.
2. If the media client can find a release, it delivers a download request to SABnzbd via its API, in the appropriate category (tv, movies etc).
3. SAB begins downloading into the configured incomplete downloads folder.
4. When the download is complete, it moves the download into the configured completed downloads folder for the category (tv, movies etc).
5. All this time, the media client has been polling the completed downloads folder for the category. It knows which folder to poll either from querying the SAB API (Sonarr), or because you configured it (CP, Sick...).
6. Now that it sees a new completed download, the media client renames the media file and moves it to the folder where it will be played from, e.g. by Plex.

There are 5 folders involved here (assuming no anime):

• incomplete downloads
• completed tv downloads
• completed movie downloads
• playback client tv
• playback client movies

SAB only touches the first 3. Each media clients only touches 2 of the last 4. Playback clients only touch the last 2.

One thing that can go wrong is that if you let completed downloads for more than one SAB category go to the same folder, the wrong media client can grab a completed download, e.g. CP might grab a TV show and rename it as a movie.

 

[ChiknNutz]

Now having some trouble with Plex not recognizing the locations. There was a FreeNAS update that I installed which restarted the server. Now after the restart, Plex seems "broken" in regards to just the movie location. Seems something might be up with the jail storage, but it was all working just fine until the server restart. Cr@p!

 

[ChiknNutz]

Appreciate all the feedback. I am sure I will get it going eventually. I must sound like a spas, but I have been working with home Windows PCs for many years (since the days of DOS), so I do get the basics and some intermediate aspects. I've dabbled in some programming like C++ and VisualBasic. However, all this as related to FreeNAS is new to me. Really do appreciate all the help.

Here are screenshots of my configs for CP and SAB if that is of help for diagnosing or confirming what's going on.

 

[ChiknNutz]

Just checked the CP log and have these two error messages which I am not sure how to rectify:

FIXED: 12-05 11:44:38 ERROR [tato.core.plugins.renamer] Missing `cd` or `cd_nr` in the renamer. This will cause multi-file releases of being renamed to the same file. Please add it in the renamer settings. Force adding it for now.

12-05 11:44:39 ERROR [chpotato.core._base._core] OpenSSL installed but 0.15 is needed while 0.14 is installed. Run `pip install pyopenssl --upgrade`

 

[Robert Trevellyan]

What are your SAB category settings?

 

[ChiknNutz]

Ok so I got Plex working again, just deleted the Movie folder and created a new one.

Still seeing odd behavior with file renaming and deletion. I left for a few hours and came back to find that most, but not all, of the recent batch of movies had been deleted from the Download folder, but none were copied or moved to the new "/media/movies" folder" Again, thanks to snapshots, I have easily raised them back from the dead :)

Here is my SAB category settings.



Several of the folders have a file dropped in there by CP and say this:

This file is from CouchPotato
It has marked this release as "failed_rename"
This file hides the release from the renamer
Remove it if you want it to be renamed (again, or at least let it try again)

And some have this file "unknown":
This file is from CouchPotato
It has marked this release as "unknown"
This file hides the release from the renamer
Remove it if you want it to be renamed (again, or at least let it try again)

 

[ChiknNutz]

As a test, I deleted the file in one of the movie folders added by CouchPotato (which signifies not to do anything further with the movie). I let it do it's thing (looks like 1 hour is the smallest increment you can choose for the renaming function). I came back and the movie file itself is gone and there is another CP "failed_rename" file, yet the movie file was not moved or copied, seems it was totally deleted. I checked the CP log file again and find the following two entries.

12-05 16:45:18 ERROR [tato.core.plugins.renamer] Couldn't move file "/media/downloads/Ghost and the Darkness, The (1996)/The Ghost and the Darkness.mkv" to "/media/movies/Ghost and the Darkness, The (1996)/The Ghost and the Darkness_720p.mkv": Traceback (most recent call last):
File "/usr/pbi/couchpotato-amd64/share/couchpotato/CouchPotatoServer/couchpotato/core/plugins/renamer.py", line 842, in moveFile
shutil.copy(old, dest)
File "/usr/pbi/couchpotato-amd64/lib/python2.7/shutil.py", line 119, in copy
copyfile(src, dst)
File "/usr/pbi/couchpotato-amd64/lib/python2.7/shutil.py", line 82, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/media/downloads/Ghost and the Darkness, The (1996)/The Ghost and the Darkness.mkv'

12-05 16:45:18 ERROR [tato.core.plugins.renamer] Failed renaming the file "The Ghost and the Darkness.mkv" : Traceback (most recent call last):
File "/usr/pbi/couchpotato-amd64/share/couchpotato/CouchPotatoServer/couchpotato/core/plugins/renamer.py", line 625, in scan
self.moveFile(src, dst, use_default = not is_torrent or self.fileIsAdded(src, group))
File "/usr/pbi/couchpotato-amd64/share/couchpotato/CouchPotatoServer/couchpotato/core/plugins/renamer.py", line 842, in moveFile
shutil.copy(old, dest)
File "/usr/pbi/couchpotato-amd64/lib/python2.7/shutil.py", line 119, in copy
copyfile(src, dst)
File "/usr/pbi/couchpotato-amd64/lib/python2.7/shutil.py", line 82, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/media/downloads/Ghost and the Darkness, The (1996)/The Ghost and the Darkness.mkv'

I am not sure if this is a file/folder permission issue or something else. Currently, all the relevant folders use the "nobody:nogroup" ownership (not those in the jails, but those in the main dataset).

 

[Robert Trevellyan]

Are your folders spread across more than one dataset?

 

[ChiknNutz]

I do have two "media-type" datasets, but as far as CP and SAB are concerned, they only access one of them. I first created a media dataset to hold all of my photos, music, home videos and the like, where they are all stored now. I created this second one specifically for the purpose of CP and SAB. This more or may not be necessary, but at the beginning of this, I wanted to be able to do some testing and just wanted to be safe and separate them. Now for Plex, it accesses both of them and seems to be working just fine.

However, I would like to point out that my experience in this has shown that jails and jail storages are quite finicky (not just when it comes to permissions). I was pulling my hair out with how these were reacting. I would go to edit one by changing a path and then CP, SAB or Plex would not find the data (I saw this more noticeably with Plex). So what I ended up doing when I needed to change a jail storage, right or wrong, was as follows because I found just trying to EDIT them did not produce the desired result and led to odd and undesirable behavior.

1. Turn off the plug-in
2. Delete the jail storage
3. Recreate the jail storage
4. Turn the plug-in back on
5. Recreate the appropriate link within the plug-in GUI.

 

[Robert Trevellyan]

as far as CP and SAB are concerned, they only access one of them

I asked because mounting a dataset in a jail does not mount child datasets, so that could be a source of error.

I'm puzzled by the extent of your problems. I assume you've triple-checked ownership and permissions.

 

[ChiknNutz]

The latest download I tried *almost* did what it should. It downloaded and copied the movie to the correct folders. However, the incomplete folder is populated with several items that aren't going away. Further, I have tried to manually delete them to no avail. It looks like this folder has identical permissions as all the others (media:nogroup). Still slogging away, but it's improving!

Also, it seems like almost every movie I try to download is either AVI (most common I've seen so far) or WMV. There seem to be very few MKV files, but those have so far played every time. I have so far only found one other AVI file that will play in either Windows or Plex. All the others (I've probably tried a dozen or so by now) simply won't play. Further, I've tried to convert them using other Windows programs like Handbrake and the programs don't even recognize them or say they are invalid in some way. This ranges from old to new movies.

 

[Robert Trevellyan]

media:nogroup

My dataset and all subfolders are media:wheel, and media's primary group is wheel.

 

[ChiknNutz]

So I just caught onto what an "Indexer" is all about. I couldn't figure out why I kept getting all of these crap downloads. Now I get that you need to pony up for a "real" indexer, or bear with the limitations of the free accounts. I also discovered that a Usenet provider is also NOT an indexer. Wow, call me slow, but this is all brand new stuff for me. Again, would be nice to have some guides out there that go into this w/o having to just kinda figure it all out.

 

[pirateghost]

So I just caught onto what an "Indexer" is all about. I couldn't figure out why I kept getting all of these crap downloads. Now I get that you need to pony up for a "real" indexer, or bear with the limitations of the free accounts. I also discovered that a Usenet provider is also NOT an indexer. Wow, call me slow, but this is all brand new stuff for me. Again, would be nice to have some guides out there that go into this w/o having to just kinda figure it all out.

There's about a thousand guides out there spelling all this out in detail. I believe one such site was linked earlier.

Honestly, all of that is outside the context and intention of these forums. This one thread is the most any of it has been talked about here, in such detail, due to the nature of the legalities of such activities (depending on where you hail from).

Simply search 'fully automated htpc' in your favorite search engine.

 

[danb35]

Now I get that you need to pony up for a "real" indexer, or bear with the limitations of the free accounts.

I haven't paid a dime for usenet-crawler.com, and seem to get a very high percentage of good downloads.

 

[vilane]

Hey Chikn, sorry to see you've had so much trouble with getting this operational. Like many others have already mentioned, the issues really sound to be tied to renaming operations. I had to recreate my jail/plugin installs about 5 times the first time I was setting up FreeNAS because I didn't comprehend how the permissions and groups really worked, even with using a guide. From what you are saying about "failed to rename", it sounds extremely similar to my first experiences with using SickRage and Transmission. I botched it by not having the jail running/using the same permissions group/user across each application interaction. So essentially, my problem was I could download, it would show up, and the processing/renaming/copy/move operations would fail because the system didn't have the permission to copy/rename/move the source data to the permanent location.

maybe double check all of the ownership across that? If you are using Windows to try and administrate all of the permissions, I would really suggest maybe trying something like PuTTy to directly connect in to the FreeNAS with SSH and sticking to the native 'nix permissions hierarchy on the folders. It's cleaner for the plugins from my experience, but leaves you stranded with trying to administer and move files around from Windows. I had to learn to get outside my comfort zone there and use some command line in putty, which I recommend for you also if you aren't using it. understanding how to use commands like cp and rm manually can help you clean up a lot of the mess that can arise from getting your settings working correctly.

Best of luck to you in your continued fight to get this working smoothly.

 

[ChiknNutz]

I think I have this working finally. I believe the source of the issues came from the jails and storages. When I thought I was fixing them by editing them, it seems that really wasn't the case after all. It wasn't until I went thru the process I last mentioned about the jails that I finally got things straightened out. So far I believe it is working as intended.