sabnzbd / sickbeard / couchpotato pbi's

[bbddpp]

I feel like a bit of a ninny here but I'm throwing up the white flag on a permissions problem. SickBeard can't seem to create new show folders for me.

I am sure I'm missing something stupid here. I'd like just SABNZBD and SickBeard to use the same user, and ultimately I'd like it to be "guest" if possible instead of "media" because that's what my Windows shares seem to be liking. SickBeard seems to be having issues creating new show folders when I create them and I'm not quite sure why. I had always previously left the username/password fields blank in both SABNZBD and SickBeard. Do I now need to be putting freenas server passwords into these spots? So if I have a user called "guest", with password "guest", setup on freenas, and that's what my CIFS shares authenticate to, is that the same user/pass I need to use in both SickBeard and SABNZBD in order for permissions and folders to all create properly?

I have been scouring this and other threads for hours, and permissions setting seems to be the least thing covered and I think I am just missing something simple and obvious.

I think the key thing I see mentioned here is..."SickBeard is running as guest"...Or "SickBeard running as root"...etc...Same with SABnzbd. How do I control what user these servers are running as so they in turn can have the proper write permissions to the folders I want them to access?

These plugins ROCK, btw.

 

[Joshua Parker Ruehlig]

I feel like a bit of a ninny here but I'm throwing up the white flag on a permissions problem. SickBeard can't seem to create new show folders for me.

I am sure I'm missing something stupid here. I'd like just SABNZBD and SickBeard to use the same user, and ultimately I'd like it to be "guest" if possible instead of "media" because that's what my Windows shares seem to be liking. SickBeard seems to be having issues creating new show folders when I create them and I'm not quite sure why. I had always previously left the username/password fields blank in both SABNZBD and SickBeard. Do I now need to be putting freenas server passwords into these spots? So if I have a user called "guest", with password "guest", setup on freenas, and that's what my CIFS shares authenticate to, is that the same user/pass I need to use in both SickBeard and SABNZBD in order for permissions and folders to all create properly?

I have been scouring this and other threads for hours, and permissions setting seems to be the least thing covered and I think I am just missing something simple and obvious.

I think the key thing I see mentioned here is..."SickBeard is running as guest"...Or "SickBeard running as root"...etc...Same with SABnzbd. How do I control what user these servers are running as so they in turn can have the proper write permissions to the folders I want them to access?

These plugins ROCK, btw.

Thanks, BTW I hope to work on these soon. gonna add 'lazy librarian', gamez', and fix a few bugs I found in maraschino's shutdown. Also want to work out dependencies and make them a bit cleaner.

---

Permissions are confusing but let me do my best to make things clearer.
*The user/password for sabznbd/sickbeard are only for their webinterfaces and have no relation whatsoever to the underlying OS / filesystem.
*The plugins run as a user 'media' in the FreeNAS Plugins jail userland and I don't recommend changing that.
*This user media will have a certain uid number assigned to it when first being created, this is what is stamped onto the files/folders permissions in unix. the usernames don't span from the FreeNAS Plugins Jail userland to the FreeNAS Base OS userland, only the UIDs.
*CIFS runs in the FreeNAS Base OS userland and will not know/care about users in the FreeNAS jail, only relying on UIDs to figure out file permissions.
*I don't have experience setting up CIFS on FreeNAS but I know you can set what users can access a share and user they actually act as. But you also need to think about the underlying UID level below that.

I suggest you make the 'media' user (in the jail) owner of your media/download folders
you can then see what uid that gives to the files in your FreeNAS Base OS userland 'ls -l'
then in FreeNAS gui create a user with that uid (name it guest or whatever you want)
then you can set CIFS to use this user for that share (I haven't don't this through the freenas gui but have done it on regular CIFS on FreeBSD)

Goodluck

 

[bbddpp]

HUGELY helpful post, thanks so much. When you said about the second user, a light bulb went off...Of course, there's a different userbase for the JAIL than the FREENAS. That wasn't clicking but now it does. Now it makes sense why I'd want to manually create that media user from the ssh prompt while in the jail so that user exists for the plugins to use.

I have gotten much further now...Still a couple minor quibbles, and I'm going to keep scouring this and other threads, but if anyone has any quick answers to either of these two, you might help me get more sleep :)

1. SABnzbd isn't deleting what it downloads into the complete folder once it's finished processing and moves the file to its final home. So it's got a copy of whatever it brought down with the NZB (NFO files, the unzipped video file, etc) and then a second copy over in my actual share drive where I want it. Does the autoProcess TV script need a mod to tell it that it's ok to delete or is this a permissions thing? I've been having to ssh into /mnt/software/pbi/.../etc/data/download/tv/complete and doing a "rm -r *" every few days. I'd rather it just completely delete it and I'm pretty sure I have everything set right in the GUI for SAB unless I'm missing something.

2. Does Couch Potato have a similar script that I can give to SABnzbd to tell it what to do when it's grabbed something out of the "movies" category? I'd like to tell it where in my library to put the file and again to delete the stuff in the download work directory in the jail.

It's possible I missed a mount point or something in freenas that needs to be made to connect the jail to freenas that may be causing this too. But any suggestions would be welcome from anyone reading this thread who did this.

I really love this, it's been my favorite project of 2012 and the plugins have made freenas, for me, the best OS I've had the pleasure of trying in a long time. Such a powerful platform and I love these plugins!

 

[Karnage]

Unable to run extra_script: 13

Let me iterate this multiple times, to make it easier for other people in the future who might have the same problem to find it in google.

Unable to run extra_script: 13
Unable to run extra_script: 13
Unable to run extra_script: 13

OK, I am trying to run an extra script (end of post)that tells XBMC to update its library, and I am getting the following error from the debug.

Updatelibrary.py works when run from the command line. So its a permissioning problem, right? I have chown'd the script folder and the sickbeard and sab folders, but the error remains. This is the last step for SB on Freenas to have completely replaced my previous process.

Any help appreciated!

Code:

Dec-13 23:12:29 INFO     CP Server Thread-5 :: Unable to run extra_script: 13
Dec-13 23:12:29 DEBUG    CP Server Thread-5 :: Unknown value passed in, ignoring it:  (13:)
Dec-13 23:12:29 DEBUG    CP Server Thread-5 :: Absolute path to script: /usr/pbi/sickbeard-amd64/SickBeard/autoProcessTV/Updatelibrary.py
Dec-13 23:12:29 INFO     CP Server Thread-5 :: Executing command ['/usr/pbi/sickbeard-amd64/SickBeard/autoProcessTV/Updatelibrary.py', u'/usr/pbi/sabnzbd-amd64/data/mediamount/TV Shows/X/SeasonY/X.S05E50.mkv', u'/usr/pbi/sabnzbd-amd64/data/mediamount/Temp/Complete/tv/X/SeasonY/X.S05E50.mkv', '75644', '5', '50', '2010-11-12']

Here is the extra_script:

Code:

#!/usr/local/bin/ python

settings = {
    'hostname': '192.168.1.20',
    'port': '99',
    'username': 'SD',
    'password': 'FR'
}

http_address = 'http://%s:%s/jsonrpc' % (settings['hostname'], settings['port'])
username = settings['username']
password = settings['password']

try:
    import json
except ImportError:
    import simplejson as json
import urllib2, base64

class XBMCJSON:

    def __init__(self, server):
        self.server = server
        self.version = '2.0'

    def __call__(self, **kwargs):
        method = '.'.join(map(str, self.n))
        self.n = []
        return XBMCJSON.__dict__['Request'](self, method, kwargs)

    def __getattr__(self,name):
        if not self.__dict__.has_key('n'):
            self.n=[]
        self.n.append(name)
        return self

    def Request(self, method, kwargs):
        data = [{}]
        data[0]['method'] = method
        data[0]['params'] = kwargs
        data[0]['jsonrpc'] = self.version
        data[0]['id'] = 1

        data = json.JSONEncoder().encode(data)
        content_length = len(data)

        content = {
            'Content-Type': 'application/json',
            'Content-Length': content_length,
        }
   
        request = urllib2.Request(self.server, data, content)
        base64string = base64.encodestring('%s:%s' % (username, password)).replace('\n', '')
        request.add_header("Authorization", "Basic %s" % base64string)

        f = urllib2.urlopen(request)
        response = f.read()
        f.close()
        response = json.JSONDecoder().decode(response)

        try:
            return response[0]['result']
        except:
            return response[0]['error']


xbmc = XBMCJSON(http_address)
xbmc.VideoLibrary.Scan()

 

[Joshua Parker Ruehlig]

HUGELY helpful post, thanks so much. When you said about the second user, a light bulb went off...Of course, there's a different userbase for the JAIL than the FREENAS. That wasn't clicking but now it does. Now it makes sense why I'd want to manually create that media user from the ssh prompt while in the jail so that user exists for the plugins to use.

I have gotten much further now...Still a couple minor quibbles, and I'm going to keep scouring this and other threads, but if anyone has any quick answers to either of these two, you might help me get more sleep :)

1. SABnzbd isn't deleting what it downloads into the complete folder once it's finished processing and moves the file to its final home. So it's got a copy of whatever it brought down with the NZB (NFO files, the unzipped video file, etc) and then a second copy over in my actual share drive where I want it. Does the autoProcess TV script need a mod to tell it that it's ok to delete or is this a permissions thing? I've been having to ssh into /mnt/software/pbi/.../etc/data/download/tv/complete and doing a "rm -r *" every few days. I'd rather it just completely delete it and I'm pretty sure I have everything set right in the GUI for SAB unless I'm missing something.

2. Does Couch Potato have a similar script that I can give to SABnzbd to tell it what to do when it's grabbed something out of the "movies" category? I'd like to tell it where in my library to put the file and again to delete the stuff in the download work directory in the jail.

It's possible I missed a mount point or something in freenas that needs to be made to connect the jail to freenas that may be causing this too. But any suggestions would be welcome from anyone reading this thread who did this.

I really love this, it's been my favorite project of 2012 and the plugins have made freenas, for me, the best OS I've had the pleasure of trying in a long time. Such a powerful platform and I love these plugins!

Never have had a problem with sab not being able to delete. If sab can write the file it can likely delete it so it might be a problem with a setting. i have notice sab doesn't always cleanup sickbeard's files, but this is usually when sickbeard downloads a whole season pack and leaves some of the sample video files around.

 

[Joshua Parker Ruehlig]

Let me iterate this multiple times, to make it easier for other people in the future who might have the same problem to find it in google.

Unable to run extra_script: 13
Unable to run extra_script: 13
Unable to run extra_script: 13

OK, I am trying to run an extra script (end of post)that tells XBMC to update its library, and I am getting the following error from the debug.

Updatelibrary.py works when run from the command line. So its a permissioning problem, right? I have chown'd the script folder and the sickbeard and sab folders, but the error remains. This is the last step for SB on Freenas to have completely replaced my previous process.

Any help appreciated!

Code:

Dec-13 23:12:29 INFO     CP Server Thread-5 :: Unable to run extra_script: 13
Dec-13 23:12:29 DEBUG    CP Server Thread-5 :: Unknown value passed in, ignoring it:  (13:)
Dec-13 23:12:29 DEBUG    CP Server Thread-5 :: Absolute path to script: /usr/pbi/sickbeard-amd64/SickBeard/autoProcessTV/Updatelibrary.py
Dec-13 23:12:29 INFO     CP Server Thread-5 :: Executing command ['/usr/pbi/sickbeard-amd64/SickBeard/autoProcessTV/Updatelibrary.py', u'/usr/pbi/sabnzbd-amd64/data/mediamount/TV Shows/X/SeasonY/X.S05E50.mkv', u'/usr/pbi/sabnzbd-amd64/data/mediamount/Temp/Complete/tv/X/SeasonY/X.S05E50.mkv', '75644', '5', '50', '2010-11-12']

Here is the extra_script:

Code:

#!/usr/local/bin/ python

settings = {
    'hostname': '192.168.1.20',
    'port': '99',
    'username': 'SD',
    'password': 'FR'
}

http_address = 'http://%s:%s/jsonrpc' % (settings['hostname'], settings['port'])
username = settings['username']
password = settings['password']

try:
    import json
except ImportError:
    import simplejson as json
import urllib2, base64

class XBMCJSON:

    def __init__(self, server):
        self.server = server
        self.version = '2.0'

    def __call__(self, **kwargs):
        method = '.'.join(map(str, self.n))
        self.n = []
        return XBMCJSON.__dict__['Request'](self, method, kwargs)

    def __getattr__(self,name):
        if not self.__dict__.has_key('n'):
            self.n=[]
        self.n.append(name)
        return self

    def Request(self, method, kwargs):
        data = [{}]
        data[0]['method'] = method
        data[0]['params'] = kwargs
        data[0]['jsonrpc'] = self.version
        data[0]['id'] = 1

        data = json.JSONEncoder().encode(data)
        content_length = len(data)

        content = {
            'Content-Type': 'application/json',
            'Content-Length': content_length,
        }
   
        request = urllib2.Request(self.server, data, content)
        base64string = base64.encodestring('%s:%s' % (username, password)).replace('\n', '')
        request.add_header("Authorization", "Basic %s" % base64string)

        f = urllib2.urlopen(request)
        response = f.read()
        f.close()
        response = json.JSONDecoder().decode(response)

        try:
            return response[0]['result']
        except:
            return response[0]['error']


xbmc = XBMCJSON(http_address)
xbmc.VideoLibrary.Scan()

this script is for couchpotato? do you use couchpotato's notification feature (i don't know if that does a full update like you want though). i haven't had problems with that before.

 

[Karnage]

this script is for couchpotato? do you use couchpotato's notification feature (i don't know if that does a full update like you want though). i haven't had problems with that before.

sorry, its for sickbeard. "Update XBMC" doesnt work with Frodo (I am using RaspBMC on my Pi).

 

[Joshua Parker Ruehlig]

sorry, its for sickbeard. "Update XBMC" doesnt work with Frodo (I am using RaspBMC on my Pi).

Ahh, I see. actually I noticed that too (updated my openelec boxes to frodo 2 weeks ago). There's already a patch you can do to have the native sickbeard do the updating for the new frodo api, the developer doesn't want to add it into to sickbeard cause frodos not in production yet. I haven't patched and just use the xbmc autoupdate addon for now, waiting for sickbeard to merge the patch.

 

[Karnage]

Ahh, I see. actually I noticed that too (updated my openelec boxes to frodo 2 weeks ago). There's already a patch you can do to have the native sickbeard do the updating for the new frodo api, the developer doesn't want to add it into to sickbeard cause frodos not in production yet. I haven't patched and just use the xbmc autoupdate addon for now, waiting for sickbeard to merge the patch.

Cool. I would still like to get extra_scripts working though. Any suggestions?

 

[Joshua Parker Ruehlig]

Cool. I would still like to get extra_scripts working though. Any suggestions?

so you can execute the script from shell?
test 'su media' (changes your user to media user)
then execute the script and see if it works.

Goodluck

 

[Karnage]

so you can execute the script from shell?
test 'su media' (changes your user to media user)
then execute the script and see if it works.

Goodluck

Thanks for that, I will add it to the knowledge bank.

Edit: problem remains. I can run the script from cli as media though. Gonna try a couple of others things, stabs in the dark though.

 

[Joshua Parker Ruehlig]

Thanks for that, I will add it to the knowledge bank.

Edit: problem remains. I can run the script from cli as media though. Gonna try a couple of others things, stabs in the dark though.

A quick google brought this up.
http://sickbeard.com/forums/viewtopic.php?f=4&t=4303

 

[Karnage]

A quick google brought this up.
http://sickbeard.com/forums/viewtopic.php?f=4&t=4303

Yeah, I saw that too, he was getting a different error. I get the same error when I mis-specify the location of the script. I have specified the full path.

 

[sdspieg]

Can somebody please point me to a guide on how to backup the datafolder outside of the jail? Thanks!

-Stephan

 

[toddos]

Can somebody please point me to a guide on how to backup the datafolder outside of the jail? Thanks!

-Stephan

Assumptions:

• In your jail, you've mounted a dataset at /mnt/tank
• You've stopped services before doing this

To backup:

1. Open your jail -- jexec `jls jid` csh
2. Make a directory to store your backup -- mkdir /mnt/tank/sbdata
3. Copy everything -- cp -a /usr/pbi/sickbeard-`uname -m`/data/* /mnt/tank/sbdata
4. Repeat 2-3 for everything (sabnzbd, couchpotato, headphones), obviously changing the name of the target directory each time (cpdata, sabdata, hpdata following my sbdata naming convention, or name it whatever you want).

To restore:

1. Open your jail -- jexec `jls jid` csh
2. Copy everything from your backup back to the pbi data folder -- cp -a /mnt/tank/sbdata/* /usr/pbi/sickbeard-`uname -m`/data
3. Optional, make sure ownership of files is correct -- chown -R media:media /usb/pbi/sickbeard-`uname -m`/data
4. Repeat 2-3 for everything

 

[sdspieg]

Thanks toddos. Things are being backed up as we speak (quite slowly, but still steadily). Unfortunately, the "jexec `jls jid` csh" command didn't work for me. So I first created the backup directory, then went manually to my jail directory from the shell prompt and opened csh there. Then I went to my (completed) sabnzbd download dir and did did "cp -a * /mnt/HomeNAS/sbdata", and so now everything is being copied. Seems like it will take a few hours now. But is this an acceptable way too? Or did I miss sthg important by not doing the jexec `jls jid` csh?Thanks again!

 

[toddos]

Just to be clear, you used backtick (`) characters, not apostrophe (') characters, right? If that's not working, you can do it manually. Run "jls", find your jail id, and use that. For example, my jid is 1, so I would use jexec 1 csh.

Edit: Also, the data directories should not be large. Copying them should take seconds, not minutes or hours. If yours are taking hours to copy, you should investigate why.

Edit 2: And to answer your last question, the only thing that you must do from inside the jail is the chown at the end of the restore. I guess technically you could do that from outside the jail if you know the uid:gid values, but it's easier/safer/better to do it from within the jail.

 

[Joshua Parker Ruehlig]

Thanks toddos. Things are being backed up as we speak (quite slowly, but still steadily). Unfortunately, the "jexec `jls jid` csh" command didn't work for me. So I first created the backup directory, then went manually to my jail directory from the shell prompt and opened csh there. Then I went to my (completed) sabnzbd download dir and did did "cp -a * /mnt/HomeNAS/sbdata", and so now everything is being copied. Seems like it will take a few hours now. But is this an acceptable way too? Or did I miss sthg important by not doing the jexec `jls jid` csh?Thanks again!

if it's taking a few hours I think your not setting up your folders the suggested way.
most people make a separate dataset on freenas for their media files and use a nullfs mount to have it accessible in their jail. If your just backing up a plugins data folder (where no videos/music live) backup should be essentially instant. I'm guessing you didn't make a separate dataset for your media files and just have sabnzbd use the default folder living inside it's data folder which works but isn't as portable cause it doesn't partition your media files form your freebsd jail as much.

 

[Joshua Parker Ruehlig]

Version 1.1

CHANGES
- dropped sickbeard anime branch (you can always change your branch by changing you /usr/pbi/sickbeard-`uname -m`/Sickbeard folder to a different branch's source files)
- dropped 32bit support for now until I have a good way to test it
- removed all external links outside of the plugin's directory, everything is handled by changing the PATH and LD_LIBRARY_PATH environmental variables
- fixed up bugs in maraschino's shutdown code
- added ffmpeg to headphones for transcoding

NOTE
- sabnzbd downloads / headphones transcoding - needs to be tested

 

[sdspieg]

Joshua, you are right - I had my data folder in the default sabnzb folder. I now have everything backed up and running again (more or less). I found http://mybroadband.co.za/vb/showthr...SABnzbd-Sickbeard-and-Headphones-Setup-Guide? quite useful in setting up the folders the way they're supposed to - at least I hope so. My sickbeard is still not talking to my sabnzbd (OR, for that matter, not finding any shows that I am trying to add), so I can't check everything yet, but I manually loaded a nzb file, and that one DID end up nicely in the 'movies' directory outside of the jail, so I guess that's encouraging.