Trying to find missing library packages for software installed in jail

[keboose]

I went cheap when I bought an IP security camera for my home. It thankfully has a feature to save files in numbered sequence to an FTP server (running on my FreeNAS box) but it only saves images. If I want to save even 1 frame per second, I end up with nearly 100,000 files per day, and with block sizes being large, the files end up taking extremely large amounts of space on the NAS.

My idea was to run a script every 30 minutes or so that would take all images in the FTP folder and combine them into a video, then delete the images. The first step looks like it can be done with the package 'mencoder,' but when I install it into a 10.3-standard-release jail, I get errors about missing libraries:

The command, taken from the FreeBSD page on mplayer/mencoder:

Code:

mencoder "mf://*.jpg" -mf fps=25 -o output.avi -ovc lavc -lavcopts vcodec=mpeg4

The error:

Code:

Shared object "libpng16.so.16" not found, required by "mencoder"

In messing around, I managed to install that one, but then another pops up. I don't remember how, but I managed to make pkg list the missing dependancies, and there were at least a dozen (I did not copy down the list, unfortunately.) Would anyone be able to help me find and install the missing dependancies?

Thanks!

 

[nightshade00013]

https://www.freshports.org/multimedia/mencoder/

Take a look there for the dependencies, but from what I can glance at it has issues and may not work correctly if at all.

 

[keboose]

Thank you for the help! I went and installed all those dependancies with pkg, except for LAME, which I had to build to install (See the LAME page.)

After that, I was able to run the command in my first post, and it created an .AVI video that is exactly what I was expecting.

I'm going to build a script that runs every 30 minutes:
-Move all *.jpg's to a subfolder (so it won't mess with the camera as it's still writing)
-make a video with the images, then timestamp it.
-Delete all *.jpg's from the folder

I'll let you know if it works out.

 

[keboose]

I got it working, in a very "it's enough" kind of way. I set up a script that contains this (I have the FTP folder mounted at /cam, and the camera puts pictures in the /Images subdirectory):

Code:

#!/bin/sh                                                                                                                       
                                                                                                                               
cd /cam/Images                                                                                                                 
mv -f *.jpg /cam/Videos                                                                                                         
cd /cam/Videos                                                                                                                 
mencoder "mf://*.jpg" -mf fps=5 -o "$(date +"%Y%m%d_%H%M%S")_5x_speed.avi" -ovc lavc -lavcopts vcodec=mpeg4                     
rm -f *.jpg

cd /                                                                                                                              
find /cam/Videos -type f -mmin +129600 -size 10k -delete : 'Added in later. Removes files older than 90 days and larger than 10kB'

The camera saves 1 image per second, so the video plays back at 5x speed. The script moves all available *.jpg files to the Videos folder, makes the video, then deletes the images. I move the images in the script because I don't know how to only effect the images that were originally picked out in the 'mencoder' command, as the camera will continually write to the Images folder while the video is being made.

I set this script to run every 30 minutes. Two things that I would like to add, but don't know how: give video the timestamp of the first image's creation date, and delete videos older than, say, 3 months.

*Edit: Should add that in the crontab my command is "jexec [jail_id#] ./path/to/script.sh" and I had to redirect errors to /dev/null because there are some font loading errors that pop up every time the script is run:

Code:

Fontconfig failed to select a font. Trying without fontconfig...
New_Face failed. Maybe the font path is wrong.
Please supply the text font file (~/.mplayer/subfont.ttf).
subtitle font: load_sub_face failed.
Fontconfig failed to select a font. Trying without fontconfig...
New_Face failed. Maybe the font path is wrong.
Please supply the text font file (~/.mplayer/subfont.ttf).
subtitle font: load_sub_face failed.

[mjpeg @ 0x8049a3be0]error count: 268435458
[mjpeg @ 0x8049a3be0]error y=27 x=17
Skipping frame!

idk what those mean, but my command has nothing to do with fonts, and the videos turn out fine, so I'm going to ignore it unless someone mentions a way to fix it.

 

[nightshade00013]

It is probably trying to place an overlay on the video is my guess. If the output works then I would probably ignore it unless you start to see performance issues.

 

[keboose]

I fixed the font errors by installing mplayer and all 48 of its dependancies. Ran it once letting it email me the error and now it's:

Code:

[mjpeg @ 0x8049a3be0]overread 8
[mjpeg @ 0x8049a3be0]EOI missing, emulating

[mjpeg @ 0x8049a3be0]overread 8
[mjpeg @ 0x8049a3be0]EOI missing, emulating

[mjpeg @ 0x8049a3be0]error count: 268435459
[mjpeg @ 0x8049a3be0]error y=19 x=19

Skipping frame!

No more font errors, but it seems to have problems with one frame for some reason.

No overlay text by installing mplayer, but it would be really nice if I could put the filename of each jpg over its frame, then I wouldn't have to worry about how accurate the video's timestamp is.

 

[fracai]

I have used mencoder to assemble timelapse videos exactly like this with timestamp overlays. I don't have the commands on hand right now, but I should be able to grab those and post it tonight. One option that I think I considered is to just put together a very simple SRT file. I don't think that's the solution I've ended with.

What is the camera that you selected? I've been thinking about cameras myself and would much rather be dependent on my own system rather than some subscription cloud like Nest runs.

 

[fracai]

I found two methods that I used. First, here's a script that uses ImageMagick to annotate the images:

Code:

#!/bin/bash

SOURCE="$1"
OUTPUT="$2"

find "$SOURCE" -name "*.JPG" -print | while read FILE
do
    INPUT="$(basename "$FILE" .JPG)"
    convert -fill '#0008' -draw 'rectangle 5,128,114,145' -fill white -annotate +10+141 "$INPUT" "$SOURCE"/"$INPUT".JPG "$OUTPUT"/"$INPUT".JPG
done

You can then use the annotated images as the source for constructing your movie.

Another method uses python to construct an SRT file. The following script includes a comment with the command to use to burn the subtitles in to the movie when stitching the images. In my opinion, the SRT method looks better and it's faster with fewer temporary files:

Code:

#!/usr/bin/env python

import argparse
import datetime

import exifread
import os

# mencoder -nosound -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=21600000 -o output.avi -mf type=jpeg:fps=12 mf://input-images/*.jpg -vf scale=1440:1080 -sub out.srt -subfont-text-scale 3
# http://ubuntuforums.org/showthread.php?t=2029067&p=12128251#post12128251

if __name__ == '__main__':
    parser = argparse.ArgumentParser(
        description='output an SRT subtitle file containing EXIF timestamps from a collection of timelapse images')
    parser.add_argument('-p', '--prefix', dest='prefix', help='the prefix directory where images are located')
    parser.add_argument('-r', '--frame-rate', dest='rate', type=int, help='the frame rate used for the timelapse')

    args = parser.parse_args()

    timestamp = datetime.datetime(1901, 1, 1)
    timedelta = datetime.timedelta(milliseconds=1000 / args.rate)
    srtformat = "%H:%M:%S"
    count = 0
    for curfile in sorted(os.listdir(args.prefix)):
        filepath = os.path.join(args.prefix, curfile)
        timestamp = timestamp + timedelta
        timestamp2 = timestamp + timedelta
        count += 1
        with open(filepath, 'rb') as f:
            tags = exifread.process_file(f)
            values = {'count': count, 'ts1': timestamp.strftime(srtformat), 'ms1': timestamp.microsecond / 1000,
                      'ts2': timestamp2.strftime(srtformat), 'ms2': timestamp2.microsecond / 1000,
                      'dto': tags['EXIF DateTimeOriginal']}
            print '{count}\n{ts1},{ms1:03d} --> {ts2},{ms2:03d}\n{dto}\n'.format(**values)

 

[keboose]

Thank you for the suggestion. I merged the lines from your script into mine, so now it reads like so:

Code:

#!/bin/sh                                                                                                                          
                                                                                                                                  
SOURCE="$1"                                                                                                                        
OUTPUT="$2"                                                                                                                        
                                                                                                                                  
cd /cam/Images                                                                                                                    
mv -f *.jpg /cam/Videos                                                                                                            
cd /cam/Videos                                                                                                                    
                                                                                                                                  
find "$SOURCE" -name "*.JPG" -print | while read FILE                                                                              
do                                                                                                                                
        INPUT="$(basename "$FILE" .JPG)"                                                                                          
        convert -fill '#0008' -draw 'rectangle 5,128,114,145' -fill white -annotate +10+141 "$INPUT" "$SOURCE"/"$INPUT".JPG "$OUTPUT"/"$INPUT".JPG
done                                                                                                                              
                                                                                                                                  
mencoder "mf://*.jpg" -mf fps=5 -o "$(date +"%Y%m%d_%H%M%S")_5x_speed.avi" -ovc lavc -lavcopts vcodec=mpeg4                        
rm -f *.jpg                                                                                                                        
                                                                                                                                  
cd /                                                                                                                              
find /cam/Videos -type f -mmin +129600 -size 10k -delete        

I installed ImageMagick, but the end result is that the images do not have their file names burned on them. If I integrated the code incorrectly, please let me know. I'm not really familiar with the syntax, so I've been pretty much guessing up to this point.

I will also give the python solution a try, though I'll admit that I have no experience with python, so if there is anything beyond adding the code to a script and calling it, I'll probably need some pointers.

 

[fracai]

I should have added some additional explanation.

The bash script has references to $1 and $2. Those are arguments passed on the command line:

Code:

script.sh /path/to/source/ /path/to/destination/with/timestamp/

The convert command doesn't change the files in-place, though I think you could switch to using "mogrify" instead.

The python script wouldn't be integrated in to your bash script, you'd just call it to generate the SRT and then call mencoder (as you already are, but add the SRT argument) to build the movie.

I can put together a complete solution later today if you need it.

 

[keboose]

I can put together a complete solution later today if you need it.

I would appreciate that. I'm pretty much shooting blindly at this point. I called the script like so:

Code:

./cammerge.sh /cam/Images/timestamp /cam/Videos/

and I also changed the move command in my script to move the images to the timestamp folder first:

Code:

mv -f *.jpg /cam/Images/timestamp

but it doesn't seem to do the conversion. The script moves the images to the "timestamp" folder successfully, but the conversion never happens, and thus no images appear in the Videos folder, and the video is never created (mencoder stops the script with an error).

 

[fracai]

Here's a working script that uses ImageMagick to annotate the images. If the point size and annotation offset don't work for you, try adjusting those (the offset is +horizontal+vertical from the top left). mogrify modifies files in place, so it's simpler, but make sure it's doing what you want before you start using it regularly. Also, the python / SRT method would be quicker; I've included that example below. I think the only python package you'd need to install is exifread.

Code:

#!/bin/sh

cd /cam/Images
mv -f *.jpg /cam/Videos
cd /cam/Videos

find . -name "*.JPG" -print | while read FILE
do
    FILE_BASE_NAME="$(basename "$FILE" .JPG)"
    mogrify -fill white -pointsize 64 -stroke black -strokewidth 3 -annotate +10+70 "$FILE_BASE_NAME" "$FILE"
done

mencoder "mf://*.jpg" -mf fps=5 -o "$(date +"%Y%m%d_%H%M%S")_5x_speed.avi" -ovc lavc -lavcopts vcodec=mpeg4
rm -f *.jpg
cd /
find /cam/Videos -type f -mmin +129600 -size 10k -delete

Code:

#!/bin/sh

cd /cam/Images
mv -f *.jpg /cam/Videos
cd /cam/Videos

gen-srt.py --prefix /cam/Videos --frame-rate 5 > dates.srt
mencoder "mf://*.jpg" -mf fps=5 -o "$(date +"%Y%m%d_%H%M%S")_5x_speed.avi" -ovc lavc -lavcopts vcodec=mpeg4 -sub dates.srt -subfont-text-scale 3

rm -f *.jpg
cd /
find /cam/Videos -type f -mmin +129600 -size 10k -delete

 

[keboose]

I apologize for not replying sooner, I never got around to testing the scripts until today, along with the exifread python package. I saved your python script as gen-srt.py in the same directory as the scripts.

Unfortunately, neither of them are working. I think this is due to the fact that, earlier today, I updated my freenas install from 9.10.2-U1 to -U2. Running your first script gives the following output:

Code:

root@imgconverter:/cam # sh newcam.sh										 
: not found																	
: No such file or directory													
: No such file or directorydeos												
: No such file or directory													
: not found																	
newcam.sh: 16: Syntax error: end of file unexpected (expecting "do")

the python version is the same, but it also cannot find gen-srt.py. I update all the packages in the jail to make sure they are up to date, but the scripts still do not complete correctly. Also, my original script is still running perfectly fine every half-hour like I set it to in the scheduled tasks UI (I would expect it to also be broken right now.)

 

[fracai]

I can take a longer look another time, but my quick take is that maybe #!/bin/sh needs to change to #!/bin/bash

not being able to find the commands should be able to be solved by specifying the full path to each command or script

I can take another look at this later as well.