Installing Shinobi (CCTV software) in an FN11 jail

[danb35]

A comment in another thread mentioned Shinobi as an open-source CCTV app which seems to compare very favorably with Zoneminder. Their docs don't say anything about installing under FreeBSD, but the relevant software sounded like it should work there as well.

So I just was playing around with it, and got it up and running in a jail on FN11-U4. Here are my very rough notes, modeled on the instructions at https://shinobi.video/docs/start#content-ubuntu--the-harder-way:

• Create a new jail, call it shinobi
• jls, find the number, jexec # tcsh
• Run the following:

Code:

pkg upgrade
pkg install node npm mariadb100-server llvm40 cairo opencv-core openjpeg tmux yasm gettext-tools binutils gmp texinfo help2man bison m4 texi2html cmake py27-Jinja2 py27-Babel py27-docutils py27-snowballstemmer rhash libarchive lzo2 ninja libass schroedinger orc libsoxr libogg libvorbis libva libvdpau libvpx webp giflib libx264 bash nano xvid nasm sekrit-twc-zimg gnutls nettle libtasn1 libunistring trousers
portsnap fetch extract
tmux new
cd /usr/ports/multimedia/ffmpeg
make install clean <- this will take a long time
choose options to match docs
sysrc mysql_enable=YES
service mysql-server start
/usr/local/bin/mysql_secure_installation
Set root password, remove anonymous users and test data
cd /usr/local
fetch https://github.com/ShinobiCCTV/Shinobi/tarball/master
tar zxf master
mv ShinobiCCTV-Shinobi-3bbdba7/ shinobi
chmod -R 755 shinobi
cd shinobi/sql
mysql -u root -p < user.sql
mysql -u root -p < framework.sql
mysql -u root -p ccio < default_data.sql
cd ..
cp conf.sample.json conf.json
cp super.sample.json super.json
npm install && npm install pm2 -g
pm2 start camera.js

The options selected for ffmpeg should match as closely as possible the compile flags given in the instructions at the link above.

The pkg install line includes all the dependencies that you'd otherwise have to configure during the build of ffmpeg. It's ugly and inefficient, and probably could be reduced quite a bit, but it does the job at least for now.

Once the pm2 start is completed, you can browse to http://(IP of jail):8080 and log in with ccio@m03.ca and password.

That's as far as I've gotten so far--I can log in and get the Shinobi web GUI. I haven't tried anything further yet.

TODO:

• work on adding cameras
• add storage to a suitable location, configure Shinobi to use that location for video storage
• Set to start up automatically on boot
• Add motion detection and object detection plugins
  
• ???

Edit: As fair warning, I know very little about this software, having only learned of it yesterday. This worked for me on my test system, but I can't promise anything on yours.

 

[m0nkey_]

Hey @danb35, good write up. Why not post it in the resources section? :)

 

[Redcoat]

The options selected for ffmpeg should match as closely as possible the compile flags given in the instructions at the link above.

I've installed but presently don't get to the web server. I matched the ffmpeg options as you indicated, which entailed turning off quite a few of the defaults - was this correct?

 

[danb35]

I didn't turn off anything for ffmpeg, just turned on things that looked like they matched the compile flags mentioned. Maybe I should have taken notes of which options I used--don't see a direct way to get that back out of the ports directory.

Edit: Here's what I think I changed from defaults. Checked these:

• AMR_NB
• AMR_WB
• ASS
• FRIBIDI
• LAME
• OPENJPEG
• OPUS
• SOXR
• SPEEX
• VIDSTAB
• WEBP
• ZIMG

Of the radio buttons at the bottom, I set these:

• LIBRTMP
• GNUTLS

When you're asked to select options for lame and x265, I just accepted the defaults.

 

[Redcoat]

I didn't turn off anything for ffmpeg, just turned on things that looked like they matched the compile flags mentioned. Maybe I should have taken notes of which options I used--don't see a direct way to get that back out of the ports directory.

OK, thx, I'll back track here and see what happens...

 

[danb35]

See my edit above.

 

[Redcoat]

Got it, thx.

 

[Redcoat]

Got it, thx.

An edit on your code sample:

Looks to me like this line "cp conf.sample.json" should read "cp conf.sample.json conf.json"

With that change I was able to get to the web server

 

[danb35]

Looks to me like this line "cp conf.sample.json" should read "cp conf.sample.json conf.json"

Oops. Yes, you're right. Fixed.

 

[White95supra]

@danb35 have you made any progress on this setup?

 

[BarnaM7]

Hi,
Thanks the post, it helps me to install it to jail.
Have you tried to install motion part? https://shinobi.video/docs/motion
How have you translate it to FreeBSD ?
thx
Barna

 

[danb35]

After getting this running, I got busy with other things and haven't spent much time on it. On the motion part, I haven't tried it, but a first guess would be to replace the "apt-get install" or "yum install" commands with "pkg install". Once the necessary binary packages are installed, the rest of the steps should work the same on FreeBSD as under any version of Linux.

 

[AfroUSA]

Hi,

I manage to add script to start pm2 on jail boot:

Code:

#!/bin/sh

# PROVIDE: shinobi
# REQUIRE: mysql sshd
# BEFORE:
# KEYWORD:
#
# Add the following line to /etc/rc.conf to enable script on boot:
#
#  shinobi_enable="YES"

. /etc/rc.subr

name="shinobi"
rcvar=shinobi_enable

start_cmd="${name}_start"
stop_cmd=":"

load_rc_config $name
: ${shinobi_enable:=no}
: ${shinobi_msg="HTTP server starts ..."}

shinobi_start(){
	echo $PATH
	export PATH=$PATH:/usr/local/bin/
	echo $PATH

	### Start server with PM2 ###
	/usr/local/bin/pm2 start /usr/local/shinobi/camera.js
	echo "$shinobi_msg"
}

run_rc_command "$1"

Please remember to add shinobi_enable="YES" to /etc/rc.conf.
cd /usr/local/etc/rc.d/
chmod +x shinobi
You can check if this works by typing /usr/local/etc/rc.d/shinobi onestart
Should have this:

root@Shinobi:/usr/local/etc/rc.d # /usr/local/etc/rc.d/shinobi onestart
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin:/usr/local/bin/
[PM2][ERROR] Script already launched, add -f option to force re-execution
┌──────────┬────┬──────┬───────┬────────┬─────────┬────────┬─────┬────────────┬──────┬──────────┐

│ App name │ id │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ user │ watching │

├──────────┼────┼──────┼───────┼────────┼─────────┼────────┼─────┼────────────┼──────┼──────────┤

│ camera │ 0 │ fork │ 24790 │ online │ 0 │ 43m │ 0% │ 105.9 MB │ root │ disabled │

└──────────┴────┴──────┴───────┴────────┴─────────┴────────┴─────┴────────────┴──────┴──────────┘
Use `pm2 show <id|name>` to get more details about an app
HTTP server starts ...


TODO:

• ???

Thanks for great tutorial, all is working well, till...
Sometimes after login to WebUI of Shinobi, I'm getting stuck on blank page. Nothing is getting loaded, so i need to flush MySQL database (delete files from /var/db/mysql) and start again from building new database.
First need to stop jail, than delete all files from /var/db/mysql. Starting my jail again and than jexec 3 tcsh to jail:

Code:

cd /usr/local/shinobi/sql
service mysql-server start
/usr/local/bin/mysql_secure_installation
mysql -u root -p < user.sql
mysql -u root -p < framework.sql
mysql -u root -p ccio < default_data.sql
cd ..
cp conf.sample.json conf.json
cp super.sample.json super.json
pm2 start camera.js

Not sure what is causing this?

Update: 19.Jan.2018
Shinobi is still running (3 days now), looks like messing with conf files makes some problems with mysql database.

 

[BarnaM7]

Have you look at the log ? pm2 logs camera --lines xxx
I have used mariadb, that was compiled from source like the ffmpeg without any issue. (11.0 jail problem)
But I need the motion too, I have installed, but met zlib issue after start. There is no solution for that at this moment.
So at this moment I use an debian vm for Shinobi.

 

[wywywywy]

How are you guys getting on with this?

So overall is Shinobi any good?

I have ZoneMinder set up in a jail but could really do with something more modern.

 

[raidflex]

Hi,

I manage to add script to start pm2 on jail boot:

Code:

#!/bin/sh

# PROVIDE: shinobi
# REQUIRE: mysql sshd
# BEFORE:
# KEYWORD:
#
# Add the following line to /etc/rc.conf to enable script on boot:
#
#  shinobi_enable="YES"

. /etc/rc.subr

name="shinobi"
rcvar=shinobi_enable

start_cmd="${name}_start"
stop_cmd=":"

load_rc_config $name
: ${shinobi_enable:=no}
: ${shinobi_msg="HTTP server starts ..."}

shinobi_start(){
	echo $PATH
	export PATH=$PATH:/usr/local/bin/
	echo $PATH

	### Start server with PM2 ###
	/usr/local/bin/pm2 start /usr/local/shinobi/camera.js
	echo "$shinobi_msg"
}

run_rc_command "$1"

Please remember to add shinobi_enable="YES" to /etc/rc.conf.
You can check if this works by typing /usr/local/etc/rc.d/shinobi onestart
Should have this:

root@Shinobi:/usr/local/etc/rc.d # /usr/local/etc/rc.d/shinobi onestart
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin:/usr/local/bin/
[PM2][ERROR] Script already launched, add -f option to force re-execution
┌──────────┬────┬──────┬───────┬────────┬─────────┬────────┬─────┬────────────┬──────┬──────────┐

│ App name │ id │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ user │ watching │

├──────────┼────┼──────┼───────┼────────┼─────────┼────────┼─────┼────────────┼──────┼──────────┤

│ camera │ 0 │ fork │ 24790 │ online │ 0 │ 43m │ 0% │ 105.9 MB │ root │ disabled │

└──────────┴────┴──────┴───────┴────────┴─────────┴────────┴─────┴────────────┴──────┴──────────┘
Use `pm2 show <id|name>` to get more details about an app
HTTP server starts ...





Thanks for great tutorial, all is working well, till...
Sometimes after login to WebUI of Shinobi, I'm getting stuck on blank page. Nothing is getting loaded, so I need to flush MySQL database (delete files from /var/db/mysql) and start again from building new database.
First need to stop jail, than delete all files from /var/db/mysql. Starting my jail again and than jexec 3 tcsh to jail:

Code:

cd /usr/local/shinobi/sql
service mysql-server start
/usr/local/bin/mysql_secure_installation
mysql -u root -p < user.sql
mysql -u root -p < framework.sql
mysql -u root -p ccio < default_data.sql
cd ..
cp conf.sample.json conf.json
cp super.sample.json super.json
pm2 start camera.js

Not sure what is causing this?

Update: 19.Jan.2018
Shinobi is still running (3 days now), looks like messing with conf files makes some problems with mysql database.

I had this same problem with Shinobi loading a blank page after logging in. I found that the startup script you provided was the cause, not sure why though. If I do not use the script and start the server manually its fine.

 

[Nick2253]

I stumbled across this post today when it hit the top of "New Posts", and it really got me excited. I've been using ZoneMinder for years, but ZoneMinder is getting extremely long in the tooth, and either some serious development work, or a new program was needed. And here we have Shinobi.

However, then I started reading about the project and its short history, and my heart fell. It sounds like Moe, the original creator of Shinobi, started with the right intentions. He set out to create a better ZoneMinder to power a hosted service providing a CCTV platform. And Shinobi was born.

Unfortunately, Moe has clearly demonstrated that he doesn't know anything about open source software. About six months after he started, he split the project into a Pro and CE edition, re-licensing the code to Creative Commons NC for the Pro version. He segregated the support network into a CE chat and a private, invite-only Pro chat (and made explicitly clear that no developers would provide CE support). And then immediately shut down any community pull requests to the CE edition (there have only been 6 commits since its been relicensed). Instead of using the community edition as the platform to develop a stable, monetizeable product, he's using the Pro version to control everything, and only feeds down to the CE whenever a "critical release" happens.

The Shinobi community, though small, was understandably upset by this move. They had contributed (whether in code, support time, or publicity) in good faith, and Moe had pulled the rug out from under them. His response to the issue was basically "F you, it's my code, read the contribution guidelines." His blog posts are full of anecdotes about "people wasting [his] time" and "users ... taking earnings". His closing remark as justification for this whole fiasco is a bitter "I didn't pick up a fork and start editing it, I started from line 1."

I don't begrudge anyone the ability to make money on software. Developing code is not an easy job, and I think that you should be able to sell your efforts. However, I do get heartburn about people marketing themselves as "the open source CCTV solution" when they are anything but. And what's really unfortunate here, is that Moe started down the right path. He recognized many of the failings of the ZoneMinder project, and is making a worthy replacement. His early blog posts include lines like "When someone cares about something they will give their time and effort for that. I care about Shinobi, that is why it has progressed so quickly." My favorite is from his "The Aim of Shinobi" article:

I named it how I saw it. Shinobi (Ninja). One who watches from the shadows. Then threw it on Github and began my journey. I never really expected to make a replacement for the current self hosted solutions. It sort of just happened.

But instead of recognizing the power of the community to bolster his product, he saw the community as people out to take advantage of him. And, ironically, he's making one of the biggest mistakes that hurt ZoneMinder over the years: in his own words, "developer arrogance". He could have gone down many different paths to monetize his software, but when his "I'm going to rely on community donations to fund my life" didn't pan out, he turned his back on the community. And to make matters worse, he still solicits community donations on his website.

I'm really disappointed, because I think Shinobi was just the project that the CCTV community needed. If the CE edition is updated during the next "critical release" (whatever that means), perhaps there is hope for Shinobi as a real open source project. But with the direction the project is heading now, I'm not holding out hope.

 

[LandMineHare]

Anyone able to get this working? I just get "unsupported platform" like, 16 times in an error log file.

 

[AfroUSA]

Anyone able to get this working? I just get "unsupported platform" like, 16 times in an error log file.

Have the same error when starting PM2 with camera.js :(

Code:

0|camera	| Unsupported platform.
PM2 App	 | [camera] with id [0] and pid [53696], exited with code [1] via signal [SIGINT]
PM2  Script |  /usr/local/shinobi/camera.js had too many unstable restarts (16). Stopped. "errored"

Errors while npm install && npm install pm2 -g

Code:

node-pre-gyp ERR! stack	 at maybeClose (internal/child_process.js:936:16)														
node-pre-gyp ERR! stack	 at Process.ChildProcess._handle.onexit (internal/child_process.js:220:5)								
node-pre-gyp ERR! System FreeBSD 11.1-STABLE																						
node-pre-gyp ERR! command "/usr/local/bin/node" "/usr/local/shinobi/node_modules/sqlite3/node_modules/.bin/node-pre-gyp" "install" "
--fallback-to-build"																												
node-pre-gyp ERR! cwd /usr/local/shinobi/node_modules/sqlite3																	   
node-pre-gyp ERR! node -v v9.10.0																								   
node-pre-gyp ERR! node-pre-gyp -v v0.6.38																						   
node-pre-gyp ERR! not ok																											
Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to
-build --module=/usr/local/shinobi/node_modules/sqlite3/lib/binding/node-v59-freebsd-x64/node_sqlite3.node --module_name=node_sqlite
3 --module_path=/usr/local/shinobi/node_modules/sqlite3/lib/binding/node-v59-freebsd-x64' (1)									   
npm WARN shinobi@1.0.37 license should be a valid SPDX license expression														   
																																	
npm ERR! code ELIFECYCLE																											
npm ERR! errno 1																													
npm ERR! sqlite3@3.1.13 install: `node-pre-gyp install --fallback-to-build`														 
npm ERR! Exit status 1																											 
npm ERR!																															
npm ERR! Failed at the sqlite3@3.1.13 install script.																			   
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.								 
																																	
npm ERR! A complete log of this run can be found in:																				
npm ERR!	 /root/.npm/_logs/2018-04-03T22_58_08_726Z-debug.log

 

[jonah.aberle]

I stumbled across this post today when it hit the top of "New Posts", and it really got me excited. I've been using ZoneMinder for years, but ZoneMinder is getting extremely long in the tooth, and either some serious development work, or a new program was needed. And here we have Shinobi.

However, then I started reading about the project and its short history, and my heart fell. It sounds like Moe, the original creator of Shinobi, started with the right intentions. He set out to create a better ZoneMinder to power a hosted service providing a CCTV platform. And Shinobi was born.

Unfortunately, Moe has clearly demonstrated that he doesn't know anything about open source software. About six months after he started, he split the project into a Pro and CE edition, re-licensing the code to Creative Commons NC for the Pro version. He segregated the support network into a CE chat and a private, invite-only Pro chat (and made explicitly clear that no developers would provide CE support). And then immediately shut down any community pull requests to the CE edition (there have only been 6 commits since its been relicensed). Instead of using the community edition as the platform to develop a stable, monetizeable product, he's using the Pro version to control everything, and only feeds down to the CE whenever a "critical release" happens.

The Shinobi community, though small, was understandably upset by this move. They had contributed (whether in code, support time, or publicity) in good faith, and Moe had pulled the rug out from under them. His response to the issue was basically "F you, it's my code, read the contribution guidelines." His blog posts are full of anecdotes about "people wasting [his] time" and "users ... taking earnings". His closing remark as justification for this whole fiasco is a bitter "I didn't pick up a fork and start editing it, I started from line 1."

I don't begrudge anyone the ability to make money on software. Developing code is not an easy job, and I think that you should be able to sell your efforts. However, I do get heartburn about people marketing themselves as "the open source CCTV solution" when they are anything but. And what's really unfortunate here, is that Moe started down the right path. He recognized many of the failings of the ZoneMinder project, and is making a worthy replacement. His early blog posts include lines like "When someone cares about something they will give their time and effort for that. I care about Shinobi, that is why it has progressed so quickly." My favorite is from his "The Aim of Shinobi" article:

But instead of recognizing the power of the community to bolster his product, he saw the community as people out to take advantage of him. And, ironically, he's making one of the biggest mistakes that hurt ZoneMinder over the years: in his own words, "developer arrogance". He could have gone down many different paths to monetize his software, but when his "I'm going to rely on community donations to fund my life" didn't pan out, he turned his back on the community. And to make matters worse, he still solicits community donations on his website.

I'm really disappointed, because I think Shinobi was just the project that the CCTV community needed. If the CE edition is updated during the next "critical release" (whatever that means), perhaps there is hope for Shinobi as a real open source project. But with the direction the project is heading now, I'm not holding out hope.

Very interesting info Nick. I've been running Zoneminder for a few years but after moving it into my FreeNAS jail, it's wildly unstable. Crashes and stays down every 3 or 4 hours... Was very much hoping for an open source upgrade to run in a ZM jail. Are there any options what-so-ever?