How to install Boinc with World Community Grid in a FreeNAS 9.3 Jail

[ChrisFor]

This tutorial shows how to get boinc with the world community grid running in a freenas 9.3 jail. You can manage this boinc-client via boinc-manager from any other computer.

1. Prepare the FreeNAS-Host
1.2. Prepare Kernel-Params (sysctl) via FreeNAS-Gui

Code:

variable: kern.elf32.fallback_brand
value: 3
type: sysctl

-> for linux compatibility, allows ELF32 application to run, needed for tasks from WorldCommunityGrid

Code:

variable: security.bsd.unprivileged_idprio
value: 1
type: sysctl

-> allowing boinc as a non-root user to set an idle priority​

By setting them at the host, they will automatically be set for every jail as well. You can check that, by performing a 'sysctl security.bsd.unprivileged_idprio' in the jail's root-shell.
As always when one sets params like this, everybody has to make the decision themselves, whether its worth the risk. Its not just some option one can fiddle with. I recommend to google every sysctl parameter before enabling it in the host. Also put something like 'For BOINC' in the comment of the tunable, so that you remember later what it was for. Besides, I tried to put them directly at the Jail, which prevented the Jail from starting. Don't have a clue why. I will check that again under 9.10.
(My boinc jail runs fine for quite a while now. Once I had tasks having problems with shmget() but I just aborted them. I am very refusive to set jail_sysvipc_allow=yes and so far its not needed I think)
Some further info on the matter:
http://netlab.linkpc.net/download/software/msd/tunings/FreeBSD/sysctl.conf
https://www.freebsd.org/doc/en/books/arch-handbook/jail-restrictions.html​

1.2. Add an Init-Script via FreeNAS-Gui

Code:

Command: kldload linux
When: PostInit

-> This loads the linux module into the kernel which is needed by WCG's tasks. Since we can't do it inside the Jail, it has to be done in the host.​

1.3. Create a standard Jail and name it 'boinc'. I left everything to default, maybe enabled DHCP.


2. Prepare the Jail
2.1. Shell into your Jail

In your freenas-rootshell perform

Code:

jls
jexec <Id> csh

If you want to login directly you need to enable sshd. Best is to follow the documentation on 'How to log into Jails via SSH' if you want to do this.​

2.1. Install Packages inside the Jail

Code:

pkg update
pkg upgrade
pkg audit -F

The last command performs a security check. Enjoy the good feeling of having results = 0

Code:

pkg install wget
pkg install mc
pkg install nano
pkg install linux_base-c6
pkg install boinc-client

Installing mc and nano is optional. I like mc for peeping around and you might prefer vi or ee over nano.
You can also put names in one row like pkg install mc nano wget ... Sometimes you should read and follow the remarks and hints coming up directly after installation, (e.g. if you install bash its mandatory) so I left them separatly that you don't miss them.

If I remember right, packages are recommended over ports. But we need to adjust a makefile later,
so ports are the way to go here for the boin-client unfortunatly. But since we want to avoid having everything compiled that boinc needs we just install the boinc-client package.
​

2.2. Prepare Ports Collection

Take a look at: https://www.freebsd.org/doc/handbook/ports-using.html

Code:

portsnap fetch extract

​

2.3. Install boinc-client

Code:

cd /usr/ports/net/boinc-client
nano -w Makefile

Find the first line and adjust it to look like the second one

Code:

CONFIGURE_ARGS= --disable-server
CONFIGURE_ARGS= --disable-server --with-boinc-platform=x86_64-pc-freebsd --with-boinc-alt-platform=i686-pc-linux-gnu

Same here:

Code:

OPTIONS_DEFAULT= CLIENT MANAGER X11 USER
OPTIONS_DEFAULT= CLIENT MANAGER X11 LINUX USER

When done hit Ctrl-X and 'y'.

Perform:

Code:

make BATCH=yes install clean
make deinstall
make BATCH=yes reinstall

The BATCH=yes will implicitly hit ok on every config-dialog. Leave it out, if you want to enable/disable certain features. Since I don't have a clue about them, I just left everything to default.
Enjoy the compile-show!
​

2.4. Prepare Linux compatibility

Add this line to /etc/sysctl.conf cause WCG needs 2.6.18 instead of 2.6.16

Code:

compat.linux.osrelease=2.6.18

​

2.5. Prepare /etc/rc.conf

Add these lines to /etc/rc.conf

Code:

linux_enable="YES"
boinc_enable="YES"
boinc_client_enable="YES"
boinc_client_flags="-allow_remote_gui_rpc"
boinc_flags="-allow_remote_gui_rpc -redirectio"

You can later (not now!) start and stop the boinc-client daemon via:

Code:

/usr/local/etc/rc.d/boinc-client start/stop

But we will start boinc as a process not as a daemon during the configuration, so that we can check what boinc complains about directly. Important is that the current path is '/var/db/boinc' when you do that, cause starting boinc just like that will create some files in the current directory and we only want those files in '/var/db/boinc'.
​

2.6. Install boinc_curses (for checking what the client is doing)

Code:

cd /usr/ports/net/boinc_curses
make install clean

​

2.7. Update packages again

For some (to me) unknown reason, packages need to be updated again. Also boinc-client needs to be recompiled. I love to know why. Anyhow, this step will also be needed, if you ever want to update packages in your boinc jail. (Make sure via 'top' that no boinc-process is running. Stop the daemon if necessary)

Code:

pkg update
pkg upgrade
cd /usr/ports/net/boinc-client
make BATCH=yes reinstall

​

2.8. Fix certificate problem

If we don't do this step, the following line will appear in the logs:
Scheduler request failed: Peer certificate cannot be authenticated with given CA certificates
I don't know whats wrong with the existing link to /usr/local/share/certs/ca-root-nss.crt. I just grabed one from certifie.com. If anybody has an idea about that, please share it.

Code:

cd /var/db/boinc
mv ca-bundle.crt ca-bundle.crt_previous
wget http://certifie.com/ca-bundle/ca-bundle.crt.txt
mv ca-bundle.crt.txt ca-bundle.crt

​

2.9. Load linux kernel module (avoiding freenas-restart)

Go into freenas root-shell and perform:

Code:

kldload linux

​

2.10. Make the first testrun

Code:

cd /var/db/boinc
boinc

​

2.11. Attach to a WCG project

Open another shell into the jail

Code:

cd /var/db/boinc
boinccmd --lookup_account www.worldcommunitygrid.org <YourLogin> <YourPassword>
boinccmd --project_attach www.worldcommunitygrid.org <YourAccountKey>

​

2.12. Adjust file permissions

Hit Ctrl-C on the boinc process

Code:

cd /var/db/boinc
chown -R boinc:boinc *

2.13. Roundup

Restart the boinc jail. Boinc-client will start as daemon now. To check that everything is fine you can start 'boinc_curses' and check if tasks are calculating. You can also run a 'pkg audit -F' to check if everything is fine.
​

3. Connect with boinc-manager

This file contains the password to connect with the boinc-manager:

Code:

cat /var/db/boinc/gui_rpc_auth.cfg

In Boinc-Manager you go to 'Advanced -> Select Computer' and enter your jails hostname or ip-address together with the password. If you run boinc at home I suggest that you change this very long key to something you can remember. (daemon restart required!)
I had the case that my instance was calculating only one task even though I have 8 HT-cores. So I aborted it via the manager and waited the 1:30m. After that 8 tasks were picked up and calculation started.

​

4. Maintenance
4.1 Adjust the jail after migration to FreeNAS 9.10 stable

Inside the boinc jail perform:

Code:

/usr/local/etc/rc.d/boinc-client stop
pkg update
pkg upgrade

By now our boinc-client has been replaced by the one from the packages, so we need to recompile it. We continue with:

Code:

cd /usr/ports/net/boinc-client
make BATCH=yes reinstall

This fails now with UNAME_r complaining about the major version number. So we need to add this line to /etc/make.conf:

Code:

OSVERSION=1003000

Now try again and everything should be fine so that you can end with:

Code:

/usr/local/etc/rc.d/boinc-client start
boinc_curses

​

5. Resources

The following links helped me on the journey:

https://forums.freenas.org/index.php?threads/boinc-installation-is-kicking-my-a.25707/
http://people.freebsd.org/~pav/boinc.html
http://sjoosten.nl/tag/jails.html (unfortunatly dead)
http://setiathome.berkeley.edu/forum_thread.php?id=78453
https://bugs.pcbsd.org/issues/7291
http://boinc.berkeley.edu/wiki/Installing_BOINC

Controlling Boinc Remotely:
http://boinc.berkeley.edu/wiki/Controlling_BOINC_remotely

Boinc-client params:
http://boinc.berkeley.edu/wiki/Boinccmd_tool

ca-bundle.crt
https://boinc.berkeley.edu/trac/wiki/Error/Scheduler request failed
http://certifie.com/ca-bundle/

Configuration
http://www.worldcommunitygrid.org/forums/wcg/viewthread?thread=25302#225698
https://www.freebsd.org/doc/handbook/ports-using.html

Linux Ports:
http://www.freshports.org/emulators/

Linux Compatibility
https://www.freebsd.org/doc/handbook/linuxemu-lbc-install.html

How to use BOINC
http://www.oook.cz/bsd/boinc_curses/

boinccmd and error -1
http://boinc.berkeley.edu/dev/forum_thread.php?id=9178

Jails and boinc
https://forums.freebsd.org/threads/14086/

UNAME_r and OSVERSION do not agree on major version number
https://forums.freebsd.org/threads/49295/
https://www.freebsd.org/doc/en/books/porters-handbook/book.html#versions
​

Any corrections, improvements and the like are most welcome!
Enjoy!

P.S. I migrated this jail successfully to 9.10 stable.

 

[cyberjock]

This is... amazing! Good job! Moved to the guides. :D

 

[joeschmuck]

I haven't tried this yet but I like the write-up. Nice job.

 

[ChrisFor]

Thanks! You're welcome! Love to hear from someone who made it through the end successfully.

 

[danb35]

It's working for me. A couple of things I'd change, though:

• You can group all the 'pkg install's together: 'pkg install nano mc ...'
• You can put portsnap fetch and portsnap extract together ('portsnap fetch extract'), and there's no reason to do an update immediately afterward
• The Batch=yes flag didn't work as described; I think the correct one might be 'batch=yes' (lower-case)

...and one question: I also run the BOINC client on my desktop machine, to the BOINC manager is already there. It's easy enough to enter the IP of my jail, but is there a way to make it save the RPC password? It's a little inconvenient to have to drop into the jail each time I want to check out what it's been doing.

 

[ChrisFor]

Thanks for your hints danb35, I adjusted the HowTo accordingly.
I googled this BATCH=yes param and it seems that everybody is using it uppercase.
About the boinc-manager remembering the RPC password: I think it has just not been implemented. The only thing I discovered is an MRU-List for the hostname edit-field. But you can change the content of gui_rpc_auth.cfg and after a boinc-daemon restart, make use of a password you can remember easily. (If you're in a company or university you better put that key as it is into some password safe tool like keepassx)
Depending on the DHCP-server you use, it should be possible to put in the name of the jail as hostname. (e.g this works with my Fritzbox 7390)

About the second pkg update/upgrade. At the very beginning of the tutorial I didn't do this trick of installing boinc-client via pkg. As a result my jail spent 40+ minutes compiling everything boinc-client depends on. During the second run of pkg update/upgrade a dozen packages got upgraded. With that trick of installing boinc-client via pkg before it was only one or two packages beeing updated (for some reason) but I don't remember which. Anyhow, I left this section in in case someone regularly performs pkg update/upgrade + pkg audit -F in their jails.

Thanks again!

 

[danb35]

About the second pkg update/upgrade.

I wasn't referring to that, I was referring to your having 'portsnap fetch extract' immediately followed by 'portsnap fetch update'. That's unnecessary. You'd do 'fetch update' at some later time to update the ports tree, but there's no reason at all to do it immediately after fetching and extracting the ports tree. It doesn't hurt anything, but it doesn't help anything either.

Thanks for the tip on changing the gui_rpc_auth password--that should make things easier.

 

[ChrisFor]

Thanks for the clarification. I removed the extra 'portsnap fetch update'.

 

[danb35]

BATCH in all caps worked as expected. Only other suggestion I'd make is to add the -w flag to nano, especially when editing the Makefile for boinc-client (so nano -w Makefile). The -w flag avoids line wrapping, which could be an issue for the CONFIGURE_ARGS changes.

 

[ChrisFor]

Done! Thanks again ...

 

[Green750one]

Thanks for the write-up - all set up and running perfectly!

 

[ChrisFor]

Thanks for the write-up - all set up and running perfectly!

Thanks! You're welcome!

 

[smokintbird]

I appreciate your taking the time to list this out so thoroughly. It saved me a ton of time figuring this out on my own!

 

[niktak11]

Does the "suspend boinc when CPU usage is above XX%" feature work? Even if the usage comes from another jail?

 

[ChrisFor]

Yes. I have it set to 60% and the CPU usage looks like down there in the pic. I don't 'feel' any speed-penalty because of boinc running in the background the whole time. We use our fileserver moderatly (Xeon E3-1220, 32GB-RAM, 6*2TB WDRed, RaidZ2). Once in a blue moon I need to copy a big 150GB vm to and fro and get constant 100MB/s both ways. So our GB Ethernet is nicely saturated.

Anhow, a jail can take as many cpu time as it wants. It would be much better to have it set to 100% inside the boinc jail and have the jail's CPU usage limited from outside the jail.
I assume (and hope) that this is going to be possible with bhyve under FreeNAS 10.
Then of course, setting up a boinc-vm with debian or ubuntu is going to be a piece of cake. Exciting times ahead!

 

[AgentJuno]

@ChrisFor

This guide worked very well even for a Linux noob like me! The only thing I had to do was reboot the server before creating the Jail. I did it without a restart and many of the commands didn't work out. Perhaps I just misread the post or that is simply a noob oversight on my part.

All in all, it is a great guide and I am happy that I can move BOINC off my main PC and to the server!

Cheers!

 

[ChrisFor]

Hey Juno,
thanks for the feedback! Glad to hear that your boinc-jail is up and running ...

 

[riesling]

It would be much better to have it set to 100% inside the boinc jail and have the jail's CPU usage limited from outside the jail.
I assume (and hope) that this is going to be possible with bhyve under FreeNAS 10.
Then of course, setting up a boinc-vm with debian or ubuntu is going to be a piece of cake. Exciting times ahead!

I see that FreeNAS is currently on version 9.10 and FreeNAS 10 is available as a Beta 2 release. I'm very interested in seeing the updated procedure for installing BOINC on FreeNAS 10, which I hope will be a lot closer to the painless, quick and easy process for installing BOINC on Linux Mint!