Guide: How to Build FreeNAS (Part 2: Install and Build FreeNAS)

[joeschmuck]

Install the FreeNAS development software
**** NOTE: The repository location has changed since FreeNAS 9.x so these instructions will not work as indicated however they can still be a guide ****

Did you copy your .vmdk file before you started this section?

1a) To do all future work we will use SSH via Putty because it is much easier to run from a terminal than the VM directly. You can minimize the VM to get it off the desktop. For my DHCP my IP address is 192.168.1.29, yours will likely be different. You should get a security alert, say Yes.


1b) (Screen Shot 2)


2) Now log on as yourself or Joe in my case. Next type “su” and hit Enter and insert the root password. This will make you root.

3) Time to install required packages. Just copy from here and in the Putty window right click your mouse and it will paste the entire text. The time to add these packages will depend on your internet speed.

Code:

pkg_add -r subversion
pkg_add -r cdrtools
pkg_add -r python

4) Type “rehash” and Enter.

5) Type “mkdir /usr/local/freenas” and Enter.

6) Type “reboot” or “shutdown –p now”. The new files installed need to be loaded via rebooting before you can build.

7) If you haven’t heeded my advice to make a back of the VM hard drive file, you need to make a copy here. The VM hard drive file size can get crazy and the easiest way to recover is just restore the VM file .vmdk then continue. To make a copy you need to shut down the VM and then copy the .vmdk. I like to ZIP the file as it’s much smaller.

 
Building your FreeNAS

1) Log in as super user or root into your SSH terminal (Putty for me).
NOTE: I have created a script listed below which will automate the following steps. You can use the script or do it manually. I do provide the manual method since it demonstrates the steps to build the FreeNAS files.

2) These are the manual commands to download the main trunk files:

a. Type “cd /usr/local/freenas/” and Enter.​

b. Type “svn co https://freenas.svn.sourceforge.net/svnroot/freenas/trunk” and Enter. You will get this message and type “p” and Enter.​

​

And a few minutes later you end up with a screen like so which indicates the revision you will be building.


3) Actual Build manual steps:

a. Type “cd trunk” and Enter.​

b. Type “setenv FREEBSD_CVSUP_HOST cvsup3.freebsd.org” and Enter.​

c. Type “sh build/do_build.sh” and Enter. A lot of source files will be downloaded so this will take some time the first go around. Take a break for a few hours. If it doesn’t look like it’s doing anything, check the hard drive light on the VM Player, it should be flashing periodically. This does take a long time to compile.​

​

d. Type “sh build/create_iso.sh” and Enter. This is you last thing to build if you want an iso image.​

​

4) Locate your files at “cd /usr/local/freenas/trunk/obj.amd64”. This is where Filezilla comes in handy because you can copy the files directly to your computer (out of the VM).

Script

The script should be located in the /usr/local/freenas/ directory. To run it type "cd /usr/local/freenas" and "sh fbuild.sh" for a listing of switches.

Here are the current options:
Useage: sh fbuild.sh [-switch]
-b [-386] = Build, do not update any SVN files.
-bu [-386] = Build, Update BSD Source & Port files.
-buildrev revision [-386] [Server#] = Builds a designated revision of FreeNAS.
-clean [-386]= Delete compiled files (.xz .iso .sha)
-delete = Delete ALL Trunk files.
-deleteobj = Delete both obj.amd64 and obj.i386 directories if they exist.
-deletesvn = Delete all FreeNAS (SVN) source code.
-freebsd = Update FreeBSD and installed ports collection.
-h [-m] = Displays Full Help Info. -m displays one page at a time.
-list = List major build files created.
-loadrev revision [Server#] = Same as -buildrev, loads SVN files, no building.
-s [Server#] = Update SVN Trunk files only - No build.
-sb [-386] [Server#] = Update SVN Trunk files only then build.
-sbu [-386] [Server#] = Update SVN Trunk, BSD Source/Port, and build.
-sha = Create SHA256 hash for all .xz build files.

Building Branches (New Functions)
-brs branch# [-u] = Download branch/update.
Add 'x' after any build option to use local server LOCALCVSUP
-bx or -sbx or -sbux or buildrevx.
-bbu[x] = Build Branch Update: Different build switches from trunk builds.

Note: Default build is 64-bit. Switch -386 Creates 32-bit code.

And example usage would be: sh fbuild.sh -sb
This would download the current trunk and build it.

sh fbuild.sh -s
Just download the main FreeNAS program. This is handy to allow tweaking the code before building it.

sh fbuild.sh -delete
Delete everything so you can start from scratch.

sh fbuild.sh -deleteobj
Handy to remove all compiled data so you can force compiling everything again.

sh fbuild.sh -deletesvn
Maybe you tweaked on something and just want to start from scratch again, but without downloading all the FreeBSD files. This just deletes the base FreeNAS code.

sh fbuild.sh -buildrev 8515
This will build r8515. This is great to build an older revision because sometimes the developers break the software this helps at times when your testing.

sh fbuild.sh -brs 8.0.3
This will build the branch for 8.0.3.

If you setup a local CVSUP or just have an alternate CVSUP you would like to use often, modify the script LOCALCVSUP to your alternate site. I have a small local CVSUP which saves a great deal of time downloading source and ports code. To use this alternate location just append "x" to your build function where specified. Example:
sh fbuild.sh -bx

If you find the script is not working, odds are the developers made another change in how the build process is performed. Toss a message up here and someone will guide you and it will prod me into updating the script.

 

[robertdwatson]

trouble with build

following your instructions but in middle of build I get:

...
NanoBSD image FreeNAS-8r7318-amd64 build starting
Clean and create object directory (/usr/local/freenas/trunk/obj.amd64)
Construct build make.conf (/usr/local/freenas/trunk/obj.amd64/make.conf.build)
run buildworld
log: /usr/local/freenas/trunk/obj.amd64/_.bw
inet[858]: ssh/tcp: bind: Address already in use
inet[858]: ssh/tcp: bind: Address already in use
inet[858]: ssh/tcp: bind: Address already in use
..... and so on every 10 mins

 

[joeschmuck]

That occurs due to how you setup the network configuration. You can safely disregard those messages. And yes, it does occur every 10 minutes. These messages will occur after you start FreeBSD. You don't need to build, it will occur even if you just sit there, or so my experience has been. If it didn't produce a usable file, let me know.

-Mark

 

[Durkatlon]

I already have a completed build, but now I want to update the kernel with a customized version. I've edited FreeBSD/src/sys/i386/conf/GENERIC to include the option I want.

What are the steps to force the build process to build the kernel and repackage everything? I tried just kicking off the "do_build.sh" (and planning to do_build -b and do_createiso after this) but I'm not sure if the change to the kernel config will be picked up this way. I am building on a slow VM instance, so if I'm doing something wrong, I'd love to hear about it, so I can preempt the build and make additional fixes!

Thanks for any help you can provide.

 

[ProtoSD]

Durk,

Mark and I talked about this some other place and agreed that editing the do_build.sh and commenting out the 'IF' with the -b option and then running it will should do what you want. I also noticed that in a recent 'nightly' they added a -f option for just doing 'build world' and the kernel if I'm not mistaken.

Edit: Here's the comment:

Add a -f src option to just do a rebuild of buildkernel and buildworld. …

 

[Durkatlon]

That leads to my next questions I guess :D. What is the -b option to do_build and would simply doing a do_build.sh -b have done the trick?

I was trying to read the fbuild.sh script and it keeps echoing "Building First File" which is "do_build.sh" and "Building Second File" which is "do_build.sh -b". What are these first and second files? Is that like the first and second partition in nanobsd?

 

[ProtoSD]

I'm not exactly sure what the -b does, it just seemed to do what I wanted without it. It seemed like the -b part was getting done regardless of whether you added it to the command line. So by commenting it out in the script, and then specifying it on the command line, it did the second part. I build in a VM also, but haven't done it recently. Maybe Mark can offer some more insight.

 

[joeschmuck]

Durk,
I'm not that smart when it comes to compiling FreeNAS but I am smart enough write a script making repetitive tasks easier for myself. I used the README file as my guide on what is required to compile FreeNAS. Maybe reading that would help out. Based on the recent changes I've seen I believe the build process will wipe out all the files and download everything from scratch. Since those changes (earlier this week) I have not tried to compile anything, just too busy. I'll try another compile today to see how it works out.

What are you trying to do? Maybe if I understand I could help or at least learn from the experience?

 

[Durkatlon]

Thanks, I'll go over the readme to understand the build process better.

What I want to accomplish is to have an i386 build which is able to use more than 512MB of the 4GB of RAM that I give it. The default kernel only gives 1GB of the RAM to the kernel and out of that you then have to take a "safe" slice for the kmem map.

By rebuilding the kernel with "option KVA_PAGES=512" this effectively doubles to 2GB, at which point it should be safe to assign about 1.5GB to the kmem map. Still a ways away from the effectiveness with which the amd64 version uses RAM, but better than the current situation hopefully.

I plan to replace my i386 FreeNAS 8 with amd64 as soon as I can get a working mobo (my latest E35M1-I refuses to boot from USB). But I still want to do the exercise because I also have some i386 FreeNAS 7 machines which suffer from the same limitation. I ultimately want to build a custom kernel for those machines which should be a bit easier since I have long stopped upgrading the build on those boxes, so my custom kernel won't get overwritten every time I grab a new official build.

 

[ProtoSD]

I understand, I went thru the process of rebuilding the kernel many times for my system before I realized I could use the AMD64 image. I'm not sure that you be able to keep the same kernel when you do an upgrade to a new version, I guess it depends on what changed, but it could cause unexpected problems. It almost seems like KVA_PAGES=512 should be the default for i386 because anyone wanting to use ZFS is going to need the extra RAM and having it set that way doesn't really prevent you from using less RAM.

 

[Durkatlon]

Yeah, on the motherboard that I'm using in this NAS amd64 is not an option. The CPU is a Via C7 which only does x86. But I'm replacing that motherboard because of other problems (AHCI time-outs) which I do not have with FreeNAS7 on the same board.

Anyways, I agree the KVA_PAGES thing is strange. They should make it easier to configure the kernel so it can use more memory though a sysctl instead of a compile time option.

The box is however very stable with just using the very small kmem map. It's just not very fast. I did tweak the loader.conf to set vm.kmem_size and vm.kmem_size_max both to 512MB. Before that the machine would crash under high loads.

Still seems like a waste of 4 gigs of RAM right now :D

 

[joeschmuck]

I'd like to know how you make out with your build. If it's stable and the benefits are there, maybe that could be incorporated into the FreeNAS 8.1 i386 builds, or at least a variant.

 

[Durkatlon]

Yeah, I'll let you know how it works out. Probably going to take a little time as my main priority is actually getting amd64 on this NAS since it has the time-out issues *and* it's "production".

I did read the do_build.sh and the nanobsd.sh scripts and I do have support for the "force" option. I have kicked it off with options to "build kernel, don't clean, don't build world":

# build/do_build.sh -f -n -w

Oh and also based on this reading of the scripts, the correct file to modify to set kernel options is not FreeBSD/src/sys/i386/conf/GENERIC, but nanobsd/FREENAS.i386. This file gets copied into the aforementioned directory by nanobsd.sh.

So now we wait :D.

 

[Hi-Liter]

I get an "illegal option -b" error when trying to run the build script as described in step 3d.

 

[ProtoSD]

I usually just edit the do_build.sh script, search for the -b, and # comment out the whole 'IF" statement, it's like 3 or 4 lines.

Works like a charm every time.

 

[joeschmuck]

I'm sorry, I haven't updated my original posting since the developers made some changes in the build process. I've updated my posting and added the new script.

 

[warri]

Great guide, thank you very much!

Just one small addition: Before being able to build the iso, I also had to install cdrtools with "pkg_add -r cdrtools".

 

[joeschmuck]

Hum... At one time that was already added during the build then it was removed and I was told it was added again, maybe not. I'll update the guide to reflect adding cdrtools again.

Thanks.

 

[editmode]

Having problems with compiling the latest build, here's the error:

01:50:29 ## customize "add_port_net_netatalk"
01:50:29 ### log: /usr/local/freenas/trunk/obj.amd64/_.cust.add_port_net_netatalk
do_build.sh: ERROR: FreeNAS build FAILED; please check above log for more details

Any help is much appreciated, thanks.

 

[Hi-Liter]

I'm getting the exact same error, here's the last lines from the logfile (_.cust.add_port_net_netatalk):

Code:

===>  Building package for netatalk-2.2.1,1
tar: bin/add_netatalk_printer: Cannot stat: No such file or directory
tar: bin/showppd: Cannot stat: No such file or directory
tar: libexec/etc2ps.sh: Cannot stat: No such file or directory
....
tar: libexec/tfwpap: Cannot stat: No such file or directory
tar: libexec/tfwpaprev: Cannot stat: No such file or directory
tar: share/netatalk/pagecount.ps: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors.
pkg_create: make_dist: tar command failed with code 256
*** Error code 1

Stop in /usr/ports/net/netatalk.

Judging from the rest of the log, the package seems to have compiled just fine.

Edit: Just did an svn update and tried do build again, this time the build fails in "add_pkg_devel_dbus". Here's the log:

Code:

+ cd /usr/ports/packages/All
+ pkg_add -F dbus-1.4.6.tbz
pkg_add: could not find package kbproto-1.0.5 !
pkg_add: could not find package xproto-7.0.16 !
pkg_add: could not find package libXdmcp-1.0.3 !
pkg_add: could not find package libXau-1.0.6 !
pkg_add: could not find package libxcb-1.7 !
pkg_add: could not find package libX11-1.3.6,1 !