Unable to build FreeNAS with make release.

[Kingedgar]

I have cloned the repo from git; made the FreeBSD directory; fetched and extracted src and ports in the FreeBSD directory; and then re-ran make release. This is where I am at now...

Code:

root@freenas-build:~/freenas/freenas # make release
NOTICE: You are building from the EXTERNAL git repo.
Doing executing target release on host: freenas-build
Build directory: /root/freenas/freenas
env NANO_LABEL=FreeNAS VERSION=9.2.2-ALPHA GIT_LOCATION=EXTERNAL BUILD_TIMESTAMP=20140308 script -a release.build.log build/build_release.sh
Script started, output file is release.build.log
+ set -e
+ dirname build/build_release.sh
+ mydir=build
+ sh build/do_build.sh -z
Sourcing /root/freenas/freenas/nanobsd/os-base
Nano media size rounded down for SWAP backing 3906250 -> 3903488
Not setting MASTER_SITE_OVERRIDE, value: ''
Package is not yet built: net_mDNSResponder
cd: /root/freenas/freenas/FreeBSD/ports/dns/py-bonjour: No such file or directory
do_build.sh: ERROR: FreeNAS /root/freenas/freenas/nanobsd/os-base build FAILED; please check above log for more details
 
Script done, output file is release.build.log
*** [release] Error code 1
 
Stop in /root/freenas/freenas.

Here is release.build.log...

Code:

root@freenas-build:~/freenas/freenas # cat release.build.log
Script started on Sat Mar  8 01:18:10 2014
command: build/build_release.sh
+ set -e
+ dirname build/build_release.sh
+ mydir=build
+ sh build/do_build.sh -z
cd: /root/freenas/freenas/FreeBSD/src: No such file or directory
 
Script done on Sat Mar  8 01:18:10 2014
Script started on Sat Mar  8 01:19:32 2014
command: build/build_release.sh
+ set -e
+ dirname build/build_release.sh
+ mydir=build
+ sh build/do_build.sh -z
Sourcing /root/freenas/freenas/nanobsd/os-base
Nano media size rounded down for SWAP backing 3906250 -> 3903488
Not setting MASTER_SITE_OVERRIDE, value: ''
cd: /root/freenas/freenas/FreeBSD/ports/net/mDNSResponder: No such file or directory
do_build.sh: ERROR: FreeNAS /root/freenas/freenas/nanobsd/os-base build FAILED; please check above log for more details
 
Script done on Sat Mar  8 01:19:33 2014
Script started on Sat Mar  8 02:09:26 2014
command: build/build_release.sh
+ set -e
+ dirname build/build_release.sh
+ mydir=build
+ sh build/do_build.sh -z
Sourcing /root/freenas/freenas/nanobsd/os-base
Nano media size rounded down for SWAP backing 3906250 -> 3903488
Not setting MASTER_SITE_OVERRIDE, value: ''
Package is not yet built: net_mDNSResponder
cd: /root/freenas/freenas/FreeBSD/ports/dns/py-bonjour: No such file or directory
do_build.sh: ERROR: FreeNAS /root/freenas/freenas/nanobsd/os-base build FAILED; please check above log for more details
 
Script done on Sat Mar  8 02:09:27 2014

My reason for "rolling my own" is to eliminate the line in FREENAS.amd64 kernel config "IPFIREWALL_DEFAULT_TO_ACCEPT"

Are there any more specific instructions on how to build freenas from the github repo other than the README?

 

[jgreco]

That looks to me like the FreeBSD source didn't get extracted properly... let me play

 

[jgreco]

bleugh. the current instrs are a train wreck. oh, and great. there's no py-boujour, wtf is that.

i'm going to put in a bug report.

 

[jgreco]

bug 4509. i expect extra points for general incoherence due to no coffee. could you please indicate to me where you found a set of instructions that included "fetched and extracted src and ports in the FreeBSD directory;" because while i saw several sets of instrs, *none* of them listed that currently.

 

[Kingedgar]

I manually fetched and inserted the ports and src tarballs.

 

[dkran]

You have to run the other command that make recommends first in order to get the source. not the release the other one. Bleh sorry it's early morning and I'm fried.
If you try to go from making git-external to the release, I believe I had the same issue with no source

 

[Kingedgar]

Right now I am cloning a fresh copy and will try some new things, but the hangup seems to be py-bonjour missing... It is also early here and I'm quite tired. Was up until 3am messing with this, now back up at 7 to get right back at it. :(

 

[jgreco]

You're correct about py-bonjour. I was going to look at the other ix repos (trueos, pc-bsd) for that but kind of lost momentum. I filed a bug against the problem and a few other nits.

 

[dkran]

Building freenas (I'm a newb) was not as straightforward as it was written for me at all. It needs specific versions of specific apps, the freebsd repos were all screwed up in the 9 release because they were transferring to pkg install instead of pkg_add, and a whole bunch of things. Want me to fire up my vm and you can ask about packages I have? Maybe I got further than you

EDIT: btw thanks, jgreco. I was hesitant to talk about this in the forums because I was like oh I've troubleshooted package issues in build environments before, and I figured it was something I should just be doing. lol.

 

[jgreco]

Yeah, the developers will likely be working in a consistent and established build system which might make assumptions that are not accounted for in the instructions to build an external copy of the repo and all that.

Having worked on stuff like this I appreciate the problem.

I'll see if I can fix my FreeNAS buildbox to pump out nightlies or something, it rotted after FreeNAS 8 and now basically has the same problem you outlined above. Having to figure out the git stuff too to do periodic updates.

 

[dkran]

Yeah probably doesn't help to try to pump out a fresh clean build on a new vm like every once in a while I guess at least for the instructions. Thanks! I'll run mine and like pastebin my error

edit: @jgreco, here is my paste from my terminal output. I get to the virtualbox section and it seems to crash. On a side note, my i386 portion of the build seems to die almost immediately, the build continues on x64, which is fine because that's all I want anyway (can I build that alone as a make target?) Anyway,

http://pastebin.com/j72aG6g5

 

[Kingedgar]

I would definitely like to check out a branch other than 9.2.2-ALPHA. I would rather checkout 9.2.1.2 and build on that.

I got past the py-bonjour problem by setting an environment variable and doing make git-verify or something similar before starting the make release.

I will update my post with the specifics when the build either fails, or get's done.

EDIT: This is what I ran to pull down FreeBSD sources and ports and some other things necessary.

Code:

env NANO_LABEL=FreeNAS make update

 

[Kingedgar]

I'm not sure how long this took to error out because I left it unattended, but now it looks like it's erroring on building devel/talloc. Here is a snippet of the error.

Code:

===> Fetching all distfiles required by tevent-0.9.18 for building
===>  Extracting for tevent-0.9.18
=> SHA256 Checksum OK for tevent-0.9.18.tar.gz.
===>  Patching for tevent-0.9.18
===>  Applying FreeBSD patches for tevent-0.9.18
===>  tevent-0.9.18 depends on package: talloc>=2.0.8 - not found
===>    Verifying install for talloc>=2.0.8 in /usr/ports/devel/talloc
===>  License LGPL3 accepted by the user
/!\ WARNING /!\
PYTHON_DEFAULT_VERSION is defined, consider using DEFAULT_VERSIONS=python=2.7 instead
 
===> Fetching all distfiles required by talloc-2.0.8 for building
===>  Extracting for talloc-2.0.8
=> SHA256 Checksum OK for talloc-2.0.8.tar.gz.
===>  Patching for talloc-2.0.8
===>  Applying FreeBSD patches for talloc-2.0.8
===>  talloc-2.0.8 depends on file: /usr/local/bin/python2.7 - found
===>  talloc-2.0.8 depends on executable: pkgconf - found
===>  Configuring for talloc-2.0.8
 
===>  talloc-2.0.8 needs port lang/python27 to be build with THREADS support.
 
*** [pre-configure] Error code 1
 
Stop in /usr/ports/devel/talloc.
*** [build-depends] Error code 1
 
Stop in /usr/ports_dir/devel/tevent.

Python should be compiled with THREADS support.

 

[jkh]

Urk! You're doing too much work here and it's messing you guys up!

First, FreeNAS isn't built from FreeBSD src and ports. It is built from TrueOS src/ports, which is FreeNAS's own branch of both (with extra stuff). This is why the instructions explicitly don't mention to grab FreeBSD, because you want FreeNAS's build process to check out all the other repos for you, and it does.

If you follow the instructions at the base of the FreeNAS repo, namely the README file, and you install all of the dependent ports or packages it lists first (I did add a note about building Python with THREADS - thanks for noting the omission) on a stock FreeBSD 9.2-RELEASE system, things should work fine. I have done this multiple times on a vanilla FreeBSD build machine to validate that the README actually works. This is also why I added the "make git-external" hooks, so that folks could pull from github instead of our internal server. Should work, really, or at least it did last time I tested it. I suppose I could always set up another FreeBSD 9.2 VM from scratch and re-verify...

 

[Kingedgar]

I can verify that you HAVE to run "env NANO_LABEL=FreeNAS make update" between make git-external and make release. Otherwise it will not pull in src or ports from TrueOS.

On a side note, how do I pull in 9.2.1.2-RELEASE branch instead of 9.2.2-ALPHA?

I'm trying to build this box for a client who needs a couple key features that requires me to build a custom FreeNAS.

 

[jkh]

Actually, you just run "make checkout" - I forgot that Craig had changed the build process to decouple checking out the code from doing the release build (the release target used to do both). NANO_LABEL is set to FreeNAS by default - you don't have to set it. I've changed the README to mention the "make checkout" step, thanks.

 

[Kingedgar]

Actually, you just run "make checkout" - I forgot that Craig had changed the build process to decouple checking out the code from doing the release build (the release target used to do both). NANO_LABEL is set to FreeNAS by default - you don't have to set it. I've changed the README to mention the "make checkout" step, thanks.

Thank you, this clears up a lot!

 

[Kingedgar]

Ok, after many many hours put into this, the build process failed with compiling megacli, it apparently couldn't fetch the distfile. I omitted this from nanobsd/osbase because I couldn't fetch it manually either. I restarted the build (without make clean), expecting the build to pick up where it left off. No, it couldn't be that easy. It started with building world again, which takes HOURS to complete.

Why can't the build script pick up where it left off?

I really really need to build my own FreeNAS with a few edits that would never be in standard FreeNAS. This is for a client and my future as a contractor with the company depends on it.

EDIT: It seems to be just verifying everything it has already done and is much faster, although I ran into another problem with virtualbox-ose-kmod, which I don't see why FreeNAS needs this if virtualbox-ose is not installed. Oh well, back to the build.

 

[jgreco]

Urk! You're doing too much work here and it's messing you guys up!

Honestly, Jordan, accusing me of doing too much work is pretty over the top...

There were several documented instances of it stopping with some apparent failure within a "FreeBSD" subtree, so it probably wasn't too much of a stretch (obviously not for the OP either) to try installing FreeBSD src in there.

I haven't actually gotten a successful build with the new instructions either; apparently git has to be built with curl support or something or other so there's more dependencies to track down.

 

[Kingedgar]

jgreco, I think I am close to a complete build. I had to eliminate a few lines in nanobsd/os-base to get the build to run correctly. One was for hardware I don't use, and the other was the virtualbox-ose-kmod. (why is this needed?). I will update when the build is done.

My only limitation has been the hardware I'm building this on. It's a virtualbox vm on my windows machine with 2 cores and 2gb of ram allocated for the machine. It's been slow going but I think I'm getting there.