[HOWTO]FreeNAS as a Ubuntu/Debian patch server

Status
Not open for further replies.

YBO Tong

Cadet
Joined
May 3, 2014
Messages
3
This is a tutorial about how to use FreeNAS as a Ubuntu update mirror. Jails are the main tools used to achieve this. Forgive my glitchy English if you found any language errors, since I am not a native English speaker.

My approach is to run 2 jails, one with Ubuntu running “apt-mirror” periodically,and the other one with plugin jail that runs apache2 server.

Prologue:
Following are some infos about how I met Linux and the motivation for doing this tutorial, you can skip this section if you only want the tutorial.

I strongly recommend you to run step first, then head back if you want read all these personal stuff. Step will take a long time and there is nothing you can do about it.

First time I met Linux was about 12 years ago, when I was still a 10 years old kid. At that time, the only distro of linux I know is Redhat and I found it hard to use it since I was so used to window 95. I cared about running “Red Alert” more than anything,XD. Although I had a lot experience with CLI but it was with MS-DOS. Linux is very different and the documentation at that time is pretty poor. I never touched linux again until half years ago. I started to work on my home router because of the evil P2P downloading that made my home network pretty unusable. This is typical problem among the students who lives together. During this process, I found a very powerful platform —— OpenWRT. It is a very easy to use linux distro specially modified to run on routers.

Finally my network problem was solved using “tc” and “iptables” built in linux kernel, with other minor modifications. I was able to play League of Lengend with very stable latency (8~15ms more than idle) even when my roommates are pushing their P2P software to the limit(Unlimited Upload/Download+1000 concurrent connections per user). I have to thank them for their help, tho. They are very helpful when I was doing the test. The whole project took me about 3 months and I even wrote a client running in Windows so that I can run P2P download while I am playing games, but that’s not the topic here.

In this process I started to use Ubuntu. I started using linux because the terminal in MacOS won’t run nano in linux(xterm256 errors), and I chose ubuntu over other distros because of the AppCenter. Ubuntu is very easy to start with for a windows user. Switching from GUI to CLI is not a easy task,XD.

After that I gradually switched from GUI to server edition, which has only CLI. Command lines are actually faster if you have a clear idea what you are doing(except moving files around, which I found very annoying to work with in CLI).

I like to do some tests on linux system now and then, sometimes I will break the whole OS and it won’t boot again. Normally I will just wipe everything and install a new one(habit from using windows, once you get affected by virus, the simplest way to have a clean system is to reinstall it). To accelerate the process of reinstall Ubuntu, I created scripts to doing all the pre-config stuff. It is really not that hard since pretty much everything is automated and all I need to do is leave it running for a while.

And… the last command in my scripts is

sudo apt-get update && apt-get -y upgrade,which will fetch about 300MB of archives from us.archive.ubuntu.com

after that, I have to run another script which will set up a build environment for Xen or xenserver-core, whose archive size is significant. Xen’s environment is fine, 200MB. For xenserver-core, about 900MB of archives need to be fetched, not to mention the cross compile environment the xenserver-core compiling tools tries to set up(basically it will build a chroot environment, with its own binutils and GCC compiler. You will know how painful it is if you have ever tried “Linux from scratch”). The whole process will take more than 2 hours for me,which is way too much.

And… I still haven’t successfully compiled xen or xenserver-core once,:D. Xen4.4 will complain about broken toolchain and xenserver-core will complain that nothing will be installed.

It is hella fun to compile something from source, especially the lasted edition with little or none documentation. The error info will just bury you,:D. Still trying very hard!!!

Pre-request:
OK, enough craps. This post is aiming at noobs like me, so I will try to explain everything in detail. As long as you feel comfortable with CLI in linux or Unix(FreeBSD), you should be fine. This is not rocket technology,:D. If you’re familiar with the command, you can skip the explanation.

“apt-mirror” is a tool provided by linux community to mirror Debian/Ubuntu archives. This post focuses on ubuntu, but it should work with Debian,too. What it does is downloading everything on the server that is related to your distro and store it at a local place. For example, I run ubuntu-server-1404(Code “Trusty”), apt-mirror will fetch all the updates/patches related to Trusty and download them to local disk.
WARNING:apt-mirror is very space demanding. For Ubuntu-14.04(Trusty), the archive is about 180GB. For Ubuntu-12.04(Precise), the archive will take about 160GB. I am not sure about the number for Ubuntu-13.04, but it should be around 160GB. Reserve enough space for this before you start this whole process.

I am running FreeNAS 9.2.1.3.
(PS: Oh,damn, I just checked the website and it seems that 9.2.1.5 is released, zzzzzzzzzz. I guess it won’t make any difference for this. I will check the release note and update this post later)

Last but not least, you need internet access since a bunch of packages will be downloaded.

Procedure:
1. Create a Ubuntu jail(Ubuntu 13.04 32bit on my FreeNAS box)

This jail is the platform for "apt-mirror" tool.

Login the FreeNAS web management page of your FreeNAS and click the big “Jails” button on the top, then click “Add Jails” below it.

In the prompt windows, give your Jail a easy to remember name, filled it in the "Jail Name" and chose “ubuntu-13.04” in "type". Chose a IP address you like(do not conflict with any existing device), you can simply leave it as default since this IP is not important. Leave everything else as default and click “OK”. Then you can grab yourself some drink and popcorn and enjoy a movie. It will take a quite long time to download some archive from internet.

2. Create a plugin jail
This jail is the platform for apache2 server.

Repeat step 1, the difference is that we chose "Plugin jail" instead of "Ubuntu-13.04" this time. Also we need to give it a different name. I use “AptServerPluginJail” here. The IPv4 address in this configuration window is the IP address of our future Update server. I am not sure if we can change the IP address once we click the “OK” button, thus we better do it right first time. It is perfectly fine to just leave it as default, tho. Only change it when you want a specific IP address.

3. Create a new dataset in ZVOL
To make it possible for 2 jails to communicate, we can leverage the nullfs provided by FreeNAS. We mount a dataset to both our jails and use it as a media to store all the archives. It is also much easier to rebuilt the whole thing if your usb stick screw up and you need to build all the jails from ground.

Click the “Storage” button on the top of the FreeNAS management page, add a new dataset using the button at the bottom of the window. Give it a easy to recognize name and leave everything as default. In my case, I called it “UbuntuAptMirror”.

What's next is to set proper permission to allow OSs in jail doing all the stuff. My configuration is to allow everyone(owner/group/others) do everything (read/write/execute). You only need to set this on the dataset where archives are stored. It is not the best practice in terms of security but it is fine for me. Consult other documents if you want to enforce specific permission for the jails. I had some test in it but I am not familiar with jail system, thus I might not be able to help you on this.

4. Install apt-mirror in Ubuntu Jail
To gain access to jails, two methods can be used. One is use the shell of the jail in webpage, the other one is to use SSH and issue commands in the tunnel. Both works, chose one you like. Personally I prefer using SSH,since it is much more responsive.

To SSH into jail:
1)SSH into FreeNAS
My FreeNAS’s IP is 192.168.2.192, thus issue following command in terminal (putty if you use windows)

ssh root@192.168.2.192

following the prompt and login

2)List all jails
Issue “jls” in terminal, and you should get something like
JID IP Address Hostname Path
1 - Penskey /mnt/YBONASVol/jails/Penskey
2 192.168.2.9 Ubuntu1304 /mnt/YBONASVol/jails/Ubuntu1304
3 - owncloud_1 /mnt/YBONASVol/jails/owncloud_1
4 - transmission_1 /mnt/YBONASVol/jails/transmission_1

“jls” stands for j(ail) l(i)s(t).Find the JID of your Ubuntu jail. In my case, it is 2. JIDs are not always the same. If you shutdown a jail and restart it, JID may change. So make sure you check the JID every time you are doing this.

3)Switch to Ubuntu
From step 2), we know the JID of my Ubuntu jail is 2, thus issue following command to switch to ubuntu jail

jexec 2 /bin/bash

Number 2 amid of the command stands for PID you got above. Use your own PID in your command. You should see the beginning of your command line changes to [root@ubuntu1304] from [root@freenas].

Once you successfully switched to Ubuntu jail, just issue command

apt-get install apt-mirror

There may be some errors poping up but they are not critical, thus can be ignored.

Do not close the terminal now, you can continue use this window for next step

5. Chose a place in Ubuntu jail to store archive files
Since we want to mount the external dataset to the jail, we have to provide a place to mount to. Personally I use /root/LocalRepo. SSH into Ubuntu jail(not needed if you kept the terminal open in step 5), issue command

mkdir -p /root/LocalRepo

the -p option after mkdir will create all the intermediate directories for you if they don’t exist.You can chose whatever path you like, as long as you can find it.

Still, leave the terminal window open.

6. Configure apt-mirror in Ubuntu jail
This is the most tricky part, may need some more knowledge about text editing in Linux. I am not going to spend much time here explaining how to do this since everyone has their own taste.
1)Open config file for apt-mirror
Issue the following command (in Ubuntu jail)

vi /etc/apt/mirror.list

This will open the file in vi. I am a huge fan of vim,:D.

2)Configure the storing path
You should find "#set base_path /var/spool/xxxxxx" at the beginning of the file. Uncomment this line(delete the "#") and change "/var/spool/xxxxxx" to your designated path. In my case, it is /root/LocalRepo(created in step 5)

3)Configure what archive to mirror
The reason choosing apt-mirror over rsync is that it is more flexible and easier to configure. Here we are going to make apt-mirror download exactly what we want to download.

When apt-mirror is installed in Ubuntu jail, it will generate a mirror.list automatically according to the host(Ubuntu jail here). For example, when apt-mirror is installed in Ubuntu-13.04, it will add content in the form of

deb http://us.archive.ubuntu.com/ubuntu/ raring main restricted

deb = form of the package
htt://xxx = address of the archive
raring = Release of Ubuntu(13.04,code raring)
main restricted = group of package

for more information about what these lines means, please read /etc/apt/sources.list or consult google.com.

What we need to do here is to fit our release repo in it and let apt-mirror download deb packages for that specific release.

To be perfectly clear, the sources.list file mentioned below is the /etc/apt/source.list file located in you WORKING UBUNTU disk(The operating system you want to run apt-get update && apt-get upgrade). The mirror.list file mentioned below is the /etc/apt/mirror.list file located in UBUNTU JAIL you just created. You will also find a sources.list file in ubuntu jail but it won’t do you any good unless you’re running the same version of ubuntu both in working and in jail.

(Update*: If you only use a single release(13.04 or 14.04 or any other one), an easier way is to install apt-mirror in your WORKING UBUNTU and copy the /etc/apt/mirror.list in WORKING UBUNTU to UBUNTU JAIl and replace it.)

When we run “sudo apt-get update”, APT will pull index from addresses stored in /etc/apt/sources.list, thus the easiest way to modify the mirror.list is copy whatever in /etc/apt/sources.list to /etc/apt/mirror.list. But if you open the sources.list by yourself, you will see a much larger collection of addresses stored in it than in mirror.list. Reason for that is simple, sources.list includes 3rd-party packages as well as security updates. It’s up to you whether to download them all or just core packages provided by Ubuntu. Copy whatever you need from source.list to mirror.list and proceed to next step. I will recommend you delete all the lines below “########### End of config ############” except “clean http://archive.ubuntu.com” in mirror.list,then copy what you need from sources.list to it.

When the copy-paste stuff is finished, one thing still has to be done. If only something like deb http://xxxx is stored in mirror.list, apt-mirror will try to download packages that match the host it runs on. For example, recent ubuntu releases come with 32bit and 64bit versions. They’re called ubuntu-i386 for 32bit version and ubuntu-amd64 for 64bit version. While the ubuntu jail we just created is a 32bit one, run apt-mirror on it will force it to download ONLY 32bit packages. That means, if you want to run “sudo apt-get update” on a 64bit version ubuntu, 404 error will occur. It will complain that deb-amd64 packages can’t be found. Thus, we have to tell apt-mirror to download both 32bit and 64bit packages to make sure nothing’s left behind.

To achieve this, we need to do some modification on the mirror.list file we just worked on. In that file, lines should be in the form of
deb http://xxx.archive.ubuntu.com/ubuntu/ yourRelease yourPackageGroup
deb-src http://xxx.archive.ubuntu.com/ubuntu/ yourRelease yourPackageGroup

These 2 lines will always appear in couple. To make sure both 32bit and 64bit packages are downloaded, we need to change it to:
deb-i386 http://xxx.archive.ubuntu.com/ubuntu/ yourRelease yourPackageGroup
deb-amd64 http://xxx.archive.ubuntu.com/ubuntu/ yourRelease yourPackageGroup
deb-src http://xxx.archive.ubuntu.com/ubuntu/ yourRelease yourPackageGroup

What I did is copy the address following deb and deb-src, and make 2 new lines with deb-i386 and deb-amd64 as new headers. You can do all these manually, or you can use the ultimate weapon —> sed
issue the following command in Ubuntu jail after you finished all the previous steps

sed -i 's/^deb \(.*\)/deb-i386 \1\ndeb-amd64 \1/' /etc/apt/mirror.list

You won't see any output in bash but the mirror.list file has been modified. run the following command to confirm

cat /etc/apt/mirror.list

WARNING:DO NOT START APT-MIRROR NOW. If you started it here, you won’t be able to mount dataset to the folder(/root/LocalRepo) in jail. Clear everything in /root/LocalRepo if you ran it by accident.

4)Change timezone if needed
Timezone in Jail may not sync with the FreeNAS host. My local time is CDT while the timezone in jails are PDT. It is essential to sync them if you want to setup cron jobs. Issue the following command to change your timezone

dpkg-reconfigure tzdata

7. Mount dataset to Ubuntu jail
In jail's setting, click “add storage” and mount it to /root/LocalRepo in Ubuntu jail. Make sure you don’t check the “Read Only” option or the Ubuntu jail won't be able to write to this dataset.

If you want to save some time, you can start step 10 (run apt-mirror) now and come back since step 8 do take some time.

8. Install apache2 in Plugin jail
I chose apache2 over nginx or any other http server because this is the one I am most familiar with. Actually I tried nginx as a file server but I failed. Apache2 is really easy to set up for this task, almost no effort needed. I tried apache2 in ubuntu jail,too. It turned out that install apache2 from apt-install won’t work. Jail is not a fully virtualized environment thus its functionality is limited.

Plugin jail is a "FreeBSD" environment, thus we need to switch to freebsd commands in it. Linux and unix are close brothers, only minor difference will be noticed here.

To install apache2 in Plugin jail, issue following command one by one:

portsnap fetch
portsnap extract
cd /usr/ports/www/apache24
make install clean

Notice:You will be asked to select what pre-request package to install during the compiling process. If you don’t know what they are, just press enter to use default settings. Change it as needed if you have special requirement.

The compiling may take some time, but not awfully long(try to compile /usr/ports/editors/vim if you don’t know what I am talking about,:D).For me, it is about 10 minutes (Xeon E3 1245v3). Compiling is very CPU intensive,tho. It will saturate one of your cores during the process. When the compiling is finished, we need to enable it in the system. Open /etc/rc.conf with vi and add the line below at the end of the file

apache24_enable="YES"

or you can simply issue this command

echo 'apache24_enable="YES"' >>/etc/rc.conf

After configuration, issue the following command to start it

service apache24 start

You should see no error in the prompt or something is wrong.Apache2 will start automatically when the jail starts so you can shutdown the jail without any issue in the future.

The last step here is to create a folder as a mount point for external dataset. You can create whatever folder you like. In my case, I created a folder at root named "LocalRepo"

mkdir /LocalRepo

Now you can actually test the apache2 server from a web browser. Type the IP address of the Plugin Jail in your web browser and you should see a blank page with "It works!" at top of the page. If you see nothing or the browser tells you address not found, there must be something wrong.

9. Mount UbuntuAptMirror dataset to /LocalRepo in Plugin jail
This is literally the same as it is in step 7, the only difference is that we mount the dataset to Plugin jail instead of Ubuntu jail. In this way we let two jails share file and storage freely.

10. Download archive
(*Only do this once, if you started this step immediately after step 8, skip it this time.)

SSH into Ubuntu jail(see step 4), and issue the following command:

apt-mirror

Once you started it, you will first see it downloading index, then translation. The numbers decrease from 20 to 1 is the indicator how many downloading thread you are running. If you run "top" command now, you will see a lot of "wget" thread in it. The number of wget matches the number of thread. The number of thread is defined in /etc/apt/mirror.list.

Since this will run for a very long time, you will probably want to run it on the background. But if you close you ssh session now, the downloading will stop. If you want to detach the process from current session, thus the downloading will keep going if you close you SSH session, issue the following command in written order:

Ctrl+z(this is a key combo, control + z, you should see the process is paused or something similar. You should be able to run new commands now)
bg
disown

Now you can close your SSH session(terminal window) safely. The first and best tool for detaching a session ought to be "screen", but sadly it won't work in ubuntu jail.

Note that you can't login and attach that session again. If you really want to stop downloading, the only way I found is shutdown the whole Jail. You can either restart FreeNAS box or you can stop it in web management page.

It is perfectly fine to stop it in the middle of the downloading and run it again later. I have done that several times without any issue. Do keep in mind that you will not be able to use the mirror until your first download is finished. You can't even test it since an important folder "/mirror/WEBSITENAME/dists" is downloaded at last, which contains the index for releases. Thus run apt-get update in any other client will not be able to fetch anything from this server even all the packages are there.

11. Finalize apache2 server configuration
If you have successfully go through step 1~10, your Ubuntu mirror is just 1 step away. Now we have the mirror from ubuntu jail, and web server from plugin jail. This very last step is to tell the apache2 server where to find your mirror and serve it to clients.

Still, SSH into Plugin jail(remember the difference between bash/csh), and go to /usr/local/www/apache24/data. Check the file in the folder you will see an index.html file in it. This is actually what we see when we type the IP address of the jail in web browser. If you don't see this folder, there must be something wrong.

Then we need to go to /LocalRepo folder(or the folder you created in Plugin jail. Since now we have mounted the dataset to plugin jail, this folder should contain a folder called "mirror". Open it and you will see a list of folders named after websites name. In my case, I live in US, thus the folder name is called "us.archive.ubuntu.com". This name is determined by the mirror you defined in /etc/apt/mirror.list file.If you have multiple mirror to download, you will have exactly same number of folders.

What we need to do now is to link this folder to apache2 server so that we can get access to these files in browser. Actually, what we are really linking is the subfolder in it. Issue the following command:

ln -s /LocalRepo/mirror/us.archive.ubuntu.com/ubuntu/ /usr/local/www/apache24/data/ubuntu/

Now we need to test if it succeeded. Visit http://Your-PluginJail-IP/ubuntu in any web browser. You should be able to see the files on the server and you should be able to download any of them.

12. Wait for the downloading to complete
Now everything is up and running. You will need to configure your ubuntu client at /etc/apt/sources.list to enable this server or do some dirty work redirect all the traffic pointing at ubuntu server to your local server(what I did). Do be patient about the downloading,tho. It may feels forever.

This tutorial seems to be so long..............I really didn't expect this.

I am still a green finger at FreeNAS and FreeBSD, the first edition I used is FreeNAS 9.2.1.1, so if there is anything wrong in this post or you find anything can be improved, I'd love to hear from you.
 

c32767a

Patron
Joined
Dec 13, 2012
Messages
371
We have multiple versions of Ubuntu around, so I did not build a jail machine to run apt-mirror but rather just created an rsync job to drop the entire ubuntu tree into a dataset that could be mounted on a jail running Apache.

The rsync task config is very simple:
path : path to the dataset accessible in the jail
remote host: rsync.ubuntu.com
mode: rsync module
remote module name: ubuntu
direction: pull
short description: ubuntu

pick a scheduled time to run the rsync. Be nice and don't run it too often.

checkboxes:
recursive : yes
times: yes
compress: no
archive: yes
delete: yes
quiet: no
preserve: no
preserve extended: no
extra options : "--delete-after"
enabled: yes

That's it..

It's been working for months for me.

note that this will download approximately 1T from the archive.. Make sure you won't overrun your internet connection's transfer limits.
You can use an additional "extra option" to rsync to force it to only use a limited amount of bandwidth, "--bwlimit=xx" where xx is a number in KBytes/s
 
Status
Not open for further replies.
Top