pkg update fails after installing jail from own template

[Junicast]

Hi,

I created my own template and after overcoming some obstacles it worked. The jail template is marked as FreeBSD, x64.
After using that very template to create a new jail I have trouble to do a
pkg update

Some additional information:
host:
FreeNAS-11.0-U4 (54848d13b)
jail:

Code:

root@sql001:/etc/pkg # uname -a
FreeBSD sql001 11.0-STABLE FreeBSD 11.0-STABLE #0 r321665+25fe8ba8d06(freenas/11.0-stable): Mon Sep 25 06:24:11 UTC 2017	 root@gauntlet:/freenas-11-releng/freenas/_BE/objs/freenas-11-releng/freenas/_BE/os/sys/FreeNAS.amd64  amd64
root@sql001:/etc/pkg # pkg -vv
Version				 : 1.10.1
PKG_DBDIR = "/var/db/pkg";
PKG_CACHEDIR = "/var/cache/pkg";
PORTSDIR = "/usr/ports";
INDEXDIR = "";
INDEXFILE = "INDEX-11";
HANDLE_RC_SCRIPTS = false;
DEFAULT_ALWAYS_YES = false;
ASSUME_ALWAYS_YES = false;
REPOS_DIR [
	"/etc/pkg/",
	"/usr/local/etc/pkg/repos/",
]
PLIST_KEYWORDS_DIR = "";
SYSLOG = true;
ABI = "FreeBSD:11:amd64";
ALTABI = "freebsd:11:x86:64";
DEVELOPER_MODE = false;
VULNXML_SITE = "http://vuxml.freebsd.org/freebsd/vuln.xml.bz2";
FETCH_RETRY = 3;
PKG_PLUGINS_DIR = "/usr/local/lib/pkg/";
PKG_ENABLE_PLUGINS = true;
PLUGINS [
]
DEBUG_SCRIPTS = false;
PLUGINS_CONF_DIR = "/usr/local/etc/pkg/";
PERMISSIVE = false;
REPO_AUTOUPDATE = true;
NAMESERVER = "";
HTTP_USER_AGENT = "pkg/1.10.1";
EVENT_PIPE = "";
FETCH_TIMEOUT = 30;
UNSET_TIMESTAMP = false;
SSH_RESTRICT_DIR = "";
PKG_ENV {
}
PKG_SSH_ARGS = "";
DEBUG_LEVEL = 0;
ALIAS {
}
CUDF_SOLVER = "";
SAT_SOLVER = "";
RUN_SCRIPTS = true;
CASE_SENSITIVE_MATCH = false;
LOCK_WAIT = 1;
LOCK_RETRIES = 5;
SQLITE_PROFILE = false;
WORKERS_COUNT = 0;
READ_LOCK = false;
PLIST_ACCEPT_DIRECTORIES = false;
IP_VERSION = 0;
AUTOMERGE = true;
VERSION_SOURCE = "";
CONSERVATIVE_UPGRADE = true;
PKG_CREATE_VERBOSE = false;
AUTOCLEAN = false;
DOT_FILE = "";
REPOSITORIES {
}
VALID_URL_SCHEME [
	"pkg+http",
	"pkg+https",
	"https",
	"http",
	"file",
	"ssh",
	"ftp",
	"ftps",
	"pkg+ssh",
	"pkg+ftp",
	"pkg+ftps",
]
ALLOW_BASE_SHLIBS = false;
WARN_SIZE_LIMIT = 1048576;
METALOG = "";
NFS_WITH_PROPER_LOCKING = false;


Repositories:
  FreeBSD: {
	url			 : "pkg+http://pkg.FreeBSD.org/freebsd:11:x86:32/latest",
	enabled		 : yes,
	priority		: 0,
	mirror_type	 : "SRV",
	signature_type  : "FINGERPRINTS",
	fingerprints	: "/usr/share/keys/pkg"
  }

Update:
The 32 bit thing is obviously wrong.
The file /usr/local/etc/pkg/repos/FreeBSD.conf actually contains the wrong content. How can that have happened? I just did a tar -czf from the original jail and when I inspect that .tgz file it contains the right file.

Update 2:
Might my .tgz be cached somewhere? But even so, I could not imagine that the string ever could have been "32" as there are no 32 bit systems involved? What's the deal here? To me it looks like the mechanism of conserving files through to the destination system is broken, don't you think?

 

[joeschmuck]

I created my own template

What exactly do you mean by this? Also, did you not like the provided FreeBSD template that comes with FreeNAS?

 

[Junicast]

I like the default templates but I just got my own configuration with default user and password and my own software / service configurations.
With mtree and tar I created my own jail template files.
So this is the overall procedure:
- create default jail
- modify jail to my likings
- shut down that jail
- create mtree of that jail
- create tgz of that jail
- put those files on a webserver
- created a new jail template on FreeNAS using those new files
- create a new jail based up that new template

This links I used:
https://forums.freenas.org/index.ph...-based-on-an-existing-jail.38791/#post-236468
http://doc.freenas.org/11/jails.html#managing-jail-templates

 

[joeschmuck]

I can validate your steps if you like but I'm going to need an exact step by step procedure so I can do the exact same thing because like you, I find it hard to believe that if you started out with a 64 bit reference, how could it be changed to a 32 bit reference. It blows my mind too so I thought it would be interesting to look into, plus it is scheduled to rain all day tomorrow so I'll be locked up in the house for the most part.

EDIT: Please do not assume I know anything in the steps, build it so an idiot can follow it. I've found this the best way to help troubleshoot any problem. When you say "format the drive" to someone, does that mean to offline the drive first and then format the drive and would the other person know to do that. Assumptions kill. I know the example above was terrible but it gets the point across.

 

[Junicast]

Appreciate your help. Let me try to bring as much detail to my steps as I can.
- create default jail with name myjailname
- start that jail
- modify jail to my likings (install packages like git, screen, mc tcpdump, nmap, sudo, vim / add a local user and game him sudo privileges <<username>> ALL=(ALL) ALL with command visudo
- shut down that jail
- create mtree of that jail within the FreeNAS cli:
mtree -c -p /path/to/jails/myjailname -k sha256digest > /root/myjail-template.mtree
- create tgz of that jail
cd /path/to/jails/myjailname/
tar -czf /root/myjail-template.tgz .
- put those two files on a webserver, let's say
http://my.url.net/myjail-template.mtree and
http://my.url.net/myjail-template.tgz
- created a new jail template on FreeNAS using those new files
Goto "Jail - Templates - Add Jail Template"
Name: mynewjailtemplate
OS: FreeBSD
Architecture: x64
URL: http://my.url.net/myjail-template.tgz
Mtree: http://my.url.net/myjail-template.mtree
- create a new jail based up that new template
Goto "Jail - Jails - Add Jail"
Advanced Mode
Under Template, choose the new template.
- new jail is being created
- start that new jail
- access command line
- run "pkg update"
- take a look into "/usr/local/etc/pkg/repos/FreeBSD.conf" and see if it's 32 or 64 bit.

 

[joeschmuck]

Thanks. I will get this done today and send you my results.

 

[joeschmuck]

So I had no problems, well I didn't use a web server or ftp server, I manually added the template.

What I did was create a 64 bit jail (don't have any other option) and ensured it was a 64 bit jail.
Then I ran mtree, tar, and I added it manually using warden.
Next I created a new jail based off the template I created and ran it and verified it was still 64 bit.
Laslty I updated the jail using pkg update and pkg upgrade.

I'll try one more thing, this time I'll make some changes to the jail before I make a template out of it.

 

[joeschmuck]

The next test was to update all the packages and upgrade the jail, then I installed MiniDLNA and configured it. Then I created the template and went through all the paces and it still is a 64 bit template and references the 64 bit freebsd repo.

I have no idea why the jail would change the repo location. Could it be due to one of the packages you installed? Maybe it was a 32 bit only package? This is pure speculation, I have no idea why it would change.

Sorry, I've basically exhausted testing on my end. I'd recommend that you start over from scratch and after each change you make to your original jail, verify the repo location and check to see if it was changed.

The commands I used on my testing were:
1) Created a generic FreeBSD jail, 64bit (the only thing you can make by default in FreeNAS 11.0-U4) called "peanut".
2) I entered the jail and ran pkg update with no errors.
3) I ran pkg upgrade with no errors (accepted the "Y" to make the changes).
4) I exited the jail and then stopped the jail.
5) In SSH I performed the following commands:
mtree -c -p /mnt/farm/jails/peanut -k sha256digest > /root/peanut2-template.mtree
cd /mnt/farm/jails/peanut
tar -czf /root/peanut2-template.tgz .
warden template create -tar /root/peanut2-template.tgz -nick peanut2-template
6) I entered the FreeNAS GUI and manually added the template using:
Name: "peanut2-template"
OS: FreeBSD
Architecture: x64
URL: "ftp://192.168.1.51/peanut2-template.tgz"
MTREE: "ftp://192.168.1.51/peanut2-template.mtree"
7) I nexted created the jail using the new template of course and I defined the gateway under the advanced mode.
8) I deleted the template files in /root/ to clean up my mess.

The jail automatically started and all was good in the world.

The URL data is not important in step 6 other than something must be there because the template is already installed during the last command of step 5, but if you don't do step 6 then FreeNAS has no idea the template is present.

 

[Junicast]

Well I can see than in the tgz file from my webserver the content is exactly right. So I guess starting over from scratch doesn't seem necessary, right?
What do I need to do in order to clean off all cached tgz files? I understand that FreeNAS caches earlier downloaded tgz / mtree files. Any idea?

 

[joeschmuck]

What do I need to do in order to clean off all cached tgz files? I understand that FreeNAS caches earlier downloaded tgz / mtree files. Any idea?

Sure, that is easy... Delete the wardens associated with your template. In my example it is called "warden-template-peanut2-template". So first delete the template from the Jails -> Template area and then go to Storage and ensure that if the warden is still listed, you remove it. All of this is done via the GUI. Now your system is like it's never had the template in the first place so add it again and hopefully all works well.

Good Luck.

 

[Junicast]

Well it's just messed up.
I deleted all jails referring to my self made template, then completely removed the template and recreated a new one.
Before that I made sure that the tgz and mtree are fine:
1) In the original jail I used for the template went into /usr/local/etc/pkg/repos and did a sha256 on FreeBSD.conf:
d089146205709d0d5160dc3e23989c4d35114aa78b33571bc363fde35a5b8ecb
Also checked the content. It was ok, set to 64 bit.
2) extracted tgz and verified checksum. They are the same.
3) Looked into the mtree file, checksum is ok, too.
The problem remains the same. When I create a new jail with that template the file then contains the string "32" instead of "64". The checksum is different then, too.

So your file doesn't change after making use of such a self made template @joeschmusck?
I think it must be a bug honestly. Can't find an error on my side.

 

[SweetAndLow]

Well it's just messed up.
I deleted all jails referring to my self made template, then completely removed the template and recreated a new one.
Before that I made sure that the tgz and mtree are fine:
1) In the original jail I used for the template went into /usr/local/etc/pkg/repos and did a sha256 on FreeBSD.conf:
d089146205709d0d5160dc3e23989c4d35114aa78b33571bc363fde35a5b8ecb
Also checked the content. It was ok, set to 64 bit.
2) extracted tgz and verified checksum. They are the same.
3) Looked into the mtree file, checksum is ok, too.
The problem remains the same. When I create a new jail with that template the file then contains the string "32" instead of "64". The checksum is different then, too.

So your file doesn't change after making use of such a self made template @joeschmusck?
I think it must be a bug honestly. Can't find an error on my side.

You should try skipping the web upload part and see if it still is 32bit. If it's not you can easily file a bug against the upload template feature. Creating your own template is a feature no one uses so it's not crazy to have a bug there.

 

[joeschmuck]

So your file doesn't change after making use of such a self made template @joeschmusck?

The steps I listed above are what I did and it works perfectly. As @SweetAndLow recommended, skip the webserver and follow my steps, it could be the web server portion however I honestly don't see how that can be. My last recommendation is to do the following, install VMWare Workstation Player on a Windoze PC and create a new FreeNAS installation. Create your template on this system and see how it goes. Do not nake any changes to the jail such as add new programs or update anything, just make a clean template. If this doesn't work for you then you need to treat me like a total idiot and create an exact step by step procedure so I can replicate your steps because something is very wrong if you can't duplicate my results.

And use FreeNAS 11.0-U4, that is exactly what I'm using and I am using my main FreeNAS to do this work, it's faster than my play system but I will give this a try as well today just to ensure there is not something weird with a clean install.