Several Questions VM issue / Memory / SSD install / Jails

[spacecabbie]

Hi all,

Been working with Freenas for about a year now maybe.
I have some questions i saved up:

1. I have 16gb of memory I run Plex / nzbget / sonarr / radarr / transmission / plexpy / webserver (spotnet)
CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 99.9% idle
Mem: 53M Active, 542M Inact, 505M Laundry, 13G Wired, 1110M Free
ARC: 11G Total, 2551M MFU, 8031M MRU, 29M Anon, 83M Header, 492M Other
9746M Compressed, 11G Uncompressed, 1.16:1 Ratio
Swap: 8192M Total, 1019M Used, 7172M Free, 12% Inuse
This is my load currently.
I am wondering should i start thinking of of more memory 32gb ? (note that 8gb is reserved atm for nothing see question 4.)

2. Jails I now run everything in separate jail, Can this apart from hdd space have a impact on cpu, memory, and power Usage. Instead of running all programs in one jail ?

3. Ssd install I have all my 8 sata ports in use i really like to install a ssd for par and unpacking (run the nzbget jail on that mount) and a Maintenance VM.
I could install a sata card. i prefer not to because of extra power usage. A usb stick could do it but i hear they cannot handle the continue reading and writing ?
So i am thinking a ssd via a usb 3.0 adapter is this viable ?
The Board has:
1 PCI-E 3.0 x8 (in x16),
1 PCI-E 3.0 x8,
2 PCI-E 3.0 x4 (in x8)
So maybe an PCI-E Ssd but i have used a ocd before i was not happy with it is there any PCI-E ssd that can be used in this setting ?
Any other suggestions as always are welcome.

4. I have created a VM with windows 10 on it but it runs horribly slow not sure what's going on maybe related to question 1
Also when i tinker with it and try to improve it when i reboot or change items and it shuts down. This is not a big deal it happens but when restart i get this message below when trying to boot it again.
[2018/07/06 15:24:36] (WARNING) VMService.__init_guest_vmemory():830 - ===> Cannot guarantee memory for guest id: 1
It keeps the memory reserved atm i can only clear it by rebooting freenas.
Is there a way to free up this memory without reboot?

Some additional questions about the vm:
4.1 what os is lightweight but has graphical smb explorer unrar par chrome browser support. It's supposed to be remotely run via teamviewer to do manual tasks connected to programs above it has to be graphical i am not a fan of command line work.
4.1 The vm can it be slow cause i run it on the RAIDZ2 file system ? Task Manager cpu show 100% usage and drive close to 99 at start later around 60-80%.
4.2 I would like to be able to start this vm remotely without having to allow access to freenas web interface.

==Current specs and versions (I am on nightly's atm)===

Code:

OS Version:
FreeNAS-11.2-MASTER-201807010447
(Build Date: Jul 1, 2018 7:47)

Processor:
Intel(R) Xeon(R) CPU E3-1230 v6 @ 3.50GHz (8 cores)

Memory:
16 GB

Uptime:
5:56PM up 5 days, 1:16, 1 user


last pid: 50001;  load averages:  0.19,  0.20,  0.17																															  up 5+01:20:32  18:00:32
78 processes:  1 running, 77 sleeping
CPU:  0.1% user,  0.0% nice,  0.0% system,  0.0% interrupt, 99.8% idle
Mem: 98M Active, 601M Inact, 486M Laundry, 13G Wired, 945M Free
ARC: 11G Total, 2502M MFU, 8076M MRU, 2244K Anon, 82M Header, 491M Other
	 9754M Compressed, 11G Uncompressed, 1.17:1 Ratio
Swap: 8192M Total, 973M Used, 7218M Free, 11% Inuse

  PID USERNAME	 THR PRI NICE   SIZE	RES STATE   C   TIME	WCPU COMMAND
  241 root		  18  49	0   277M   177M kqread  2  37:53   0.55% python3.6
57429 spacecabbie   21  52	0   447M   148M uwait   5  25:25   0.28% Plex Media Server
 6563 spacecabbie	7  20	0   104M 18596K nanslp  7  10:24   0.11% nzbget
57430 spacecabbie   13  52   15   268M 59336K piperd  6   6:08   0.10% Plex Script Host
57522 spacecabbie   14  52	0   341M   127M piperd  5  94:31   0.07% Plex Script Host
49455 root		   1  20	0  7956K  4272K CPU0	0   0:00   0.06% top
49437 root		   1  20	0  7956K  3856K select  6   0:00   0.05% top
57516 spacecabbie   10  52	0   169M  3872K piperd  2   1:52   0.03% Plex Script Host
 7299 spacecabbie   15  20	0   491M   143M uwait   4  16:21   0.02% mono-sgen
 7306 spacecabbie   14  20	0   324M   138M uwait   4  20:55   0.01% mono-sgen
 2984 www			1  20	0 12856K  2636K kqread  7   0:00   0.01% nginx
44376 root		   1  20	0 13252K  8360K select  2   0:00   0.01% sshd
83463 root		   1  20	0   176M 20680K select  4   0:01   0.01% smbd
 2364 root		   1  20	0 12496K 12604K select  4   0:12   0.01% ntpd
57435 spacecabbie   13  52	0   218M 20540K uwait   0   1:23   0.00% Plex DLNA Server
 3051 root		   1  20	0   135M  4824K kqread  2   0:21   0.00% uwsgi-3.6
 3199 root		   1  20	0 15668K  4424K select  6   0:11   0.00% httpd
 5324 root		   1  20	0 10472K   584K select  2   0:00   0.00% syslogd
 3179 root		  12  20	0   155M 37164K nanslp  3  15:03   0.00% collectd
 2043 root		   1 -52   r0  3528K  3592K nanslp  7   1:08   0.00% watchdogd
 3833 root		  15  20	0   183M 62872K umtxn   1   0:08   0.00% uwsgi-3.6
95842 spacecabbie	1  20	0 23588K	 0K kqread  0   0:08   0.00% <EasyAudioEncoder>
 2039 root		   2  20	0 21224K  3204K kqread  3   0:07   0.00% syslog-ng
 2571 root		   1  20	0 85436K 47608K select  3   0:05   0.00% winbindd
 2562 root		   1  20	0 37200K  5676K select  4   0:05   0.00% nmbd
 2566 root		   1  20	0   170M  5556K select  2   0:02   0.00% smbd
  351 root		   2  21	0 77476K 34704K usem	1   0:02   0.00% python3.6
 1183 nobody		 1  20	0  7152K  2280K select  3   0:02   0.00% mdnsd
 2996 nobody		 1  20	0  7152K  2340K select  1   0:02   0.00% mdnsd
  350 root		   2  20	0 73812K 29084K piperd  4   0:02   0.00% python3.6
 4985 root		   1  52	0  6504K   776K nanslp  6   0:02   0.00% cron
 5203 root		   1  52	0 75604K  4588K ttyin   3   0:01   0.00% python3.6
 2789 root		   1  21	0  7108K   904K wait	5   0:01   0.00% sh
 2763 root		   1  20	0  8736K  1996K nanslp  0   0:01   0.00% smartd
 2576 root		   1  20	0 45196K  7928K select  3   0:01   0.00% winbindd
 3203 root		   1  20	0 15668K   284K kqread  3   0:01   0.00% httpd
 6915 root		   1  20	0 12592K   488K nanslp  6   0:01   0.00% cron
 7313 root		   1  20	0 12592K   400K nanslp  0   0:01   0.00% cron
 6570 root		   1  20	0 12592K   452K nanslp  4   0:01   0.00% cron
 2577 root		   1  20	0   128M  5052K select  5   0:01   0.00% smbd
 2685 root		   1  20	0 46324K  6088K select  3   0:01   0.00% winbindd
 2684 root		   1  20	0 45444K  4928K select  6   0:01   0.00% winbindd
 6849 root		   1  20	0 10500K   652K select  4   0:00   0.00% syslogd
 6513 root		   1  20	0 10500K   504K select  3   0:00   0.00% syslogd
 7245 root		   1  20	0 10500K   608K select  6   0:00   0.00% syslogd
 2296 root		   1  52	0 12508K  1772K rpcsvc  0   0:00   0.00% rpc.lockd
 5386 root		   1  20	0 12568K   448K nanslp  3   0:00   0.00% cron
 2290 root		   4  52	0  6264K  1804K rpcsvc  3   0:00   0.00% nfsd
 2578 root		   1  20	0   128M  5528K select  3   0:00   0.00% smbd
 2063 root		   1  20	0  6408K  1916K select  2   0:00   0.00% rpcbind
 2293 root		   1  20	0   268M  1824K select  7   0:00   0.00% rpc.statd
 1705 root		   1  20	0  9176K  2040K select  2   0:00   0.00% devd
  249 root		   1  52	0 16976K  2260K piperd  1   0:00   0.00% python3.6

root@FileServer:~ # zpool status -v
 pool: BlackHole
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
		still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
		the pool may no longer be accessible by software that does not support
		the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0 days 08:47:26 with 0 errors on Sun Jun  3 08:47:30 2018
config:

		NAME											STATE	 READ WRITE CKSUM
		BlackHole									   ONLINE	   0	 0	 0
		  raidz2-0									  ONLINE	   0	 0	 0
			gptid/4c76ceb7-a3d4-11e7-80c8-ac1f6b233196  ONLINE	   0	 0	 0
			gptid/74ca7afe-9bcd-11e7-bed9-ac1f6b233196  ONLINE	   0	 0	 0
			gptid/7607e004-9bcd-11e7-bed9-ac1f6b233196  ONLINE	   0	 0	 0
			gptid/773238c7-9bcd-11e7-bed9-ac1f6b233196  ONLINE	   0	 0	 0
			gptid/782f40cb-9bcd-11e7-bed9-ac1f6b233196  ONLINE	   0	 0	 0
			gptid/78f09b23-9bcd-11e7-bed9-ac1f6b233196  ONLINE	   0	 0	 0
			gptid/79b809b6-9bcd-11e7-bed9-ac1f6b233196  ONLINE	   0	 0	 0
			gptid/7a7f7415-9bcd-11e7-bed9-ac1f6b233196  ONLINE	   0	 0	 0

errors: No known data errors

  pool: freenas-boot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:00:29 with 0 errors on Mon Jul  2 03:45:29 2018
config:

		NAME		STATE	 READ WRITE CKSUM
		freenas-boot  ONLINE	   0	 0	 0
		  da0p2	 ONLINE	   0	 0	 0

errors: No known data errors

root@FileServer:~ # zpool iostat 1
				 capacity	 operations	bandwidth
pool		  alloc   free   read  write   read  write
------------  -----  -----  -----  -----  -----  -----
BlackHole	 15.7T  6.06T	  3	 45   268K   558K
freenas-boot  2.71G  12.2G	  0	  0  13.7K	372
------------  -----  -----  -----  -----  -----  -----
BlackHole	 15.7T  6.06T	  0	  0	  0	  0
freenas-boot  2.71G  12.2G	  0	  0	  0	  0
------------  -----  -----  -----  -----  -----  -----
BlackHole	 15.7T  6.06T	  0	  0	  0	  0
freenas-boot  2.71G  12.2G	  0	  0	  0	  0
------------  -----  -----  -----  -----  -----  -----
BlackHole	 15.7T  6.06T	  0	197	  0  1.38M
freenas-boot  2.71G  12.2G	  0	  0	  0	  0
------------  -----  -----  -----  -----  -----  -----
BlackHole	 15.7T  6.06T	  0	  0	  0	  0
freenas-boot  2.71G  12.2G	  0	  0	  0	  0
------------  -----  -----  -----  -----  -----  -----
BlackHole	 15.7T  6.06T	  0	  0	  0	  0
freenas-boot  2.71G  12.2G	  0	  0	  0	  0
------------  -----  -----  -----  -----  -----  -----
BlackHole	 15.7T  6.06T	  0	 12	  0   443K
freenas-boot  2.71G  12.2G	  0	  0	  0	  0
------------  -----  -----  -----  -----  -----  -----

 

[HoneyBadger]

Short answer: You are starved for memory, add more.

1. That is far too many jails/plugins to be running on 16GB of RAM, as evidenced by the fact that you are swapping to disk. Disable some of those plugins and/or upgrade to 32GB of RAM ASAP.

2. Managing each jail will have some overhead, if you collapse multiple non-conflicting plugins into one it may improve things. I would recommend to stop/disable some of them until your swap returns to 0% used.

3. I would not recommend running ZFS over USB at all. There are several PCIe SSDs from Samsung, Intel, or other vendors (that aren't OCZ) that will work fine - alternatively you could use a $10 PCIe-to-M.2 adapter card and install a small NVMe SSD such as an Optane drive. Most

4. You are already starved for memory, a bhyve VM will eat into that even more significantly. It is definitely related to question #1.
4.1 Linux with a lightweight X server (XFCE) may suffice, but you need to get more RAM regardless as you are already swapping.
4.2 RAIDZ is a poor choice for VM use but in this case the memory is a far bigger problem.
4.3 You can connect a terminal session via ssh and build a script to launch the bhyve VM with the appropriate parameters. If you are simply looking for "VM powers on with FreeNAS" then you could add this script to the post-boot section.

You'll note that 3/4 of your answers return to the core issue of "insufficient RAM" - while 16GB is fine for a system that's not doing much aside from serving files, you're definitely asking a lot more from your box. Feed it the RAM it needs and I imagine you'll find all of those problems rapidly resolve themselves.

 

[spacecabbie]

Short answer: You are starved for memory, add more.

1. That is far too many jails/plugins to be running on 16GB of RAM, as evidenced by the fact that you are swapping to disk. Disable some of those plugins and/or upgrade to 32GB of RAM ASAP.

2. Managing each jail will have some overhead, if you collapse multiple non-conflicting plugins into one it may improve things. I would recommend to stop/disable some of them until your swap returns to 0% used.

3. I would not recommend running ZFS over USB at all. There are several PCIe SSDs from Samsung, Intel, or other vendors (that aren't OCZ) that will work fine - alternatively you could use a $10 PCIe-to-M.2 adapter card and install a small NVMe SSD such as an Optane drive. Most

4. You are already starved for memory, a bhyve VM will eat into that even more significantly. It is definitely related to question #1.
4.1 Linux with a lightweight X server (XFCE) may suffice, but you need to get more RAM regardless as you are already swapping.
4.2 RAIDZ is a poor choice for VM use but in this case the memory is a far bigger problem.
4.3 You can connect a terminal session via ssh and build a script to launch the bhyve VM with the appropriate parameters. If you are simply looking for "VM powers on with FreeNAS" then you could add this script to the post-boot section.

You'll note that 3/4 of your answers return to the core issue of "insufficient RAM" - while 16GB is fine for a system that's not doing much aside from serving files, you're definitely asking a lot more from your box. Feed it the RAM it needs and I imagine you'll find all of those problems rapidly resolve themselves.

Fair enough RAM Ordered. but.
After reboot the swap returns to 0 used

Code:

last pid:  6524;  load averages:  0.66,  0.22,  0.08																															  up 0+00:01:44  21:02:49
88 processes:  2 running, 86 sleeping
CPU:  7.1% user,  0.0% nice,  1.3% system,  0.0% interrupt, 91.6% idle
Mem: 765M Active, 226M Inact, 1130M Wired, 13G Free
ARC: 603M Total, 167M MFU, 414M MRU, 1070K Anon, 3286K Header, 17M Other
	 157M Compressed, 449M Uncompressed, 2.86:1 Ratio
Swap: 8192M Total, 8192M Free

This is du to the vm crashing and keeping the memory of 8gb reserved for some reason. So its not yet so dramatic even with vm running the swap stay at 0% but close.
any recommendations for type of pci-E cheap ssd ?

 

[HoneyBadger]

I'm guessing that the VM is crashed/nonresponsive from the OS perspective, but the virtual machine itself is still consuming bhyve resources, which is why it isn't freeing the memory. 16GB is still not enough if you want to have an 8GB VM and still run all of the jails though. 32GB should be sufficient.

any recommendations for type of pci-E cheap ssd ?

"Cheap" and "PCIe SSD" don't tend to go together. Your best option might be to use an NVMe M.2 SSD and a PCIe-to-M.2 adapter card - StarTech makes a good one, search "PEX4M2E1" on Amazon. I've had mixed results with the cheaper $10-15 ones; compatibility is less solid I've found but you at least know immediately if they work or not.

Personally though I wouldn't be too concerned with the power consumption of an additional low-end HBA, as you'll likely spend much more from a $/GB perspective buying an NVMe M.2 card and adapter card versus an HBA, SATA SSD, and the extra $10/yr in power it might consume.

 

[Chris Moore]

I would get a SAS HBA and put all the mass storage drives on it. The SATA ports would be fine for some small, inexpensive SATA SSDs.
It is only 7 to 10 watts for a good, reasonably new, SAS controller.

Sent from my SAMSUNG-SGH-I537 using Tapatalk