12.0-BETA2.1 Now Available

Kris Moore

SVP of Engineering
Administrator
Moderator
iXsystems
Joined
Nov 12, 2015
Messages
1,471
Note: 12.0-BETA2.1 is a hot-fix release to address a ZFS permissions issue discovered during BETA testing. Details included in release notes.



iXsystems is pleased to announce the general availability of TrueNAS 12.0-BETA2! This is the second beta version of the unified FreeNAS and TrueNAS release, renamed to TrueNAS CORE and TrueNAS Enterprise, respectively. More than 175 bugs have been fixed in this release, representing a significant step toward the full TrueNAS 12.0-Release.

TrueNAS 12.0
The TrueNAS 12.0 release is a major new update to the TrueNAS family of storage products. TrueNAS CORE is the successor to FreeNAS, which is still fully open-source and offering the same functionality and more features than ever before. Included in version 12 are major changes to ZFS filesystem, bringing all the features of ZFS on Linux (ZoL) into the upcoming OpenZFS 2.0 release. Additionally, version 12 includes these major changes:

  • ZFS Native Crypto - Per dataset encryption and support for encrypted replication to untrusted targets.
  • 2 Factor Authentication - Allows using Applications such as “Google Authenticator” to provide an extra level of security before logging into the UI or SSH access.
  • KMIP Support - TrueNAS Enterprise now adds the ability to interface with KMIP servers for the storage and retrieval of passwords and encryption keys.
  • Performance Improvements - Virtually every area of the platform has been updated and includes some major performance improvements, including SMB, iSCSI, ZFS and more.
  • API Keys - All of TrueNAS can be configured with TrueNAS API v2.0, and now supports keys for remote access. TrueNAS API v1.0 is no longer supported.
  • Fusion Pools - Allows creating pools with all-flash vdevs that store metadata and small-block IO. This provides flash-like performance on all metadata operations, but with the cost effectiveness of spinning disk for storage blocks.
  • VPN Support - Support for TrueNAS acting as both an OpenVPN Server and Client, depending on your VPN needs.
These major changes are also paired with numerous other bug fixes and quality-of-life type improvements to the UI and APIs for users. We’ve also revamped our Documentation, focusing more content on how to accomplish specific tasks, reducing the learning curve and time normally spent on deploying TrueNAS in a variety of storage environments.

Release Notes:

Download:
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,112
We're still syncing with upstream OpenZFS, so 12.0-RC1 should be built on the recently tagged branches of OpenZFS 2.0/RC as well.
Sweet. Persistent L2ARC was in RC1, yet another thing to get excited for. Sounds like it's time for another bulk SSD order.
 
Joined
Jan 4, 2014
Messages
1,644
Idle CPU use on 12 is more than double that for 11.3. This is somewhat worrying.

screenshot.508.png
 
Joined
Jan 4, 2014
Messages
1,644
Would be useful to know any specifics... is there any process that is consuming abnormal amounts of CPU in your case. It's unlikely to be all of them.
It's a bare-bone, unadulterated test server. No jails, VMs and so on, just FreeNAS or TrueNAS.

A minute-long video displaying system processes for the test server running 12.0-beta2.1. Now, the same server running 11.3-U4.1. Note there's a lot more idle CPU available in 11.3.
 
Last edited:

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
@Basil Hendroff, on Beta2.1, it appears Python is stalling quite a bit. I wonder if it's just-in-time compiling all middleware modules, but not caching them? Are there *.pyc files in /usr/local/lib/python3.8/site-packages/middlewared/plugins/__pycache__/? Likewise throughout the python3.8 tree, there should be __pycache__ directories with *.pyc compiled Python modules.
 
Joined
Jan 4, 2014
Messages
1,644
on Beta2.1, it appears Python is stalling quite a bit. I wonder if it's just-in-time compiling all middleware modules, but not caching them?
You figured that out just by looking at system processes?! I bet you're one of those guys that can read the Matrix.

Are there *.pyc files in /usr/local/lib/python3.8/site-packages/middlewared/plugins/__pycache__/?
Yes, there are.

Likewise throughout the python3.8 tree, there should be __pycache__ directories with *.pyc compiled Python modules.
Where do I look? I'm having to go offline now. will check later.
 

Samuel Tai

Never underestimate your own stupidity
Moderator
Joined
Apr 24, 2020
Messages
5,399
Where do I look? I'm having to go offline now. will check later.

All throughout /usr/local/lib/python3.8/. However, the fact the middlewared section does have compiled *.pyc files refutes my hypothesis. Nevertheless, Python (process 71) is the most active process. What does ps -auxww -p 71 display?
 
Last edited:

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
It's a bare-bone, unadulterated test server. No jails, VMs and so on, just FreeNAS or TrueNAS.

A minute-long video displaying system processes for the test server running 12.0-beta2.1. Now, the same server running 11.3-U4.1. Note there's a lot more idle CPU available in 11.3.
"top -SHIz" might be more useful. Also check /var/log/middlewared.log for errors.
 
Joined
Jan 4, 2014
Messages
1,644
Output from top -SHIz for 12.0-beta2.1 in this minute-long video.

The middlewared log file is attached. I'm not sure what I'm looking at, but there's a recurring task blocked warning and a couple of other errors.

I have noticed if I wait long enough (hours maybe?), 12.0 does eventually sort itself out and idle CPU use drops to a more reasonable level (though never as low as 11.3).

screenshot.512.png
 

Attachments

  • middlewared.txt
    40.5 KB · Views: 606
Last edited:
Joined
Jan 4, 2014
Messages
1,644
These three TrueCommand resource graphs taken at different points in time show how long it takes for 12.0 to drop below 50% idle CPU use. I basically rebooted from 11.3 to 12.0 at around 3 PM. At 5:10 PM it had just crossed the 50% idle CPU use boundary. It's still going down.

screenshot.515.png

screenshot.516.png

screenshot.517.png
 
Joined
Jan 4, 2014
Messages
1,644
Around 6:45 PM local time and it's still dropping.

screenshot.518.png


Around 8:34 PM local time and it's starting to plateau.

screenshot.519.png


A check around 11 PM local time shows that idle CPU use has plateaued at around 42%.

screenshot.520.png


Compare this with idle CPU use of 29% for the same hardware running 11.3-U4.1...

screenshot.523.png
 
Last edited:

Stilez

Guru
Joined
Apr 8, 2016
Messages
529
A quick bit of feedback. Beta2.1 is looking really good.

A couple of the big/really annoying bugs I reported against beta2 looks tentatively fixed - THANK YOU!!
  • Limted zfs holds causing replication (zfs send) to fail - fixed.
  • Middleware client-server/front-back end disconnections - at the least a hell of a lot lower, may well be fixed, but too soon to tell. Not seen ANY yet.
 
Top