FreeNAS installer + SanDisk SSD = checksum errors

[Jacopx]

Still some problem?
I have just bought one of this SanDisk SSD 120GB without reading this... Someone have solved the problem? I think i will use it for pfSense and i will buy a Kingston for FreeNAS.

 

[Krowvin]

I too stumbled on this thread after installing a SanDisk SSD Plus 120GB for my boot drive.

I ran a scrub on the drive after installing it from the GUI and was greeted with 100+ checksum errors.

I will most likely end up returning this drive and ordering another.

I have x2 - Samsung SSD 850 EVO 250GB
that I use for jail storage. I didn't see a 120GB model for Samsung.
Perhaps I will try Kingston for the replacement.


Here's an output for camcontrol, if someone can make heads or tails of it:

Show : camcontrol identify /dev/ada0 -v


camcontrol identify /dev/ada0 -v

camcontrol: sending ATA ATA_IDENTIFY with timeout of 30000 msecs
pass16: Raw identify data:
0: 0040 3fff c837 0010 0000 0000 003f 0000
8: 0000 0000 3138 3333 3335 3830 3538 3230
16: 2020 2020 2020 2020 0000 0000 0000 5545
24: 3435 3030 524c 5361 6e44 6973 6b20 5353
32: 4420 504c 5553 2031 3230 4742 2020 2020
40: 2020 2020 2020 2020 2020 2020 2020 8001
48: 4000 2f00 4000 0200 0000 0006 3fff 0010
56: 003f fc10 00fb 9101 8000 0df9 0000 0007
64: 0003 0078 0078 0078 0078 5e00 0000 0000
72: 0000 0000 0000 001f 850e 0004 0148 0040
80: 03f0 0110 346b 7d09 4123 3469 bc01 4123
88: 407f 0001 0000 0000 fffe 0000 0000 0000
96: 0000 0000 0000 0000 8000 0df9 0000 0000
104: 0000 0008 4000 0000 5001 b448 b940 c444
112: 0000 0000 0000 0000 0000 0000 0000 411c
120: 401c 0000 0000 0000 0000 0000 0000 0000
128: 0009 0000 0000 0000 0000 0000 0000 0000
136: 0000 0000 0000 0000 0000 0000 0000 0000
144: 0000 0000 0000 0000 0000 0000 0000 0000
152: 0000 0000 0000 0000 0000 0000 0000 0000
160: 0000 0000 0000 0000 0000 0000 0000 0000
168: 0003 0001 0000 0000 0000 0000 0000 0000
176: 2020 2020 2020 2020 2020 2020 2020 2020
184: 2020 2020 2020 2020 2020 2020 2020 2020
192: 2020 2020 2020 2020 2020 2020 2020 2020
200: 2020 2020 2020 2020 2020 2020 0000 0000
208: 0000 4000 0000 0000 0000 0000 0000 0000
216: 0000 0001 0000 0000 0000 0000 10ff 0000
224: 0000 0000 0000 0000 0000 0000 0000 0000
232: 0000 0000 0010 0010 0000 0000 0000 0000
240: 0000 0000 0000 0000 0000 0000 0000 0000
248: 0000 0000 0000 0000 0000 0000 0000 44a5

camcontrol: sending ATA READ_NATIVE_MAX_ADDRESS48 with timeout of 1000 msecs
pass16: Raw native max data:
0: 5000 ff00 f97f 0d40 0000 0000
error = 0x00, sector_count = 0x0000, device = 0x40, status = 0x50
pass16: <SanDisk SSD PLUS 120GB UE4500RL> ACS-2 ATA SATA 3.x device
pass16: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes)

protocol ATA/ATAPI-9 SATA 3.x
device model SanDisk SSD PLUS 120GB
firmware revision UE4500RL
cylinders 16383
heads 16
sectors/track 63
sector size logical 512, physical 512, offset 0
LBA supported 234455040 sectors
LBA48 supported 234455040 sectors
PIO supported PIO4
DMA supported WDMA2 UDMA6
media RPM non-rotating

Feature Support Enabled Value Vendor
read ahead yes yes
write cache yes yes
flush cache yes yes
overlap no
Tagged Command Queuing (TCQ) no no
Native Command Queuing (NCQ) yes 32 tags
NCQ Queue Management no
NCQ Streaming no
Receive & Send FPDMA Queued no
SMART yes yes
microcode download yes yes
security yes no
power management yes yes
advanced power management yes no 0/0x00
automatic acoustic management no no
media status notification no no
power-up in Standby no no
write-read-verify no no
unload no no
general purpose logging yes yes
free-fall no no
Data Set Management (DSM/TRIM) yes
DSM - max 512byte blocks yes 8
DSM - deterministic read yes any value
Host Protected Area (HPA) yes no 234455040/234455040
HPA - Security no


Here's what smartctl returned after running a short and long test

Show : smartctl output


=== START OF INFORMATION SECTION ===
Device Model: SanDisk SSD PLUS 120GB
Firmware Version: UE4500RL
User Capacity: 120,040,980,480 bytes [120 GB]
Sector Size: 512 bytes logical/physical
Rotation Rate: Solid State Device
Form Factor: 2.5 inches
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: ACS-2 T13/2015-D revision 3
SATA Version is: SATA 3.2, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is: Mon Oct 29 12:50:35 2018 CDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED


If I look at the drive errrors with
zpool status -v freenas-boot
I see the following errors

Show : errors

errors: Permanent errors have been detected in the following files:

<metadata>:<0x22>
<metadata>:<0x23>
<metadata>:<0x34>
<metadata>:<0x3d>
<metadata>:<0x49>
<metadata>:<0x55>
freenas-boot/ROOT/default:<0x0>
//conf/base/etc/rc.d/rctl
//conf/base/etc/rc.d/resolv
//conf/base/etc/rc.d/rtadvd
freenas-boot/ROOT/default@2018-10-28-21:14:05:<0x0>
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/migrate93/django/conf/locale/kn/__pycache__
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/conf/locale/sw
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/licenselib
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/migrate93/django/conf/locale/ko
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/migrate93/django/conf/locale/ko/LC_MESSAGES
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/bcrypt-3.1.3-py3.6.egg-info
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/share/zsh/5.4.2/functions/Completion/Debian/_cdbs-edit-pa tch
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/atom/widget/nls/zh
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/lockfile-0.12.2-py3.6.egg-inf o
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/bcrypt
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/migrate93/django/db/models/sql
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/site_perl/mach/5.24/Unicode/Map/IBM
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/bcrypt/__pycache__
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/migrate93/django/db/models/sql/__pycache__
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/site_perl/mach/5.24/Unicode/Map/ISO
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/jinja2/__pycache__/_stringdef s.cpython-36.opt-1.pyc
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/tastypie/__pycache__/http.cpy thon-36.opt-1.pyc
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/pyasn1_modules-0.0.9-py3.6.eg g-info
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/form/nls/pt-pt
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/atom/widget/templates
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/share/zsh/5.4.2/functions/Completion/Debian/_dput
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/pyasn1_modules
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/atom/widget/templates/css
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/admin/locale/l t/LC_MESSAGES
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/5.24/mach/auto/Filter/Util
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/form/nls/pt
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/freenasUI/dojango
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/av
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/admin/locale/l v/LC_MESSAGES
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/freenasUI/fnstatic
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/freenasUI/freeadmin
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/5.24/mach/auto/I18N/Langinfo
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/5.24/ExtUtils/Command
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/admin/locale/m l
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/admin/locale/m l/LC_MESSAGES
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/5.24/ExtUtils/Constant
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/share/locale/ja_JP.SJIS/LC_COLLATE
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/admin/locale/m n
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/form/nls/ro
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/gis/static/gis /js
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/share/locale/ja_JP.UTF-8/LC_COLLATE
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/gis/templates
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/gis/templates/ gis
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dijit/nls/nb
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/gis/templates/ gis/admin
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/5.24/ExtUtils/Liblist
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/libexec/netdata/python.d/python_modules/urllib3/packages
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/gis/templates/ gis/google
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/libexec/netdata/python.d/python_modules/urllib3/packages/ backports
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/form/nls/ru
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dijit/nls/nl
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/contrib/gis/templates/ gis/kml
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/libexec/netdata/python.d/python_modules/urllib3/packages/ ssl_match_hostname
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/share/locale/da/LC_MESSAGES/e2fsprogs.mo
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/libexec/netdata/python.d/python_modules/urllib3/util
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/google/cloud/storage/__pycach e__/_http.cpython-36.opt-1.pyc
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dijit/nls/pl
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/jinja2/_stringdefs.py
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/form/nls/sk
freenas-boot/ROOT/default@2018-10-28-21:14:05:/conf/base/etc/rc.d/rctl
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dijit/nls/pt-pt
freenas-boot/ROOT/default@2018-10-28-21:14:05:/conf/base/etc/rc.d/resolv
freenas-boot/ROOT/default@2018-10-28-21:14:05:/conf/base/etc/rc.d/rtadvd
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/south/tests/emptyapp/migratio ns
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/form/nls/sl
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dijit/nls/pt
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/south/tests/emptyapp/migratio ns/__pycache__
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/south/tests/fakeapp
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/boto3/resources/__pycache__/b ase.cpython-36.opt-1.pyc
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/requests/__pycache__/api.cpyt hon-36.opt-1.pyc
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/idna/__pycache__/intranges.cp ython-36.opt-1.pyc
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/site_perl/mach/5.24/Net/SSLeay.pod
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dijit/nls/ro
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/perl5/site_perl/mach/5.24/Unicode/CharName.pm
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/db/migrations/operatio ns
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dijit/nls/ru
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/share/locale/hu_HU.UTF-8/LC_COLLATE
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/www/dojo/dojox/atom/widget/nls/zh-tw
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/conf/locale/sv
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/conf/locale/sv/LC_MESS AGES
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/google_cloud_storage-1.4.0-py 3.6.egg-info
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/conf/locale/sv/__pycac he__
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/libzfs.so
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/migrate93/django/conf/locale/kn
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/migrate93/django/conf/locale/kn/LC_MESSAGES
freenas-boot/ROOT/default@2018-10-28-21:14:05:/usr/local/lib/python3.6/site-packages/django/core/checks/__pycache_ _/registry.cpython-36.opt-1.pyc
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/df-mnt-ssd_pool-jails-.warden-template-pluginjail-11 .0-x64/df_complex-used.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/interface-igb0/if_errors.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/processes/ps_state-stopped.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/df-mnt-tank-RESTORE_QT_FILES/df_complex-free.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/df-mnt-tank-RESTORE_QT_FILES/df_complex-reserved.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/df-mnt-tank-RESTORE_QT_FILES/df_complex-used.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/aggregation-cpu-sum/cpu-nice.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/zfs_arc_v2/gauge_arcstats_raw_evict-evict_skip.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/cpu-4/cpu-nice.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/zfs_arc_v2/gauge_arcstats_raw_l2write-l2_write_buffe r_bytes_scanned.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/disk-da12/disk_io_time.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/cputemp-0/temperature.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/zfs_arc_v2/gauge_arcstats_raw_mru-mru_ghost_hits.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/ctl-ioctl/disk_octets-0-0.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/zfs_arc_v2/gauge_arcstats_raw_prefetch-prefetch_data _hits.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/ctl-ioctl/disk_ops.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/zfs_arc_v2/gauge_arcstats_raw_prefetch-prefetch_meta data_misses.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/disk-da3/disk_io_time.rrd
/var/db/system/rrd-f6da24756e2f4dee86c3a9c9fb75829f/localhost/disk-da4/disk_io_time.rrd

Regarding the last few lines,
I have noticed my RRD data is now missing. Attempting to move my system > system dataset from from freenas-boot to another volume gave me an error in the gui. When I exit and go back to the system dataset it shows the new pool as the location. However it is not logging data.

So I believe the files are actually corrupted.
(I was thinking I could move my system dataset to the boot drive now that it was an SSD and could handle more read/writes someone correct me if it's still a good idea to have it on a primary volume if your boot data persists)

I don't know if the WD Reds spin down (I assume no b/c server drives), but this might also free your drives up of read/writes having the dataset on the boot drive/ssd pool.[/cmd]

 

[Krowvin]

It looks like the folks in this thread are having similar issues with the SanDisk SSD 120GB. I was not sure how to cross post and let them know too.

https://forums.freenas.org/index.php?threads/smart-error-followed-by-boot-failure.62309/

I too am getting 122 errors when trying to run commands. I cannot update the system either (GUI or CLI) i'm on U4, trying to get to U6

 

[Ericloewe]

You'll need either a different SSD or to install with TRIM disabled. They seem to have a firmware bug in newer versions. I have one of them and one of the equivalent WD Green ones, but both have an older firmware that seems to work fine.

 

[Ericloewe]

On further inspection, my WD Green unit with UE300000 firmware has this problem too.

 

[wreedps]

I am having lots of Sandiskt Fits false positive fail also.

 

[wreedps]

Using a Kingston now and its working perfectly for days.

 

[Andrew Ostrom]

I see that the bug report for this issue has been closed. It doesn't say it was fixed, though - does anyone know? I don't want to go through the pain of setting my server back up with the SSDs just to have it fail again... Thanks.

 

[Redcoat]

All outstanding Redmine bugs were closed out with the transition to Jira - it's "unresolved" there at https://jira.ixsystems.com/browse/NAS-100276

 

[RegularJoe]

FreeNAS 11.2-U6

I have the curse, the latest firmware from Sandisk and the AMS1061 PCIe 2.0 x1 card that runs 2 hard drives. The installer does not detect 4k sectors and adds quirks :

ahci1: quirks=0xc00000<NOCCS,NOAUX>
ahci1: quirks=0xc00000<NOCCS,NOAUX>

zdb | grep ashift
ashift: 9


In BIOS mode it took longer to get errors, I had to do a scrub of the boot volume where in UEFI mode doing a diskinfo -citv /dev/ada0 and ada1 made issues.

Here is my error :

[root@freenas ~]# zpool status -v
pool: freenas-boot
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub repaired 3K in 0 days 00:00:04 with 5 errors on Thu Nov 7 17:23:21 2019
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 5
mirror-0 ONLINE 0 0 11
ada0p2 ONLINE 0 0 12
ada1p2 ONLINE 0 0 11

errors: Permanent errors have been detected in the following files:

freenas-boot/ROOT/default@2019-11-07-23:06:38:<0x0>
freenas-boot/ROOT/default@2019-11-07-23:06:38:/usr/local/bin/delv
freenas-boot/ROOT/default@2019-11-07-23:06:38:/usr/local/bin/dig
freenas-boot/ROOT/default@2019-11-07-23:06:38:/usr/local/bin/mtools
freenas-boot/ROOT/default@2019-11-07-23:06:38:/usr/local/lib/python3.6/site-packages/markdown/__pycache__/postprocessors.cpy
thon-36.opt-1.pyc
[root@freenas ~]#