Can not create snapshot: out of space error

M

mahmud100

Cadet
Joined
May 11, 2023
Messages
6
I am new to TrueNAS Core, and have spent the last couple of days on the forums and YouTube to try and solve this issue. I do not know what I may be missing and need a little help.

1683836623702.png


Code:

Error: concurrent.futures.process._RemoteTraceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 1019, in do_create
    ds.properties['freenas:vmsynced'] = libzfs.ZFSUserProperty('Y')
  File "libzfs.pyx", line 402, in libzfs.ZFS.__exit__
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 1016, in do_create
    ds.snapshot(f'{dataset}@{name}', recursive=recursive, fsopts=properties)
  File "libzfs.pyx", line 3798, in libzfs.ZFSDataset.snapshot
libzfs.ZFSException: out of space

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/concurrent/futures/process.py", line 246, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 111, in main_worker
    res = MIDDLEWARE._run(*call_args)
  File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 45, in _run
    return self._call(name, serviceobj, methodobj, args, job=job)
  File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 39, in _call
    return methodobj(*params)
  File "/usr/local/lib/python3.9/site-packages/middlewared/worker.py", line 39, in _call
    return methodobj(*params)
  File "/usr/local/lib/python3.9/site-packages/middlewared/schema.py", line 979, in nf
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/plugins/zfs.py", line 1024, in do_create
    raise CallError(f'Failed to snapshot {dataset}@{name}: {err}')
middlewared.service_exception.CallError: [EFAULT] Failed to snapshot Storage_Pool/Company_Data/Company_Data@manual-2023-05-11_14-51: out of space
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 139, in call_method
    result = await self.middleware._call(message['method'], serviceobj, methodobj, params, app=self)
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1236, in _call
    return await methodobj(*prepared_call.args)
  File "/usr/local/lib/python3.9/site-packages/middlewared/service.py", line 496, in create
    rv = await self.middleware._call(
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1244, in _call
    return await self._call_worker(name, *prepared_call.args)
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1250, in _call_worker
    return await self.run_in_proc(main_worker, name, args, job)
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1169, in run_in_proc
    return await self.run_in_executor(self.__procpool, method, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/middlewared/main.py", line 1152, in run_in_executor
    return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs))
middlewared.service_exception.CallError: [EFAULT] Failed to snapshot Storage_Pool/Company_Data/Company_Data@manual-2023-05-11_14-51: out of space

********************************************************************************************************************************

root@SIVERNAS1P[~]# zfs list
NAME                                                            USED  AVAIL     REFER  MOUNTPOINT
Storage_Pool                                                   4.43T  2.49T      151K  /mnt/Storage_Pool
Storage_Pool/.system                                           52.2M  2.49T      163K  legacy
Storage_Pool/.system/configs-2663265f094743bfbd6dcc41661bf096  12.1M  2.49T     12.1M  legacy
Storage_Pool/.system/cores                                      140K  1024M      140K  legacy
Storage_Pool/.system/rrd-2663265f094743bfbd6dcc41661bf096      38.1M  2.49T     38.1M  legacy
Storage_Pool/.system/samba4                                     442K  2.49T      442K  legacy
Storage_Pool/.system/services                                   140K  2.49T      140K  legacy
Storage_Pool/.system/syslog-2663265f094743bfbd6dcc41661bf096   1.01M  2.49T     1.01M  legacy
Storage_Pool/.system/webui                                      140K  2.49T      140K  legacy
Storage_Pool/Company_Data                                      4.43T  2.49T      140K  /mnt/Storage_Pool/Company_Data
Storage_Pool/Company_Data/Company_Data                         4.43T  4.36T     2.56T  -
Storage_Pool/iocage                                            9.41M  2.49T     8.59M  /mnt/Storage_Pool/iocage
Storage_Pool/iocage/download                                    140K  2.49T      140K  /mnt/Storage_Pool/iocage/download
Storage_Pool/iocage/images                                      140K  2.49T      140K  /mnt/Storage_Pool/iocage/images
Storage_Pool/iocage/jails                                       140K  2.49T      140K  /mnt/Storage_Pool/iocage/jails
Storage_Pool/iocage/log                                         140K  2.49T      140K  /mnt/Storage_Pool/iocage/log
Storage_Pool/iocage/releases                                    140K  2.49T      140K  /mnt/Storage_Pool/iocage/releases
Storage_Pool/iocage/templates                                   140K  2.49T      140K  /mnt/Storage_Pool/iocage/templates
boot-pool                                                      3.88G   142G       24K  none
boot-pool/ROOT                                                 3.87G   142G       24K  none
boot-pool/ROOT/13.0-U3.1                                        183K   142G     1.29G  /
boot-pool/ROOT/13.0-U4                                         3.87G   142G     1.29G  /
boot-pool/ROOT/Initial-Install                                    1K   142G     1.29G  legacy
boot-pool/ROOT/default                                          182K   142G     1.29G  legacy

******************************************************************************************************************************

root@SIVERNAS1P[~]# zfs list -o space,quota
NAME                                                           AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD  QUOTA
Storage_Pool                                                   2.49T  4.43T        0B    151K             0B      4.43T   none
Storage_Pool/.system                                           2.49T  52.2M        0B    163K             0B      52.0M   none
Storage_Pool/.system/configs-2663265f094743bfbd6dcc41661bf096  2.49T  12.1M        0B   12.1M             0B         0B   none
Storage_Pool/.system/cores                                     1024M   140K        0B    140K             0B         0B     1G
Storage_Pool/.system/rrd-2663265f094743bfbd6dcc41661bf096      2.49T  38.1M        0B   38.1M             0B         0B   none
Storage_Pool/.system/samba4                                    2.49T   442K        0B    442K             0B         0B   none
Storage_Pool/.system/services                                  2.49T   140K        0B    140K             0B         0B   none
Storage_Pool/.system/syslog-2663265f094743bfbd6dcc41661bf096   2.49T  1.01M        0B   1.01M             0B         0B   none
Storage_Pool/.system/webui                                     2.49T   140K        0B    140K             0B         0B   none
Storage_Pool/Company_Data                                      2.49T  4.43T        0B    140K             0B      4.43T   none
Storage_Pool/Company_Data/Company_Data                         4.36T  4.43T        0B   2.56T          1.87T         0B      -
Storage_Pool/iocage                                            2.49T  9.41M        0B   8.59M             0B       837K   none
Storage_Pool/iocage/download                                   2.49T   140K        0B    140K             0B         0B   none
Storage_Pool/iocage/images                                     2.49T   140K        0B    140K             0B         0B   none
Storage_Pool/iocage/jails                                      2.49T   140K        0B    140K             0B         0B   none
Storage_Pool/iocage/log                                        2.49T   140K        0B    140K             0B         0B   none
Storage_Pool/iocage/releases                                   2.49T   140K        0B    140K             0B         0B   none
Storage_Pool/iocage/templates                                  2.49T   140K        0B    140K             0B         0B   none
boot-pool                                                       142G  3.88G        0B     24K             0B      3.88G   none
boot-pool/ROOT                                                  142G  3.87G        0B     24K             0B      3.87G   none
boot-pool/ROOT/13.0-U3.1                                        142G   183K        0B    183K             0B         0B   none
boot-pool/ROOT/13.0-U4                                          142G  3.87G     2.58G   1.29G             0B         0B   none
boot-pool/ROOT/Initial-Install                                  142G     1K        0B      1K             0B         0B   none
boot-pool/ROOT/default

*******************************************************************************************************************************

root@SIVERNAS1P[~]# zpool status
  pool: Storage_Pool
 state: ONLINE
config:

        NAME                                            STATE     READ WRITE CKSUM
        Storage_Pool                                    ONLINE       0     0     0
          raidz2-0                                      ONLINE       0     0     0
            gptid/40114790-ec47-11ed-b577-fcaa1499c7c2  ONLINE       0     0     0
            gptid/404532db-ec47-11ed-b577-fcaa1499c7c2  ONLINE       0     0     0
            gptid/40283f57-ec47-11ed-b577-fcaa1499c7c2  ONLINE       0     0     0
            gptid/403703be-ec47-11ed-b577-fcaa1499c7c2  ONLINE       0     0     0

errors: No known data errors

  pool: boot-pool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:10 with 0 errors on Fri May  5 03:45:10 2023
config:

        NAME        STATE     READ WRITE CKSUM
        boot-pool   ONLINE       0     0     0
          ada4p2    ONLINE       0     0     0

errors: No known data errors

*******************************************************************************************************************************

root@SIVERNAS1P[~]# zpool list
NAME           SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
Storage_Pool  14.5T  5.29T  9.26T        -         -     0%    36%  1.00x    ONLINE  /mnt
boot-pool      151G  3.88G   147G        -         -     0%     2%  1.00x    ONLINE  -
root@SIVERNAS1P[~]#
 

Attachments

  • 1683837001173.png
    1683837001173.png
    41.5 KB · Views: 372
NugentS

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,947
Well I would say you have run out of disk space. What makes you think it isn't that?

Look at the dashboard - the Pool widgit.
Or
Storage/Pools and post a screenshot of what you see there
 
NugentS

NugentS

MVP
Joined
Apr 16, 2020
Messages
2,947
Can you post your hardware please - as per forum rules
 
M

mahmud100

Cadet
Joined
May 11, 2023
Messages
6
Datto S3B2000
Intel(R) Core(TM) i3-4150T CPU @ 3.00GHz
16MB RAM
1 167.68 GiB SSD HDD - System boot
4 4TB WD Red HDD
Storage_Pool RAID2 4.4 TiB
iocage RAID2 10 MiB
TrueNAS-13.0-U4 (Core)
Use Case: Primary File and Media Storage
 
Last edited:
Whattteva

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
Is Company_Data a zvol or a dataset? That distinction matters when you're taking a snapshot.
 
Whattteva

Whattteva

Wizard
Joined
Mar 5, 2013
Messages
1,824
AHA! That's your problem. You're trying to take a snapshot of a 4 TiB zvol with only 2.49 TiB space left.
Yes I know, snapshots are supposed to be lightweight, but zvols are an exception unfortunately.

Basically, if you do a zfs list Storage_Pool/Company_Data/Company_Data you might get something like this:
Code:
NAME                          USED  AVAIL     REFER  MOUNTPOINT
Storage_Pool/Company_Data/Company_Data  4.43T   <some_amount>    4.04T  -


In order for that zvol to be snapshottable, you must have more space available than what's listed in the REFER section.
 
Last edited:
M

mahmud100

Cadet
Joined
May 11, 2023
Messages
6
Whattteva said:
AHA! That's your problem. You're trying to take a snapshot of a 4 TiB zvol with only 2.49 TiB space left.
Yes I know, snapshots are supposed to be lightweight, but zvols are an exception unfortunately.

Basically, if you do a zfs list Storage_Pool/Company_Data/Company_Data you might get something like this:
Code:
NAME                          USED  AVAIL     REFER  MOUNTPOINT
Storage_Pool/Company_Data/Company_Data  4.43T   <some_amount>    4.04T  -


In order for that zvol to be snapshottable, you must have more space available than what's listed in the REFER section.
Click to expand...
Thanks, I will have to upgrade the disk set.
 
You must log in or register to reply here.

Important Announcement for the TrueNAS Community.

The TrueNAS Community has now been moved. This forum will now become READ-ONLY for historical purposes. Please feel free to join us on the new TrueNAS Community Forums.

Related topics on forums.truenas.com for thread: "Can not create snapshot: out of space error"

Similar threads

D
  • Locked
Snapshot out of space
Replies
8
Views
4K
jgreco
jgreco
V
Cannot create snapshot - out of space
Replies
11
Views
5K
vicpassy
V
Aaron C. de Bruyn
  • Locked
Cannot create snapshot: out of space
2
Replies
31
Views
15K
Kostya Berger
K
J
cannot create snapshot out of space
Replies
19
Views
6K
Ericloewe
Ericloewe
L
  • Locked
cannot create snapshot - out of space
Replies
13
Views
13K
human
human
Top