11TB disk import terminated at 2.5 TB.

[pnunn]

Hi there,

I've been importing an 11TB data set using a usb drive, formatted as ntfs and it all just seemed to work. However, I've just now discovered that after about 12 hours the import has simply stopped after 2.5 odd TB's of data.

I tried starting it again, but was presented with a bunch of python exceptions.

I had a look at the end of the log and found

Oct 19 09:13:01 cwpstorage ugen0.4: <JMicron External Disk 3.0> at usbus0
Oct 19 09:13:01 cwpstorage umass0 numa-domain 0 on uhub0
Oct 19 09:13:01 cwpstorage umass0: <MSC Bulk-Only Transfer> on usbus0
Oct 19 09:13:01 cwpstorage umass0: SCSI over Bulk-Only; quirks = 0x8100
Oct 19 09:13:01 cwpstorage umass0:11:0: Attached to scbus11
Oct 19 09:13:01 cwpstorage da8 at umass-sim0 bus 0 scbus11 target 0 lun 0
Oct 19 09:13:01 cwpstorage da8: <JMicron Tech 9128> Fixed Direct Access SPC-4 SCSI device
Oct 19 09:13:01 cwpstorage da8: Serial Number RANDOM40143833EEF5
Oct 19 09:13:01 cwpstorage da8: 400.000MB/s transfers
Oct 19 09:13:01 cwpstorage da8: 15259648MB (31251759104 512 byte sectors)
Oct 19 09:13:01 cwpstorage da8: quirks=0x2<NO_6_BYTE>
Oct 19 09:29:41 cwpstorage 1 2022-10-19T09:29:41.466167+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 82992 - - Version 2017.3.23 external FUSE 29
Oct 19 09:29:41 cwpstorage 1 2022-10-19T09:29:41.466354+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 82992 - - Mounted /dev/da8p2 (Read-Only, label "VOLANS", NTFS 3.1)
Oct 19 09:29:41 cwpstorage 1 2022-10-19T09:29:41.466359+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 82992 - - Cmdline options: ro
Oct 19 09:29:41 cwpstorage 1 2022-10-19T09:29:41.466363+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 82992 - - Mount options: ro,allow_other,nonempty,relatime,fsname=/dev/da8p2
Oct 19 09:29:41 cwpstorage 1 2022-10-19T09:29:41.466366+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 82992 - - Ownership and permissions disabled, configuration type 1
Oct 19 18:54:44 cwpstorage 1 2022-10-19T18:54:44.726215+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 82992 - - Unmounting /dev/da8p2 (VOLANS)
Oct 19 20:36:04 cwpstorage 1 2022-10-19T20:36:04.139787+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 90553 - - Version 2017.3.23 external FUSE 29
Oct 19 20:36:04 cwpstorage 1 2022-10-19T20:36:04.139977+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 90553 - - Mounted /dev/da8p2 (Read-Only, label "VOLANS", NTFS 3.1)
Oct 19 20:36:04 cwpstorage 1 2022-10-19T20:36:04.139983+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 90553 - - Cmdline options: ro
Oct 19 20:36:04 cwpstorage 1 2022-10-19T20:36:04.139986+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 90553 - - Mount options: ro,allow_other,nonempty,relatime,fsname=/dev/da8p2
Oct 19 20:36:04 cwpstorage 1 2022-10-19T20:36:04.139990+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 90553 - - Ownership and permissions disabled, configuration type 1
Oct 19 20:36:15 cwpstorage 1 2022-10-19T20:36:15.795985+10:00 cwpstorage.marketdispatch.com.au ntfs-3g 90553 - - Unmounting /dev/da8p2 (VOLANS)

which doesn't really tell me much.

Can I find out a) why it stopped, and b) can I restart it somehow?

The Nas is running 13.0-U1.

Ta.

Peter.

 

[ChrisRJ]

Please describe how you did the import (SMB, local connection, ...)

 

[Patrick M. Hausen]

He wrote

- local USB
- NTFS

So probably the import function in the UI.

 

[pnunn]

Yes that is correct @Patrick M. Hausen. I was via a usb connection. Is there a way to remount the disk and perhaps manually transfer the files with rsync or some such I I can't re-do the import?

 

[pnunn]

I just tried re-starting the import but the result is as shown here

 

[MrGuvernment]

Maybe related?

How to avoid "ValueError: Separator is not found, and chunk exceed the limit"

I have a server that uses asyncio in the following way: proc = await asyncio.subprocess.create_subprocess_exec( command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.STDOU...

stackoverflow.com