SOLVED Importing disk question

[ericg75]

Hello-

I'm a little lost trying to IMPORT 2 WD 4GB HDD's that hold my 'photo_library'. The two HDD's have about 25MB of photos on them- and I am wanting to IMPORT them into my TrueNAS box.
SYSTEM CONFIG: Gigabyte Z87X-UD4H motherboard: Intel core i5-4670K CPU; 16GB RAM; 1-Crucial CT275M 275GB SDD which holds the TrueNAS application; and the 2 WD 4GB HDD's: TrueNAS Generic 12.0-U7

Comes right down to the fact that I do not know what file system type the 2 WD HDD's are. They were created on my Windows 10 desktop. Furthermore it looks to me like they each have 3 partitions (p1, p2 & p3) IF I am reading the information correctly. I simply do not know which partition(s) to import.

Screen shots attached.

I appreciate any input! Thank you- /E

 

[Ericloewe]

That doesn't sound like much data. It's probably simpler to use your Windows client to copy the stuff over.

 

[ericg75]

Hello-

I'm a little lost trying to IMPORT 2 WD 4GB HDD's that hold my 'photo_library'. The two HDD's have about 25MB of photos on them- and I am wanting to IMPORT them into my TrueNAS box.
SYSTEM CONFIG: Gigabyte Z87X-UD4H motherboard: Intel core i5-4670K CPU; 16GB RAM; 1-Crucial CT275M 275GB SDD which holds the TrueNAS application; and the 2 WD 4GB HDD's: TrueNAS Generic 12.0-U7

Comes right down to the fact that I do not know what file system type the 2 WD HDD's are. They were created on my Windows 10 desktop. Furthermore it looks to me like they each have 3 partitions (p1, p2 & p3) IF I am reading the information correctly. I simply do not know which partition(s) to import.

Screen shots attached.

I appreciate any input! Thank you- /E

I'd still like to know if I can (rather: how I can) import these 2 HDD's into my TrueNAS- when they each show ada1 (p1-3). When I look at the drives on my Windows10 PC they do not show any partitions- but over here on TrueNAS they appear to have 3 partitions each. I just do not want to guess at this when my photo_library is at risk. Thank you- /E

 

[Ericloewe]

When I look at the drives on my Windows10 PC they do not show any partitions-

I suspect you're just not looking at the Disk Management MMC plugin, which would show the partitions. As for what the partitions are, we don't have much to go on. The Windows installer creates a bunch of partitions (at least an EFI system partion and a recovery partition), but that only applies to disks you boot from. If they're just plain data disks, you either created the partitions yourself or they were already there.

So, you need to start by having a look at the partition tables and figuring out what's going on. I suspect that the most user-friendly environment for you to do this is going to be Windows. Search for "Disk Management".

 

[ericg75]

I suspect you're just not looking at the Disk Management MMC plugin, which would show the partitions. As for what the partitions are, we don't have much to go on. The Windows installer creates a bunch of partitions (at least an EFI system partion and a recovery partition), but that only applies to disks you boot from. If they're just plain data disks, you either created the partitions yourself or they were already there.

So, you need to start by having a look at the partition tables and figuring out what's going on. I suspect that the most user-friendly environment for you to do this is going to be Windows. Search for "Disk Management".

Disl Management plugin? I do not know where that is located. As for the partitions on the HDD's- I did use Windows10 DiskManagement utility and that only showed 1 (one) partition on each of the two HDD's.

 

[Ericloewe]

Disl Management plugin? I do not know where that is located.

Same thing.

I did use Windows10 DiskManagement utility and that only showed 1 (one) partition on each of the two HDD's.

Okay, that's getting into bizarre territory. But before we get to that, I was looking at your screenshots again (what happened there? they're awfully hard to read) and noticed you don't actually seem to have any disks connected aside from the boot device and the two disks with data you want to keep. This scenario will not work, as to import the data on the disks you'll need a ZFS pool already setup - naturally, backed by its own disks. You cannot use arbitrary filesystems with TrueNAS other than to copy data over to a ZFS pool.
Back on track: Let's see what the partitions are. What's the output of geom part list?

 

[ericg75]

Hello-

I'm a little lost trying to IMPORT 2 WD 4GB HDD's that hold my 'photo_library'. The two HDD's have about 25MB of photos on them- and I am wanting to IMPORT them into my TrueNAS box.
SYSTEM CONFIG: Gigabyte Z87X-UD4H motherboard: Intel core i5-4670K CPU; 16GB RAM; 1-Crucial CT275M 275GB SDD which holds the TrueNAS application; and the 2 WD 4GB HDD's: TrueNAS Generic 12.0-U7

Comes right down to the fact that I do not know what file system type the 2 WD HDD's are. They were created on my Windows 10 desktop. Furthermore it looks to me like they each have 3 partitions (p1, p2 & p3) IF I am reading the information correctly. I simply do not know which partition(s) to import.

Screen shots attached.

I appreciate any input! Thank you- /E

Ok on the 2 HDD's that I have my data on- I have ordered 2 more HDD's and I will attache them as soon as they arrive- then I understand I need to create a (ZFS) POOL, and then import my data into it.

I can see by the output of the GEOM command that ada0 is my 256gb system disk; ada1 is 1 and 2 are the two 3.6TB data disks.
Here is output from the GEOM command:

Code:

root@truenas[~]# geom part list
Geom name: ada0
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 537234727
first: 40
entries: 128
scheme: GPT
Providers:
1. Name: ada0p1
   Mediasize: 272629760 (260M)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 20480
   Mode: r0w0e0
   efimedia: HD(1,GPT,5879cdfa-6820-11ec-8e36-94de80a324ec,0x28,0x82000)
   rawuuid: 5879cdfa-6820-11ec-8e36-94de80a324ec
   rawtype: c12a7328-f81f-11d2-ba4b-00a0c93ec93b
   label: (null)
   length: 272629760
   offset: 20480
   type: efi
   index: 1
   end: 532519
   start: 40
2. Name: ada0p2
   Mediasize: 257597374464 (240G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 272650240
   Mode: r1w1e1
   efimedia: HD(2,GPT,588076b0-6820-11ec-8e36-94de80a324ec,0x2082028,0x1dfd0000)
   rawuuid: 588076b0-6820-11ec-8e36-94de80a324ec
   rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 257597374464
   offset: 17452519424
   type: freebsd-zfs
   index: 2
   end: 537206823
   start: 34086952
3. Name: ada0p3
   Mediasize: 17179869184 (16G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 272650240
   Mode: r1w1e1
   efimedia: HD(3,GPT,587dcce6-6820-11ec-8e36-94de80a324ec,0x82028,0x2000000)
   rawuuid: 587dcce6-6820-11ec-8e36-94de80a324ec
   rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b
   label: (null)
   length: 17179869184
   offset: 272650240
   type: freebsd-swap
   index: 3
   end: 34086951
   start: 532520
Consumers:
1. Name: ada0
   Mediasize: 275064201216 (256G)
   Sectorsize: 512
   Mode: r2w2e4

Geom name: ada1
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 7814037134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada1p1
   Mediasize: 1048576 (1.0M)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 17408
   Mode: r0w0e0
   efimedia: HD(1,GPT,9e1c13d4-7b54-11eb-9f70-f02f74188b50,0x22,0x800)
   rawuuid: 9e1c13d4-7b54-11eb-9f70-f02f74188b50
   rawtype: 5808c8aa-7e8f-42e0-85d2-e1e90434cfb3
   label: LDM metadata partition
   length: 1048576
   offset: 17408
   type: ms-ldm-metadata
   index: 1
   end: 2081
   start: 34
2. Name: ada1p2
   Mediasize: 15711232 (15M)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 1065984
   Mode: r0w0e0
   efimedia: HD(2,GPT,b207f282-77a7-11eb-9f6b-f02f74188b50,0x822,0x77de)
   rawuuid: b207f282-77a7-11eb-9f6b-f02f74188b50
   rawtype: e3c9e316-0b5c-4db8-817d-f92df00215ae
   label: Microsoft reserved partition
   length: 15711232
   offset: 1065984
   type: ms-reserved
   index: 2
   end: 32767
   start: 2082
3. Name: ada1p3
   Mediasize: 4000770235904 (3.6T)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 16777216
   Mode: r0w0e0
   efimedia: HD(3,GPT,9e1c13d7-7b54-11eb-9f70-f02f74188b50,0x8000,0x1d1c03e8f)
   rawuuid: 9e1c13d7-7b54-11eb-9f70-f02f74188b50
   rawtype: af9b60a0-1431-4f62-bc68-3311714a69ad
   label: LDM data partition
   length: 4000770235904
   offset: 16777216
   type: ms-ldm-data
   index: 3
   end: 7814037134
   start: 32768
Consumers:
1. Name: ada1
   Mediasize: 4000787030016 (3.6T)
   Sectorsize: 512
   Mode: r0w0e0

Geom name: ada2
modified: false
state: OK
fwheads: 16
fwsectors: 63
last: 7814037134
first: 34
entries: 128
scheme: GPT
Providers:
1. Name: ada2p1
   Mediasize: 1048576 (1.0M)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 17408
   Mode: r0w0e0
   efimedia: HD(1,GPT,9e1c13e0-7b54-11eb-9f70-f02f74188b50,0x22,0x800)
   rawuuid: 9e1c13e0-7b54-11eb-9f70-f02f74188b50
   rawtype: 5808c8aa-7e8f-42e0-85d2-e1e90434cfb3
   label: LDM metadata partition
   length: 1048576
   offset: 17408
   type: ms-ldm-metadata
   index: 1
   end: 2081
   start: 34
2. Name: ada2p2
   Mediasize: 15711232 (15M)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 1065984
   Mode: r0w0e0
   efimedia: HD(2,GPT,16c7451c-7838-11eb-9f6d-f02f74188b50,0x822,0x77de)
   rawuuid: 16c7451c-7838-11eb-9f6d-f02f74188b50
   rawtype: e3c9e316-0b5c-4db8-817d-f92df00215ae
   label: Microsoft reserved partition
   length: 15711232
   offset: 1065984
   type: ms-reserved
   index: 2
   end: 32767
   start: 2082
3. Name: ada2p3
   Mediasize: 4000770235904 (3.6T)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 16777216
   Mode: r0w0e0
   efimedia: HD(3,GPT,9e1c13e3-7b54-11eb-9f70-f02f74188b50,0x8000,0x1d1c03e8f)
   rawuuid: 9e1c13e3-7b54-11eb-9f70-f02f74188b50
   rawtype: af9b60a0-1431-4f62-bc68-3311714a69ad
   label: LDM data partition
   length: 4000770235904
   offset: 16777216
   type: ms-ldm-data
   index: 3
   end: 7814037134
   start: 32768
Consumers:
1. Name: ada2
   Mediasize: 4000787030016 (3.6T)
   Sectorsize: 512
   Mode: r0w0e0

 

[Ericloewe]

I threw your output into CODE tags for readability.

As for your output, it looks like the disks are setup with Windows' software RAID - Windows "helpfully" hides its metadata partitions for this. There seems to be some support for this in FreeBSD, but at this point I strongly urge you to just use a Windows machine to avoid unexpected gremlins.

 

[ericg75]

I threw your output into CODE tags for readability.

As for your output, it looks like the disks are setup with Windows' software RAID - Windows "helpfully" hides its metadata partitions for this. There seems to be some support for this in FreeBSD, but at this point I strongly urge you to just use a Windows machine to avoid unexpected gremlins.

Sure- I can use M$ winders to send the data over. Where do you think the threshold is in copying data files from one place to another- before you potentially begin to loose quality?

 

[Ericloewe]

I'm not sure I understand the question. In this situation, I recommend avoiding doing it on TrueNAS because support for the weirder Windows storage stuff is not really well-understood. Importing data from a simple NTFS-formatted disk would be fine, but the weirder stuff, be it the traditional software RAID or Storage Spaces, is not something a lot of people are trying to use.

 

[GlueFactoryBJJ]

FWIW, think 3-2-1 backup strategy. Keep your working files on your windows PC. Backup all of your files (all systems on your network) to your NAS. Backup your NAS to an external drive(s). Then back up your local machine (including your external drive(s)) to some cloud system like backblaze (unlimited local machine backup for $65/year).

This gives you the 3-2-1 backup and good data security, even from ransomware.

Backup Strategies: Why the 3-2-1 Backup Strategy is the Best

A Backblaze survey revealed 54% of people said they know someone or have themselves recently lost data. Find out what you can do to prevent data loss.

www.backblaze.com

Scott

 

[ericg75]

Ok thanks for the backup recommendation.
Here is a question I have: I would like to use the Linux desktop PC on which I am currently running the TrueNAS iso image- to also run Debian Linux. As you know, the TrueNAS ISO install occupies the entire desktop PC. I see where TrueNAS Core can be run on a linux VM presence. So, is it possible to run, Debian Liunux on that PC AND install TrueNAS to run within the Debian OS? In other words I would like to have my PC do more than just run TrueNAS. Just curious.

 

[GlueFactoryBJJ]

Ok thanks for the backup recommendation.
Here is a question I have: I would like to use the Linux desktop PC on which I am currently running the TrueNAS iso image- to also run Debian Linux. As you know, the TrueNAS ISO install occupies the entire desktop PC. I see where TrueNAS Core can be run on a linux VM presence. So, is it possible to run, Debian Liunux on that PC AND install TrueNAS to run within the Debian OS? In other words I would like to have my PC do more than just run TrueNAS. Just curious.

Personally, I don't think it is so much a matter of CAN you do that (I question that you can because TrueNAS uses the ZFS file system), but rather SHOULD you do that.

If you wanted, I would think that the better way would be too run Linux through TrueNAS.

IMO, the purpose of a NAS is to have independent backup storage. Something that won't get damaged because your PC does. Something that won't be affected by a ransomware attack. And so many other reasons.

For about +/-$1000 (depending on how much old equipment you have lying around), you could put together a used parts, 24TB NAS server (with new drives) that would work well for a home backup server. If you need less storage, then you could buy fewer drives.

You said you already have the parts, but based on your comment, these parts are from your personal computer and not a separate machine?

The reason why I "harped" on the external backup drive is so that you can copy so all the data off the 4TB drives to the external drive (part of the 3-2-1 backup system), then properly use the 4TB drives you have as ZFS drive in your TrueNAS server. Then cot the data off the external drive to the TrueNAS server. However, I'd recommend you buy at least one more so you can create a ZFS1 array, giving you some drive failure protection and still have 8TB of storage.

Scott

 

[ericg75]

All done here. Thanks for the input. I decided to go with TrueNAS CORE on a dedicated PC. How to close this thread?

 

[Ericloewe]

No need to do anything.

 

[Netfreak]

1.
I have a backup laptop disk connected on my windows PC with a USB cable.
I imported the disk/data successfully on my TrueNAS CORE test system. Great!

2.
I have a second backup WD laptop disk with bad sector(s) and tried to import it also. TrueNAS SCALE sees those bad sectors and rings the alarm bell but proceed with the import. During the process I was monitoring what's going on and noticed in the process the import gauge was moving forward and backwards frequently.
Finally, FreeNAS stopped/failed with an error - in the logs I see
Error
Separator is not found, and chunk exceed the limit (whatever that means?)

I did a manual/optical comparison of what was imported and saw most of the contents of the source disk was imported (not all).

Both disks were imported with the NTFS option selected.

In my premade destination folder in which the contents were imported I saw/noticed 3 things:

a . $RECYCLE.BIN (folder with sub folders) this is also to be seen in the 1. above that was successfully imported (what's the purpose of this folder?)
b. All the folders were imported but many were left empty (that's perhaps when the system couldn't finish the task and stopped/failed).
c. An unknown empty folder (found.000) is seen which is not in the source drive.

Most likely, all the strange things were caused by the bad sector(s) of the source drive BUT I can't confirm that? Maybe other users of TrueNAS have some experience in this.

Does TrueNAS import folders alphabetically?
What's faster? Import a disk or a manual copy in Windows?
Does L2ARC play any speed role in importing disks and manual copying in CORE & SCALE?

While I am at it, I also did a test on a folder share Replication from a CORE IBM machine to another SCALE IBM machine (note 2 different versions) and that data transfer went fast but unfortunately I have no benchmark on that to share. I didn't try from SCALE --> CORE

I am still learning on my test rides not yet in real life...
My old test bike:
IBM X3100 M4
XEON @ 3.10GHz
32GB ECC RAM
4x2TB WD purple in RAID z2

 

[Netfreak]

An experience

I was adding an extra fan on one of my test PCs which is running TrueNAS CORE smoothly. When done, I start up the PC and saw only 4 disks instead of 5 (normally, 4x in RAIDz2 and 1x boot disk).

I restart the system and went into the BIOS as I’ve had this issue before where in the BIOS on disk configuration, I have to have it set to AHCI instead of IDE cos on IDE I saw only 4 disks instead of 5, and thought that is the culprit! It turned out that the setting was still correct and on AHCI.

Next step, (I’ve learned to make notes/keep track of the disk names and serial numbers connected to my test NAS) so i located the missing disk and double-check the SATA cable on it making sure it’s intact. DONE!... I start up the PC and there I have that one missing disk back (could have been that the cable was out of its connection during the addition of the fan). Back in business.

I was running through the web interface and went into Storage and saw in the heading a red alert small circle. That circle is normally green when all disks are up and running OK (RED is dangerous). With that RED alert, I thought I have to click replace. I shut down the PC instead and after a while turned it back on; that red circle turned out to be green automatically.

Although the disk came back online I don’t know if I can trust my experience. BUT what happens in such a situation when a disk goes offline, like in my case mentioned above, and was not manually set offline because when we set a disk offline, we do it on purpose to replace it for resilvering (which takes a very long time), this is not the case.

I am still learning on my test rides, not yet in real life...
One of my old test bikes:
ASUS MB
Dual XEON @2.4GHz
64gb ECC Registered RAM
4x500gb Seagate in RAID z2