Register for the iXsystems Community to get an ad-free experience

SOLVED Boot Drive Gone / Encrypted Disks / Attempt to Import + Unlock Fail [ESXi]

Western Digital Drives - The Preferred Drives of FreeNAS and TrueNAS CORE
Status
Not open for further replies.

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
My apologies for the delayed reply.
Can you get head -c1536 disk-0.vmdk | strings? It should have a contained spec for the vmdk, and maybe something in there will look abnormal.
I'd love to - where do I run this command from (and are there any required software prerequisites)?
What is the full error when you try to import the ovf?
From vSphere Client, (local install) the error recieved is "The OVF Package is invalid and cannot be deployed. | Root element is missing."
From both vSphere Web Client and Workstation Pro, the error received is " Line 1: Could not parse the document: 'no element found'. "
I've provided screen screen shots of all three attempts, but they don't tell you anything more than the above.

  • vSphere Client (local install) = "The OVF Package is invalid and cannot be deployed. | Root element is missing."
  • vCenter Client 1.jpg vCenter Client 2.jpg
  • vSphere Web Client = " Line 1: Could not parse the document: 'no element found'. "
  • vCenter Web 1.jpg vCenter Web 2.jpg vCenter Web 3.jpg vCenter Web 4.jpg vCenter Web 5.jpg
  • Workstation Pro = " Line 1: Could not parse the document: 'no element found'. "
  • Workstation 1.jpg Workstation 2.jpg
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
I think your OVF file may be corrupt. It seems small. But the vmdk is most likely in a compressed stream format. It can be changed into a usable format with vmkfstools, and possibly other ways I don't know. If you can run command line on your ESXi, you need to do vmkfstools -i disk-0.vmdk -d thin usable.vmdk and the result should be able to attach to a VM.
I will attempt this evening and report back. Thank you!
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
This might've been caused from a different controller options in the VM.
There is an option to run virtualized drives on different types of controllers. I believe default is depending on what VM template is used. IIRC FreeBSD template doesn't give you a SATA controller, but for example Windows templates do.
  • In the interest of clarity let me note that I didn't use a template, I created a custom VM. (When I think template, I think .ovf that can be imported without further configuration.)
  • Settings used for new VM: Configuration = Custom | Guest OS = Other, FreeBSD (64-bit)
The SCSI error has plagued people before to get FreeNAS running on ESXi judging from PM's I recieved.
With some luck, that lack of configuration may be what causes your vmdk import to not be accepted.

[*]Create a new VM
[*]Make sure there is a SATA controller listed (else add it "add another device")
[*]Change the type of controller underlying the virtualized drive. ("Virtual device node")
[*]Attempt the same vmdk maneuver.

Now that I think of it, you should probably replicate your previous configuration. If that was using IDE or what not and it worked - do it.
Else, try the above.

Mine are all <2kb too.

I think we may be talking about two different issues here, but I know what you are referring to:
  • When I first attempted to install FreeNAS in a VM, I encountered the issue which if memory serves, it required the SCSI Controller Type to be changed from "LSI Logic Parallel" to "LSI Logic SAS", but the need to do so went away in later updates of the FreeNAS iso. Reference below Image SCSI controller 0.
  • I believe the SCSI error I received was related to Hard disk SCSI Device Node (now that I look "SCSI (0:0)" rings familiar as part of the error. Not the SATA controller type. Reference below Image Hard disk 1.
Kindly note, if we are attempting to say the same thing in two different ways, I have misunderstood and apologies for that.

Just to be clear, I attempted to restore the prior VM in two different manners: (a) "Import" of .ovf / .vmdk and (b) Create new VM, delete created .vmdk in Datastore, swap in "backed up" .vmdk. The former (a), resulted in the errors provided two posts back. The later (b), resulted in the SCSI error I made reference to. If I had to guess it is because the config was expecting device X, but the .vmdk I manually substituted had device Y in it. At any rate, I will retry that approach later this evening and report back with the exact error.

Image SCSI controller 0:
SCSI controller 0.jpg


Hard disk 1:
Hard disk 1.jpg
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,335
Kindly note, if we are attempting to say the same thing in two different ways
I think you are on the ball right there.
Good.

I'd imagine it would cause problems if the vmdk file expects SCSI for virtual device node, while the VM is configured for SATA or the opposite.
If you're sure what you've done on the first occasion, then fine.
To me it is the typical little configuration detail that easily gets forgotten down the line, echoing its absence when needed again.
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
To me it is the typical little configuration detail that easily gets forgotten down the line, echoing its absence when needed again.
Agreed - I think this echos @rs225 's guidance:
Can you get head -c1536 disk-0.vmdk | strings? It should have a contained spec for the vmdk, and maybe something in there will look abnormal.
If the exact configuration can be extracted, then a 1:1 "replica" of the VM could be created, allowing for substitute of that "backed up" .vmdk without error. [hopefully]

I just need some clarity on how to run that suggested command.
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
Can you get head -c1536 disk-0.vmdk | strings?

From ESXi shell

Attempt #1
Code:
[root@ESXi-Host:/vmfs/volumes/593d75bc-201bbad2-7d00-0025902da22c/test2] head -c1536 disk-0.vmdk | strings

Output
Code:
-sh: strings: not found

So Attempt #2 without "| strings"
Code:
[root@ESXi-Host:/vmfs/volumes/593d75bc-201bbad2-7d00-0025902da22c/test2] head -c1536 disk-0.vmdk

Output
Code:
KDMV▒2▒*▒

# Disk DescriptorFile
version=1
CID=98950747
parentCID=ffffffff
createType="streamOptimized"

# Extent description
RDONLY 16777216 SPARSE "generated-stream.vmdk"

# The Disk Data Base
#DDB

ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "1044"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.longContentID = "270ac8761ee94bd121421e42799a886e"
ddb.toolsVersion = "2147483647"
ddb.virtualHWVersion = "11"
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
I think your OVF file may be corrupt. It seems small. But the vmdk is most likely in a compressed stream format. It can be changed into a usable format with vmkfstools, and possibly other ways I don't know. If you can run command line on your ESXi, you need to do vmkfstools -i disk-0.vmdk -d thin usable.vmdk and the result should be able to attach to a VM.

Code:
[root@ESXi-Host:/vmfs/volumes/593d75bc-201bbad2-7d00-0025902da22c/test1] vmkfstools -i disk-0.vmdk -d thin usable.vmdk
Destination disk format: VMFS thin-provisioned
Cloning disk 'disk-0.vmdk'...
Clone: 100% done.

Results in ...

Clone.jpg


Now to give it a try ...
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
  1. Ran vmkfstools -i disk-0.vmdk -d thin FreeNAS.vmdk, where FreeNAS = prior vmdk name.
  2. Create new VM in VMware Worstation (using existing disk cloned in #1). Relevant screenshots below for reference.
  3. Configure the same as previous.
1.png 2.png 3.png 4.png 5.png

Fire it up!

Booyah.png


@rs225: Thank you so very much for leading me in the right direction.
@Jailer: ;)
 

Jailer

Not strong, but bad
Joined
Sep 12, 2014
Messages
4,796
Nice work! It does leave me a bit uneasy that it wasn't that difficult to restore without having decryption key.

Then again maybe I'm just being paraniod.......
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
Nice work! It does leave me a bit uneasy that it wasn't that difficult to restore without having decryption key.

Then again maybe I'm just being paraniod.......

I don't think one can be too paranoid regarding security. Someone else can probably explain it better, but to create a parallel:
  • You: Attacker gains physical access to your FreeNAS server, steals boot drive | Me: I "repaired" a broken virtual image on a physical boot drive.
  • You: Attacker would need to know your login / passphrase, etc. to generate a key | Me: Already known in my scenario as I was the "attacker" and created them.
  • Both: As described in the guide, the encryption key is to mitigate the security risk of a "physical" theft, so say an attacker has either your boot drive, or my virtual image,
  • manages to generate a key, then hopefully (or not) they stole the storage disks too, otherwise they have a key to nothing.
  • Of course, nothing is un-hackable, it is simply a question of: Do the resources that need to be expended justify the object of a hack?
  • I would think most individuals on this forum aren't storing next generation stealth fighter turbofan schematics on their server, which China may like to have, so if the adversary is not a nation state, then it is Joe Blow thief who wouldn't know the difference between a boot drive and a storage drive and thinks that GELI goes with peanut butter between bread.
Does that make sense a bit? I think the only item for concern regarding this is to store that key in a safe place (not taped inside your desk drawer like any people do with passwords, OK bad joke) ...
 

Jailer

Not strong, but bad
Joined
Sep 12, 2014
Messages
4,796
Good points. Thanks again for sharing your experience with this, I'm sure it will help others.
 

Dice

Wizard
Joined
Dec 11, 2015
Messages
1,335
Interesting.
Hopefully I won't have any business returning to this thread ;)
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
Interesting.
Hopefully I won't have any business returning to this thread ;)
Agreed, let's close this chapter (but, at least we know how to fix it should the need arise in the future). I think this demonstrates to build server #2 for replication. Especially with AMZN Cloud Drive terminating the unlimited plan (what I used for encrypted / cloud backup) ... :rolleyes:
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
There's always Google Drive...
For unlimited? I think that requires Gbiz / Apps account with 5 users @ $10/month/each (if memory serves). I believe my only two options to achieve similar pricing for unlimited (ACD @ $60/year) are Backblaze and Crashplan. I'd love your take (admittedly I stopped paying close attention being happy with ACD - but I think we all knew it would come to an end).

Sent from my HTC6545LVW using Tapatalk
 

rs225

Guru
Joined
Jun 28, 2014
Messages
878
Glad you got it working.

There seem to be a number of unlock failure that need some investigation, and probably also a bigger warning that every time a recovery key is downloaded, any prior recovery keys become unusable. In other words, if you do a disk replace on your pool, you need a new recovery key.
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
13,538
For unlimited?
Well, as I understand it (and I'm far from an expert on the subject), there are a few Google Drive possibilities:
  • I've heard that a single-user Gbiz account, despite having a stated 1 TB limit, can exceed that storage by quite a bit without penalty. That's still $10/mo, or double what ACD charged.
  • There's at least one legitimate reseller of Gbiz accounts, again for $10/mo, but since there are over five users, it's definitely unlimited.
  • Then there are the folks on eBay selling the "unlimited lifetime" Gbiz accounts for somewhere around $10 outright. I wouldn't trust them for much, and certainly not for anything business critical.
 

svtkobra7

Patron
Joined
Jan 12, 2017
Messages
202
Well, as I understand it (and I'm far from an expert on the subject), there are a few Google Drive possibilities:
  • I've heard that a single-user Gbiz account, despite having a stated 1 TB limit, can exceed that storage by quite a bit without penalty. That's still $10/mo, or double what ACD charged.
  • There's at least one legitimate reseller of Gbiz accounts, again for $10/mo, but since there are over five users, it's definitely unlimited.
  • Then there are the folks on eBay selling the "unlimited lifetime" Gbiz accounts for somewhere around $10 outright. I wouldn't trust them for much, and certainly not for anything business critical.
Thanks for the info:
1. In my instance (not true for all), I'd rather adhere tightly to acceptable use, as even playing by the rules I now have to pull 50+ TBs off ACD in a few months.
2. Legitimate or shady account provisioner, they still have one thing in common: There is a middle man between you and Google (the admin) that * could * wipe your account without recourse, right? While I'm sure the legit seller would not be inclined to do so, they still could (and I'd rather be in control there). Further, while low probability their account could get hacked, they may no longer be around for one reason or another without a contingency plan, etc. (Please correct me if this is wrong)

Sent from my HTC6545LVW using Tapatalk
 

danb35

Hall of Famer
Joined
Aug 16, 2011
Messages
13,538
Status
Not open for further replies.
Top