Proxmox with Truenas VM - Broke it, now just Truenas

[Blueduck3285]

I was running Truenas in Proxmox where I had 4 1tb drives attached as LVM's in Proxmox to the Truenas VM. Long story short, I blew up that proxmox install on accident. I installed Truenas on bare metal skipping Proxmox.

Is there a way to re attach those drives in Truenas or because they were wrapped in an LVM in proxmox am I just screwed?

For context, zpool and zfs show nothing in shell.

 

[artlessknave]

because they were wrapped in an LVM in proxmox am I just screwed?

you would need to recreate your proxmox environment (doesnt have to be exact but would have to be close enough), backup the data, destroy the proxmox disaster, spin up TrueNAS, make a new pool, and restore the data

as your environment was poorly architect-ed you will have to jump through hoops to get the data back, demonstrating one of the ways TrueNAS as a VM can go VERY wrong, particularly when you have no backups (RAID(z) is not a backup)

also, you haven't followed the forum rules of including your hardware

 

[Blueduck3285]

you would need to recreate your proxmox environment (doesnt have to be exact but would have to be close enough), backup the data, destroy the proxmox disaster, spin up TrueNAS, make a new pool, and restore the data

as your environment was poorly architect-ed you will have to jump through hoops to get the data back, demonstrating one of the ways TrueNAS as a VM can go VERY wrong, particularly when you have no backups (RAID(z) is not a backup)

also, you haven't followed the forum rules of including your hardware

If you saw this is my first post, way to be a jerk about the forum rules (find below).

When I did recreate the Proxmox environment, when I tried to attached the 4 drives to the Truenas VM it wouldn't attach them. Even if it did, I do not have space to pull off that data as its over 2tb on what was a raidz1.

HP DL360p Gen8 Dual E5-2620 Xeon's
32GB Ram HP Genuine
Smart Array P420i Controller
4 1tb scsi HP Genuine
2 300gb scsi HP Genuine
1 320gb
1 750gb
1 512gb nvme
1 300gb ssd usb
1 2tb hdd usb

PS, everyone knows raid isn't a backup.

 

[artlessknave]

well, your hardware looks solid, so at least it shouldn't be a hardware failure.

Smart Array P420i Controller

were you using RAID or HBA mode? if it was RAID, did you delete any of it? if so, there isn't really a way to get it back.

if you blew away the LVM with the proxmox, then that's probably very gone.

unfortunately, most of the VM knowledge on the forums is TrueNAS with ESX. I'm not very familiar with proxmox myself.

jerk about the forum rules

If you think fairly neutral pointer to the forum rules, which are required before posting, is being a "jerk" I don't know what to say.
many of the people most able to help will not get involved if the poster clearly didn't read the forum rules, as a minimum.

PS, everyone knows raid isn't a backup.

considering how many people lose ZFS pools and have no backups.....

 

[Blueduck3285]

well, your hardware looks solid, so at least it shouldn't be a hardware failure.

were you using RAID or HBA mode? if it was RAID, did you delete any of it? if so, there isn't really a way to get it back.

if you blew away the LVM with the proxmox, then that's probably very gone.

unfortunately, most of the VM knowledge on the forums is TrueNAS with ESX. I'm not very familiar with proxmox myself.

If you think fairly neutral pointer to the forum rules, which are required before posting, is being a "jerk" I don't know what to say.
many of the people most able to help will not get involved if the poster clearly didn't read the forum rules, as a minimum.

considering how many people lose ZFS pools and have no backups.....

I believe I set and have left it in HBA mode. I haven't deleted anything.

I wanted to move my install from a 300gb drive that was part of the HBA array to one that wasn't so that I could pass the HBA through to Truenas. I was in the process of getting a 12tb drive and a nas enclosure to set up backups. It arrived after I blew up proxmox.

I figured now would be a good time to just install truenas onto bare metal so I wouldnt have to worry about hypervisor bs.

I guess at this point I will have to make images of the drives, toss them on the backup and try to reconstruct the data later.

 

[jgreco]

Smart Array P420i Controller

The P420i RAID controller is not suitable for use with TrueNAS. Please see

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

and please note there's no need to tell me that it has a "HBA mode". A lobotomized RAID controller is still a RAID controller, note points 4), 5), and 6) in the linked article.

Is there a way to re attach those drives in Truenas or because they were wrapped in an LVM in proxmox am I just screwed?

Since you didn't follow the guide on how to virtualize TrueNAS, sorry to say, you're somewhat screwed. See

"Absolutely must virtualize FreeNAS!" ... a guide to not completely losing your data.

[---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some specific things known to be problematic ----] [---- 2014/12/24: Note, there is another post discussing how to deploy a small FreeNAS VM instance for basic file...

www.truenas.com

There is a possibility that you could reinstall Proxmox, copy the disk images out of the virtual disks and onto physical disks using dd, and I would expect that to work. But it would be a lot of effort; only you can tell us whether you want to pursue that. The wonderful thing about ZFS is that it will put your pool back together if there is any reasonable way to do so, but hidden away in an LVM isn't one of the things it will understand.

If you saw this is my first post, way to be a jerk about the forum rules (find below).

As for this, everyone is reminded to try to be kind to the new users. There is already a painful enough learning curve here. New users often don't realize that they are asking a LOT out of community members, and that the forum rules exist because ten years have shown that hardware often plays a significant role in problem resolution. It gets frustrating to ask each and every new user for the info. But also remember that you don't have to engage with new users if you don't want to.

I do have a sort-of formulaic template for new users who have forgotten stuff, that anyone is welcome to use. It's intended to be friendly yet firm. I generally try to replace "hardware" with the specific things I suspect may be relevant to a user's problem.

Welcome to the forums.

Sorry to hear you're having trouble. Please take a few moments to review the Forum Rules, conveniently linked at the top of every page in red, and pay particular attention to the section on how to formulate a useful problem report, especially including a detailed description of your hardware.

You've basically given no one anything to work with, so the responses will tend to be random guesses rather than anything useful.

I'm fine with people using this, or even better, if we can come up with a better version, I am fine with that too. We do want new users to feel welcome, but also to clue them in that they play a crucial role in helping solve their problems, as they are the eyes and fingers in front of their system.

 

[Blueduck3285]

The P420i RAID controller is not suitable for use with TrueNAS. Please see

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

and please note there's no need to tell me that it has a "HBA mode". A lobotomized RAID controller is still a RAID controller, note points 4), 5), and 6) in the linked article.



Since you didn't follow the guide on how to virtualize TrueNAS, sorry to say, you're somewhat screwed. See

"Absolutely must virtualize FreeNAS!" ... a guide to not completely losing your data.

[---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some specific things known to be problematic ----] [---- 2014/12/24: Note, there is another post discussing how to deploy a small FreeNAS VM instance for basic file...

www.truenas.com

There is a possibility that you could reinstall Proxmox, copy the disk images out of the virtual disks and onto physical disks using dd, and I would expect that to work. But it would be a lot of effort; only you can tell us whether you want to pursue that. The wonderful thing about ZFS is that it will put your pool back together if there is any reasonable way to do so, but hidden away in an LVM isn't one of the things it will understand.



As for this, everyone is reminded to try to be kind to the new users. There is already a painful enough learning curve here. New users often don't realize that they are asking a LOT out of community members, and that the forum rules exist because ten years have shown that hardware often plays a significant role in problem resolution. It gets frustrating to ask each and every new user for the info. But also remember that you don't have to engage with new users if you don't want to.

I do have a sort-of formulaic template for new users who have forgotten stuff, that anyone is welcome to use. It's intended to be friendly yet firm. I generally try to replace "hardware" with the specific things I suspect may be relevant to a user's problem.



I'm fine with people using this, or even better, if we can come up with a better version, I am fine with that too. We do want new users to feel welcome, but also to clue them in that they play a crucial role in helping solve their problems, as they are the eyes and fingers in front of their system.

the P420i is what came with the server. I was looking for a server that I could use for a home lab and for data storage. It was 125 bucks on amazon and came with the dual xeon's, 32gb ddr3 ecc ram and 2 300gb drives.

When I was looking up Proxmox vs Truenas, the article you linked, never once came up in my googling. My google fu is pretty good but never saw that.

Sadly, the stuff that does come up are things from LTT, Craft Computing, or other channels with "guides". The other issue is that I purchased drives in steps, so when I first installed Proxmox and a vm of Truenas, I had like 4 total drives (stuff scavenged from dead laptops that I repurposed as they were 2.5 drives. So yeah, not set up correctly is certainly the case.

Sounds like my best option is to do what you suggested, after I make my images, repurpose the hard drives into... not a raidz1... and try to reconstruct my data later.

I had a "kinda" snap shot of some of the data on the 2tb drive, but scattered across many of the drives are more data, the nvme drive has photo dumps, the 750gb drive has vm data from the proxmox install, and on and on. I have a lot of work ahead of me.

 

[artlessknave]

I had a "kinda" snap shot of some of the data on the 2tb drive, but scattered across many of the drives are more data, the nvme drive has photo dumps, the 750gb drive has vm data from the proxmox install, and on and on. I have a lot of work ahead of me.

ouch.
TrueNAS is powerful, but it's like driving a semi; if you get the gears wrong you ain't gonna make it up the hill.

LTT, Craft Computing, or other channels with "guides"

these have brought many people to the TRUE NAS but if you look at the LTT video(s) about how they lost their entire vault.....
many of their videos, while entertaining, are just that; entertainment, often showing the best ways NOT to do things.

unfortunately, what tends to happen is people don't realize this until after they lost a whole pool of data.

 

[Whattteva]

Sounds like my best option is to do what you suggested, after I make my images, repurpose the hard drives into... not a raidz1... and try to reconstruct my data later.

Quite frankly, I'd take a baremetal RAIDZ1 over whatever Frankenstein contraption setup you have with Proxmox any day even if it was RAIDZ2+.

 

[jgreco]

the P420i is what came with the server.

Yeah. It's a nice RAID controller for certain purposes, but unfortunately TrueNAS is not one of them. Already explained in the linked article.

When I was looking up Proxmox vs Truenas, the article you linked, never once came up in my googling. My google fu is pretty good but never saw that.

Sadly, the stuff that does come up are things from LTT, Craft Computing, or other channels with "guides".

I can't stand the entertainment tech channels. This may be because a lot of my early professional experience with computers was in the medical electronics field; you play for keeps because if you don't, people may die.

If you want to do dumb things, that's fine, you should be able to, but when you pass that off as expertise, that is bad. We have one guy rattling around here who had dozens of old SSD's, and we showed him how to set it up in a Franken-server because it was fun and funny. But it's not a serious server.

I wish we could catch everyone before they wasted money on the wrong thing. Fortunately (trying to look on the bright side) it doesn't sound like you are out any significant money; old servers are relatively cheap.

 

[Blueduck3285]

Yeah. It's a nice RAID controller for certain purposes, but unfortunately TrueNAS is not one of them. Already explained in the linked article.



I can't stand the entertainment tech channels. This may be because a lot of my early professional experience with computers was in the medical electronics field; you play for keeps because if you don't, people may die.

If you want to do dumb things, that's fine, you should be able to, but when you pass that off as expertise, that is bad. We have one guy rattling around here who had dozens of old SSD's, and we showed him how to set it up in a Franken-server because it was fun and funny. But it's not a serious server.

I wish we could catch everyone before they wasted money on the wrong thing. Fortunately (trying to look on the bright side) it doesn't sound like you are out any significant money; old servers are relatively cheap.

Its just the data. We have a side business making stuff with a CNC laser and the files were kept on the raidz1 pool along with 200gb of photos dumped from google photos (i still have the zips on the 500gb nvme, hoping I can image that drive and get that data back). The photos I have the uncompressed copies on the 2tb drive but the backups are really all I care about. It would be nice to just have everything to dump on the 12tb I just grabbed and start fresh.

 

[Blueduck3285]

Quite frankly, I'd take a baremetal RAIDZ1 over whatever Frankenstein contraption setup you have with Proxmox any day even if it was RAIDZ2+.

At the time I didnt have two spare drives which is why I went with a raidz1 but ill look into new configurations as I get things sorted.

 

[Blueduck3285]

ouch.
TrueNAS is powerful, but it's like driving a semi; if you get the gears wrong you ain't gonna make it up the hill.

these have brought many people to the TRUE NAS but if you look at the LTT video(s) about how they lost their entire vault.....
many of their videos, while entertaining, are just that; entertainment, often showing the best ways NOT to do things.

unforatunetly, what tends to happen is people don' realize this until after they lost a whole pool of data.

I do wish there was a way to mount existing drives like the 2tb ntfs and share it via smb. I understand the reason why not, I am just wanting one solution to do everything.

 

[artlessknave]

I do wish there was a way to mount existing drives like the 2tb ntfs and share it via smb. I understand the reason why not, I am just wanting one solution to do everything.

it's not a simple matter of mounting them. because of way you designed the stack, you need about 2-3 different technologies all running in the correct order for it to work at all. this is why virtualizing is so risky; it's REALLY easy to do this.

some people are lucky enough to have minor issues that lead them to find out the problems before anything catastrophic, but this is not the position you are in, and, there is usually not much we can do about it.

there just isn't a lot of experience with unfubar-ing such a thing because usually by this point it's so mangled that it's pretty much throw %*^$ at it and see if it works.

you're going to need to decide how important this data is to you, because the best way to try and recover it will be to DD the disks over to new disks, keep those disks very safe, and muck around with now essentially throwaway disks, to try and reproduce what you nuked; you would ALSO have to have disks ready and waiting, because if you DO get it functional, you want to backup it up last century.

the other option is paid services, but you'd be looking at thousands at least likely for them to try and figure out the structure of what occured.

if you go that route.....you kind of ransomware'd yourself

 

[Whattteva]

At the time I didnt have two spare drives which is why I went with a raidz1 but ill look into new configurations as I get things sorted.

I think you misunderstood me. I'm saying a RAIDZ1 that's properly setup on a bare-metal non-virtualized TrueNAS is BETTER than whatever monstrosity you setup with Proxmox LVM's passed through to TrueNAS VM.

@artlessknave summarized it best here:

it's not a simple matter of mounting them. because of way you designed the stack, you need about 2-3 different technologies all running in the correct order for it to work at all. this is why virtualizing is so risky; it's REALLY easy to do this.

 

[samarium]

While I haven't played with proxmox extensively, it seems just a thin veneer, a gui, and a few tools on top of debian, as is truenas really. Assuming you haven't actually destroyed the LVM metadata it should be possible to reconstruct. I never can understood why use LVM when ZFS was available. I retired LVM, even thinpools, many years ago in favor of ZFS on my servers. Send me a message, and we can talk about it, reconstruction might be an interesting project. If you have some idea what you did to bork proxmox, even better.

Its just the data. We have a side business making stuff with a CNC laser and the files were kept on the raidz1 pool along with 200gb of photos dumped from google photos (i still have the zips on the 500gb nvme, hoping I can image that drive and get that data back). The photos I have the uncompressed copies on the 2tb drive but the backups are really all I care about. It would be nice to just have everything to dump on the 12tb I just grabbed and start fresh.

 

[Blueduck3285]

While I haven't played with proxmox extensively, it seems just a thin veneer, a gui, and a few tools on top of debian, as is truenas really. Assuming you haven't actually destroyed the LVM metadata it should be possible to reconstruct. I never can understood why use LVM when ZFS was available. I retired LVM, even thinpools, many years ago in favor of ZFS on my servers. Send me a message, and we can talk about it, reconstruction might be an interesting project. If you have some idea what you did to bork proxmox, even better.

I got everything back up and running and purchased a 12tb drive and a two bay nas box to back everything up. I want to rebuild with everything done a better way.

 

[WI_Hedgehog]

As a note to @Blueduck3285, we really do want you to succeed in building a solid, reliable, and affordable system. We do come off as harsh at times though it's out of frustration that new members spent a bunch of money that they could have saved and had a better system if they'd read the documentation (the tab at the top of the screen) and asked questions first...we're basically frustrated for you, not necessarily at you.

Welcome, by all means, and let's get this set up right and at minimal cost & hassle for you.

 

[Blueduck3285]

As a note to @Blueduck3285, we really do want you to succeed in building a solid, reliable, and affordable system. We do come off as harsh at times though it's out of frustration that new members spent a bunch of money that they could have saved and had a better system if they'd read the documentation (the tab at the top of the screen) and asked questions first...we're basically frustrated for you, not necessarily at you.

Welcome, by all means, and let's get this set up right and at minimal cost & hassle for you.

Well the expectation that someone would read all the documentation of all the available products out there before making a hardware purchase on a budget is a bit much though I do agree I am the idiot but was able to get things back in a working state to fix it moving forward.

 

[jgreco]

Well the expectation that someone would read all the documentation of all the available products out there before making a hardware purchase on a budget is a bit much

Yes, let's say I even agree with this, but the problem is if you buy a Ford Pinto to haul a trailer, it just isn't going to work well even if your trip is all downhill. At a certain point, no one else can fix a bad purchase. I really don't care for my frequent role as messenger of doom on these forums, people don't like hearing that their RAID controller won't work or their ethernet controller is garbage, but when push comes to shove, I have not heard a better way to handle the bad news.

That said, there are lots of clever people here who will try to get you going in the right direction. Perhaps it is just the price of being a free product.