Problem, TrueNas CORE is dead

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
Hi all, not sure what went wrong where, tried to log onto flex, did not respond, went onto the NAS, saw some errors on some of the pools, but nothing seen before, that does not clear itself.

tried to restart plex, did not want to... so issued a reboot of the NAS,

it now comes up, BIOS, then kicks over to LSI card, shows 4 buses as per attached and then cycles back to bios boot up screen.
MB: Gigabyte B250M-D3H

slightly panicking, not sure where to start.

thinking, need to check/confirm the boot up volume is in tact, this is on a microsd card that plugs into a usb port at the back.
for now i've disconnected the cable coming out of LSI controller to the HDD's

get the distinct feeling something broke, as i don't think it's getting anywhere close to accessing the microsd yet.
although I eventually got into my bios, it does not want to move away from the update bios / save bios screen.
thinking of removing all HDD's (including those plugged directly into the MB), and writing a bootable image onto a usb and see if i can get it to boot.

going to need some hand holding here...

G
IMG_4538.jpg


IMG_4539.jpg


IMG_4540.jpg


IMG_4541.jpg
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
That's not a BIOS screen, it's a QFLASH screen that can only be used to either update or save your BIOS.

Your boot pool could have died.
 

Patrick M. Hausen

Hall of Famer
Joined
Nov 25, 2013
Messages
7,776
Write a known good USB drive with FreeBSD 13.2, check if you can boot that.
 

joeschmuck

Old Man
Moderator
Joined
May 28, 2011
Messages
10,996
If the above recommendations do not fix the problem.
1. Power off and unplug the electrical cord. Wait 5 minutes (2 will probably be good but it's only 3 more minutes and you should locate your motherboard user manual and be prepared to switch your BIOS).
2. Plug the power cord back in and power on the computer.
3. If the same problem occurs, are you able to enter the BIOS screen (read your manual for which key to press). If you can get into the BIOS setup screen verify your boot drive is correct.
4. If the same problem occurs, since you have Dual BIOS, switch to the other BIOS.
5. If this fails, Clear the CMOS (again, read your manual), did this fix the problem.
6. If the same problem occurs, unplug your PCIE cards from your motherboard and does that fix the problem, meaning it does not stop at the BIOS Flash screen.
7. If the problem still occurs I would Flash the current firmware to your BIOS (read the user manual and make sure you grab the correct file).
8. If you have had no success and continue to end on the BIOS flash screen, disconnect everything from the motherboard that is all add-ons. Leave the RAM, CPU, Keyboard, Mouse, Monitor. Power it back on, it should not end up at the BIOS Flash screen.
9. If you still have a problem, I would suspect the motherboard is faulty. You could start moving RAM around or anything else you can possibly think of, at this point you don't have much to loose. I would not touch the CPU unless you have reason. And you might try to reseat the power supply cables to the motherboard. At this time I have no reason to believe the power supply is the culprit.

If you need to provide feedback, please be very descriptive. Remember that we are not there with you and you do not want us to make assumptions as it takes longer to isolate the problem.

Best of luck to you.
 

Constantin

Vampire Pig
Joined
May 19, 2017
Messages
1,829
Check all power cables. Twice. I had several import failures thanks to a bad power connection on one backplane leading to a bad import pool event, then the lovely message that the pool was dead and to rebuild it from backup.
 

NickF

Guru
Joined
Jun 12, 2014
Messages
763
Good news is I think your data is probably safe.
The bad news is I think you have either a bad power supply (most likely) or a bad motherboard.
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
That's not a BIOS screen, it's a QFLASH screen that can only be used to either update or save your BIOS.

Your boot pool could have died.
please explain this a bit more, i know about bios on mb and i know fw on expander cards like a LSI.
I like to understand what I'm told so excuse the 101 Questions.

G
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
Write a known good USB drive with FreeBSD 13.2, check if you can boot that.
Will do, was a thinking by me also to confirm the hw excluding the TrueNAS stack. but sadly as per others, what I'm seeing looks like well before the HW is reaching out for OS to boot it.
thank BSD link is stuck at searching, will do search.
G
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
If the above recommendations do not fix the problem.
1. Power off and unplug the electrical cord. Wait 5 minutes (2 will probably be good but it's only 3 more minutes and you should locate your motherboard user manual and be prepared to switch your BIOS).
2. Plug the power cord back in and power on the computer.
3. If the same problem occurs, are you able to enter the BIOS screen (read your manual for which key to press). If you can get into the BIOS setup screen verify your boot drive is correct.
4. If the same problem occurs, since you have Dual BIOS, switch to the other BIOS.
5. If this fails, Clear the CMOS (again, read your manual), did this fix the problem.
6. If the same problem occurs, unplug your PCIE cards from your motherboard and does that fix the problem, meaning it does not stop at the BIOS Flash screen.
7. If the problem still occurs I would Flash the current firmware to your BIOS (read the user manual and make sure you grab the correct file).
8. If you have had no success and continue to end on the BIOS flash screen, disconnect everything from the motherboard that is all add-ons. Leave the RAM, CPU, Keyboard, Mouse, Monitor. Power it back on, it should not end up at the BIOS Flash screen.
9. If you still have a problem, I would suspect the motherboard is faulty. You could start moving RAM around or anything else you can possibly think of, at this point you don't have much to loose. I would not touch the CPU unless you have reason. And you might try to reseat the power supply cables to the motherboard. At this time I have no reason to believe the power supply is the culprit.

If you need to provide feedback, please be very descriptive. Remember that we are not there with you and you do not want us to make assumptions as it takes longer to isolate the problem.

Best of luck to you.
hmmm, manual.. good points...
some questions.
as you mentioned, dual bios...
then why did it (2nd bios) not kick in and bring the system up, (will need to figure out how to instruct it to switch bios's)
I know F12 and F9 kicks me into various interfaces, if i recall from last night (I'm GMT+2) did not get much response out of the KB...
main power cable have been changed, to work on machine I needed to move it which then introduced a different, longer power cable as required where it is stamina vs where it was in my rack.
also thinking #6... if you see one of the photos shoes where it is showing scanning by the LSI card and then output from that, making me think we've gone pass ignition bios startup.

G
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
Good news is I think your data is probably safe.
The bad news is I think you have either a bad power supply (most likely) or a bad motherboard.
like the Good news bit, i'm focusing on that part...
G
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
so... a little update

I first just unplugged the cable with the HDD that goes into my LSI controller... no change in behaviour. still does LSI scan, showing the devices plugged into it, then cycles back to bios boot screen,

I then removed the LSI controller which now (much faster) took me to a line that said
"This is a FreeNAS data disk and can not boot system. System Halted"

Now i restarted system, went to bios, reselected my boot device => external USB
(I plugged in a Ubuntu bootable USB.) and let it boot onto this (I want to see if the Ubuntu installer see 3 drives (plugged directly into MB) or all 7, 6 x 4TB HDD + 1 x 128GB M.2 SSD, excluding the boot usb), and surprise... it saw 6 HDD's, no mention of the M.2 SSD,
Guess this is good news, it means my LSI controller works.
What is/was/suppose to be on the M.2 it is a App boot pool, so things like Plex, qBittorrent, Unify controller and a FreeBSD Jail running some random apps like network speed tested + influxdb.
so bit perplexed, if that's whats on the M.2 why is it not booting... it should just have a problem with the pool.
some more fault testing.
G
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
m
You entered the motherboard's utility to flash its own BIOS, not the BIOS interface.
right have hit think end or del which does that... while it was all dead, trying to see if there is life anywhere... if the KB batteries are good... it's a BTLE KB... and was worried some of my it's not listening to me might be the KB, since just switched over to a cabled KB to make sure i'm not fighting myself.
ok so all indications are the MB is ok,
G
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
interesting, after removing the Ubuntu usb, and plugging the TrueNas boot USB back, booting and going back into bios to select boot device.
(it's seeing all 6 HDD's + the M.2 and the external TrueNas boot device).

Notice the list of possible Boot devices having change from early... to later. only thing i did was removed TrueNAS usb put in Ubuntu check it booted and then went back to TrueNAS boot device.

Can't recall if the boot device was tagged UEFI or not, having to try both. first try still took me to "This is a FreeNAS data disk and can not boot system.... "

G
 

Attachments

  • IMG_4562.jpg
    IMG_4562.jpg
    72 KB · Views: 103
  • IMG_4561.jpg
    IMG_4561.jpg
    53.7 KB · Views: 118
  • IMG_4559.jpg
    IMG_4559.jpg
    40.8 KB · Views: 126
  • IMG_4558.jpg
    IMG_4558.jpg
    65.3 KB · Views: 104
  • IMG_4557.jpg
    IMG_4557.jpg
    64.6 KB · Views: 138
  • IMG_4563.jpg
    IMG_4563.jpg
    72.6 KB · Views: 123

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
It's Booting.... :)

G
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
wondering if the TrueNAS USB boot device is on the way out... - will look at replacing this. to be sure,
see attached, do believe my M.2 is gone that house App pool. - have some brand new 500GB SSD's I can put in. have 2, so can do a mirror pair.

options ?
happy to consider switching to scale, while we at it.

to consider :
  1. Plex install - MOST Important , the data is spread over 2 pools located on the 6 x 4TB HDD's, app is on the M.2 SSD
  2. Unify controller [have a export/backup],
  3. qBittorent - no sweat, simple reinstall
  4. BSD Jail, can rebuild
G
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
No that it's been up once... lets check stability... going to do a total reboot.
See if we end with what we have currently.
G
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
so a boot took me back to this is a FreeNAS data disk, and a halt of system.
doing another power cycle
reselecting the boot device.
need boot stability before anything else.
G
 

georgelza

Patron
Joined
Feb 24, 2021
Messages
417
Not really a need to mirror the boot pool.
the device that's gone is not a boot pool, it was my app pool, where apps was installed.
think i happen to run into multiple things here.
the app pool failed, and at the same time it seems to be having a issue sticking to the correct boot device=> being a smallish USB
G
 
Top