Register for the iXsystems Community to get an ad-free experience and exclusive discounts in our eBay Store.

SOLVED Mounting /bin/ as writable in rescue shell to replace wrong /bin/sh

Western Digital Drives - The Preferred Drives of FreeNAS and TrueNAS CORE

gesshoku

Member
Joined
Jun 23, 2012
Messages
46
Hi there,

I did something as stupid as overwriting /bin/sh (not on purpose of course ;-)). Now my FreeNAS installation doesn't boot anymore. It will get stuck here (for obvious reasons):

Code:
"Enter full pathname of shell or RETURN for /bin/sh:"


I can access the rescue shell by specifying /rescue/sh as shell.

As per this link I have mounted the filesystem read-write:

Code:
/rescue/mount -uw /


Since my installation is not working anyway, I thought of taking a copy of /bin/sh of my other working FreeNAS system to overwrite the faulty one. I try to do this by issuing this command:

Code:
mv working-sh /bin/sh

(working-sh being on an ISO mounted into my FreeNAS vm)

This results in the following error:

Code:
mv: rename working-sh to /bin/sh: Read-only file system



Any ideas on how I can accomplish this? Will this solve my issue at all? I am still not sure, if /bin/sh contains system specific entries.


Edit: So excited that I forgot to add some information on my build...

My FreeNAS Version is 11.2 (recently upgraded from 9.3).
It's running in a VM on ESXi 6.7 U1 with an IBM1015 (LSI IT Mode) passed through.
Hope this is all the information needed.


Thanks in advance.

gesshoku
 

Chris Moore

Wizened Sage
Joined
May 2, 2015
Messages
10,062
Why not just reinstall FreeNAS and load your configuration db to get back where you were? It should be super easy.
I did something as stupid as overwriting /bin/sh
What kind of customization are you trying to do?
 

gesshoku

Member
Joined
Jun 23, 2012
Messages
46
Hi Chris. Thank you for your time.

I am currently looking for a recent backup of my configuration on my Backup-NAS and other computers I worked with recently, but so far unfortunately I can only find an old configuration from before upgrading to 11.2. (and the configuration of my Backup-NAS which is running thankfully). Still searching though.

Would booting an older version help as well? This was what I had in mind should my plan of replacing /bin/sh with a working copy not work.


What kind of customization are you trying to do?
One of my drives failed and I was about to replace it. I wanted to make sure I would replace the correct one and tried to run this script as double check. In the process I managed to foolishly overwrite /bin/sh with the script :-(
I only noticed when I tried to reboot for one last check prior to replacing the faulty drive.
 

Meyers

Member
Joined
Nov 16, 2016
Messages
210
Maybe just mount root read write and copy sh back?
 

gesshoku

Member
Joined
Jun 23, 2012
Messages
46
Maybe just mount root read write and copy sh back?
That was kinda what I had in mind.

I assume /rescue/mount -uw / isn't sufficient for this? Because I get the "Read-only" error with just this.

Would you be so kind to tell me which command would mount it as read-write or point me to a link?

Thanks.
 

Meyers

Member
Joined
Nov 16, 2016
Messages
210
Would you be so kind to tell me which command would mount it as read-write or point me to a link?
Good question. If that command doesn't work then I don't know, and I don't have any way to test it out. If you don't have a recent backed up config then I would try rolling back to your previous version and upgrading to 11.2 again (select the previous boot instance from the boot menu). That should get you back up and running on 11.2!
 

Chris Moore

Wizened Sage
Joined
May 2, 2015
Messages
10,062

gesshoku

Member
Joined
Jun 23, 2012
Messages
46
Thanks Chris. Unfortunately the folder "system" doesn't exist for me:

2019-02-23 22_53_50-FreeNAS - VMware Remote Console.png


2019-02-23 22_54_37-FreeNAS - VMware Remote Console.png


I will search further. I am pretty sure I made another backup last week....
And will also still contemplate booting a previous version.
 

gesshoku

Member
Joined
Jun 23, 2012
Messages
46
My bad.
I was being stupid. I don't work with FreeBSD/Linux often. I used "mv" when I should have used "cp".
Not quite there yet, but wanted to let you know before you put more thought into it.

Will write again when I make more progress (hopefully :smile:).

Thanks again for helping.
 

gesshoku

Member
Joined
Jun 23, 2012
Messages
46
Thank you once more Chris and Meyers for your time and help! I got the system back up and running.
I'm angry at myself for making such a noob-y mistake, but it helped me learn a bit more and I will also create regular backup copies (there's a thread on that I saw earlier :)) of my configuration now.
And now to replace that faulty drive...

Here's a little write up in case someone else has a similar problem:


Problem: /bin/sh got modified/destroyed/lost/foolishly overwritten.

Get yourself a working copy of /bin/sh (I used the one from my working FreeNAS machine with the same version running).

FreeNAS boots up until this point:

"Enter full pathname of shell or RETURN for /bin/sh:"

Load the rescue shell:

/rescue/sh

Mount the filesystem read-write:

mount -uw /

As my FreeNAS system is a VM I got the working copy of "sh" into the VM by creating an ISO image (Folder2ISO) and mounting it into the VM.

Mount the CD into the rescue shell session:

mkdir -p /media/cdrom
mount_cd9660 /dev/cd0 /media/cdrom


Copy the working copy to the right path:

cp /media/cdrom/working-sh /bin/sh

Modify permissions (root:wheel should already be set as owner:group):

chmod 555 /bin/sh

Reboot System.

???

Profit
 

Chris Moore

Wizened Sage
Joined
May 2, 2015
Messages
10,062
I appreciate the documentation for what you did. I hope it can help someone else down the line.
 
Top