Announcement

Collapse
No announcement yet.

mount: / is busy and system hangs without shutting down

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    mount: / is busy and system hangs without shutting down

    It seems this problem is back again:

    https://www.kubuntuforums.net/showth...ght=mount+busy


    I'm about ready to pull my hair....

    The more I try, it seems the worse it gets. Please see screenshot.

    I tried a solution suggested here:

    http://askubuntu.com/questions/55897...hutdown-reboot

    The last message in that thread suggests:

    I had the same problem with Kubuntu 14.10. My solution was:
    1. I changed this line in file /etc/default/grub to look like this: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash init=/lib/systemd/systemd"
    2. After that, I ran sudo update-grub in a Terminal

    Now the system shuts down again...:-)
    This did not appear to do anything on the first machine, the screenshot of which I have attached.

    It worked once on the second machine (more about that later), but now that second one won't boot at all. It stops midway with a kdm panic of some description.

    I can only get these machines to shut off if I press and hold the power button, and that messes up the system drive. Restarting normally then gives a kernel panic. Restarting again in safe mode, and running fsck, then continuing the boot brings it back up again, but the next shutdown is the same thing all over again.

    The first machine has a brand-new fresh install of 14.04 with the latest updates all installed. Boot disk is a brand new Samsung 850 SSD. It has a second mechanical hdd, but I have not been able to get to the point of using it yet, as I can't resolve the shutdown issue.

    I would chalk this up to a specific issue with this machine if my second desktop machine were not displaying similar problems. It too now hangs on shutdown, with similar error messages. It is a mature install from two years ago, and it too was recently updated to the latest software available. Trying to boot into safe mode on that second machine and running fsck never seems to terminate, and rebooting again just gives the kernel panic. I can't get it going again at all. Please see second screenshot.

    I tried booting a previous kernel on both machines, but no change on exit.

    Both machines are running Kubuntu 14:04 with 3.13.0-83-generic, and are both 64 bit machines with 64 bit kernels.

    I am down to my laptop now, and, for the time being, I will NOT update it.

    At this point, at least the one machine works if I run fsck in recovery mode. As noted, the second one seems to be hung in fsck at the moment. I could probably drop to a command shell and try running fsck that way, but I will need some help with that.

    Any help or suggestions appreciated.

    Thanks.

    Frank.


    Attached Files
    Last edited by Frank616; Apr 19, 2016, 04:18 PM.
    Linux: Powerful, open, elegant. Its all I use.

    #2
    Could be a process still has an open file and is preventing umountroot from remounting the root filesystem as read-only (this is part of the shutdown process). I see a couple Network Manager messages there on your screen shot. It could be the culprit. Just for fun, boot up that system and kill network manager completely then try shutting down. If that doesn't work, maybe adding this to the umountroot script will give some useful output:

    ps ajx or maybe lsof

    As always, have a liveUSB bootable handy before doing any system edits.

    The other screenie shows a kernel panic (not KDM panic?). Is that what you meant? We haven't used KDM by default in quite awhile.
    Last edited by oshunluvr; Mar 15, 2016, 01:47 PM.

    Please Read Me

    Comment


      #3
      oshunluvr,

      I thought I had edited that KDM comment. Sorry. It is indeed a kernel panic, and I knew that when I posted. Just missed editing it out, I guess.

      OK, how can I kill network manager before trying a shutdown?

      Where will I find the unmountroot script?

      Thanks for the suggestions.

      Frank.
      Linux: Powerful, open, elegant. Its all I use.

      Comment


        #4
        next time it hangs ,,,,,,try ,,,,,press and hold "ctrl+alt+SysRq" wile holding them down press one at a time S then U then B ,,,,,,,,,,the system should reboot.


        VINNY
        i7 4core HT 8MB L3 2.9GHz
        16GB RAM
        Nvidia GTX 860M 4GB RAM 1152 cuda cores

        Comment


          #5
          Vinny:

          I'll try that.

          I booted from a Kubuntu USB stick, and ran fsck on the offending sdb1 partition. It found a block number error. I asked it to fix it, and now it comes back 'clean', but I still get the kernel panic. I'll try gparted tomorrow or Thursday when I have some time off again.

          Frank.
          Linux: Powerful, open, elegant. Its all I use.

          Comment


            #6
            Originally posted by Frank616 View Post
            Vinny:

            I'll try that.

            I booted from a Kubuntu USB stick, and ran fsck on the offending sdb1 partition. It found a block number error. I asked it to fix it, and now it comes back 'clean', but I still get the kernel panic. I'll try gparted tomorrow or Thursday when I have some time off again.

            Frank.
            if this is the same one you did this to ,,,,,,,

            1. I changed this line in file /etc/default/grub to look like this: GRUB_CMDLINE_LINUX_DEFAULT="quiet splash init=/lib/systemd/systemd"
            2. After that, I ran sudo update-grub in a Terminal
            ,,,,,,,then undo it ,,,,their is no systemd in 14.04

            OOPS wrong about that ,,,,,,,,,,but I would still try undoing it.

            VINNY
            Last edited by vinnywright; Mar 15, 2016, 08:43 PM.
            i7 4core HT 8MB L3 2.9GHz
            16GB RAM
            Nvidia GTX 860M 4GB RAM 1152 cuda cores

            Comment


              #7
              You should be able to kill networkmanager from ksysguard.

              My system shows umountroot at:

              :~$ locate umountroot
              /etc/init.d/umountroot
              /etc/rc0.d/K11umountroot
              /etc/rc6.d/K11umountroot
              /lib/systemd/system/umountroot.service

              The last one is a link to null, so must be a place holder for future use. The 2nd and 3rd both link to the first, so edit /etc/init.d/umountroot. try adding the lsof and/or ps ajx commands below the three lines that contain the remount command.

              Please Read Me

              Comment


                #8
                oshunluvr and vinny:


                next time it hangs ,,,,,,try ,,,,,press and hold "ctrl+alt+SysRq" wile holding them down press one at a time S then U then B ,,,,,,,,,,the system should reboot.
                Did nothing. System is shut down, right? Should the keyboard do anything at this point?


                Just for fun, boot up that system and kill network manager completely then try shutting down
                As soon as I kill it, it restarts.


                I will try yet another update today and see if that changes anything. If not, then I'll try as you suggest. I just looked up those commands and now understand that they should give more information so we know where to go from here. However, I cannot find an 'ajx' command Is there a spelling error there?

                Correction: I see that ajx are flags. sorry.


                try adding the lsof and/or ps ajx commands below the three lines that contain the remount command.
                Finally, do I understand correctly that the commands should go after the last UNMOUNT command? I see no 'remount' commands in there. The file should look like:


                Code:
                :~$ locate umountroot
                /etc/init.d/umountroot
                /etc/rc0.d/K11umountroot
                /etc/rc6.d/K11umountroot
                lsof ps ajx
                /lib/systemd/system/umountroot.service
                I thank you both for taking the time to help out.

                Correction: I totally misread the above LISTING. sorry. Another post to follow this one....

                Frank.
                Last edited by Frank616; Mar 17, 2016, 01:59 PM.
                Linux: Powerful, open, elegant. Its all I use.

                Comment


                  #9
                  oshunluvr:

                  OK, I finally stopped and READ you example of your SEARCH for umount. I have opened /etc/init.d/umountroot

                  The script is as follows:

                  Code:
                  #! /bin/sh
                  ### BEGIN INIT INFO
                  # Provides:          umountroot
                  # Required-Start:
                  # Required-Stop:
                  # Should-Stop:       halt reboot kexec
                  # Default-Start:
                  # Default-Stop:      0 6
                  # Short-Description: Mount the root filesystem read-only.
                  ### END INIT INFO
                  
                  PATH=/sbin:/bin
                  . /lib/init/vars.sh
                  
                  . /lib/lsb/init-functions
                  
                  do_stop () {
                      [ "$VERBOSE" = no ] || log_action_begin_msg "Mounting root filesystem read-only"
                      # Ask init to re-exec itself before we go down if it has been
                      # upgraded this cycle.  It'll lose all its state, but at least
                      # it won't hold open files on the root filesystem (lp:#672177).
                      if [ -f /var/run/init.upgraded ]
                      then
                          old_map=$( cat /proc/1/maps )
                          map=$old_map
                          telinit u || :
                          i=0
                          timeout=5
                          while [ "$map" = "$old_map" ]
                          do
                              sleep 1
                              map=$( cat /proc/1/maps )
                              /usr/bin/logger "waiting for init to respawn"
                              i=$((i+1))
                              if [ $i -eq $timeout ] ; then
                                  break
                              fi
                          done
                  
                          if [ "$map" = "$old_map" ] ; then
                              /usr/bin/logger "FAIL: init failed to respawn in $timeout seconds - unmounting anyway"
                          else
                              /usr/bin/logger "SUCCESS: init respawned after $i seconds (within $timeout seconds timeout)"
                          fi
                      fi
                  
                      MOUNT_FORCE_OPT=
                      [ "$(uname -s)" = "GNU/kFreeBSD" ] && MOUNT_FORCE_OPT=-f
                      # This:
                      #     mount -n -o remount,ro /
                      # will act on a bind mount of / if there is one.
                      # See #339023 and the comment in checkroot.sh
                      mount    $MOUNT_FORCE_OPT -n -o remount,ro -t dummytype dummydev / 2>/dev/null \
                      || mount $MOUNT_FORCE_OPT -n -o remount,ro              dummydev / 2>/dev/null \
                      || mount $MOUNT_FORCE_OPT -n -o remount,ro                       /
                      ES=$?
                      [ "$VERBOSE" = no ] || log_action_end_msg $ES
                  }
                  
                  case "$1" in
                    start)
                      # No-op
                      ;;
                    restart|reload|force-reload)
                      echo "Error: argument '$1' not supported" >&2
                      exit 3
                      ;;
                    stop)
                      do_stop
                      ;;
                    *)
                      echo "Usage: $0 start|stop" >&2
                      exit 3
                      ;;
                  esac
                  
                  :
                  So, If I have finally understood correctly, I should end up with the following snippet, right?

                  Code:
                  MOUNT_FORCE_OPT=
                      [ "$(uname -s)" = "GNU/kFreeBSD" ] && MOUNT_FORCE_OPT=-f
                      # This:
                      #     mount -n -o remount,ro /
                      # will act on a bind mount of / if there is one.
                      # See #339023 and the comment in checkroot.sh
                      mount    $MOUNT_FORCE_OPT -n -o remount,ro -t dummytype dummydev / 2>/dev/null \
                      || mount $MOUNT_FORCE_OPT -n -o remount,ro              dummydev / 2>/dev/null \
                      || mount $MOUNT_FORCE_OPT -n -o remount,ro                       /
                      lsof ps ajx
                      ES=$?
                      [ "$VERBOSE" = no ] || log_action_end_msg $ES
                  }
                  Sorry, its been a long day already....

                  Thanks for your patience.

                  Frank.
                  Linux: Powerful, open, elegant. Its all I use.

                  Comment


                    #10
                    Its two commands, so try

                    lsof
                    ps ajx

                    Please Read Me

                    Comment


                      #11
                      OK, just got back to sleuthing this issue.

                      I edited /etc/init.d/umountroot to look as follows:

                      Code:
                      MOUNT_FORCE_OPT=
                          [ "$(uname -s)" = "GNU/kFreeBSD" ] && MOUNT_FORCE_OPT=-f
                          # This:
                          #     mount -n -o remount,ro /
                          # will act on a bind mount of / if there is one.
                          # See #339023 and the comment in checkroot.sh
                          mount    $MOUNT_FORCE_OPT -n -o remount,ro -t dummytype dummydev / 2>/dev/null \
                          || mount $MOUNT_FORCE_OPT -n -o remount,ro              dummydev / 2>/dev/null \
                          || mount $MOUNT_FORCE_OPT -n -o remount,ro                       /
                          lsof       >  /home/irma/lsof.txt
                          ps ajx    >  /home/irma/ps.txt
                          ES=$?
                          [ "$VERBOSE" = no ] || log_action_end_msg $ES
                      }
                      As you can see, I played with the commands in an effort to pipe the output to a text file that I could read later, as otherwise the output goes by on the screen too quickly to read. Anyway, the system did not like my little effort, calling my command a 'garbage option'! (snif). The output I got is on the attached screenshot.

                      Suggestions? Corrections?

                      Thanks.

                      Frank.
                      Attached Files
                      Linux: Powerful, open, elegant. Its all I use.

                      Comment


                        #12
                        Could still use some help with this, thanks.

                        Did the pipe fail because the machine is mostly shut down? It worked on my own machine when I tried it from a console. Should I try

                        lsof | more

                        ?

                        Frank.
                        Last edited by Frank616; Mar 25, 2016, 03:50 PM.
                        Linux: Powerful, open, elegant. Its all I use.

                        Comment


                          #13
                          Can't hurt. Likely cant redirect to a file this late in the shutdown.

                          You could remove the redirect, record the scrolling text with your phone, play it back in slo-mo

                          Please Read Me

                          Comment


                            #14
                            Hadn't thought of taking a video. Though I think I'll try lsof | more first, and take pictures one screen at a time.

                            Once I have the output, what am I looking for?

                            Frank.
                            Linux: Powerful, open, elegant. Its all I use.

                            Comment


                              #15
                              Some process has a file open. You want to know what processes are running that have open files. I'm still wagering it's network manager. You should really try and kill it and all network processes then shutting down manually to see if it breezes past the remount.

                              Please Read Me

                              Comment

                              Working...
                              X