Announcement

Collapse
No announcement yet.

security updates & external hard disk woes...

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    security updates & external hard disk woes...

    this is now solved! hooray!

    -----

    i'm using precise, i don't know how this got posted in raring, but there you go... maybe someone who has the ability to do so will move it to the correct location, whatever that may be... 8/

    a while back — a year and a half ago, or thereabouts — i decided that i was going to make a separate directory, on an external hard disk, that contained all of my essential data, data that i had created, so that when i upgraded my system, i wouldn’t face the potential of losing everything when a system upgrade didn’t go the way it was supposed to… and, the last time i did a major upgrade of my system, it worked exactly the way i expected it to work, and i was very happy.

    basically, instead of my “home” directory being at /home/salamandir on the primary hard disk of the computer, i partitioned my 3TB external hard disk and made a home directory at /media/home/salamandir and everything was fine... i just had to remember to change directories when i was downloading something, because i knew (from first-hand experience) that if i forgot, and downloaded to /home/salamandir/Downloads i would have difficulty finding the data afterwards.

    so i did the intermediary system upgrades on my linux box, yesterday. it wasn’t a full upgrade, but there were some more things than normal, and it “hung up” when it got to the point where it asked me if i wanted to keep the configuration file that “i” had “modified”, or if i wanted to download a new configuration file, but when i answered that i wanted to keep the configuration file that “i” had “modified”, the installation procedure proceded as normal.

    then i rebooted my computer, because i like to do that, to make sure there aren’t any delayed configurations, or things that don’t get installed at reboot that aren’t working the way they’re supposed to, and that’s where the problems started.

    i rebooted, and the first thing i noticed was that the external disks, including the 3TB disk that i have partitioned into “home” and “backup” partitions, weren’t mounting the way they were supposed to. i didn’t notice this until later, but the reason why the “home” and “backup” partitions weren’t mounting correctly was because, for some (as yet unknown) reason, they had been converted from /media/home and /media/backup to /media/salamandir/home and /media/salamandir/backup

    what this meant in the short run is that it couldn’t find the directory that contains my wallpaper graphics, and my “dropbox” folder was “moved” from where it was supposed to be, and these errors were what clued me in that something was amiss.

    this would have been an entirely different problem if i had noticed that immediately, but as it is, i tried to re-establish my directories where they should be, using system settings, and succeeded only in overwriting (or something, i still don’t know what) the directory in /media/salamandir/home/salamandir — which is where ALL my data from the LAST time this happened, now lives… because i was under the impression that if i kept my data on an external hard drive, that it would be safe from random deletion and/or disasters… like this one is turning out to be.

    /media/salamandir/backup (which contains backups of my “real” home directory - /home/salamandir - i.e. configuration files and nothing else) is still intact, /media/salamandir/home/salamandir-new (a directory i created when i was still trying to deal with the loss of data a year and a half ago) is still intact, but /media/salamandir/home/salamandir is NOT intact… which includes all of the printing clients that i have worked for in the past year-and-a-half, ALL of my genealogy data, a huge pile of my (irreplaceable) music files, and who knows what else.

    the only good part about this whole thing is that i BELIEVE all of the data is still there: usually it takes a while for the computer to actually delete a couple of gigabytes of information (which /media/salamandir/home/salamandir certainly is), and it didn’t actually do anything that seems like actual deleting, so what i suspect is that i just overwrote the file that tells the computer where everything is, and if i could just replace that, everything would be cool…

    but, so far, i haven’t been able to figure out how to do that.

    does anybody have any clues?
    Last edited by przxqgl; Aug 18, 2013, 05:56 PM. Reason: solved!
    ⇑ Hybrid Elephant
    http://www.hybridelephant.com/
    ⇓ The world's finest exotic incense

    #2
    Originally posted by Teunis
    Update: I just found this post explaining what happened:
    http://www.kubuntuforums.net/showthr...-my-USB-drives
    thanks! this helps a lot, since it describes precisely what happened. i am even more confident than ever that i can eventually recover my lost data.

    Originally posted by Teunis
    All I can contribute is you should NEVER have put your /home on an external- and thus removable- drive, a suggestion for next time...
    surprisingly, that is the first time i have ever heard that. i talked to a number of high level linux nerds for a number of years before i decided to do what i did... and apart from this particular incident, it has worked exactly as advertised. i would be very interested in hearing your reasonings for this conclusion, and any suggestions you might have for, in my instance, a /home directory that is larger than the primary hard disk.

    Originally posted by Teunis
    Further, having a back-up on the same disk or even another disk in the same machine is only marginally better than no back-up, really, a back up should be physically separated from the original!
    the /media/backup contains zip files which i also burn to CD, so i keep files separate from the actual machine. i don't know how to set it up to burn CDs directly, but i've got a script that runs on cron that backs everything up to a zip file, which i then manually burn to CD, twice a month.

    Originally posted by Teunis
    You don't mention it but I assume this is a USB drive, plugging it in to a running (Kubuntu) computer should auto-mount it somewhere under /media and you *should* still be able to find your old data on it.
    i don't own another computer running kubuntu, and i don't know anybody locally who runs it... i'm sure there must be someone, but the only other computers i have access to are macs, which, i'm guessing, will not work correctly.

    Originally posted by Teunis
    The way you seem to have set up this computer it's possibly hard or even impossible to boot it up with this disk disconnected
    actually, i can and have booted the computer with all of the (USB) drives detached, because they all failed to mount when i ran the updates.

    Originally posted by Teunis
    But before all of this, why not have the computer do the search for you?
    Open Dolphin, find the disk under /media and press Ctrl + F, in the top of Dolphin a search box will open and you type in the name of one of the files you seem to be missing.
    This should make the system search for it, it might take quite some time on a 3 TB disk.

    Another test is to see how many files are on the disk and how much free space is available, again indicators if your data is still present.
    Right click the disk (or one of it's directories) and select Properties.
    it sounds fairly simple, but reality is more difficult... i don't remember specific names, and even if i were to guess a specific name (which is entirely possible), the files that were in, for example, /media/salamandir/home/salamandir/Documents, aren't there any more, because the directory doesn't exist.


    Originally posted by Teunis
    What is the format of the disk, ext3 or 4?
    ext3 - i believe... it may have been ext4, but i actually created the partitions long enough ago that i don't remember any longer.

    Originally posted by Teunis
    In case the data has indeed disappeared from view there still is a chance you can recover some of it:
    http://askubuntu.com/questions/24037...xt4-partitions


    Success!
    thanks for the help and encouragement. i was fairly sure that the data was recoverable, i just hadn't the first clue how to go about it. i'll let you know how it comes out.
    ⇑ Hybrid Elephant
    http://www.hybridelephant.com/
    ⇓ The world's finest exotic incense

    Comment


      #3
      Moved to the Precise subforum.

      Comment


        #4
        Originally posted by Teunis
        I am glad to hear you were (in the past) able to start the machine with the disk containing your /home detached and not run into issues the next time you booted up with it re-attached. Normally this would seriously upset the 'logic expectations' in the file tree...
        i'm afraid i may have miscommunicated the difference between what i have set up as /home/salamandir as compared to /media/home/salamandir...

        i have never (before 3 days ago) attempted to actually "move" my /home directory. i have kept /home/salamandir intact, i have just created another directory, that i use as /home which is at /media/home/salamandir... this directory contains all of the data that i have created, apart from system configuration files, which reside at /home/salamandir

        i keep /media/home/salamandir on a separate 2TB partition, because there is a lot more data than will fit on my primary hard disk, in my /home directory, so i keep that information on a separate, external disk where it will all will fit. the system configuration files (which are the ones that get backed up) live at /home/salamandir

        two days ago, when my /media/home partition became /media/salamandir/home, was the first time i tried to "move" the /home directory, using the "System Settings" control panel, and that is when things started to disappear.

        does that make it more clear?

        Originally posted by Teunis
        In Kubuntu most, if not all, file operations will warn before removing, deleting or overwriting user generated data.
        So try to remember if you at any point knowingly allowed the removal of your data.
        And have a look in the trash can!
        i'm fairly sure that nothing told me that the data was about to be deleted. there was a notificaiton from the "System Settings" control panel that asked if i wanted to keep the files or move them, and it was after i said that i wanted to move them that things started to look different...

        which is why i don't think using the "Find" feature in dolphin is going to find the data... 8/

        Originally posted by Teunis
        The exceptions are mainly the (hidden) system configuration files in your ~/home.
        That's one of the potential catastrophic issues you run into with a removable /home, where else would the system have to find the needed configurations?

        It is well known you can in case of a serious configuration problem remove the ~/.kde directory and the system will on reboot generate a default new one but I wonder where it would end up in
        your scenario where the /home has been removed c.q. does no longer exist, possibly on the system partition?
        given that i have not actually removed the /home directory, i have just re-directed (most of) its contents, i wonder how removing the ~/.kde directory may help the problem... however i'm confused... you made reference to ~/home which i am going to assume means /home/salamandir (in my case) and ~/.kde means /home/salamandir/.kde - if i am incorrect in this assumption, please let me know.

        Originally posted by Teunis
        You don't need a second Linux machine, you can boot your present machine from a Live device to investigate your external disk.
        i don't keep a live disk around. i would have to download and burn one on my (faulty, defective, old) mac, which is unreliable at best... i plan on buying a more up-to-date mac, eventually, but i'm hoping to have my kubuntu system more or less fixed before that happens...

        Originally posted by Teunis
        Lastly, I find it hard to impossible to believe you don't remember (part of) some names of the files you are missing.
        A typical Kubuntu install pre-populates the /home directory with some generic names like /Desktop, /Pictures, /Documents, Music etc, as a start look for them!
        Or use the well-known wild cards like *.jpg for pictures, check on your back-up CD's what the typical names of your missing files look like.
        using a wild-card will definitely help, as will the fact that most of the pre-populated directories were used for the purposes for which they are pre-populated, but the main reason i don't remember specific filenames is because i have a brain injury that affects my ability to remember things in the short term.

        however, i am sure that a search for /media/home/salamandir/Documents/graphics/*.jpg will come up with something... once there's actually a directory at /media/home/salamandir/Documents/graphics... 8)

        once again, thank you most profusely for your help and advice.
        Last edited by przxqgl; Aug 17, 2013, 09:22 PM.
        ⇑ Hybrid Elephant
        http://www.hybridelephant.com/
        ⇓ The world's finest exotic incense

        Comment


          #5
          Originally posted by SteveRiley View Post
          Moved to the Precise subforum.
          thanks...

          hey, another kubuntu geek in the seattle area! wotcher, mate? 8)
          ⇑ Hybrid Elephant
          http://www.hybridelephant.com/
          ⇓ The world's finest exotic incense

          Comment


            #6
            "Wotcher"?

            Comment


              #7
              a greeting from "Asterix and The Cauldron" used by Chief Whosemoralsarelastix... 8)
              ⇑ Hybrid Elephant
              http://www.hybridelephant.com/
              ⇓ The world's finest exotic incense

              Comment


                #8
                Oh! Teh Googelz only showed me Harry Potter stuff. But hey, I'd be in the dark anyway -- haven't read Potter or any of the adventures of Whosemoralsarelastix. I spend too much time here on KFN! lol

                Comment


                  #9
                  it's actually something that i learned a long, long time ago... before the age of cell phones, internet tubes and the evil koch brothers... 8)
                  ⇑ Hybrid Elephant
                  http://www.hybridelephant.com/
                  ⇓ The world's finest exotic incense

                  Comment


                    #10
                    Originally posted by SteveRiley View Post
                    Oh! Teh Googelz only showed me Harry Potter stuff. But hey, I'd be in the dark anyway -- haven't read Potter or any of the adventures of Whosemoralsarelastix. I spend too much time here on KFN! lol
                    http://www.phrases.org.uk/meanings/wotcher.html
                    ⇑ Hybrid Elephant
                    http://www.hybridelephant.com/
                    ⇓ The world's finest exotic incense

                    Comment


                      #11
                      Originally posted by Teunis
                      I appreciate your explanation around remembering file names and it's good to see you have a way around it using your computer's options.
                      Like I can't remember a person's name unless she has long hair and is curvy
                      yours is a software problem, mine is a hardware problem... but they both have the same results. 8)

                      Originally posted by Teunis
                      OK, you confirmed you selected the 'move' option to get the original home to the new one.
                      Move will normally only delete the original after the new version has been created meaning there should still be a version of your files on the disk.
                      i'm wondering, as well... since the "new" /home directory was actually created at /media/salamandir/home and the data that i was hoping would not get deleted was at /media/home is it possible, after i did the "/etc/udev/rules.d/99-udisks2.rules" thing, that if i removed the ~/.kde directory with the USB drive disconnected, rebooted, and then connected the USB drive after the reboot, if the /media/home would not just "show up" along with the "new" /media/salamandir/home
                      directory?

                      or is it possible that i'm expecting too much?

                      Originally posted by Teunis
                      About /home and ~/.kde, I accidentally/erroneously included the tilde (~) in the /home description.
                      The tilde stands for your /home directory so ~/.kde is the hidden kde directory in your home folder which in my case would be /home/teunis/.kde
                      that's what i thought, but another affect of the brain injury is that, sometimes, i have trouble determining what people really mean when they say/write things... especially if i don't know them that well... ;-)

                      Originally posted by Teunis
                      As stated I am not at all familiar with Macs, does a G5 or Macbook have the ability to boot from a USB drive?
                      not that i know of... i've been a mac-head ever since i first started using computers, back in the dark ages (my first computer was a "Lisa"), but i've never tried to do things that macs weren't specifically designed to do... which is one of the big reasons i decided to start delving into linux, around 2000 or so...

                      Originally posted by Teunis
                      Success with searching your disk.
                      i'm hoping for the best, but prepared for the worst... i'll know which it is soon...
                      Last edited by przxqgl; Aug 18, 2013, 10:53 AM.
                      ⇑ Hybrid Elephant
                      http://www.hybridelephant.com/
                      ⇓ The world's finest exotic incense

                      Comment


                        #12
                        and, as with many other things where i've been "hoping for the best and expecting the worst", what actually happened is somewhere in between.

                        i did the "/etc/udev/rules.d/99-udisks2.rules" thing, which worked, but it didn't restore my data.

                        i then re-named the ~/.kde directory (to ~/.130818-kde) and rebooted with the USB drive disconnected.

                        when it came up, i had a fresh, clean KDE, but none of my preferences were there, and when i started up email, it started at the very beginning, as though i don't have an account... i shut down email, because i don't want to re-create my accounts, and some really interesting, but for the current subject, pointless things happened...

                        which is sort of what i expected, but here's the interesting part for the current subject:

                        /media/home is there, as it should be. /media/home/salamandir is also there, but the only VISIBLE contents are an empty /Desktop directory, and an empty /Downloads directory... however, when i right click and select "Properties", it tells me that there are 31.3GB taken up by 45,277 files in /media/home

                        which is a very strong indication that my data is still extant... once again, however, i haven't the first clue how to access it... so here's what i'm thinking.

                        /media/backup contains backups of my /home/salamandir directory. is there any reason why i cannot unzip one of the backed-up archives, retrieve the /home/salamandir/.kde directory from one of those, and replace my current /.kde directory with that?

                        my impression is that it would restore my defaults, paths, and everything to what they were before this disaster happened. is that an incorrect assumption?

                        UPDATE: i just couldn't be that lucky... while it did restore a lot of my defaults, it didn't restore the visibility of my files... they're still there when i select "Properties", but they're not visible or accessible... 8/
                        Last edited by przxqgl; Aug 18, 2013, 01:38 PM.
                        ⇑ Hybrid Elephant
                        http://www.hybridelephant.com/
                        ⇓ The world's finest exotic incense

                        Comment


                          #13
                          Originally posted by Teunis
                          Losing all your settings when renaming .kde is as expected.
                          The fact you lost your mail tells me you are using Kmail, because Thunderbird would be in a hidden .thunderbird file in your home, not in .kde.
                          i didn't actually lose any of my email, but it appears that i may have lost my filters, or, perhaps, some of the directories have been "changed" or something.

                          at this point, after crashing twice, once when i booted after renaming ~/.kde and once after replacing it with a ~/.kde from a backup, kmail is functioning the way i would expect.

                          Originally posted by Teunis
                          If this had any beneficial influence on recovering the lost data fine
                          it didn't have any effect on the recovery of my lost data, but it did have a significant effect on the recovery of my preferences, which it would have taken me hours, if not days to reconstruct...

                          Originally posted by Teunis
                          Dolphin properties telling you there are x-thousands of files will be based on hard fact, a good sign indeed
                          i figured it was... now to figure out how to make them visible.

                          Originally posted by Teunis
                          It seems you don't see them in Dolphin?
                          You know how to show hidden files? (press Alt+. that's Alt plus a dot)

                          Another way is to use the CLI, when in Dolphin and on the 'empty' directory you press F4.
                          This will open a terminal in the bottom half of Dolphin with the cursor in the directory that has focus in the top half.

                          Type
                          Code:
                          ls -n
                          and hit return, you should get a list of the content with two columns of the number 1000, that's you the regular owner and group you belong to.
                          In case you see rows of different numbers like 1001 or 27 we'll have to fix the ownership.
                          In case you see nothing you can try again with
                          Code:
                          ls -a -n
                          , the -a makes hidden files visible.
                          nope. NONE of this did anything to show me where that 31.3GB is hiding... 8/

                          it's still there, i just can't see (or access) any of it.
                          Last edited by przxqgl; Aug 18, 2013, 02:35 PM.
                          ⇑ Hybrid Elephant
                          http://www.hybridelephant.com/
                          ⇓ The world's finest exotic incense

                          Comment


                            #14
                            Originally posted by Teunis
                            Forgive me when I sometimes ask for what might be obvious to you but as you noted we don't know each other...
                            it's okay... how else are we going to get to know each other? 8)

                            Originally posted by Teunis
                            You are sure the properties that report the thousands of files were done on the directory that otherwise seems empty?
                            correct. when i view /media/home/salamandir, i see two empty directories, /Desktop and /Downloads, however when i right-click in the window and choose "Properties" it tells me that 31.3GB are used for 45,227 files, which is about 13% of the 913.8GB available on the partition.

                            Originally posted by Teunis
                            When Dolphin reports 'something' surely the list command should show at least similar.
                            one would think so...

                            Originally posted by Teunis
                            Back to the terminal (or Konsole)
                            Code:
                            ls -R -1 | wc -l
                            This is a Recursive list command, it should give you the number of all files in this and the subdirectories.
                            what i get is

                            Code:
                            salamandir@Gingko:/media/home/salamandir$ ls -R -1 | wc -l
                            13770
                            Originally posted by Teunis
                            Adding -a will include hidden files.
                            Code:
                            ls -R -a -1 | wc -l
                            that gives me:

                            Code:
                            salamandir@Gingko:/media/home/salamandir$ ls -R -a -1 | wc -l
                            62188
                            Originally posted by Teunis
                            Another command would be:
                            Code:
                            find | wc -l
                            This should really show a similar number to what the Dophin properties report...
                            that gives me

                            Code:
                            salamandir@Gingko:/media/home/salamandir$ find | wc -l
                            48662
                            somewhat confusing, and definitely a step into a realm that i know nothing about... what's next? 8)
                            Last edited by przxqgl; Aug 18, 2013, 04:09 PM.
                            ⇑ Hybrid Elephant
                            http://www.hybridelephant.com/
                            ⇓ The world's finest exotic incense

                            Comment


                              #15
                              Originally posted by Teunis
                              Around my bedtime I would one more time use Dolphin and it's Cntr+F.
                              This time I suggest to use the wildcard
                              Code:
                              *.
                              this should make a huge list of files.

                              SNIP!

                              Now or before, in the top of Dolphin's main window you can right-click on 'name' or 'size', from the drop down menu you select Others and then put a check in front of Path.
                              This will show you where these files are hiding.
                              i did the *.jpg search, and found A WHOLE BUNCH of files, most of which i remembered immediately...

                              and then, WHAM! and suddenly the empty /Desktop directory had all of my stuff in it. i believe, at this point, all i have to do is move it from /Desktop one level down, and everything should be as it was.

                              i appreciate the help! 8) 8) 8)
                              ⇑ Hybrid Elephant
                              http://www.hybridelephant.com/
                              ⇓ The world's finest exotic incense

                              Comment

                              Working...
                              X