Announcement

Collapse
No announcement yet.

The Saga of Installing 15.04 on a USB Flash Drive

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [SOLVED] The Saga of Installing 15.04 on a USB Flash Drive

    My residence is in South Australia, however I am now in the UK for about 3 months.

    We found out the day before Christmas that our Daughter-in-Law in the UK had bone cancer in her spine which was a secondary cancer from breast cancer that she had some years ago. She has one child who is now 2 years old. So we had to, at short notice, travel to the UK to lend support to our UK family.

    I only have my trusty Laptop (ASUS R501VM) with me and no other Computer on which to try new distributions. I do not want to alter the partitions on this Laptop, so I decided to install 15.04 on a 32GB USB 3.0 flash drive. The following list of problems may be related to the write performance of the Kingston Datatraveler 111 flash drive which has a write speed about 20% slower than the read speed. Even if this is the case, I think the software should behave correctly and allow for slow drive I/O conditions.

    I must say that after overcoming most of the following problems, I am very impressed with the speed of vivid and how quickly it is developing. I hope that this posting helps improve the performance of vivid.


    MY PROBLEM LIST

    The first problem that I encountered was that for each installation attempt, I found that the installation hung when I advance to the Disk Setup. It gives no options for disk setup and when I looked at the processes with System Monitor, I could not see any processes related to the disk setup. Every time this happens, I just exit the process with the “X” and start it again and, voila, I get the disk setup menu, so I can advance as normal – well maybe.

    The second problem that I encountered was that Screen Locking was enabled and I had to constantly come and unlock it to find out the progress. Because of the frustration that this caused me, I removed Screen Locking in System Settings > Desktop Behaviour. So I could now watch the progress.

    The third problem that I encountered was related to the fact that I no longer needed to nurse the installation. I could go away and get on with other activities. Under these conditions, when I returned, the screen was black and non-responsive to any keyboard or mouse input. I did yet another download of a more recent iso file and had another go. The stall of the installation happened again. So I learned from this to just sat through the installation process, making sure that I did something on the keyboard or mouse every now and then. Thankfully the installation went through to the end – hurray!!!

    The fourth problem occurred on my first installation attempt. I decided to use the option “Guided – use entire disk”, ie my Kingston DataTraveller 111 32GB flash drive. After using this system, I was puzzled by the fact that I could only access about 13GB of memory. After checking what was going on, I found that about half the disk was allocated to a swap file! This was solved by downloading another iso and installing with the manual option for the disk partitions. After subsequent re-installations related to the above problems, I found that I did not need to have a swap file on the flash drive as 15.04 was using the swap file on my Laptop's SSD. Using that option gave more memory for my home partition which then resulted in more problems.

    The fifth problem resulted from copying large directories from my SSD home directory into the home directory for vivid. When the message came that the copy was complete, I then closed down and when I rebooted, the system failed to come up, just a black screen. On one occasion I waited a while and the hplip-gui reported that there was no system tray. It was not after repeated re-installations did I realize that after a copy is completed there is a process that utilizes about 30% of my CPU for some time (ie eight 2.3GHz processors). I can close down if this has been running for some time and then start again. This time I let the grub 10s time expire and then proceed with login and wait for that to complete. I then have a usable system. The lesson I learned from this was that after a copy is complete, start the System Monitor and wait until the CPU usage drops to less than 10%. On some occasions vivid fails to start after the login step and I get a black screen. If this happens, I wait for the completion of grub and then let the login menu be there for a few minutes before login.

    The sixth problem is that Dolphin does not complete the copy of individual files. I noticed this particularly when I had files that could not be copied due to protection issues. When I removed these problems, the directory sizes are still not the same in some cases, but the number of files is. This is illustrated in the attached graphic. I only notice this problem with copying large directories. The iso file that I used for this system is dated 30 January 2015.

    Click image for larger version

Name:	vivid_Dolphin_Utopic.jpg
Views:	1
Size:	46.0 KB
ID:	648880Click image for larger version

Name:	vivid_Dolphin.jpg
Views:	1
Size:	43.0 KB
ID:	648881

    #2
    Baloo Hogs Processor Resources

    I decided to repeat copying the My_Documents directory onto Vivid and once again my system became very sluggish but after some time it becomes usable. As System Monitor gives limited information on what processes are running I decided to use htop. The attached image shows what is happening well after the completion of copying the directory and that baloo_file is utilizing 100% of one CPU. I have not checked the installation "disk" to see if it has "top" available. If I do another installation I will definitely use it to check on the culprit for problems with installation.

    Click image for larger version

Name:	baloo_hog1.jpg
Views:	1
Size:	108.1 KB
ID:	642722
    Last edited by NoWorries; Feb 03, 2015, 11:53 AM.

    Comment


      #3
      I have witnessed this also. Going up to 100% for a longer time. I once killed it and it didn't return to life at that time.

      I am not sure whether it was doing anything on USB. I may have had some connected, always an USB stick in this thing (laptop).
      Last edited by xennex81; Feb 04, 2015, 12:51 PM.

      Comment


        #4
        Baloo is the file indexer, which would definitely bog down a flash drive, at least until it finished indexing. I'll bet a portion of the problems you have may be due to the slowness (relatively speaking) of a flash drive - I wonder if even a usb3 drive can handle the volume of read/writes that an ssd can?

        Have you updated the OS since install? I did notice a speed improvement and some bug fixes relating to my screen locking up regularly on my laptop in recent days.
        .

        Comment


          #5
          It would be very weird if the file indexer would take up 100% cpu if the USB link was the bottleneck, right?

          I mean, that would suppose the CPU consumption would go down, not up, right? .

          Comment


            #6
            I have a full install (not a LiveCD) on a USB3 flash drive for fun. The difference in speed is barely noticeable, but I do nearly everything in a browser these days, so nothing that is write intensive. Desktop Search is turned off of course.

            Comment


              #7
              Sorry I have taken so long to get back to you all. I have had a number of updates and I cannot see baloo_file being used. Looks like the developers have put it in its place as I no longer have any process using 100%. I deleted two large directories on vivid and reloaded the largest one again. During the load plasmashell was the largest CPU activity which was very modest as shown in the htop image taken during the copy.
              Click image for larger version

Name:	no_baloo.jpg
Views:	1
Size:	91.4 KB
ID:	642723

              The only outstanding problem is that the file size is different between the vivid and utopic. I have only checked two hidden files between on the two systems and they were the same. That does not mean that all hidden files are the same. So I will wait and see if updates fix this problem as they did with the baloo problem. An example of the differences that I am now getting is shown below.

              Click image for larger version

Name:	vivid_Dolphin.jpg
Views:	1
Size:	43.1 KB
ID:	642724Click image for larger version

Name:	vivid_Dolphin_Utopic.jpg
Views:	1
Size:	48.2 KB
ID:	642725

              In conclusion, I have a Birthday this month and I will be putting in a suggestion to the family members that I get a Sandisk Extreme 64GB USB 3.0 Flash Memory. This will be about 2 weeks from now, so after installing Vivid on that, I should be zipping along.

              Comment


                #8
                Of course, not saying I really know what you have been busy with at this point... like, I haven't read everything as attentively, but I just would like to suggest that perhaps a folder's file size is really the block space being used. If there are different blocks used (different filesystem, or whatever) you would get different output. I would always check both "du -hs <foldername>" and what you get for instance when you tar the thing.

                Comment


                  #9
                  The only difference being 8192 bytes? Which = 4096 X2 or 512 x16. Sounds like a difference in block sizes on the two devices.

                  For problem 3 - I bet your USB port powers down in power save mode.

                  Please Read Me

                  Comment


                    #10
                    The problem here was Ubuntu's choice of default I/O scheduler. For many years, the default scheduler for Linux distributions has been CFQ. Ubuntu changed the default to deadline in 12.10. This went mostly unnoticed until Baloo appeared.

                    CFQ exposes a mechanism for programs to indicate an I/O priority. Baloo uses this mechanism and sets its priority to "nice." This means that Baloo will engage its indexer only when the system is idle. Unfortunately, deadline has no such mechanism: it will absolutely always serve all I/O requests. So on 14.04 and 14.10, Baloo's attempt to quiet itself is ignored and thus Baloo receives I/O time it doesn't want. You can fix this by reverting Ubutu's change and returning to CFQ. Add elevator=deadline to the kernel's boot options in grub.cfg.

                    Kubuntu successfully petitioned to change the default back to CFQ for rotating media only. This is handled by the file /etc/udev/rules.d/60-ssd-scheduler.rules from the package kubuntu-settings-desktop. The change was backported in October to 14.04 and is present in 14.10 and 15.04.

                    If you should ever notice Baloo hogging the CPU, run:
                    Code:
                    grep . /sys/block/sd*/queue/scheduler
                    If deadline is indicated on any rotating drives, something on your system probably changed it. Double-check that the Udev rules file exists, or consider the kernel boot option. A quick and dirty (and non-persistent) fix is:
                    Code:
                    echo cfq | sudo tee /sys/block/sd[i]X[/i]/queue/scheduler
                    where X is the drive in question.

                    Comment


                      #11
                      On my HP laptop (spinning HDD):

                      paul@tanagra:~$ grep . /sys/block/sd*/queue/scheduler
                      noop deadline [cfq]

                      Baloo behaves properly here; 14.10.
                      Windows no longer obstructs my view.
                      Using Kubuntu Linux since March 23, 2007.
                      "It is a capital mistake to theorize before one has data." - Sherlock Holmes

                      Comment


                        #12
                        Originally posted by SteveRiley View Post
                        The problem here was Ubuntu's choice of default I/O scheduler...
                        Thanks again, SteveRiley, for an illuminating explanation.

                        ... The change was backported in October to 14.04...
                        I'm on trusty and /sys/block/sd*/queue/scheduler shows "noop [deadline] cfq" for all the sd* devices. Do I have to "enable backports" to get this fix? It affects me if I do large copies to a USB backup disc.
                        Regards, John Little

                        Comment


                          #13
                          Update on File Transfer

                          I have only addressed a couple of the comments below. For all these operations, baloo_file was not a problem and plasmashell seems to be doing the indexing and continues to be busy after the completion of the file activity which makes the system sluggish.

                          I shall start with the reason for checking the transfer of large directories. Soon after overcoming all the problems I encountered with installing Vivid on my USB 3 flash drive, I transferred multiple large file directories from my Utopic disk to my Vivid USB drive. I also had problems with my system hanging during this transfer which I found later was from baloo hogging the system. I was interested to test the system by loading a large presentation file and found that it was not a complete file and shorter than the source file. Hence my interest in checking the properties of the directories and found discrepancies.

                          Zip results
                          Following the suggestion to compress the directories, I selected 3 directories having a total of 7.7GiB and copied them from Utopic to the Vivid. I compressed the directories on both systems and the total numbers were 7.2 GiB (7,696,121,884) on Utopic and 7.2 GiB (7,698,281,999) on Vivid. The file size is 2,157,774 greater on Vivid. By comparison the Properties from the original uncompressed files were 7.7 GiB (8,260,134,766) on Utopic and 7.7GiB (8,260,126,574) on Vivid. Thus the compressed file was larger by 2,160,115 B on Vivid and the uncompressed files were larger by 8192 B on Utopic. None of these numbers are multiples on the block size of the drives. Also, during these operations baloo_file popped in and out with about 1% processing load.

                          I have not checked the changes proposed by Steve Riley as baloo is now behaving itself. I will be interested to know if others have found that Steve's approach solves this problem.

                          Sector Size
                          On the subject of Sector size being different and thereby causing different total number of bytes, I used the KDE Partition Manager on Utopic (Note: the Vivid version does not work.) and found that both the Logical and Physical Sector Size was 512 B on both my USB and SSD drives. My system does not go to standby during these operations as I have my Laptop connected to mains power for all operations.
                          Last edited by NoWorries; Feb 07, 2015, 02:37 PM.

                          Comment


                            #14
                            Hey boys, leaving this thread. I am not sure what is going on, but i do think you should test whether there are really discrepancies in the files themselves (using diff-s) before you can say there is an issue here. Regards..

                            Comment


                              #15
                              Originally posted by xennex81 View Post
                              Hey boys, leaving this thread. I am not sure what is going on, but i do think you should test whether there are really discrepancies in the files themselves (using diff-s) before you can say there is an issue here. Regards..
                              I have checked using diff. The problem that I had was that diff would not compare files in the directories on Vivid, so I had to copy the directory from Vivid to Utopic. I did the comparison so that all files in the subdirectories were compared using the command:
                              Code:
                              diff -ur "Directory_Name"/ "Directory_Name_Vivid"/
                              I repeated this for all directories that I copied.

                              I am pleased to report that there was no difference in the files for all directories. I also removed the three directories on Vivid and redid the copy. Once again there was no difference reported by diff. One interesting event associated with this copy was baloo_file appeared again and stayed there using 100% of one of the 8 processors.

                              I hope to be able to do another installation of Vivid in about a week on a higher performance USB 3.0 flash drive. If this goes through without any of the problems that I reported at the start of this thread, I will be very happy to make this thread resolved.

                              Comment

                              Working...
                              X