Copying files freeze/not responding

[Argent]

I run kubuntu 8.10

I have been attempting to copy files from my windows NTFS drive to my ext3 linux drive. Specifically this is around 4.5GB of many files. There's around 2,000 folders and 38,000 files in the 4.5GB. During the transfer the hard drive activity stops and linux freezes at a random point usually early in the transfer.

If it try control alt backspace the system tries to rest but i just get a plain cursor at the top left of the screen. Hard rebooting does work.

I have tried this on the gome desktop and the same occurs. I have also tried a ubuntu 8.10 livecd and the same occurs.

I have had success copying around 10GB of much larger files. no more than 10 files here.

I tried copying the files into an archive from the source in the NTFS directly to ext3 and that worked. However extraction of the archive from the ext3 drive to the same drive made it freeze the first time but worked the second time.

I checked both hard drives with smartctl and there were no apparent issues. I did not run the manual scan.

I don't really know what's going on.

[mfabry]

Re: Copying files freeze/not responding

What sort of driver do you use to mount ntfs partition? Personally I would recommend ntfs-3g.
What software do you use to copy the files? Dolphin? Midnight Commander?
Did you checked logs after reboot? (messages, syslog, kernel log)

[Argent]

Re: Copying files freeze/not responding

I use ntfs-3g.

In kubuntu (my install and the live cd) i used dolphin. In ubuntu gnome I used nautilus.

no i didn't check logs. I don't know where they are or what to look for.

[dibl]

Re: Copying files freeze/not responding

There's around 2,000 folders and 38,000 files in the 4.5GB. During the transfer the hard drive activity stops and linux freezes at a random point usually early in the transfer.


I don't really know what's going on.

Probably it is not really frozen, and there is a way to discover what's going on. Try it again, when you are in a position to leave the computer for some time. Before you execute the "copy" command (or drop the folder icon in Dolphin, or however you start it), open a console window and issue

top

This will show the running processes. Put this window in a corner of the screen where you can watch it. Now execute your copy command. What does top show?

Now just walk away ... and come back later (30 - 60 minutes) and see what the situation is.

[Rog131]

Re: Copying files freeze/not responding

There's around 2,000 folders and 38,000 files in the 4.5GB
...
In kubuntu (my install and the live cd) i used dolphin.

The Dolphin is a nice light point and click file manager but with lot of files or with large files i would use commandline or Krusader:

Krusader:

twin-panel (commander-style) file manager for KDE (and other desktops)

and with the commandline i would use rsync:

man rsync

rsync - faster, flexible replacement for rcp

and the rcp is

man rcp

scp - secure copy

Using:

rsync -P <Source> <Destination>

The P option will show progress information:
bits moved, percentage, speed (MB/s), time left.

[Detonate]

Re: Copying files freeze/not responding

Copying large files, or a large amount of files requires a lot of system resources. One thing that might help is to increase the size of your swap partition. Make it really big. You can always resize it again later if disk space is an issue.

[Argent]

Re: Copying files freeze/not responding

dibl and rog131:

I installed krusader.

I opened top as dibl asked. Did the copy and it worked in this case.

Before the copy top showed most of my 2Gb memory was used with 186,496k free, most of my swap was free. Not much cpu load.

During pre copy there was even less free memory.
the top cpu users were
mount-ntfs 33%
krusader 19%
Xorg 17%

When running memory was 53,780k free and roughly steady
CPU was also roughtly steady at
17% krusader
7% Xorg
7% mount-ntfs

I saw several kio-file in the list.

I then used dolphin instead.
Pre copy cpu load a bit higher than in the krusader case, memory about the same
During copy memory about the same as for Krusader cpu load
29% dolphin
9% xorg
6% mount-ntfs

again around 5 kio-file were in the list.

dolphin copy window said 'stalled' although it was still copying and then stalled shortly after at 44% this time. the top window kept working. 53,976k free memory. All cpu use was 1% or less. top users were dolphin, kwin, plasma, top and xorg. About 4 minutes later the dolphin window appeared to close. Moving the mouse caused parts of the desktop to disappear until i had a black screen except for top which was still working. dolphin was no longer on the list and free memory was hihger at 137,300k free.

I waited for 30 mins and there was no change to the situation so i did control alt backspace which gave a completely black screen. Then I hard rebooted the computer.

[dibl]

Re: Copying files freeze/not responding

It looks like maybe you have discovered a bug in Dolphin KDE 4 -- it should not compromise your video display that way. You can file it on Launchpad, if you want.

However, it's good that the copy process works as intended.

[Argent]

Re: Copying files freeze/not responding

I checked the destination and indeed only around 44% was copied.

It may be a bug in dolphin but if nautilus did similar it might be something else right? I might also check my ext3 drive is properly connected later.

[Argent]

Re: Copying files freeze/not responding

I've now had this problem during a file extraction in a wine based install. I want to fix this problem.

My system is dual boot kubuntu 32bit 8.10/ubuntu 32bit 8.10 (both the KDE and gnome desktops) with Windows Vista 64

I tried Copying 38,974 of 4.5Gb of files on NTFS drive to NTFS drive using windows explorer on Windows Vista.

That gave 3/3 successes

I tried Copying 38,974 of 4.5Gb of files on NTFS drive to NTFS drive using dolphin on a Live CD of Kubuntu 8.10 64bit. I did it on a liveCD to ensure the system was clean and did not run off the EXT3 drive.

That gave 3/3 successes

I tried Copying 38,974 of 4.5Gb of files on EXT3 drive to EXT3 drive using dolphin on a Live CD of Kubuntu 8.10 64bit.

That gave 1/3 successes. One stopped at 60% the other at 50%

I tried Copying 38,974 of 4.5Gb of files on EXT3 drive to EXT3 drive using Krusader on my installed Kubuntu 8.10 32bit.

That gave 2/3 successes. One stopped at 74%

When it' freezes' the mouse is still movable for a while sometimes the icon highlights will activates but the system does not respond to a click. The hard drive activity also ceases.

From these results I've gotten suspicious about my EXT3 harddrive and/or cables and checked the cables which seem ok and ran smartmon short and long tests on the drive which showed no issues. I also checked my RAM. No problems there.

I'm guessing the next best thing to do is get a new drive and reinstall linux on it. What do you reckon?

[dibl]

Re: Copying files freeze/not responding

Good job troubleshooting!

Yeah, it begins to look suspicious for the hard drive that is formatted ext3, doesn't it? But if smartcl says it is functioning correctly, I don't know what the problem could be, unless there is something dying in the electronics that isn't picked up by the SMART monitoring. It wouldn't hurt to swap the data cable -- they've been known to develop intermittent issues.

Ext3 is supposed to be the most robust general purpose filesystem presently available, according to most sources that I've reviewed, so it's odd to see a person experiencing such flakiness with it.

I guess I'd vote for more draconian experiments, like maybe format it ntfs and do some big file transfers that way (prolly cost you your Linux system, of course ....). You could just reformat it ext3 and try it again, but that sounds dangerously close to "Doing the same thing repeatedly, and expecting different results ...".

[Argent]

Re: Copying files freeze/not responding

"Doing the same thing repeatedly, and expecting different results ..."

Yes certainly a point and i did reinstall kubuntu as 8.10 32bit from a verified LiveCD. And it installed. However the problem is still there (expecting different results). However it does ensure the filesystem wasn't the issue because it redid that.

I also got error 13 from grub trying to boot Windows as free entertainment but switching my drives in the bios *seems* to have fixed that.

Next I'll try a new drive.

[dibl]

Re: Copying files freeze/not responding

Is the problematic drive an IDE drive? If so, it has a "ribbon" cable -- I would swap that out for a new one and try it again, before spending money on another hard drive. Those ribbon cables are not able to survive a lot of drive-swapping, in general -- the connectors are a weak point, and so are the kinks that they develop over use.

[Argent]

Re: Copying files freeze/not responding

Yes that was an IDE drive. I have already purchased and installed a new hard drive. The new one is a sata. After reinstalling kubuntu everything seems fine. 5 tests of 38,974 files as 4.5Gb and several other tests on sets of numerous and large files.

I did have kmix crash when I installed the nvidia driver but from looking around this is a common issue and the driver still installs fine.