Data write to my external USB drives is very unreliable

[mralpen]

I would like to use an external USB disk drive as a backup device, so I bought a 500G Western Digital drive and USB2 enclosure. When I plugged it in to my USB port and tried to copy a large quantity of data the copy stalled after 50 or 60 MB (2%). After stalling, the disk disappears from the media:/. If I switch the drive off then back on again enables me to repeat this unsuccessful cycle.

The drive and enclosure work fine on Windows if the drive is formatted as NTFS, so I conclude that it isn't a hardware or interface problem. I reformatted to ext3 with the drive connected internally on an IDE interface, and everything worked fine. Then I refitted it in its external enclosure and once more the same problem occurs - the drive gets automounted, I try to write to it and the write stalls.

dmesg shows the following messages which I assume tell the sorry tale (mounting through to failure) from Linux's viewpoint:
[18706.877798] usb 5-6: new high speed USB device using ehci_hcd and address 3
[18707.011446] usb 5-6: configuration #1 chosen from 1 choice
[18707.076760] usbcore: registered new interface driver libusual
[18707.098070] Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
[18707.098198] ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
[18707.101820] Initializing USB Mass Storage driver...
[18707.101970] scsi6 : SCSI emulation for USB Mass Storage devices
[18707.102122] usb-storage: device found at 3
[18707.102125] usb-storage: waiting for device to settle before scanning
[18707.102328] usbcore: registered new interface driver usb-storage
[18707.102441] USB Mass Storage support registered.
[18712.099116] usb-storage: device scan complete
[18712.099868] scsi 6:0:0:0: Direct-Access WDC WD50 WD-WCAS83115211 1C02 PQ: 0 ANSI: 2 CCS
[18712.101469] sd 6:0:0:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[18712.102219] sd 6:0:0:0: [sdb] Write Protect is off
[18712.102222] sd 6:0:0:0: [sdb] Mode Sense: 00 38 00 00
[18712.102224] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[18712.103093] sd 6:0:0:0: [sdb] 976773168 512-byte hardware sectors (500108 MB)
[18712.103842] sd 6:0:0:0: [sdb] Write Protect is off
[18712.103845] sd 6:0:0:0: [sdb] Mode Sense: 00 38 00 00
[18712.103848] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[18712.103851] sdb: sdb1
[18712.108271] sd 6:0:0:0: [sdb] Attached SCSI disk
[18712.108307] sd 6:0:0:0: Attached scsi generic sg2 type 0
[18717.782284] kjournald starting. Commit interval 5 seconds
[18717.783643] EXT3 FS on sdb1, internal journal
[18717.783646] EXT3-fs: recovery complete.
[18717.784391] EXT3-fs: mounted filesystem with ordered data mode.
[18777.162980] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18787.401323] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18793.645874] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18823.873179] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18834.111526] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18840.356078] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18872.110539] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18872.222475] usb 5-6: device descriptor read/64, error -71
[18872.438356] usb 5-6: device descriptor read/64, error -71
[18872.654238] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18872.766177] usb 5-6: device descriptor read/64, error -71
[18872.982060] usb 5-6: device descriptor read/64, error -71
[18873.197940] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18873.605707] usb 5-6: device not accepting address 3, error -71
[18873.717653] usb 5-6: reset high speed USB device using ehci_hcd and address 3
[18874.125422] usb 5-6: device not accepting address 3, error -71
[18874.125453] sd 6:0:0:0: scsi: Device offlined - not ready after error recovery
[18874.125465] sd 6:0:0:0: [sdb] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK,SUGGEST_OK
[18874.125469] end_request: I/O error, dev sdb, sector 704070559
[18874.125474] Buffer I/O error on device sdb1, logical block 88008812
[18874.125476] lost page write due to I/O error on sdb1

sudo fdisk -l shows:

Disk /dev/sda: 750.1 GB, 750156374016 bytes
255 heads, 63 sectors/track, 91201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x30c29c7f

Device Boot Start End Blocks Id System
/dev/sda1 * 1 44619 358400000 7 HPFS/NTFS
/dev/sda2 90716 91201 3903795 82 Linux swap / Solaris
/dev/sda3 44620 90715 370266120 83 Linux

Partition table entries are not in disk order

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x57bb7c3a

Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 83 Linux

Following a failed write all reference to sdb1 disappears.

I am running gutsy, following a clean, fresh install from the downloaded live CD.

I don't see a load of people reporting similar challenges, so I assume that it's something peculiar about my set up. But can anyone offer me a clue about what's going on here?

Many thanks ...
David M

[dibl]

Re: Data write to my external USB drives is very unreliable

USB drives are tough -- the USB bus dynamically assigns device ID's (i.e. it can be different at any boot session), and that is kinda the root of all evil with USB devices.

IMO, the "best" fix for this, "best" being defined as reliable, not elegant, is to label your USB drive and set it up to be automatically mounted in /etc/fstab, using "mount-by-label. Assuming you have it connected when you boot, this will produce a reliably mounted drive that you can use like a normal hard drive. The downside is, the "hotplug" functionality goes out the window -- but I gather you're more interested in getting a good backup than hotplugging.

So, make a mount point in /media for it (mine is "NTFSTICK), make an appropriate mount line in /etc/fstab, and you should be OK. Here's the mount line for my NTFS-formatted 8GB USB stick that I use for similar purposes:

LABEL=DIBLSTUFF /media/NTFSTICK ntfs-3g user,noauto,atime,dev,exec,suid 0 0

[mralpen]

Re: Data write to my external USB drives is very unreliable

Thanks for the advice, dibl. I have tried this now, and several other things: specifying syn c/async in the fstab mount options, mounting by UUID and Label. Nothing seems to fix it.

Typically, it will be mounted at boot. When I try to write it will stall after a small amount of data has been transfered. Then it will disappear from the mounted filesystems. At the next boot a filesystem error will be reported.

Between these failed attempts I install the disk internally and reformat it using fdisk then
mkfs -t ext3 /dev/sdb1
e2label /dev/sdb1 backup01

As an internal drive it works perfectly. And as I mentioned in my first post, it works perfectly as an NTFS-formatted drive on a windows box.

Is there any endemic flakiness in USB port hardware or cables?

[SheeEttin]

Re: Data write to my external USB drives is very unreliable

There have been a few people reporting this.
My guess is that the drive is going to sleep, though I doubt it'd do that during a write...
Try going into your BIOS and toggling a "Legacy USB" option, if you have one. Sometimes that helps.
If not, I suppose you should file a bug (if one doesn't already exist). (Hmm... What would you file the bug against?)

[mralpen]

Re: Data write to my external USB drives is very unreliable

Thanks. Sounds like I'm stuffed.

It must be a strange hardware / OS interaction. I have a second gutsy box that I've just tried it on and it works fine as an external drive. So you'd think that it must be something to do with USB hardware on my main gutsy box. But no - at least not obviously - since when I dual-boot into Vista the drive is reliable.

So a combination of gutsy and this particular usb enclosure / drive just won't play nicely.