Pendrive, Files disappering in not safe remove.

[manolis.K]

After copying files on my pendrive, they disappear if I do not safely remove it (I never actually do the save remove thing). That is not happening under kubuntu 8.04 or ubuntu 9.04.
Does anybody know how to fix it?

[Qqmike]

Re: Pendrive, Files disappering in not safe remove.

You need always to "Safely Remove" the flash drive, each time.

dibl explained why at length in a recent post.

[KeithZG]

Re: Pendrive, Files disappering in not safe remove.

I've actually had this "problem" since sometime during 5.10 beta, heh (or in other words almost as long as I've been using Kubuntu; beforehand and with other distros I didn't have any USB drives). The issue I've found is:

(1) With Linux, you can trust it; safely remove!
(2) With Windows, god help you if you tell it to "safely" remove

My younger sister has also independently come to this same conclusion.

In other words, I can understand why you might feel like not choosing "safely remove", especially if you've been trained by Windows. But with Linux you can trust the coders, and in fact you really really should.

[Teunis]

Re: Pendrive, Files disappering in not safe remove.

The sync option on mount is what has in the past made you do something wholly unsafe.
There are several reasons to not use the sync option in a mount, speed is one, wear of flash disks is another.

So when you are interested in the subject please google on something like

linux usb mount options sync

[dibl]

Re: Pendrive, Files disappering in not safe remove.

Basically, the actual process of writing data into the USB stick memory cells (or onto the magnetic media of a hdd) is a background task that the OS performs for you. The screen GUI display of that action is a "foreground" task. Being the visual creatures that we are, the human tendency is to assume that when the little screen graphic indicates the data are written, then it must be so. However, the computer background task that is actually doing the writing may or may not actually be done with it. In the case of Dolphin, it seems to be quite "optimistic" in its presentation of the graphic -- sometimes I don't even think the background task has started, if the blinky lights on my USB stick are to be believed. But whether you are running Windows or Linux, the wise bunny uses "safely remove" or "eject", which has the effect of making double-damn certain that the background writing task is finished before indicating that you can remove the device.

Yep, Teunis has given a good Google search term -- thanks!

[KeithZG]

Re: Pendrive, Files disappering in not safe remove.

But whether you are running Windows or Linux, the wise bunny uses "safely remove" or "eject", which has the effect of making double-damn certain that the background writing task is finished before indicating that you can remove the device.

On the contrary though, I've had data loss and corruption from Windows' "Safely Remove" feature; I have never had it as long as the copy said it was done (although yeah the blinky lights are very important) and I just yanked it out.

If I recall there's something fundamentally different about the way that Windows handles writing to USB drives by default, in fact it's fundamentally different even from how Windows handles other types of drive (I mean IDE or SATA or etc). I wish I could find the source material again, it was actually pretty interesting reading, but the gist of it is that Windows (or at least XP, and I think Vista) actually writes the data exactly when it says it does. Part of this is that it makes no allowance for caching or any such thing, which means there's no outright need to run a safely remove if it says its done copying but that it performs far worse at the actual file copying operation.

...it's more nuanced than that, though. I wish I could find a source for the details of how Windows handles USB drives, but all my googling has come up with has been registry hacks and commercial products (reminding me again of why I ran away from the land of Windows, heh).

I remember stumbling upon this kernel of info while researching out of curiosity why Windows would, with some of my USB drives, take up to five times (!) as long to copy the same data as Linux would.

....but yes, I agree wholeheartedly with Linux that you should always safely remove.

[dibl]

Re: Pendrive, Files disappering in not safe remove.

Interesting!

It is true that Windows, being descended from a single-user architecture, has a tighter "connection" between what the GUI says is going on, and what is actually going in the background. Linux, allowing for the possibility of other users with other more important needs, takes its time scheduling activities, sometimes. You can tighten up Linux to some degree by tweaking the /etc/sysctl.conf settings:

http://rudd-o.com/en/linux-and-free-...ow-to-fix-that

There's more you can do -- fiddling with "dirtywritebacks" and such. But it's still Linux, not Windows (thankfully).

[manolis.K]

Re: Pendrive, Files disappering in not safe remove.

Thanks all for the input I 'll try to figure out how sync works.
Even if I forced myself it's certain that 7 out of ten time I'll forget to safely remove it.

[KeithZG]

Re: Pendrive, Files disappering in not safe remove.

Fascinating read, dibl In terms of the swappiness question, there's an interesting debate between some of the kernel devs at http://kerneltrap.org/node/3000

On the topic of write caching in Windows I did come across some explanations of Windows XP's behaviour; an article containing detailed info, and a short comment on Slashdot which has the salient details, and might actually have been where I first heard it.

manolis.K, you really should get in the habit of using Safely Remove! Using write caching is, in the end, a faster and safer method of copying data to USB drives. That being said, I do understand the deleterious power of habits.