Email disappeared in KMail

[Chopstick]

Hi,
Yesterday I had a strange accident happening in KMail; as a result all email in one folder was gone (except some very recent emails). I logged into my email (IMAP) account to see if they are still there, but they were gone, too. And there was nothing in the trash either.
I hit "undo" a few times, but that just brought back some spam from the trash.
I suspect that this incident was initially caused by me accidentially dragging another inbox folder into the affected folder, and then moving it back to were it was supposed to be. For some reason, every email in the moved folder was duplicated (maybe I copied it in the first incidence?). But removing duplicates is easy, so no big deal.
I was able to restore most of my mail from backups, but I still lost a few days, and as it happens, a lot of important things were discussed via email and a lot of this is lost...

So, at this point, I would like to know what caused this problem in the first place, and, if it might be possible to still recover the lost emails. I also don't really know where the offline IMAP data are kept.

Thanks in advance!

[SteveRiley]

IMAP clients usually reflect what's on the server. In KMail, IMAP information is stored in two places: an Akonadi database maintains pointers to folders and messages, while the actual message files are stored in ~/.local/share/akonadi/file_db_data/. It could be that your Akonadi database is out of sync with the message files, and the message files themselves may still exist. If you search through the files in the aforementioned subdirectory, can you find any "orphaned" emails?

[Chopstick]

Thanks Steve; I saw this folder but couldn't really make much of it initially.
After you said it is the email databse, I realized that the files are actually plain-text and sort-of human readable.
But there are about 17,000 files - I have several other email accounts.
So I greped for some email addresses of friends and find messages, but none of the ones that disappeared. There seem to be some very old orphaned messages, but nothing recent that is not already in my inbox... :-(

I'm still puzzled how I did this in the first place: even if I moved/copied a folder into this one, it should not delete the existing messages, right? And I am surprised that KMail just happily deletes all the emails on the server without even asking for confirmation... anyway... it can't be reversed...

[DoYouKubuntu]

I can't help with your immediate problem as I use neither KMail nor IMAP for mail. But I thought I'd throw out there that there's a valuable lesson to be learned here: MAKE BACKUPS!

Seriously, I have an almost obsessive/compulsive need to back things up that are important to me, including mail. The few seconds it takes to do my daily routine is well worth it. I use a combination of off-site storage (my own domains plus Dropbox), backing up to other hard drives on my network, and backing up to external drives. I know it sucks right now that you lost some important mail, but learn from this and don't let it happen again in the future.

(And I really hope this doesn't sound snarky because it's not meant that way at all.)

[Chopstick]

I can't help with your immediate problem as I use neither KMail nor IMAP for mail. But I thought I'd throw out there that there's a valuable lesson to be learned here: MAKE BACKUPS!

Seriously, I have an almost obsessive/compulsive need to back things up that are important to me, including mail. The few seconds it takes to do my daily routine is well worth it. I use a combination of off-site storage (my own domains plus Dropbox), backing up to other hard drives on my network, and backing up to external drives. I know it sucks right now that you lost some important mail, but learn from this and don't let it happen again in the future.

(And I really hope this doesn't sound snarky because it's not meant that way at all.)

I used to do backups daily, but I never needed them, so relaxed that a bit.
I still backup my code and all my research daily (or more), but I stopped doing email backups so often, because I figured the server would store them for a good while, in case I would loose something. What really got me off guard is that KMail just happily deleted the mails on the server! Is there a way one could implement some safeguard against this? (Like not delet for good, but move to trash first or something like that.)

[DoYouKubuntu]

Unfortunately, I have zero experience with KMail other than installing/using it for purposes of explaining something to someone, and that's it. I use SeaMonkey, which is a complete suite, including web browser and mail/news clients. I haven't had an IMAP account in years; all of mine are POP. But I really have a hard time imagining how mail on your IMAP server just got deleted without input from you. That is *SO* contrary to everything we expect in the Linux world. I hope Steve or someone with recent experience using IMAP can help, because I have to wonder if those e-mails are really gone or if they just APPEAR to be gone. I don't know, perhaps there's an index that needs to be rebuilt or some other type of repair process run.

[Chopstick]

Usually I would agree with you; but actually there was input from me, only that this was not the intended or expected result... and usually you don't drag folders into oneanother, so the behavior might not be well tested.

And in the Linux world there is a 'rm *' command that, in a script, can have very similarly unintended consequences ... ;-)

I can't see the messages anywhere in my imap account (also when I log into the web interface), and they are not in the akonadi database (see reply to Steves comment).

[DoYouKubuntu]

following contains a console

Usually I would agree with you; but actually there was input from me, only that this was not the intended or expected result... and usually you don't drag folders into oneanother, so the behavior might not be well tested.

Yeah, I meant in terms of you choosing to mass delete those files. BTW, in SeaMonkey's mail client you're perfectly free to drag and drop folders into other folders. (I just did it to make sure!)

And in the Linux world there is a 'rm *' command that, in a script, can have very similarly unintended consequences ... ;-)

Warning: The command following WILL DESTROY your system if you run it.

Yes, but not nearly as impressive as running rm -rf / while logged in as root.

[SteveRiley]

I'm still puzzled how I did this in the first place: even if I moved/copied a folder into this one, it should not delete the existing messages, right? And I am surprised that KMail just happily deletes all the emails on the server without even asking for confirmation... anyway... it can't be reversed...

Moving a folder from one place to another will delete the messages from their prior location. Copying will not. In either case, the action you perform in the mail client is immediately reflected on your IMAP server too.

[DoYouKubuntu]

Oh, crap. I SHOULD have had the sense to post my own warning on that rm command... Thanks, Snowhog, for stepping in.