A new privacy and log scrubber for KDE4, Firefox, & Flash

[IgnorantGuru]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Problem is that Konqueror was not open; nothing was open besides Konsole (where I was running the kscrubber command). kscrubber DID, though, seem to clean out Konqueror files.

kscrubber uses this command to see if konqueror is running:

ps -A | grep konqueror

So it probably was running - maybe a hung process in the background. kscrubber will still clean the files, but the problem with running programs is that when they do exit, they may overwrite some of the files that were cleaned, restoring the data that was removed.

Thanks for the feedback.

[sithlord48]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

this script seams to take a LONG time on my machine..... it took about an hour to finish w/ --onepass. would love to have some more I/O so i know its not frozen

[IgnorantGuru]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

this script seams to take a LONG time on my machine..... it took about an hour to finish w/ --onepass. would love to have some more I/O so i know its not frozen

Wow! You must have a LOT of private data. There may have been some very large temp files that got shredded. Probably won't take that long on future runs. I'll consider the I/O feedback - maybe a verbose mode. If I add a preview function I'll throw verbose in as well. Otherwise the hard drive light is a pretty good indicator, since srm uses a lot of syncs. And there isn't much in kscrubber that is likely to hang, so it's safe to give it some time.

I may also make 1 or 2 passes the default, since 38 pass wiping (srm's default) is a bit much for most purposes. Thanks.

[sithlord48]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

well i did have a LOT of temp data (around 2 GB.)
i will re run when i get home and let you know how it goes.. oh and yes 38 pass might be a bit much for avg. user , but if you change default(i think 2 pass would be good for most) to a lower number perhaps a switch like --passes (int) would be good for those who want more passes

And there isn't much in kscrubber that is likely to hang, so it's safe to give it some time.

exactly why i let it run for so long... (not like i didn't read the script before i ran it)

thanks again wonderful tool

[IgnorantGuru]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

@sithlord
srm can either do one pass, two pass, or 38 pass (if you're interested use 'man srm' for the details). I will indeed allow the user to choose regardless of the default.

Thanks for your feedback. If anyone has anything else on a wishlist now is a good time to tell me as I may work on this again shortly - maybe today.

[kubicle]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Kubuntu 8.04.3 (with KDE 3.5.10)
sudo myscripts/kscrubber --onepass --clean
kscrubber: Option --onepass - using one-pass wiping
WARNING: The following programs are currently running. kscrubber cannot
reliably clean the files of running programs. For best results,
quit these programs before proceeding:
konqueror

Do you want to proceed? (y/N)

Problem is that Konqueror was not open; nothing was open besides Konsole (where I was running the kscrubber command). kscrubber DID, though, seem to clean out Konqueror files.

IIRC, KDE3 has an option to preload a konqueror instance (and keep one loaded) for faster access. That would explain a running konqueror process even if no konqueror windows are open.

[IgnorantGuru]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

IIRC, KDE3 has an option to preload a konqueror instance (and keep one loaded) for faster access. That would explain a running konqueror process even if no konqueror windows are open.

True that might be it. Not sure there will be an easy way to tell the difference, but if someone likes they can run the following command twice, once with a konqueror window open and once without a konqueror window open, and post the output of each.

ps -Af | grep konqueror

Otherwise it's safe to just ignore the warning that konqueror is running.

[IgnorantGuru]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

An updated kscrubber 1.2.0 is now available...

kscrubber version 1.2.0
Cleans logs and removes recent activity information from system and
user files. This version is designed for use on Ubuntu and Arch with
a focus on KDE
Requires: secure-delete sqlite3
Must be run as root
Usage: sudo kscrubber [OPTIONS]
Options:
--clean            Clean system and user files
--keepsystem          Don't clean any system files in /var/log,
                 /var/crash, or /tmp
--keepsystemlogs        Don't clean system logs in /var/log
--keepuser           Don't alter user files in /home, /root,
                 /var/tmp, or /tmp
--killindex          Disable & clean Akonadi,Soprano,Nepomuk
                 EXPERIMENTAL - see instructions
--sim             Simulate only (also enables --verbose)
                 No files will be modified.
                 Note: If used with --backup, backup
                 files WILL be created
--force            Don't halt for any warnings
--verbose           Detailed feedback
--logfile FILE         Also append output messages to FILE
                 (place option first for best results)
--backup FOLDER        Create backup copies in FOLDER of files
                 to be removed or edited. Note: If used
                 with --sim, backup files WILL be created
--onepass           Use less secure one-pass wiping of files
                 (faster and may be better for SSDs)
                 Default is two-pass
--fullpass           Use srm default (38-pass) wiping of files
                 (more secure but much slower)
--check PATH "SEARCHSTRING"  Search recursively in PATH for files or
                 filenames that contain SEARCHSTRING
                 (text or a regular expression). You may
                 include multiple --check options to
                 search for multiple strings.
Instructions and updates:
[url]http://igurublog.wordpress.com/downloads/script-kscrubber/[/url]

Applications cleaned or tested by this version of kscrubber:
akonadi amule ark arora avidemux epdfview filelight firefox flash gftp
googleearth gqview gwenview k3b kaffeine kate kgrab kmail konqueror
kpdf krename krusader ktorrent mplayer nepomuk okular parcellite
smplayer soffice soprano speedcrunch vlc java KDE klipper krunner
plasma pidgin

I have tested it pretty thoroughly but if you do find anything not working as advertised please let me know. The actual cleaning commands issued are almost identical to the previous version 1.1.2.

[Qqmike]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

I really like it a lot!
Referring to the simulation mode (--sim), very nice!
That's all I ran for now since you said the cleaners are the same (and I'm in the middle of something else at the moment).


VACUUM: /home/mike/Desktop/M1K1PC1_2Stuff/M3_PC3/Linux/Kubuntu21_S1etup_&_Fix4/T-Bird/Profile/Profile 12-28-09/xxxxxxxx.default/urlclassifier2.sqlite

Two things about that:
> It occurs to me that the SQL db issue applies also to Thunderbird?
> It occurs to me that kscrubber goes looking (recursively) for instances of the *.sqlite. This copy of the T-Bird profile is a backup copy I keep in a folder (as you can see).

=> Very nice.

I also notice the two pass option.

Only thing might be for the user, to be clear that unless he specifies --clean, he won't get much (Nothing to do). Even for simulation: --sim --clean.
Of course, we will figure that out, but some new folks might not.

Keep up the good work, IgnorantGuru.
Thanks.

[IgnorantGuru]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Two things about that:
> It occurs to me that the SQL db issue applies also to Thunderbird?
> It occurs to me that kscrubber goes looking (recursively) for instances of the *.sqlite. This copy of the T-Bird profile is a backup copy I keep in a folder (as you can see).

kscrubber runs sqlite3 to vacuum all *.sqlite files found under /home. This saves disk space and also improves security, particularly for Firefox. Since sqlite3 does the vacuuming, this is reliable, even if the programs are running (which locks the database).

If you prefer it just vacuum Firefox databases, there is a section for that in the script which is commented out (disabled). You can enable that section and disable the section below it which vacuums all of $userhome/

Or, if you or others feel it should work differently, perhaps just being limited to Firefox, I will consider changing the default behavior if that causes a problem. Based on the research I did, vacuuming all databases in /home was recommended. Also, only files that end in ".sqlite" are vacuumed. So if you rename your backup "*.sqlite-bak" kscrubber won't vacuum it. (As a result, "*.db" files, which are sometimes SQL, are also not vacuumed.)

Only thing might be for the user, to be clear that unless he specifies --clean, he won't get much (Nothing to do).

That is by design, because I want the user to be clear on what is to be done. This isn't the type of program you want to run inadvertantly. I did add "(use --clean to clean your system)" to "nothing to do" in this version.

Also, if you get a moment, run this for me twice on your system, once while you have a konqueror window open and once when konqueror windows are closed, then post the output.

ps -Af | grep konqueror

With that I might be able to add a custom test for konqueror so you don't get a false warning. I think kubicle may be correct about the preloaded instance. Thanks for your feedback.

[Qqmike]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

No Konqueror windows open:
mike@mike-desktop:~$ ps -Af | grep konqueror
mike 23111 6123 0 13:45 ? 00:00:00 konqueror [kdeinit] --preload
mike 23172 23155 0 14:12 pts/1 00:00:00 grep konqueror

One Konqueror window open:
mike@mike-desktop:~$ ps -Af | grep konqueror
mike 23111 6123 0 13:45 ? 00:00:01 konqueror [kdeinit] --preload
mike 23177 6123 13 14:12 ? 00:00:00 konqueror [kdeinit] --preload
mike 23179 23155 0 14:12 pts/1 00:00:00 grep konqueror

Two Konqueror windows open:
mike@mike-desktop:~$ ps -Af | grep konqueror
mike 23111 6123 0 13:45 ? 00:00:01 konqueror [kdeinit] --preload
mike 23177 6123 1 14:12 ? 00:00:01 konqueror [kdeinit] --preload
mike 23189 6123 10 14:14 ? 00:00:00 konqueror [kdeinit] --preload
mike 23191 23155 0 14:14 pts/1 00:00:00 grep konqueror

[IgnorantGuru]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

@Qqmike
Okay thanks - I updated it (v1.2.1) to do a special test for konqueror. It should ignore one and only one preloaded instance, but only if it has "--preload" in its arguments. The result should be if you have no windows open it won't give you a warning.

If there is more than one preloaded instance it will give a warning, but this isn't that critical. For the sake of thoroughness, it's just best to close as many running programs as possible before running kscrubber.

You may have to clear your browser cache before you download the update - should report itself as v1.2.1.

[Qqmike]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

"You may have to clear your browser cache before you download the update - should report itself as v1.2.1."

Trying to catch me, huh? As you might surmise, my browser cache is always cleared ... isn't tours?

[Qqmike]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Deleting a file ...

With the older wiping tools like shred, wipe, etc., mention is made that they don't work with journaled file systems. While I don't know enough about the workings of ext3 specifically, it is somewhat intuitive why a tool might not work on a file system that keeps copies laying around.

I have not seen mention of such limitations for srm (with respect to deleting a file). man srm mentions RAID, NFS, and other limitations, but no mention of journaled file systems. So, apparently srm is built directly on dd?

This raises another point: apparently dd has no such limitations? Apparently (somehow) whatever copies ext3 keeps, it doesn't prevent proper deletion of files, such as:

Example:

File called testmenow.
ls -l
-rwxr-xr-x 1 mike mike 23996 2008-01-17 14:37 testmenow

Size: 23,996 bytes.
Thus:
dd if=/dev/zero of=testmenow bs= 23996 count=1 conv=notrunc
Output:
1+0 records in
1+0 records out
23996 bytes (24 kB) copied, 0.00538647 seconds, 4.5 MB/s

Then delete the file:
rm testmenow

In an ext3, can we be sure it is gone as well as any copies/artifacts thereof?

[IgnorantGuru]

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

I have not seen mention of such limitations for srm (with respect to deleting a file). man srm mentions RAID, NFS, and other limitations, but no mention of journaled file systems. So, apparently srm is built directly on dd?

I'm far from an expert on this, but to my thinking srm does work within the filesystem (accessing it via kernel calls). AFAIK it's the best wiping tool around. Not only does it use a specific pattern of passes to prevent recovery, but unlike other such tools it is at least aware of the filesystem limitations and helps avoid problems there, eg

man srm
...
As an additional measure of security, the file is opened in O_SYNC
mode and after each pass an fsync() call is done. srm writes 32k
blocks for the purpose of speed, filling buffers of disk caches to
force them to flush and overwriting old data which belonged to the
file.

I don't think there is any way to guarantee no leakage on journaled filesystems, though, especially because you don't know where else the fs may have stored pieces of the file before srm even got to it.

For a more thorough cleaning, you can convert an ext3 filesystem to ext2, wipe files, then convert it back. This article explains how. The sfill utility in the secure-delete package is also worth looking into...

man sfill
sfill is designed to delete data which lies on available diskspace on
mediums in a secure manner which can not be recovered
...
afterwards as many temporary files as possible are generated to wipe
the free inode space. After no more temporary files can be created,
they are removed and sfill is finished.

This raises another point: apparently dd has no such limitations? Apparently (somehow) whatever copies ext3 keeps, it doesn't prevent proper deletion of files, such as:
dd if=/dev/zero of=testmenow bs= 23996 count=1 conv=notrunc

There too, I think dd is writing into the ext3 filesystem via kernel calls. That will be true anytime a file is written to in the filesystem. The 'direct disk' part only comes in when dd is reading or writing to/from a device, not a file.

Then delete the file:
rm testmenow
In an ext3, can we be sure it is gone as well as any copies/artifacts thereof?

Certainly not. rm only marks the space consumed by the file as 'available', it doesn't remove the data. (That's why it's so much faster than srm, and why kscrubber took an hour to run on sithlord's system).

Just remember anytime you have a file that is on that filesystem, then it is journaled.

These filesystems were not designed with good security in mind, as least not in terms of clean removal. So all solutions are awkward at best.

A really clean option is to delete files you don't want, copy the rest to a backup, overwrite or wipe the original partition completely with dd, format it, then copy the files back. Short of dropping the drive into a bucket of acid to dissolve it and using a new one, that's probably the best you can do. It's a good idea when setting up a new OS.

Let's say sda1 is your system partition and sdb1 is a free backup area...
(WARNING: don't use these commands unless you know what they do!)

# format sdb1
mkfs.ext3 /dev/sdb1
# mount sda1 and sdb1
mount /dev/sda1 /mnt/sda1
mount /dev/sdb1 /mnt/sdb1
# copy files from sda1 to sdb1
cp -ax /mnt/sda1/. /mnt/sdb1/.
# unmount sda1
umount /dev/sda1
# overwrite all of sda1 with random data
dd if=/dev/urandom of=/dev/sda1
# format sda1
mkfs.ext3 /dev/sda1
# mount sda1
mount /dev/sda1 /mnt/sda1
# copy the data back
cp -ax /mnt/sdb1/. /mnt/sda1/.
umount /dev/sdb1
umount /dev/sda1

Even that won't do multi-pass wiping on sda1 - use sfill for that after you've formated sda1. But it will take care of file fragments and give you a filesystem that only contains what you know, at least on the first recording level. (Magnetic media can 'remember' data to several depths, or overwrites, which is how recovery is accomplished and what srm aims to defeat by overwriting it 38 times with specific patterns.)