Announcement

**Qqmike** · Nov 14, 2009, 06:49 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Very interesting, nice post IgnorantGuru.

Like you, I try to watch these things, too, simply as a matter of principle, if nothing else.

I was shocked to learn what Windows XP _does to you_ (i.e., tracks), like the rather subtle ini and index.dat's, along with the long deeply-drilling list that CCleaner gets at.

If true, your report regarding Kubuntu/Flash is disconcerting.

My faint efforts to start into this subject are chronicled here, through Kcontrol, sweeper, and a few manual clean-ups:

-- Privacy Cleanup 101
http://kubuntuforums.net/forums/inde....msg115011#new
Reply #5: kcontrol Reply #8: How to Delete Data
Reply #10: Update/Summary for Intrepid Ibex KDE4

But already, I found weird things; like kcontrol not fully clearing Konqueror History (at the time--see the how-to). I also encountered a few quirky deals with tmp files, where, as you caution may be possible, sometimes you gotta mind your p's and q's when clearing out stuff.

It seems to me that the time may have come to have a "Crap cleaner" for Kubuntu, and maybe that is what yours is positioned to do.

Imo, this topic is important, and in my book your work on it is appreciated.
If you look at the downloads at CCleaner (in the Windows world), the topic clearly is important to PC operating system users.

**Snowhog** · Nov 14, 2009, 07:16 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Your script looks impressive IgnorantGuru. Can you add a function that simulates a cleaning, writing what it finds that would be cleaned (including full paths) to a file?

**Qqmike** · Nov 14, 2009, 07:50 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Yes, that's a useful feature of CCleaner.
In addition, how about a category titled, "What we did NOT get."

**IgnorantGuru** · Nov 14, 2009, 08:10 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Thanks Qqmike. I agree it's important, even if just on principle. I feel there are two kinds of threats. There are the outright viruses and malicious programs. But there are also more subtle problems introduced either as 'accidents' or 'oversights', sometimes in well known software or an OS itself. One of the primary reasons I left Windows some years ago wasn't that I couldn't get it to do what I wanted - I actually had XP running pretty well. It was that it was a constant fight against the OS and the software just to maintain some degree of privacy and security. But when the OS itself is against you, the best you can do is reduce the issues - like Windows and it's endless viruses and anti-virus programs. And it eventually became clear to me that much of what was going on in making Windows not secure was deliberate on some level.

Unfortunately, I'm starting to have the same feeling about Ubuntu. They throw in things like Nepomuk and PulseAudio (a security nightmare), without giving us options or ways to remove them easily. And when people do find a way to remove them, the next release makes it that much harder. KDE programs throw data all over the place so that if a program on your system does become compromised (Adobe Flash anyone?), even data you thought was gone is available for review. The problem has become that much more pronounced in KDE4 - even in the tools used to build it they are creating a marriage with Windows. I think it's important to know when to jump ship.

I'm getting ready to start shopping around again for a new OS, which will probably involve finding a simpler replacement for KDE4 as well, because I think it's going to get worse not better, a la Windows. But that's a lot of work, so in the meantime I am fighting KDE4 like I used to fight Windows.

Originally posted by Qqmike

I was shocked to learn what Windows XP _does to you_ (i.e., tracks), like the rather subtle ini and index.dat's, along with the long deeply-drilling list that CCleaner gets at.

Indeed, and KDE4 isn't far behind.

My faint efforts to start into this subject are chronicled here, through Kcontrol, sweeper, and a few manual clean-ups:

Thanks for pointing that out. That thread is a good read and a good introduction - covers some of the introductory stuff more thoroughly. From reading that thread, a few things I will point out as they apply to karmic/KDE4:

In my experience, /var/tmp and /tmp can be cleared between reboots without problems in general. This is evidenced by the fact that I have both mounted to tmpfs (ramdrive), so the data is not persistent between boots. That said, some non-default programs may try to use them persistently, so your mileage may vary.

However, deleting all files in those folders while the system is running can cause a meltdown, at least until reboot. Deleting the wrong files in /tmp can cause system mayhem in general. /var/tmp is used for plasma's cache. Deleting that while logged into KDE will cause major problems until reboot. kscrubber proceeds carefully in these areas, and will vary its behavior depending on whether or not KDE is running.

I'm comfortable that kscrubber addressed most of the issues in that thread. However, kscrubber does a non-thorough job with Konqueror, mainly because I don't use it anymore. I missed the ~/.kde/share/apps/konqueror/faviconrc and ~/.kde/share/apps/khtml/formcompletions so if they are still used in KDE4 I'll add them in. I don't pretend that kscrubber addresses everything, and as noted in the instructions, if there is some data you want removed, test to make sure it was.

As for Firefox, the recommended way to use kscrubber is to first clear the history using Firefox's Clear Recent History. Then exit and run kscrubber. This is necessary because Firefox stores some of its data, such as recent URLs, in the bookmarks file, which obviously can't simply be deleted. But since Firefox doesn't vacuum the SQL database, even when it allegedly clears this and other data, it still persists. So kscrubber then vacuums, as well as deleting a few files for good measure, and the job is done.

I haven't used kcontrol, but you wrote there "Not included with 8.10 (I read somewhere that you can install it, but it is not maintained", so if that is the case it will be even more out-of-date
with karmic.

One thing I like about scripts is that you can open them up, see what they do, and change them easily to suit your purposes - no compiling needed.

Also,

Klipper – Clipboard Tool: icon in system tray > Clear Clipboard History

In addition, Klipper in karmic will save its clipboard history to disk on exit by default. kscrubber will warn you if this option is enabled in the config (for each user), but it doesn't turn it off automatically (in case you want it on). FYI you can also run this command to clear the clipboard history:

Code:

qdbus org.kde.klipper /klipper org.kde.klipper.klipper.clearClipboardHistory

Unfortunately kscrubber can't do that automatically because it must be run by the user, not root. Nor does that command work from crontab because qdbus needs the X display - a poor design.

Thanks for your input!

**IgnorantGuru** · Nov 14, 2009, 08:25 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Originally posted by Snowhog

Your script looks impressive IgnorantGuru. Can you add a function that simulates a cleaning, writing what it finds that would be cleaned (including full paths) to a file?

Thanks Snowhog. I did consider that, but I decided it was redundant because the script can be examined to see all the file and pathnames. Or maybe I was just getting tired.

I like scripts because of this - any text editor can open them and you can see and modify what they do. If there is a particular program you use, search for all occurrences of its name in the script and you'll see what files are modified.

Do you think that is sufficient, or do you think it should still have a 'simulate' mode, and why? kscrubber doesn't do much in the way of scanning and analysis. Mostly it deletes some files regardless of what they contain, and attempts narrowly focused edits on other files to remove particular information (and if the file doesn't exist a silent error is produced, and if the information doesn't exist, the edit does nothing). So a simulate mode would simply display the lines of the script more or less, which might be rather intimidating.

Also, a simulate mode isn't a trivial addition because I can't just stick "echo" in front of some of the sed edits and other commands due to the use of single and double quotes, regular expressions, etc.

Anyway, I'll give it some thought. Also, the information page on the script has a Details section which describes in plain language what will be done.

Thanks for your comments.

**Snowhog** · Nov 14, 2009, 08:40 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Thank you.

I for one, like to 'simulate' actions that will 'change' what I already have. For that reason, I use the CLI for most install/remove/purge actions so I can see what will be affected. That's just me. But, as you say, your script can be examined (viewed) to see what you have 'coded' for removal - if found - is reasonable.

**Qqmike** · Nov 14, 2009, 08:46 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Thanks for the info in your Reply #4, really good stuff.

"One of the primary reasons I left Windows some years ago wasn't that I couldn't get it to do what I wanted - I actually had XP running pretty well. It was that it was a constant fight against the OS and the software just to maintain some degree of privacy and security."

Yep, Roger that. I devised a complex check sheet for such. Maintaining it and the tools used (e.g., latest CCleaner updates; A-V updates) was a PITA, a whole page of check items.

I really find this subject interesting and certainly important, and not being a programmer, maybe learning about scripts is the way to go. Would be nice for you or someone to work on getting a package up and always available (in repos) for such, but that's a heck of a lot of work. And you are right about how the challenge changes from one release to the next, a somewhat disconcerting and time-consuming challenge for janitors. Following kcontrol was sweeper (in 9.04, I think); don't know if you've checked that out.

Lest anyone think this thread nuts, or think this topic not important or relevant to them, let them use their OS for a day or three, then poke through their system & home files to see what's left behind

Start with their /home, enable View > Show Hidden files, and poke around for 30 minutes or so, drilling down and around. (What happens to data from deleted emails in the space compacted by Thunderbird?) I'm sort of behind on things in this area, still using 8.04.3; have 9.10 set up for test/play, and your analysis on KDE4 will be helpful when I get to it soon.

On a practical note, a thought I had before installing a new OS, not a bad idea to dd zero-out your drive, or just zero-out unused space on the (old) OS partition. (In the latter case, you'd have to do something like: use a live CD to access your old OS partition. Delete everything under root / except one small toy file. Then write zeros to that file (using dd or dcfldd) until you fill up the partition. Ha! -> not a bad idea to do this every now and then and re-install the same OS in the same "space"!

)

**IgnorantGuru** · Nov 14, 2009, 09:20 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Originally posted by Qqmike

Yep, Roger that. I devised a complex check sheet for such. Maintaining it and the tools used (e.g., latest CCleaner updates; A-V updates) was a PITA, a whole page of check items.

I really find this subject interesting and certainly important, and not being a programmer, maybe learning about scripts is the way to go.

Scripts are great, and at their simplest are just a list of CLI commands - your checklist in executable format. While this is written beneath your level, some readers might find this very basic discussion of what a script is useful...
http://igurublog.wordpress.com/libra...-instructions/

One of the reasons I'm sharing my scripts is to help people get into it. I have all kinds of little scripts running on my system. And with linux, you can do so much from the command line. Scripts are a poor man's program - very easy to get started with and learn as you go. If you know how to issue CLI commands, you can write a script.

I hadn't heard of sweeper (or kcontrol), but I'll have a look.

Lest anyone think this thread nuts, or think this topic not important or relevant to them, let them use their OS for a day or three, then poke through their system & home files to see what's left behind

Start with their /home, enable View > Show Hidden files, and poke around for 30 minutes or so, drilling down and around. (What happens to data from deleted emails in the space compacted by Thunderbird?) I'm sort of behind on things in this area, still using 8.04.3; have 9.10 set up for test/play, and your analysis on KDE4 will be helpful when I get to it soon.

Well said - it's scary, even when you're not doing anything secretive, to see this dossier on all your activities. You're going to spit bricks when you see the things KDE4 does!

On a practical note, a thought I had before installing a new OS, not a bad idea to dd zero-out your drive, or just zero-out unused space on the (old) OS partition. (In the latter case, you'd have to do something like: use a live CD to access your old OS partition. Delete everything under root / except one small toy file. Then write zeros to that file (using dd or dcfldd) until you fill up the partition. Ha! -> not a bad idea to do this every now and then and re-install the same OS in the same "space"!

)

That is a good idea, and the secure-delete package used by kscrubber has some tools for that as well, including a tool to wipe free space. Not just with zeros, but several passes of random data. Install the secure-delete package and ‘man srm’, ‘man sfill’, ‘man sswap’, and ‘man smem’

Also, encrypting the /home partition is a good way to combat some of these issues too. Mounting /tmp and /var/tmp to tmpfs (ramdrive) are also helpful. I initially did so to save my SSD from excessive writes, but it also improves speed and security. (Just note this yet to be resolved bug if you plan to mount /var/tmp or /var/log to tmpfs in karmic: https://bugs.launchpad.net/ubuntu/+s...ll/+bug/479429 ) I plan to release an 'SSD-setup' script at some point to handle some of this.

Also, here is a little script, not on my site at present, which fills up a device with random data - I wrote it to fill up USB drives, but it will work on any filesystem you have write-permission on. Nice if you're not comfortable with dd since it can be run as user. It just keeps writing 4GB files to the folder you specify until it's full, then you just delete the files. Just save this in your text editor as "fillrandom" and execute it with something like:

Code:

bash fillrandom /myfoldertofill

Code:

#!/bin/bash

chunk=4000000000

if [ "$1" = "--help" ] || [ -z "$1" ]; then
	echo 'Fills empty space in filesystem with pseudo-random data'
	echo 'Usage:  fillrandom FILESYSTEM'
	echo 'Example: fillrandom /media/disk'
	echo
	exit
fi

if [ -d "$1" ]; then
	x=0
	erc=0
	while [ $erc == 0 ]; do
		(( x += 1 ))
		cnt=$x
		if (( x < 10 )); then
			cnt="0$cnt"
		fi
		if (( x < 100 )); then
			cnt="0$cnt"
		fi
		rand=`uuidgen -r`
		rand=${rand:24}
		head -c $chunk /dev/urandom > $1/fr-$cnt-$rand.tmp
		erc=$?
		sync
	done
else
	echo Invalid filesystem "$1"
fi

**Qqmike** · Nov 14, 2009, 09:44 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Thanks for your link and tips on scripts, looks good, interesting, not "below my level."
That'll have to be one of my next-in-priority Linux projects. Looks like fun.
(OT: Programming ... My programming (in various languages) goes back to 1980-ish. Though I did write a little BASIC craps simulation in 1990 to test some of my take double-odds betting strategies

Oh, and a betting simulation in 1993 for playing the horses (overlay Show bets that paid 10% fairly consistently). And some "Monte Carlo" cost simulations for a gov't contractor in '98--boring, in-vogue work that paid well at the time and taught me that life is too short to sell yourself short doing that kind of make-work work at taxpayers' expense.)
I'll look into your links on scripts. Thanks, again.

**IgnorantGuru** · Nov 14, 2009, 09:48 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Someone sent me this bug report which deals with the Firefox sqlite issue. Apparently it's been fixed in a PPA, but if/when they're going to fix it in karmic updates remains to be seen. I'll post it here so others can keep an eye on it.
https://bugs.launchpad.net/ubuntu/+s...e3/+bug/457791

**IgnorantGuru** · Nov 14, 2009, 09:53 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

You're giving me flashbacks to the 80s and my Apple ][. I've always liked BASIC - it's very readable and you can do a lot in it without a lot of overhead, unlike most modern languages. Microsoft's VB was very capable - you could churn out an app in a few minutes. Check out kbasic.com sometime - I haven't used it (lack of a good project) but it looks good.

Scripts can look a little scarier, and bash has a perplexing syntax, but they amount to console commands, which are already familiar to linux users. When you add menu items and keyboard shortcuts for them, and schedule them to run with crontab, you can really customize and automate your system nicely.

**Qqmike** · Nov 15, 2009, 08:56 AM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

I don't want to go OT here, but to illustrate how tricky, yet important it is, take XP. (It may be worse or better with Vista and 7, I don't know, but would bet it is worse.) XP users should go to CCleaner to learn what these privacy issues are, issues far bigger than those in Kubuntu. For example, if you are a normal user in XP--not admin--running CCleaner will not delete your index.dat files! To do so, you must go into the admin account and do a search through your user account for index.dat files and delete manually. Now, it's best not to use your main setup-admin account for such; it's best to create a new, clean admin account for such maintenance (and keep it pristine clean). Furthermore, after running CCleaner--the best cleaner going--index.dat's are not actually deleted until you reboot. Point is, there's all sorts of worms in this issue, and most folks don't have a clue about the overhead committed in their OS to track and trace their PC use. This is not for me an issue of doing anything illegal with your PC--if you are doing that, well, you better really know what you are doing to beat today's PC forensic guys. For me, it is principle, privacy, and maybe some personal security. If you share your PC with another user, it definitely could be an issue for you. If you use XP, hope I didn't ruin your day

And please, let's not get further OT here in the thread IgnorantGuru started (because it will be my fault!).. I suppose my point would be that if you aren't concerned about the issue he's raised in his OP, you should be--at least to some reasonable extent, and at least to the point of being informed.

**IgnorantGuru** · Nov 15, 2009, 10:41 AM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Okay I did a little more testing with both Konqueror and Arora and have updated kscrubber to version 1.0.1. Here is what it now does for those two programs:

Code:

# Konqueror
$srm $userhome/.kde/share/apps/konqueror/konq_history* 2> /dev/null
$srm $userhome/.kde/share/apps/konqueror/closeditems_saved* 2> /dev/null
$srm $userhome/.kde/share/apps/konqueror/autosave/* 2> /dev/null
$srm $userhome/.kde/share/apps/konqueror/faviconrc* 2> /dev/null #defunct?
$srm $userhome/.kde/share/config/konq_history* 2> /dev/null
$srm $userhome/.kde/share/apps/kcookiejar/cookies* 2> /dev/null
$srm $userhome/.kde/share/apps/khtml/formcompletions* 2> /dev/null #defunct?

# Arora
$srm $userhome/.local/share/data/Arora/cookies* 2> /dev/null
$srm $userhome/.local/share/data/Arora/history* 2> /dev/null
$srm $userhome/.cache/Arora/WebpageIcons.db 2> /dev/null
$srm -r $userhome/.cache/Arora/browser/http*/* 2> /dev/null
$srm -r $userhome/.cache/Arora/browser/prepared/* 2> /dev/null

# Remove Flash cookies
$srm -r $userhome/.adobe/Flash_Player/* 2> /dev/null
$srm -r $userhome/.macromedia/Flash_Player/* 2> /dev/null

# Universal
$srm -r /var/tmp/kdecache-$user/http/* 2> /dev/null
$srm /var/tmp/kdecache-$user/favicons/* 2> /dev/null
$srm -r /tmp/kde-*/* 2> /dev/null
$srm /tmp/Flash* 2> /dev/null
$srm -r $userhome/.thumbnails/* 2> /dev/null
$srm $userhome/.recently-used* 2> /dev/null
$srm $userhome/.kde/share/apps/RecentDocuments/* 2> /dev/null

(The $srm variable is set to "srm" by default, which is a secure version of "rm".)

Interestingly, Konqueror now seems to create a second konq_history file in share/config/, which is an odd place to put it. I didn't actually find the faviconrc or formcompletions files, but kscrubber will attempt to delete them just in case they are around.

Also, as for thunderbird, kscrubber currently does nothing for that program. If someone gives me a list of commands to clean it I'll consider adding them.

For any program, to find out what files it's changing, leave your computer quiet for 3 minutes, then open the program and use it. If it's a browser, visit sites with Flash and lots of advertising to drive the cookies and such up. Then issue this command:

Code:

find /var /tmp ~ -xdev -cmin -3

That will display a list of all folders and files below those top folders (/var /tmp ~) which have been modified in the last 3 minutes. That gives you a good list to examine for private data leftover. Then close the program and run that command again, to see what it may be changing as it quits (writing config files, etc). (Note: if you have parts of those folders mounted or linked to other filesystems, remove the -xdev or add those filesystems to the list of top folders to search.)

http://igurublog.wordpress.com/downl...ipt-kscrubber/
(If you downloaded the script before, empty your browser cache so you get the new one.)

**IgnorantGuru** · Nov 15, 2009, 09:15 PM

Re: A new privacy and log scrubber for KDE4, Firefox, & Flash

Having reviewed sweeper, I can tell you kscrubber does a LOT more. KDE4, and even KDE3 for that matter, need a much more intensive cleaner than what sweeper does. One example - removing recently used documents from the Applications menu is all well and good, but when plasma and the file kdeglobals are recording file access, and central apps like kate and ocular are doing the same, you are only affecting cosmetics. Anyone going beyond the Applications menu to snoop will find those history sources easily. Nor does sweeper do anything about Flash cookies - as Konqueror also uses Flash, the Flash cookies affect it as well. And of course it doesn't touch Firefox from what I can see (it seems to be KDE only).

sweeper is handy for a few things but it is not an intensive system cleaner.

Announcement

A new privacy and log scrubber for KDE4, Firefox, & Flash

A new privacy and log scrubber for KDE4, Firefox, & Flash

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment