KleanUp: Secure, automatic and komplete kleaning of Kubuntu

[lightzoo]

Update:

"KleanUp v0.3.01b"(March, 10th, 2007)

Download KleanUp and FAQ: http://www.geocities.com/lightzook/KleanUp.html




KleanUp v0.3.01b Change Log:
================================================== ==================


Major Updates:

-Wipe is now installed by KleanUp into /usr/local/bin using the source pre-compiled by me.
--Wipe is now called by "wipe [options] [file]"

-RAM disk shred option has been commented out and should not be used for the time being
--I am hoping to fix the RAM shred soon

-Size of mounted OS free space _before_ and _after_ KleanUp runs is now printed to the file "Before_After.txt"
--The file is located at ~/KleanUp/results/Before_After.txt
--Read the file for more information
--Useful to find amount space recovered by KleanUp

-The mounted OS File System type and it's configuration is now printed to the file "File_System.txt"
--The file is located at ~/KleanUp/results/File_System.txt
--Read the file for more information
--Useful for evaluating the level of secure data removal in regards to the file system


Minor Updates:

-When shredding the Free space file "scrub" Wipe uses _one_ known pass (zero).
--I have reduced the number of passes because they are redundant (to Scrub) and the extra passes take a long time

-Cleaned up the code a good bit and improved explanations (hopefully)

-Fixed an issue where the /root/.kde/cache-*-desktop/ not being shredded

-Updated the "Read_Me.txt" to reflect relevent info in this file and improved explanations (hopefully)

-Created this Change_Log.txt


News:

-KleanUp has a new home on Sourceforge at https://sourceforge.net/projects/kleanup !
--I haven't done much yet, I'll post when the project pages are ready...for now Geocities will be home
================================================== ===================






Benifits of KleanUp:
Using KleanUp and the packages KleanUp offers to install will easily free up at least a few hundred MB's of space and help you retain a lot of privacy.


Preamble:
I wrote two scripts (KleanUp.sh and KU_root.sh) to automate the process of removing old, orphaned, unused, broken, bloated and unneeded file types and directories from Kubuntu (specifically 6.10). KleanUP automatically and securely shreds files, directories, Swap partition, Free space, RAM disk, etc, etc.

In short, KleanUp started as tool to reduce the size of my install . But then it turned into a much more robust and secure tool which still reduces the size of an installed system but also offers much more. I couldn't find a tool on Linux/Kubuntu which would automatically and securely shred directories, securely clear files (eg. 'konq_history"), shred data, Swap, Free space, Slack, etc, etc. There a quite a few shredding tools for Linux (eg. Shred, Wipe, THC secure_delete, DiskScrub, etc, etc) but none automatically shred data which can compromise your privacy...so I wrote KleanUp which in it's current version uses 'Scrub' and Wipe.



Read_Me.txt:
================================================== ==================

Read_Me version 0.2.03
March, 13th, 2007

KleanUp v0.3.01b
Written by 'lightzoo'
Homepage: https://www.sourceforge
Email: lightzooK@yahoo.com
Thread: http://kubuntuforums.net/forums/inde...opic=3080222.0
GnuGP Public Key: See end of file









KleanUp




- - ---> Files:


Files included in /KleanUp-0.3.01b/:
KleanUp.sh = The main and base script of KleanUp
KU_after.sh = The script which computes and prints the total size of Free space, etc after KleanUp runs
KU_before.sh = The script which computes and prints the total size of Free space, etc before KleanUp runs
KU_FS.sh = The script which prints the type of FS and the FS's configuration for the mounted OS.
KU_root.sh = The script section of KleanUp which shreds data in non-/home/ directories, Swap, Free space, etc.


Files included in /KleanUp-0.3.01b/src/Wipe_v2.3.0-cvs/:
wipe-wip-20050509.tar.bz2 = Source files for Wipe v.2.3.0-cvs
wipe-wip-20050509.tar.bz2.sig = Author's signature for source files of Wipe
wipe-2.3.0-cvs_compiled_src.tar.gz = Pre-compied source files to facilitate installation of Wipe
wipe-2.3.0-cvs_compiled_src.tar.gz.sig = My signature for the pre-compile source file of Wipe


Files included in /KleanUp-0.3.01b/results/:
Before_After.txt = Total amount of Free space, Swap, etc before and after running KleanUp, usefull to know amount of Free space recovered.
File_System.txt = Shows the type of FS (eg. ext3) and it's configuration, important to evaluate possible level of data shredding.

- - ---> Usage:


1-Close _all_ non-essential applications.

2-Extract the file "KleanUp-0.3.01b.tar.gz"

3-Enter the newly extracted folder "/KleanUp-0.3.01b/"

4-Open a terminal and run this command: sh ./KleanUp.sh

5-KleanUp creates directory tree "~/KleanUp/..."

6-KleanUp files are 'cp'd into ~/KleanUp/

7-KleanUp runs, shredding files, folders, FS, etc

8-_DO NOT_ interrupt the shredding process

9-KleanUp exits and the computer reboots

10-Manually delete KleanUp-0.3.01b.tar.gz and it's extracted folder

11-In the future run KleanUp with:

cd ~/KleanUp
sh ./KleanUp.sh



- - ---> General Information:


KleanUp was written using Kubuntu 6.10 Edgy and I have only tested in on Kubuntu 6.10.

KleanUp does not address the issue of file slack space (inodes). When you wipe Free space with KleanUp only unallocated space will be wiped, file slack in allocated space will not be wiped.

Software wiping is more effective on modern, high-speed hard disks than on outdated (e.g., RLL) units, Zip, and floppy disks. However, there is *NO* software-based substitute for physical disk destruction. Anyone who believes that a software wipe utility can be 100% foolproof is mistaken. Software wipe tools can be better or worse, but none is perfect.

Journaling filesystems (eg. ext3) cannot be wiped with 100% accuracy and confidence due to the nature of JFS meta-data (used on current Linux Kernels). For dependable and accurate data sanitation use an ext2 filesystem with a large amount of RAM (+512) and _disable_ the Swap permanently. For more information regarding the limitations and security holes of JFS see: http://en.wikipedia.org/wiki/Shreddi...g_file_systems


For more information and a list of links see the KleanUP FAQ and downloads section on the KleanUp website:
http://www.geocities/lightzooK/KleanUp.html


- - ---> Shredding algorithm details:


>Wipe v2.3.0-cvs<
KleanUP uses Wipe to shred files (using wildcards and recursive shredding), entire directories and Swap using the following configuration:

wipe -Z[<d/D>][<n/N>]tv[<a/A>]kf<r>O -S512 -C4096 -l1 -x[<1><3>] -p1

- - -Legend-
(Some of these settings are default but I thought it's useful to show all settings used, defult and KleanUp's settings.)
Z = normal passes
d = delete file
D = do not delete file, only shred contents of file
n = delete special file
N = do not delete special file
t = enable static passes
v = very verbose
a = write until out of space
A = don't write until out of space
k = lock files
f = force, override interaction
r = recursive shredding
O = write to file
S512 = block device sector size is 512bytes
C4096 = chunk size is 4096kb
l1 = secure level 1 (I assume this means using /dev/urandom/ for pseudorandom data generation--not as secure as AES which Scrub uses)
x1 = 1 pseudoranodm pass _per_ one wipe pass (see below)
x3 = 3 pseudorandom passes _per_ one wipe pass (see next)
p1 = wipe file one time with known <n> pattern ([x]<3> combined with [p]<1> is a total of 4 passes per file, known pattern pass is last.)


*After a file (or it's contents) is shredded it's directory entry is unlinked and shredded, the file is renamed multiple times then removed by Wipe and finally KleanUp runs 'sync'. Files which are not to be removed have their contents shredded but the directory link and file names are not shredded.

*Wipe utilizes a write barrier between each pass.

*I have compiled Wipe from the cvs snapshot release "Wipe-2.3.0-cvs" as "wipe" in /KleanUp-0.3.01b/src/Wipe_v2.3.0-cvs/. There are _MANY_ improvements over version-0.20.2 which is the currently available .deb package.



>Scrub v1.8<
Scrub uses a shredding algorithm which sufficiently meets the NNSA Policy Letter NAP-14.x (see reference below) for sanitizing removable
and non-removable hard disks. This requires overwriting all locations with a pseudorandom pattern twice (AES pseudorandom) and then with a known pattern.

*After a file's (_non_wildcard and _non_recursive shredding) contents is shredded it's directory entry is unlinked and shredded. The file is then renamed multiple times with the final name of "jd92n". Then the file is shredded and removed by Wipe (see above) and finally KleanUp runs 'sync'. Files which are not to be removed (eg. .bash_history) have their contents shredded but the directory and file name are not shredded and Wipe is not called.

*Scrub utilizes a write barrier between each pass.

*Free Space is filled until an error occurs using a self-expanding file ("scrub") containing pseudorandom data (AES pseudorandom) then the file is shredded by Scrub and finally removed by Wipe. Note: When shredding the Free space file "scrub" Wipe uses _one_ pseudorandom pass and _one_ known pass.

*The NNSA algorithm is reasonable and recommended for sanitizing modern PRML/EPRML encoded disk devices.

Please refer to the NNSA document for additional constraints:
NNSA Policy Letter: NAP-14.x, "Clearing, Sanitizing, and Destroying Information System Storage Media, Memory Devices, and other Related Hardware", Unpublished Draft, 2005


>dd<
KleanUp executes three 'dd' passes of pseudorandom data from /dev/urandom/ and a final 'dd' pass of zero from /dev/zero/ with 'sync' between passes on the Swap partition.

Future releases of KleanUp will use Wipe or Scrub to shred the Swap...or you could shred it and turn it off for good (recommended).


- - ---> KleanUp Operations:

Only files present on a defult install of Kubuntu will be shredded (ex. Firefox files are not shredded). KleanUp assumes you do _not_ run programs such as 'Kaffenine' or surf the web with Konqueror as Superuser:


>Misc 1<
+Calculate and print size of mounted OS and file systems before and after running KleanUp
+Print the FS and configuration used by the mounted OS


>/home/<
+Clear the bash history in ~/...
+Clear the .xsession-errors file in ~/...
+Shred HTTP/S files in ~/cache-*-desktop/...
+Shred Kaffenine's list of viewed videos (playlists) in ~/...
+Shred 'kconf_update.log' in ~/...
+Shred the Konqueror cookie file in ~/...
+Shred the file 'formcompletions' in ~/...
+Shred files/folders in 'Recent Documents' in ~/...
+Shred file 'konq_history' in ~/...
+Shred file 'secret-cookie' in ~/...
+Shred Konqueroer crash log file in ~/...
+Shred the Trash Can in ~/...
+Shred thumbnails in ~/...
+Shred the directory '~/Examples/'...
+Shred Konqueror URL/location bar and other web traces


>Packages/modules< (KleanUp offers to download and install with [y/n] prompt)
+Download and install Alien
+Download and install Scrub (.rpm)
+Download and install Wipe
+Download and install Deborphan
+Download and install Kleansweep
+Download and install Localepurge
+Load the KDE Kontrol Privacy module
++Clear clipboard contents (using KDE Kontrol Privacy Module, _not_ Scrub)
++Clear run command history (using KDE Kontrol Privacy Module, _not_ Scrub)
+Remove packages that _were_ dependencies
+Remove downloaded and partially downloaded package files (non-locked)...
+Remove old package files that can on longer be downloaded (locked)...


>/root/<
+Clear bash history in /root/...
+Shred HTTP/S files in /root/...
+Shred KDE update log file in /root/...
+Shred files/directories in 'Recent Documents' in /root/...
+Shred Konqueror crash log file in /root/...
+Shred Trash Can in /root/...
+Shred all thumbnails in /root/...
+Shred Konqueror crash log in /tmp/...
+Shred 'secret-cookie' in /tmp/...
+Shred 'example-content' directory in /usr/...
+Shred all logs in /var/log/...
+Shred HTTP/S files in /var/...


>Partitions, File Systems, etc<
+Shred Swap partition with three passes of pseudorandom data from /dev/urandom/ and a final pass of known zero's from /dev/zero/.
++Between passes KleanUp executes the "sync" command
+Shred Free Space with Scrub (see above)


>Misc 2<
+Restart computer



--Future versions of KleanUp (To-Do):
--Shred File Slack Space (inodes)
--Shred the Swap partition with Wipe (if possible)
--Use Scrub/Wipe to shred clipboard and run command history
--Add capital and lower case [y/n] support, now KleanUp only supports _lower_ case.
--Initiate shutdown then stall (eg. -h), possibly with "Kshutdown". As so:
----Reboot > stall > run commands (shred Swap, Free, RAM) > When KleanUp finishes running the computer is automatically restarted.

================================================== ===================

[LeslieL]

Re: KleanUp: Automatic and komplete kleaning of Kubuntu!

It seems to be working for me. Thanks.

[lightzoo]

Re: KleanUp: Automatic and komplete kleaning of Kubuntu!

Hi,

I'm glad it's working for you. Do you have any suggestions, improvements or feature requests?

What version of Kubuntu are you using?


I have a new version almost ready to release that includes...


New Features/Changes: (v.0.2-b):
* The package "Wipe" is now installed and used in place of 'rm', I am using 7 passes with file name overwrite
* Check the File System in use and what mode it is running (some journaling FS are bad for data removal)
* Auto find/shred Swap (w/o reboot) using 7 pass (not w/ Wipe): Random/Zero/Random/Zero/Random/Random/Random
* Shred Free Space using Wipe
* Shred "Recent Documents" directory
* Clear "form completions"
* Clear all .log files in /var/log
* Clear Konqueror logs
* Clear file "faviconrc" which contains URLs visited
* Clear CookieJar file "cookies"
* Clear file "secret-cookie" which contains an MD5 Hash for Mcop authentication (user is prompted to choose Y/N)
* Clear Bookmarks (user is prompted to choose Y/N)
* Added many more files and directories to shred in /root/, /var/, /home/, etc, etc (re: thumbnails, bash history, etc)
* All code dealing with non-/home/, eg. '/..." directories/files are now in "KU_root.sh" to ensure proper removal of data.


Future versions:
* THC "Secure_Delete" instead of "Wipe" (Secure_Delete offers better Free space, Swap space, Slack space)
* Shred File Slack space
* Clear clipboard w/secure algorithm (currently KleanUP uses the Privacy module to do this)
* Clear RunCommand History w/secure algorithm (currently KleanUp uses the Privacy module to do this)
* Compute size of drive before AND after shredding files and show amount of space recovered

[color=green]
KleanUP version 0.2-b should be ready by Thursday/Friday...Cheers!

[Fragadelic]

Re: KleanUp: Automatic and komplete kleaning of Kubuntu!

I'm not so sure you need to use wipe.

When you create a livecd/dvd you only copy the files with contents in them and not the free or deleted space. The swap is also not copied and folks should be using a swap partition and not a swap file. This isn't windows and a swap file is very inefficient compared to a swap partition.

Neat little tool.

[lightzoo]

Re: KleanUp: Automatic and komplete kleaning of Kubuntu!

Hey Fragadelic,

The first version of KleanUp I made was simple and only used rm and was meant to be used when building a LiveCD. But then I thought it would be nice to make KleanUp into a tool I would use regularly on a real system, hence the Swap space, Free space, Slack space shredding...I know those don't get copied onto a CD but I don't like the thought of my bank account and email account passwords sitting in the clear in my File Slack or Swap space.

My script assumes the Swap is on a partition (as per Linux standard), it automatically finds the correct Swap partition and shred the partition...If I typed "Swap File" above it was a typo which I'll fix. The actual code to automatically find the swap wasn't written by me, I got it form here: (WipeSwap.sh) http://theregister.co.uk/media/945.gz ...see this article: http://www.theregister.co.uk/2002/11...r_linux_power/

So, I started adding things to the script to make it more robust and it grew bigger and bigger. I also wanted to use a secure shredding algo because I figured 'why not'?

In short, KleanUp started as tool to reduce the size of my install so I could use your scripts. But then it turned into a much more robust and secure tool which still reduces the size of an installed system but also offers much more. I couldn't find a tool on Linux/Kubuntu which would automatically and securely shred directories, securely clear files (eg. 'konq_history"), shred data, Swap, Free space, Slack, etc, etc. There a quite a few shredding tools for Linux (eg. Shred, Wipe, THC secure_delete, DiskScrub, etc, etc) but none automatically shred data which can compromise your privacy...so I wrote KleanUp which in it's current version uses 'secure_delete' and Wipe.

BTW, can you send/post me the URL to the forum for Remastersys? The email you sent with the link was accidently deleted, thanks!

Cheers,

[lightzoo]

Re: KleanUp: Secure, automatic and komplete kleaning of Kubuntu!

Update:

"KleanUp v0.3b"(March, 5th, 2007)


Please see the thread starting post for more full details and links...


Major Updates:
-Now full use of secure shredding algorithms with Scrub v1.8.1 and Wipe v2.3.0-cvs
--Now using Wipe v2.3.0-cvs with a lot of new control options enabled to enhance security
-Option to shred Swap partition, Free space and RAM disk
-Major code re-write, cleaned up a lot of lines and hopefully better organization


Minor Updates:
-Many new files and locations to shred
-Re-wrote and improved the explanations of operations and Read_Me file
-Too many to other updates mention here...read the "Read_Me" below for more info


News:
-KleanUp now has it's own homepage at http://www.geocities.com/lightzook/KleanUp.html , so check it out!
-I put a good amount of time into the FAQ, if you read it and have an opinion or suggestion let me know.
-The KleanUp homepage offers:
---A nice, indepth and informative FAQ regarding KleanUp and data sanitization in general.
---Download links for KleanUp, it's GnuPG signatures, my GnuPG key, white papers, applications and Read_Me file


Cheers!

[lightzoo]

Re: KleanUp: Secure, automatic and komplete kleaning of Kubuntu


I have noticed issues with the RAM shred option. For now don't use the RAM shred, I will update soon.

This is not too big of an issue because the RAM shred option is kind of redundant (RAM is automatically cleared upon shutdown). The reason I say "kind of" is because it has been shown that data can be retrieved from a RAM drive after a reboot. I am not sure about the amount of, or the quality of data that can be retrieved from RAM drive.

Cheers,

[lightzoo]

Re: KleanUp: Secure, automatic and komplete kleaning of Kubuntu

Update:

"KleanUp v0.3.01b" (March, 10th, 2007)


Please see the thread starting post for more full details and links...



Major Updates:

-Wipe is now installed by KleanUp into /usr/local/bin using the source pre-compiled by me.
--Wipe is now called by "wipe [options] [file]"

-RAM disk shred option has been commented out and should not be used for the time being
--I am hoping to fix the RAM shred soon

-Size of mounted OS free space _before_ and _after_ KleanUp runs is now printed to the file "Before_After.txt"
--The file is located at ~/KleanUp/results/Before_After.txt
--Read the file for more information
--Useful to find amount space recovered by KleanUp

-The mounted OS File System type and it's configuration is now printed to the file "File_System.txt"
--The file is located at ~/KleanUp/results/File_System.txt
--Read the file for more information
--Useful for evaluating the level of secure data removal in regards to the file system


Minor Updates:

-When shredding the Free space file "scrub" Wipe uses _one_ known pass (zero).
--I have reduced the number of passes because they are redundant (to Scrub) and the extra passes take a long time

-Cleaned up the code a good bit and improved explanations (hopefully)

-Fixed an issue where the /root/.kde/cache-*-desktop/ not being shredded

-Updated the "Read_Me.txt" to reflect relevent info in this file and improved explanations (hopefully)

-Created this Change_Log.txt


News:

-KleanUp has a new home on Sourceforge at https://sourceforge.net/projects/kleanup !
--I haven't done much yet, I'll post when the project pages are ready...for now Geocities will be home