Announcement

**Qqmike** · Jan 26, 2008, 04:19 PM

Re: dd Command

PART 2 dd Applications: Examples

Table of Contents

Example 1 Zero-out the hard drive or flash drive
Example 2 Zero-out the free space on a partition
Example 3 Clean up your home
Example 4 Clean a Partition Before Reusing It or Deleting It
Example 5 Clean the Master Boot Record, 1 sector; or Clean 63 sectors
Example 6 Wipe a file before deleting it (to prevent recovery)
Example 7 --blank for now--
Example 8 Cloning a [bootable] flash drive or a hard drive
Example 9 Master boot record (MBR): viewing and backup/restore
Example 10 Print a partition boot sector
Example 11 --blank for now--
Example 12 Creating files
Example 13 Check a file for errors
Example 14 Print bytes of a file to the screen
Example 15 --blank for now--
Example 16 Find the size of a drive or flash drive
Example 17 Look for non-zero entries on a drive (HDD or flash drive)
Example 18 Search for a string in your /home partition
Example 19 Check a random part of a drive to see what's there
Example 20 --blank for now--
Example 21 Write numbers to the end of a disk
Example 22 md5sums and dd
Example 23 Using the Bash command sync after a dd copy
Example 24 What's at the end of YOUR hard drive? -> See Reply #3 of this thread
Example 25 Backup your Kubuntu OS to your separate /home partition
Example 26 --blank for now--
Example 27 Bad disk sectors

Tips
> For each application, open Konsole and change to the directory in which you want to work (use cd). See Part 1, Section 1: "dd works relative to the your current directory" For example, to change to your Desktop: cd Desktop. Or, use complete paths; to change to a directory you set up in your home called TestFolder: cd /home/your_name/TestFolder.
> Midnight Commander (mc): see Part 2 for using it.
> Manual pages. To learn about command xyz, open Konsole and issue the command: man xyz.
> For drives, flash drives, we use /dev/sdx. You should run sudo fdisk -lu to see your drives and identify the correct one to use in your dd commands.
> Be careful! Don't zero-out or overwrite the wrong drive!

Example 1 Zero-out the hard drive or flash drive
-> Clears it to like-new state
From Live CD or from Kubuntu/Linux in another partition:

sudo dd if=/dev/zero of=/dev/sdx conv=notrunc

Details
Use the command fdisk -lu to check the drives, identify the drive to zero-out. Suppose it is sdx.
To write zeros to the sdx drive:
sudo dd if=/dev/zero of=/dev/sdx conv=notrunc
It took about 45 minutes to complete this on a 160 GB SATA drive. You will not get a progress meter from dd. If you use a variation of dd called dcfldd, you will get a progress meter. Install dcfldd using your package manager in the Live CD session or simply by: sudo apt-get install dcfldd. See Part 1, Section 1 "Basic principles -- dcfldd."
-> Afterwards, use GParted Live CD to re-partition the drive, beginning with setting a new Disk Label (select the standard default MSDOS Label when prompted, unless you are using GPT for 2 TB drives).

Advanced note: I can see no reason why you need conv=notrunc, but many authors use it. I think there's a lot of misinformation out there. Doesn't hurt to leave it in, but I don't use it. The statement is the same as sudo dd if=/dev/zero of=/dev/sdx bs=512 conv=notrunc, since bs=512 is the default. When using bs (instead of separate input and output block sizes), the dd command does what you expect it to do in this example: it will copy zeros all the way throughout the drive sdx; it will not truncate /dev/sdx; and it will fill up /dev/sdx fully (possibly using a partial block for the last copy).

= = = = = = = = = =
Example 2 Zero-out the free space on a partition
(Also: Clean up the root partition of your Kubuntu OS)

From live CD:
sudo dd if=/dev/zero of=/media/sdxn/bigjunkfile
sudo rm /media/sdxn/bigjunkfile

Details
We overwrite all the free (available or unused) space on a partition; this prevents recovery of past-deleted files. It does not disturb any existing files you are still using. Suppose you are working in a Kubuntu OS on your hard drive or from your Kubuntu Live CD. Assume the target drive is seen as sdx; assume the partition you want to clean up is sdxn; and that sdxn is mounted as /media/sdxn.
(Check the disks: sudo fdisk -lu; Check the partition: cd /media/disk and ls -al to see all files and their sizes in partition sdxn.)
Create a file named bigjunkfile in sdxn and write zeros to it until it fills up all the unused space in sdxn:

sudo dd if=/dev/zero of=/media/sdxn/bigjunkfile
(The output will say "dd: writing to `/media/disk/bigjunkfile': No space left on device" plus details.)
Now remove the file:
sudo rm /media/sdxn/bigjunkfile

Clean up the root partition of your Kubuntu OS
-- Do it as explained above from a live CD.
Or,
-- Do it while booted into your Kubuntu OS:
sudo dd if=/dev/zero of=bigjunkfile
sudo rm bigjunkfile

= = = = = = = = = =
Example 3 Clean up your home

Similar to Example 2.
-- If your home is on the same partition as the root partition of Kubuntu, use Example 2 ("Clean up the root partition of your Kubuntu OS").
-- If your home is on a separate partition sdxn, then exactly as in Example 2, either from a live CD or while booted into your Kubuntu OS (you don't need sudo since it is your home):

dd if=/dev/zero of=/media/sdxn/bigjunkfile
rm /media/sdxn/bigjunkfile

= = = = = = = = = =
Example 4 Clean a Partition Before Reusing It or Deleting It

From another partition or from a live CD, to zero-out partition sdxn:
sudo dd if=/dev/zero of=/dev/sdxn conv=notrunc,noerror

Advanced note: I can see no reason why you need conv=notrunc, but many authors use it. I think there's a lot of misinformation out there. Doesn't hurt to leave it in, but I don't use it.

= = = = = = = = = =
Example 5 Clean the Master Boot Record, 1 sector; or Clean 63 sectors

Zero-fill the MBR of drive /dev/sdx:
sudo dd if=/dev/zero of=/dev/sdx bs=512 count=1

Zero-fill the first 63 sectors of drive /dev/sdx
sudo dd if=/dev/zero of=/dev/sdx bs=512 count=63

(The first partition usually begins in LBA sector 63; use sudo fdisk -lu to confirm this. Since LBA counts sectors starting from zero, the first partition starts in the 64th sector. dd counts fom 1. )

= = = = = = = = = =
Example 6 Wipe a file before deleting it (to prevent recovery)

Let's get rid of the file testfile. Change into the directory containing testfile. (E.g., if it is on your Desktop, cd Desktop)

ls -al
Output:
-rwxr-xr-x 1 mike mike 23996 2008-01-17 14:37 testfile

sudo dd if=/dev/zero of=testfile bs= 23996 count=1 conv=notrunc
sudo rm testfile

(We see its size: 23,996 bytes, write zeros to it, then remove it.)

Advanced note: This will not remove all file metadata which may include the name of the file. Normally, that shouldn't be a problem. If it is for you, get in the habit of carefully selecting file names of sensitive files. After removing it, you can search for any remains of the file on partition sdxn:

sudo dd if=/dev/sdxn bs=4096 | hexdump -C | grep 'testfile'

= = = = = = = = = =
Example 7 --blank for now--

= = = = = = = = = =
Example 8 Cloning a [bootable] flash drive or a hard drive

dd if=/dev/sdx of=/dev/sdy bs=4096 conv=notrunc,noerror

sdx and sdy are physically identical (same size) drives. The result is that sdy will be identical in every way to sdx. If sdx is bootable with a MBR (and GRUB), a file system, data and programs, so will sdy.

= = = = = = = = = =
Example 9 Master boot record (MBR): viewing and backup/restore

--- To see the MBR of drive sdx:
sudo dd if=/dev/sdx count=1 | hexdump -C

This prints the MBR to the screen. Count=1 means to copy only one block size (in this case bs=512=the default, and 1 sector =512 bytes). Note the end of file signature 55 aa.
-> For more on this example, see Part 1, Section 2 on the MBR.

--- Make a backup of your MBR
sudo dd if=/dev/sdx count=1 of=MBR_copy

(That puts the copy in your home; use: of=/path/to/MBR_copy to place the copy where you want it.)

--- To restore your MBR
sudo dd if=MBR_copy of=/dev/sdx count=1

(Or sudo dd if=/path/to/MBR_copy of=/dev/sdx count=1)

= = = = = = = = = =
Example 10 Print a partition boot sector

To see the boot sector of partition sdxn, n>1 :

sudo dd if=/dev/sdxn count=1 | hexdump -C

Prints it to the screen. Count=1 means to copy only one block size (in this case bs=512= 1 sector= the default). The last row of output shows an offset of 1f0 in hexadecimal (or 0x1f0 or 1f0h) which is 496 bytes in decimal (or 496d). Then there are 16 entries across the final row, which would take us to 512 bytes (1 sector) from the start of sdxn. Note the end of file signature 55 aa.
To print it to a file located at /path/to/boot_sector_sdx_copy:
sudo dd if=/dev/sdxn count=1 of=/path/to/boot_sector_sdx_copy

The hard way when n=1 (but it demonstrates some things):
The above method works for the first partition, but here's another (harder) way. To see the boot sector of the first partition on the disk, you have to skip over the MBR (the first sector) and other bytes until you get to the first sector of the first partition. In Logical Block Addressing (LBA), sectors start from sector zero on a disk. If you run sudo fdisk -lu, you'll (probably) see the first partition of your drive starts at sector 63. Since LBA counting starts at zero (0, 1, ..., 63), sector 63 is actually the 64th sector, and to see it, you have to skip over 63 sectors; thus, to print the boot sector of sdx1:
dd if=/dev/sdx count=1 skip=63 | hexdump -C

= = = = = = = = = =
Example 11 --blank for now--

= = = = = = = = = =
Example 12 Creating files

Create a 1 megabyte file (1,000,000 bytes) containing only zeros
dd if=/dev/zero of=megfile bs=1MB count=1
(If you want 1024x1024, use bs=1024x1024 or bs=1M.)

Create a file of size 250,000 bytes containing random numbers
dd if=/dev/urandom of=bigranfile bs=250000 count=1

Create a file named file1 of size 1000 bytes that contains nothing (but metadata)
dd if=/dev/zero of=file1 bs=1000 count=0 seek=1
You will see that it contains 1000 bytes; however, if you view it in Midnight Commander (mc), you'll see there's nothing in it.

= = = = = = = = = =
Example 13 Check a file for errors

Check the partition /dev/sdxn for errors:
dd if=/dev/sdxn of=/dev/null

You can have dd copy through a file to see if it encounters any errors. The file can be any file, including device files like drives /dev/sdx and partitions /dev/sdxn. dd will copy its "reads" to the null device (see Part 1, Section 2, "Useful Linux device files"). Of course, do not use noerror since you want dd to inform you if it encounters an error. We used the default block size bs=512, but you can use any value.

Use skip and count to narrow down your probe.
Example: dd if=/dev/sdc10 of=/dev/null bs=100 skip=1M count=5000
will skip 1,000,000 bytes on partition sdc10, then copy 100 bytes at a time until it copies 500,000 bytes (5000x100) and dumps them in the black hole /dev/null. If it encounters an error, it will stop and report it.

= = = = = = = = = =
Example 14 Print bytes of a file to the screen

You can't read a binary file unless you use a hexdump or Midnight Commander.
--- To see the binary file binaryfile, change directory (cd) into the directory containing binaryfile, then
dd if=binaryfile | hexdump -C
or pipe it to hexdump and maybe to less:
dd if=binaryfile | hexdump -C
dd if=binaryfile | hexdump -C | less
See man less.
Of course, instead of using cd, you can use the full path name from any directory:
dd if=/path/to/binaryfile | hexdump -C

--- Print the first byte of a file to the screen:
dd if=myfile bs=1 count=1
(A byte is eight bits and represents a character.)
--- Print more bytes using count; e.g., print 5 bytes: dd if=myfile bs=1 count=5
--- Skip some bytes using skip.
Print the third, fourth and fifth bytes: dd if=myfile bs=1 skip=2 count=3

= = = = = = = = = =
Example 15 --blank for now--

= = = = = = = = = =
Example 16 Find the size of a drive or flash drive
You can use null to find the size (in bytes) of a drive or flash drive:
sudo dd if=/dev/sdc of=/dev/null bs=4096
123007+1 records in
123007+1 records out
503840256 bytes (504 MB) copied, 41.9472 s, 12.0 MB/s
Thus, the drive has 503,840,256 bytes.

= = = = = = = = = =
Example 17 Look for non-zero entries on a drive (HDD or flash drive)

dd if=/dev/sdx | hexdump -C | grep [^00]

You might run this after you zero-out your drive, to verify all zeros on it.

Check for non-zero entries on a disk beyond 1,031,760,000 bytes:
dd if=/dev/sdx skip=10317600 bs=100 | hexdump -C | grep [^00]

See man grep: The brackets means "find anything inside them." The ^ means "not." Thus "find anything that is not 00. The 00 is two zeros, or one byte in hexadecimal, and one byte is enough to represent one character (a letter, number of symbol). Thus a zero would show up in a hex printout as 00 (using 4 bits for each zero: 1 byte = 8 bits = 0000 0000 binary = 00 hexadecimal).

= = = = = = = = = =
Example 18 Search for a string in your /home partition

I have just one instance of the string "urmyvalentine" in my home. It is the name of an empty folder on my Desktop. Let's see how many references come up on it.

sudo dd if=/dev/sdb7 bs=4096 | hexdump -C | grep 'urmyvalentine'
Output:
381e8fab0 20 27 75 72 6d 79 76 61 6c 65 6e 74 69 6e 65 27 | 'urmyvalentine'|
45437e040 3d 75 72 6d 79 76 61 6c 65 6e 74 69 6e 65 0a 32 |=urmyvalentine.2|
45691b040 3d 75 72 6d 79 76 61 6c 65 6e 74 69 6e 65 0a 0a |=urmyvalentine..|
5120710+1 records in
5120710+1 records out
20974431744 bytes (21 GB) copied, 4770.11 s, 4.4 MB/s
(It took 79.5 minutes to search the 21 GB partition.)

This suggests that if I'm going to protect my privacy and do some cleanups of my personal files, it won't be enough to simply "delete" that folder. It also suggests that perhaps you should be careful choosing names for your files and folders for sensitive data.
You can also play with the offsets using a hex calculator (Qalculate); for example, 381e8fab0 hex = 1.5064431E10 decimal = 15,064,431,000.

= = = = = = = = = =
Example 19 Check a random part of a drive to see what's there

dd if=/dev/sdx bs=1 skip=100000 count=100 | hexdump -C

This skips the first 100,000 bytes of drive sdx, then prints the next 100 bytes to the screen (using hexdump -C so you can see it). You might also print the result to a file (say, checkfile) and then view it using Midnight Commander:
dd if=/dev/sdx of=checkfile bs=1 skip=100000 count=100

See also Example 16 Find the size of a drive or flash drive.

= = = = = = = = = =
Example 20 --blank for now--

= = = = = = = = = =
Example 21 Write numbers to the end of a disk

In this example, we write random numbers to the disk starting at 1,031,760,000 bytes and beyond to the end of the disk:
dd if=/dev/urandom of=/dev/sdx seek=10317600 bs=100
(We used a small block size bs=100 bytes knowing we were near the end of the disk (a "1 GB" flash drive that "ends" at 1,031,798,784 bytes).

See also Example 15 Find the size of a drive or flash drive.

= = = = = = = = = =
Example 22 md5sums and dd

Calculate the md5sum of drive sdx:
md5sum /dev/sdx > /path/to/mywork/sdx_md5sum.txt

/path/to/mywork is the directory where you are keeping results as you work; it may be your home or a mounted directory on another partition or disk, such as /media/sdy/mywork/sdx_md5sum.txt; sdx_md5sum.txt is a file created by the above command that contains the md5sum of drive sdx.

Write sdx to a file using dd:
dd if=/dev/sdx of=/path/to/mywork/sdx_image

Calculate the md5sum of this image:
md5sum /path/to/mywork/sdx_image > /path/to/mywork/sdx_image_md5sum.txt

This command redirects (>) the output of md5sum to the file sdx_image_md5sum.txt (which is created by this command). Now you can compare the two md5sums: sdx_md5sum.txt and sdx_image_md5sum.txt to see that they match (which implies the dd copy was correct).

= = = = = = = = = =
Example 23 Using the Bash command sync after a dd copy

(This is not the same as the dd sync option in conv=sync.)
Man sync:
sync - flush file system buffers
Force changed blocks to disk, update the super block.
You'll see some writers use sync after using dd. The idea is that invoking sync will ensure that the effect of dd is written to disk promptly, thus reducing the risk that the result of dd will be lost if the system should fail or be interrupted. Some writers use it for security.
For example, suppose testfile is 26478 bytes and you wish to write zeros to it (zero-out the file):
dd if=/dev/zero of=testfile bs=26478 count=1 conv=notrunc
sync
[optional: then remove testfile: rm testfile]
(We use notrunc because we don't want dd to truncate testfile before doing the copy; truncating testfile would mean that we lose its "place" or its bytes--we don't want those bytes returned to the file system before we write the zeros.)

= = = = = = = = = =
Example 24 What's at the end of YOUR hard drive?

A full example of this is in Reply #3 below.

= = = = = = = = = =
Example 25 Backup your Kubuntu OS to your separate /home partition

Example Suppose your home is on sda4, and your Kubuntu is on sda5.
If you want to make a backup of your root partition in your home partition, allow room for that when planning the size of your home partition.

Backup
To make a file containing the image of sda5 in your home, at Konsole, issue the following command while you are in Kubuntu on sda5:
sudo dd if=/dev/sda5 of=/home/your_name/sda5image bs=4096 conv=noerror,notrunc

To make this backup from a live CD:
sudo dd if=/dev/sda5 of=/media/sda4/your_name/sda5image bs=4096 conv=noerror,notrunc
where we are assuming your home partition is mounted as /media/sda4.

Restore
To restore your backup image of sda5 to partition sda5, issue the following command from a live Kubuntu CD:
sudo dd if=/media/sda4/your_name/sda5image of=/dev/sda5 bs=4096 conv=noerror,notrunc

Troubleshooting

-- “Permission denied” Use sudo dd.
-- “/dev/sdxn is not a directory” Use the mount point corresponding to the device /dev/sdxn (which may be something like /media/sdxn).
Errors upon rebooting
-- “Unable to resolve UUID=” “Press Control+D to end this shell etc...”
Ignore it; press Control+D to continue (or Control+D, then Enter). You got this error if you made a new partition for the root system backup but forgot to include the new partition's UUID in the /etc/fstab file of your Kubuntu OS. Try to remember to fix it.
-- Upon re-booting: “...filesystem error(s)...”
Hopefully, the system can fix those upon re-booting--it will tell you so on the screen. Otherwise, you may have to manually run the command given to you on the screen(e.g., fsck /dev/sda5). These usually resolve themselves quickly. If not, re-boot with a live Kubuntu CD, and re-try to restore the OS root partition (using the commands given to you above or try using dcfldd instead of dd).
-- Example of actual error: Dropped to a terminal prompt with messages: “sda5 contains a filesystem with errors; Inode 526117 has illegal blocks; unexpected inconsistency; run fsck manually w/o -a or -p options.” Action: To repair the filesystem, you might then type sudo fsck /dev/sda5. Then reboot with sudo shutdown -r now.

Advice (based on dozens of experiments)
-- dcfldd is best (safest, fastest). The fastest, safest way to do work like this is using the special dd command version dcfldd. See Part 1, Section 1. dcfldd is faster (about twice as fast for me). dcfldd is safe: For me, no errors; perfect reboot.
-- We did this work while booted into the OS we were making a backup of, and that's OK. Next best is to simply use your live Kubuntu CD and the dd or dcfldd command. Safe: I never got an error using the live CD method; perfect re-boot.
-- You can also do this work from another operating system partition on the same hard drive as the partition you are trying to backup & restore. I've had excellent results doing it this way(errors encountered just once, and then easily fixed automatically upon re-boot.)
-- Lastly, I got the most errors and the most serious errors when doing this work from a hard drive Kubuntu OS partition that was NOT on the same hard drive as the Kubuntu OS I was making the backup of. Is there a reason for this? I don't know; but the pattern, for me, on my system was unmistakable. Errors were easily fixed upon re-boot automatically or using fsck.

References
-- Fixing filesystems:
http://members.iinet.net.au/%7Eherman546/p10.html
-- GParted Live CD capabilities: It also has capability to copy and restore a partition
GParted: http://gparted.sourceforge.net/ Make a partition the same size and filesystem as the one you are copying; copy Kubuntu to that partition; restore using GParted again (copy/paste) or using dd.
GParted Live CD also has a Terminal (double-click the icon at the top of the Gparted screen) you can use as you would use any terminal, running commands like fdisk -lu and dd; you can also install dcfldd to your Live GParted session (sudo apt-get install dcfldd).
-- Partimage: A good program for making image backups, like what we are doing here
http://www.debianadmin.com/backup-an...partimage.html
http://www.partimage.org/Main_Page

= = = = = = = = = =
Example 26 --blank for now--

= = = = = = = = = =
Example 27 Bad disk sectors
There are six ways a disk can develop bad sectors. Two of them may be fixable (and maybe not). If the ECC fails by detecting an error but not fixing it, you'll get a sector read error (or bad sector error, or 0x40 error). By the time you start seeing bad sectors appear consistently, it it probably too late to hope to save the drive; the hard disk should have been taking care of bad sectors all along, so apparently something is amiss now. Your hard disk deals with sectors, writing whole sectors at a time, even if only one byte is changed. Disks map bad sectors to good ones, setting the bad sector aside. The file system (e.g., ext3) doesn't work with sectors but with groups of sectors called blocks or clusters. A file system can mark bad sectors so they aren't used; or remap bad blocks, say by using fsck with the badblock option.

Where does dd fit in? iflag=direct and oflag=direct

You can use dd to write zeros to a hard drive (or flash drive), and that will wipe it clean and "freshen" it up: dd if=/dev/zero of=/dev/sdx. If dd hits a bad sector, it may be able to write zeros to it and things will be OK; it may cause the disk to reallocate the bad sector, which is also a good thing. Then again, dd may not be able to fix the problem. dd must be able to read and write to the bad sector(s) (sometimes to a block of 4096 bytes), but that will not be possible if there is unrecoverable physical damage to the disk, in which case this issue is then rendered moot: you simply write zeros to as much of the disk as you can, then to prevent recovery of any remaining personal data, physically destroy the disk (burning by blow torch, physically shredding, pulverizing).

Your kernel reports hard disk errors in /var/log/messages or at Konsole by running dmesg. You might then identify a bad sector and test it using dd and try to write zeros to it using dd. For one thing, certainly, unmount the file system (from which the bad sector is being reported) because you don't want anything written to the bad area and cause loss of your data. Unfortunately for dd, your filesystem "buffers reads," so if you want to read a sector, the file system will process the request in a block of sectors; if a bad sector is in the block, that will cause dd to fail. New versions of dd have two input/output options, iflag=direct and oflag=direct, that will give an unbuffered read. See man dd or dd --help.

To give you the flavor of how dd may be useful here, suppose you learn from the kernel that LBA sector 12345 is bad. Since LBA counts sectors from zero, that means dd has to skip 12345 sectors to hit LBA 12345 (dd counts from one); then the next sector is the bad one, and this will tell you if that sector is bad (by giving you an Input/Output error, or I/O error):
dd if=dev/sdx of=/dev/null count=1 bs=512 skip 12345
(The default block size is used = 512; the data read is sent to the null black hole device so you don't have to see all that stuff on the screen.)
To write zeros to that sector and maybe cause it be reallocated ...
CAUTION: you may now lose some data or get file corruption! ... Hope you have backups? ...
dd if=/dev/zero of=/dev/sdx count=1 bs=512 seek 12345
sync
(See man sync; see Example 21 above.)
If dd gives you an error, and the kernel log says it's a write error, you may be out of luck, unless you can identify the filesystem block containing the bad sector and overwrite the entire block (of, say, 4096 bytes or whatever your filesystem uses) ... Or, hit it directly with the iflag=direct oflag=direct flags in your dd statement (see dd --help). It may turn out that you have a bad file (or more) or a bad superblock. Run SMART to investigate further. It may also happen that you can not use standard Linux tools to do the work because the bad sector(s) may belong to an unassigned block and thus they are not in any file (so debugfs tells you that no file contains that sector). You can force the disk to allocate unassigned sectors to one big file by writing zeros to a very big file that fills up the unused space on the partition: dd if=/dev/zero of=/path/mountpoint/mybigfile bs=4096, and then proceed using SMART.

The format for using iflag=direct and oflag=direct.
To read:
dd if=/dev/sdx iflag=direct skip=X of=somefile bs=Y count=Z
and to write it back:
dd if=somefile of=/dev/sdx seek=X oflag=direct bs=Y count=Z

For exact details on how to do all this, see these two papers:

Bad Sector Blues:
The Dreaded "error=0x40 { UncorrectableError }"
http://www2.uic.edu/~aciani1/sector_blues.html

Bad block HOWTO for smartmontools
http://smartmontools.sourceforge.net/badblockhowto.html

= = = = = = = = = =

**Qqmike** · Mar 02, 2008, 05:23 PM

Re: dd Command

Part 3

dd -- Wipe partitions, home, files, disks, MBR

How to clean up (wipe) your partition, home, sensitive files, hard drive, flash drive, and MBR

TABLE of CONTENTS

Summary Sheet: Quick Reference Guide
The Summary Sheet is for experienced users. Each method is explained in detail in the text (under the section headings).

SECTION 1
Introduction and philosophy
When you delete a file, you do not delete the file: Now what?!
SECTION 2
How to Clean the Deleted/Unused Space on Your Kubuntu Partition or /Home
SECTION 3
How to Clean a File Before Deleting It
SECTION 4
How to Clean a Partition Before Reusing It or Deleting It
SECTION 5
How to Clean Your Hard Drive or Flash Drive Before Reusing It
SECTION 6
How to Clean the Master Boot Record, 1 sector; and 63 sectors

-> CAUTION <-
Before running dd commands, make a BACKUP of all important files.

= = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = = = = = = = = = = =

SUMMARY SHEET: Quick Reference Guide

The Summary Sheet is for experienced users. Each method is explained in detail in the text (under the section headings).

SECTION 2
How to Clean the Deleted/Unused Space on Your Kubuntu Partition or /Home

> Delete unwanted personal files. > Use Privacy Cleanup 101 to clean up your Kubuntu partition. Or use a cleaner, like Bleachbit.
> Empty the Trash. > Re-boot (to clear /tmp directories)
Boot into Kubuntu. Open Konsole. Change into your home directory:
cd ~ (or: cd /home/your_name)
dd if=/dev/zero > bigjunkfile
[Optional: check things with df -hT, ls -al, du -sh, du -ah, Konqueror/Dolphin.]
rm bigjunkfile
That cleans up your /home.
It also cleans up your root partition if your home is on the same partition as root (/).
If your Kubuntu root partition is separate from your /home partition, then:
Clean up the root /, as follows:
Boot into Kubuntu, at Konsole (note the sudo):
cd /
sudo dd if=/dev/zero > bigjunkfile
sudo rm bigjunkfile
That cleans up your root / .
Re-boot to clear out any /tmp directories and ensure that your OS works OK.

Working from Kubuntu Live CD
Start your Kubuntu Live CD. Open Konsole.
See your partitions: sudo fdisk -lu
Look for your Kubuntu root / and your /home partition(s).
Example: your /home is on /dev/sdxn
sudo mkdir /media/sdxn
sudo mount /dev/sdxn /media/sdxn
cd /media/sdxn/home/your_name
sudo dd if=/dev/zero of=bigjunkfile
[Optional: check things with df -hT, ls -al, du -sh, du -ah, Konqueror/Dolphin.]
sudo rm bigjunkfile
cd /
sudo umount /media/sdxn
That cleans up your /home.

Technical note:
Instead of changing directory into your home (using cd /media/sdxn/home/your_name), you could simply run dd as
sudo dd if=/dev/zero of=/media/sdxn/home/your_name/bigjunkfile
(That is, in the Live CD session, supply full, absolute path names to the dd command.)
And then to remove bigjunkfile, use: sudo rm /media/sdxn/home/your_name/bigjunkfile)

-> If /home is on the root partition, that also cleans up your Kubuntu root.
Or:
If Kubuntu root is on /dev/sdxm (where m is different from n), to clean up sdxm:
sudo mkdir /media/sdxm
sudo mount /dev/sdxm /media/sdxm
sudo dd if=/dev/zero of=/media/sdxm/bigjunkfile
[Optional: check things with df -hT, ls -al, du -sh, du -ah, Konqueror/Dolphin.]
sudo rm /media/sdxm/bigjunkfile
sudo umount /media/sdxm
That cleans up your root / partition.
You are done. Exit Konsole with exit command; exit the live CD session, re-boot (to clean up /temp directories).

Troubleshooting tip: if "Cannot unmount, device busy," then exit Konsole by typing exit, then open Konsole, and do again sudo umount /media/sdxm or sudo umount /media/sdxn .

SECTION 3
How to Clean a File Before Deleting It
To clean the file oldfile and then remove it:
Assume the path to the file is /path/to/oldfile.
Open Konsole.
cd /path/to
ls -al
From the output, get the size of oldfile in bytes, say it is B.
dd if=/dev/zero of=oldfile bs=B count=1 conv=notrunc
rm oldfile
Done.
(Note: Use notrunc to hold the location and bytes of the oldfile.)

To use pseudo random numbers (instead of zeros):
dd if=/dev/zero of=oldfile bs=B count=1 conv=notrunc
rm oldfile

SECTION 4
How to Clean a Partition Before Reusing It or Deleting It
You can do this from a (Kubuntu) live CD, from another Kubuntu partition, or from GParted Live CD. If you use GParted, after you zero-out the partition, you can immediately re-format it. In Kubuntu, you use Konsole and you must use sudo dd. In GParted Live CD, open a terminal by double-clicking Terminal icon at the top, that gives you a terminal with a root prompt, so you do not have to use sudo and can simply use dd.
Here it is using sudo:

Find out exactly how the partition is named:
sudo fdisk -lu
Suppose it is /dev/sdxn.
sudo dd if=/dev/zero of=/dev/sdxn conv=notrunc

The partition /dev/sdxn will now be zerod-out (and so recovering your data would be quite impossible). Note that the partition is now FULL--full of the zeros you wrote. If you use Gparted Live CD to reformat it, it will now be mostly empty (except for filesystem overhead) and ready to use again.

SECTION 5
How to Clean Your Hard Drive or Flash Drive Before Reusing It
By writing zeros to a hard drive or flash drive, you restore it to like-new state.
sudo fdisk -lu
Identify the drive. Example: Let's say it is /dev/sdx. CAUTION: Make sure about this!
sudo dd if=/dev/zero of=/dev/sdx
The drive is now clean but FULL--full of the zeros you just wrote to it!
You must re-partition the drive, starting with a new partition table (or MBR or Disk Label).

SECTION 6
How to Clean the Master Boot Record, 1 sector; and 63 sectors

Zero-fill the MBR of drive /dev/sdx:
sudo dd if=/dev/zero of=/dev/sdx bs=512 count=1

Zero-fill the first 63 sectors of drive /dev/sdx
sudo dd if=/dev/zero of=/dev/sdx bs=512 count=63

[End of Summary Sheet: Quick Reference Guide]

= = = = = = = = = = = = = = = = = = = =
= = = = = = = = = = = = = = = = = = = =

SECTION 1
Introduction and philosophy

It comes down to this:
> When you delete a file you do not actually destroy it. You only destroy the data structure (filesystem) reference to it. The bits and bytes of file information are still on your disk.
> When you delete a partition, you do not actually destroy it. You only destroy the data structure (MBR partition table) reference to it. The bits and bytes of information comprising the partition are still on your disk.

Proof? Easy as pie. Delete a bunch of your files (documents, photos, movies), and then before messing with that disk, use PhotoRec to find them and retrieve them for you. Or, use GParted Live CD to delete your Kubuntu or /home partition, then before messing with that disk, use TestDisk to find and restore the partition(s). Or, delete a file, then use dd and grep to search for key words in that file.
=> When you delete a file, you do not delete the file

->-> My Philosophy for this how-to is simple:
I go along with Starman's conclusions regarding the literature on this and feel that simply writing zeros (or random numbers) to a disk is enough to prevent recovery in almost all normal cases. ->-> Please note what I'm talking about: We are not talking about hiding illegal activities or data. If that's your gig, you're on your own there, pal (hint: state-of-the-art computer forensics is probably ahead of you). We are talking here about deleting data that you consider to be private, confidential, sensitive or personal in some (legal) way. Examples might be the following: financial records/data, health/medical notes, sensitive business information, personal correspondence to friends, your personal diaries, family photos, and personal interest items you've downloaded from the Internet. Wiping a disk or MBR is also used to eliminate malicious code.
->-> Your philosophy about data deletion may differ from mine. You may feel it is necessary to make multiple passes of zero writes or random-number writes to your drives to prevent data recovery. That's fine. This how-to applies: you simply apply the how-to multiple times

= = = = = = = = = = = = = = = = = = = =
SECTION 2
How to Clean the Deleted/Unused Space on Your Kubuntu Partition or /Home

-> CAUTION <-
When you fill up your Kubuntu root partition or your /home partition, strange things may happen when you try to re-boot. Thus, follow these recipes carefully and do NOT re-boot until deleting the file we have called bigjunkfile!!!

-> CAUTION <-
Before running dd commands, make a BACKUP of all important files.

First:
> Delete unwanted personal files
> Use Privacy Cleanup 101 to clean up your Kubuntu partition. Or use a cleaner, like Bleachbit.
> Empty the Trash
> Re-boot (to clear /tmp directories)

Next: Now you will have a bunch of empty space in your Kubuntu OS; however, some of that empty space contains files you deleted from your filesystem but are still on your disk in bits and bytes. They could be recovered by, say, PhotoRec. So you will fill all that empty deleted/unused space with zeros. That should clean things up for you and get rid of all the data and stuff laying around in your empty space and make it very difficult--probably quite impossible--to retrieve your deleted files..

Case 1: Suppose your /home is on same partition as the Kubuntu root partition

Open Konsole.
Change into your home directory:
cd ~ (or: cd /home/your_name)
dd if=/dev/zero > bigjunkfile

(Home Full Message: ignore it.
If you get a message saying that your home (or partition) is almost full, Do you want to open Konqueror/Dolphin to check it?, say no (Cancel), and let dd continue doing its thing until done.)

The dd statement creates the file bigjunkfile in your present working directory (/home/your_name) and fills it up as far as it will go with zeros. The file bigjunkfile will grow until it fills up all the empty space in your Kubuntu partition (since in Case 1, your /home is located on the same partition as the Kubuntu root OS). Your Kubuntu partition is now FULL. You can check it using
df -hT
and
du -sh
du -ah
and
ls -al (which will show you how big bigjunkfile is), or use Konqueror or Dolphin to check things (keeping Konsole open).
Beginner note: Instead of bigjunkfile, you can name that file anything you wish, like goodbyejunk or whatever. Doesn't matter what we call it as long as you can recognize what it is as you work.

CAUTION: Before you get into any trouble, better delete bigjunkfile.
If you are not still in your home directory, change into it (cd ~) and delete the file:

rm bigjunkfile

That's it, you're done: your empty partition space is pretty clean, filled with zeros, bit by bit.
Recommended:
-> Now re-boot to clear out any /tmp directories and ensure that your OS works OK.

Case 2: Suppose your /home is on a separate partition (from the Kubuntu root partition)
Case 2a: Clean up empty space only on your /home partition
Case 2b: Clean up empty space in the Kubuntu root partition

Case 2a: Same as above. That is,
Open Konsole. Change into your home directory:
cd ~ (or: cd /home/your_name)
dd if=/dev/zero > bigjunkfile
[Optional: check things with df -hT, ls -al, du -sh, du -ah, Konqueror/Dolphin.]
Remove bigjunkfile:
rm bigjunkfile

Case 2b: Clean up empty space in the Kubuntu root partition.
After doing Case 2a, if you wish also to clean up empty space in your Kubuntu root partition, do so as follows:
Open Konsole
Change into your root directory:
cd /
sudo dd if=/dev/zero > bigjunkfile
[Optional: check things with df -hT, ls -al, du -sh, du -ah, Konqueror/Dolphin.]
The file bigjunkfile will now reside directly under the root OS directory: /bigjunkfile. That's why we need to use root privileges with sudo.
Remove bigjunkfile:
sudo rm bigjunkfile

Recommended:
-> Now re-boot to clear out any /tmp directories and ensure that your OS works OK.

Working from a live CD
The topic continues, How to Clean the Deleted/Unused Space on Your Kubuntu Partition or /Home.
Working from a live CD is another way to do this. Some users might feel it is safer, working outside the OS, or they just prefer to do such work from live CD. This is done the same as above, except you must mount the partition(s) you wish to work with.

(Advanced note: Chrooting into a partition to work as root. See my how-to in References, Commands at Konsole, Part 3, for chrooting. It is NOT necessary to chroot to do this work.)

Case 1: Suppose your /home is on same partition as the Kubuntu root partition.

Start your Kubuntu Live CD. Select the option to try the OS without changing anything (that is, do NOT select Install). Open Konsole.
Example: Assume your root partition is named sda9. If you are not sure, run
sudo fdisk -lu
to see your partitions.
Make a mount point for your root partition-- a directory (a folder) on which to mount the partition you wish to work on.
sudo mkdir /media/sda9
Mount your root partition sda9 on the mount point
sudo mount /dev/sda9 /media/sda9
Change directories into your home (see Technical note that follows):
cd /media/sda9/home/your_name
Then run the familiar dd command (with the same logic as discussed above):
sudo dd if=/dev/zero of=bigjunkfile
(you need sudo)
[Optional: check things with df -hT, ls -al, du -sh, du -ah, Konqueror/Dolphin.]
Now remove bigjunkfile:
sudo rm bigjunkfile
Change directories back to root (of the CD session):
cd /
Unmount the mount point folder (note the spelling of the command umount):
sudo umount /media/sda9

Troubleshooting tip: if "Cannot unmount, device busy," then exit Konsole by typing exit, then open Konsole, and do again
sudo umount /media/sda9

For Case 1, you are done.
You may leave Konsole by typing exit, then exit the live CD session and re-boot.

Recommended:
-> Now re-boot to clear out any /tmp directories and ensure that your OS works OK.

Technical note
Instead of changing directory into your home (using cd /media/sda9/home/your_name), you could simply run dd as
sudo dd if=/dev/zero of=/media/sda9/home/your_name/bigjunkfile
(That is, in the Live CD session, supply full, absolute path names to the dd command.)
And then to remove bigjunkfile, use: sudo rm /media/sda9/home/your_name/bigjunkfile)

Case 2: Suppose your /home is on a separate partition (from the Kubuntu root partition)
Case 2a: Clean up empty space only on your /home partition.
Case 2b: Clean up empty space in the Kubuntu root partition.

Continuing with the same example, we assume your home on /dev/sda9 is separate from your root partition, and the root partition is named /dev/sda8:
root /dev/sda8
/home /dev/sda9

First, clean up your /home just as we did above in Case 1 using the Kubuntu Live CD.
For Case 2a, you are done:
Exit Konsole with the exit command; then exit the live CD session and re-boot.
For Case 2b, continue:
You now have the option of entering your root partition where you can zero-fill its unused space. If you wish to do so, that is done as follows (assuming your Kubuntu root partition is on sda8):

sudo mkdir /media/sda8
sudo mount /dev/sda8 /media/sda8
sudo dd if=/dev/zero of=/media/sda8/bigjunkfile
[Optional: if you wish, you can change directories into /media/sda8 (cd /media/sda8) and check things with df -hT, ls -al, du -sh, du -ah, Konqueror/Dolphin.]
sudo rm /media/sda8/bigjunkfile
sudo umount /media/sda8
Troubleshooting tip: if "Cannot unmount, device busy," then exit Konsole by typing exit, then open Konsole, and do again
sudo umount /media/sda8

You are done. Exit Konsole with the exit command; then exit the live CD session and re-boot.

Recommended:
-> Now re-boot to clear out any /tmp directories and ensure that your OS works OK.

= = = = = = = = = = = = = = = = = = = =
SECTION 3
How to Clean a File Before Deleting It

Let's say you have a file you wish to delete, and (with high probability) you do not want it to be retrievable . Before deleting it in Kubuntu, you should "fill it" with zeros (or random numbers)--replace the bytes of the file with zeros (or random numbers). Actually, we will use dd to write zeros to a file of identical size and with the same name as the file you wish to be rid of. That is, the file you wish to be rid of will be overwritten with zeros.

Example
I have a file on my Desktop named discardfile.odt.
Open Konsole. Change directories to where the file is (to the Desktop).
cd Desktop
List all your files under Desktop:
ls -al
The (partial) output is
-rwxr-xr-x 1 mike mike 44425 2009-12-03 21:15 discardfile.odt
The size of discardfile.odt is 44,425 bytes.
Use dd to create a file discardfile.odt of 44,425 bytes and containing only zeros, and since discardfile.odt already exists, the latter will be overwritten by the former:
dd if=/dev/zero of=discardfile.odt bs=44425 count=1 conv=notrunc
The output is
1+0 records in
1+0 records out
44425 bytes (44 kB) copied, 0.000122628 s, 362 MB/s

When I look at it in Konqueror, it is now a Plain Text Document; when I open it with Kate it is a binary file showing all zeros (........).

Next, remove it (I'm still in the /home/mike/Desktop directory where the file is):
rm discardfile.odt

It is gone in this sense:
It is deleted from your filesystem, and its contents (on your disk in bits and bytes) has been replaced by zeros; thus you could not recover the original file (say, by using PhotoRec). (You could recover the new file discardfile.odt which contains all zeros.)
The file name Note that we used the exact file name, including its extension .odt. Had it been a photo called myhouse.jpg, you would use myhouse.jpg as the name in the dd statement.
Technical note: Although the file is gone, the file's metadata may not be gone, including the name of the file. This suggests that for sensitive files, you may wish to choose the file name carefully.

Random numbers
The following statement would write random numbers "over" discardfile.odt:
dd if=/dev/urandom of=discardfile.odt bs=44425 count=1 conv=notrunc

= = = = = = = = = = = = = = = = = = = =
SECTION 4
How to Clean a Partition Before Reusing It or Deleting It

Suppose you have been using a partition for something; for example, for data, for an operating system, or for your /home, or for a combination of these. Now you're done with it but would like to re-use the exact same partition for something else (e.g., to install a new operating system; or for a new /home partition); or you are done with it and now wish to delete the partition (so you can re-partition your drive perhaps). Here's how to wipe any evidence of past used files.

You can do this from a (Kubuntu) live CD, from another Kubuntu partition, or from GParted Live CD. If you use GParted, after you zero-out the partition, you can immediately re-format it. In Kubuntu, you use Konsole and you must use sudo dd. In GParted Live CD, open a terminal by double-clicking Terminal icon at the top, that gives you a terminal with a root prompt, so you do not have to use sudo and can simply use dd.
Here it is using sudo:

Find out exactly how the partition is named:
sudo fdisk -lu
Suppose it is /dev/sdxn.
sudo dd if=/dev/zero of=/dev/sdxn conv=notrunc

=> Now, you may re-use that partition knowing that it has been cleaned up (wiped or filled with zeros); thus you would begin by formatting it (in GParted). Or, you may delete the partition (in GParted), knowing that the data on it has been wiped (overwritten by zeros).
CAUTION: The partition will be FULL after you issue the dd command! It will be full of zeros! If you re-format it in GParted Live CD, it will be "empty" again (but clean).

= = = = = = = = = = = = = = = = = = = =
SECTION 5
How to Clean Your Hard Drive or Flash Drive Before Reusing It

By writing zeros to a hard drive or flash drive, you restore it to like-new state. Zeros are written to every writable byte. This makes it extremely difficult if not impossible to recover any data you once had on the drive. You can do this from an OS on another hard drive attached to your PC or from a live CD or live flash drive. Either way, it is easy to do and the steps are the same and apply to wiping any drive (internal HD, external HD, or flash drive):

sudo fdisk -lu
Identify the drive. Example: Let's say it is /dev/sdx. CAUTION: Make sure about this!
Then, simply do
sudo dd if=/dev/zero of=/dev/sdx

(It may take awhile, depending on your setup. For example, on an Intel D915GAVL board with a 3 GHz CPU, it took 45 minutes to zero-fill a 160 GB Seagate SATA drive.)

Next:
You may re-partition the drive, making new partitions and formatting each new partition.
When you do this, say by GParted Live CD, you will get the message that the drive needs a partition table (or MBR or Disk Label). Accept this, select the standard PC type--MSDOS partition table or Disk Label--then proceed to make new partitions. Note: Some of you may be using the newer GPT partition table type for 2 TB drives. GParted supports GPT: click the drop list arrow (in the message) to see your choices.

CAUTION: After running sudo dd if=/dev/zero of=/dev/sdx , the drive will be FULL because it is full of the zeros you just put in it. Thus, if you try to use it in some programs (e.g., usb-creator), you will get a message telling you the drive is full. To correct this and start new, you must re-partition the drive.

= = = = = = = = = = = = = = = = = = = =
SECTION 6
How to Clean the Master Boot Record, 1 sector; and 63 sectors

The Master Boot Record (MBR) is the first 512 bytes of the flash drive; that's the first sector of the flash drive. (A sector is 512 bytes.) The MBR is very important; it is the first thing your BIOS looks for when booting.

The traditional PC MBR contains 446 bytes of bootloader code; then a 64-byte partition table: enough for four primary partitions, where the fourth one may be an Extended partition; then a 2-byte end-of-file boot record signature (in hexadecimal: 55 aa). To see your MBR on drive /dev/sdx, do this:
sudo dd if=/dev/sdx count=1 | hexdump -C
(The "|" is the vertical bar which is Shift and \, or Shift+\)

Your MBR may be empty or it may have some past bootloader code in it or just some useless data. When GRUB is used, sectors following the MBR are also used for GRUB code. And when partitioning is done, usually the first partition starts at sector 63--after all this other stuff. In LBA addressing, the first sector is sector 0 (in older C/H/S addressing, the first sector is sector 1). Thus, in LBA, 63 sectors precede the first sector of the first partition. (My dd Command how-to explains more about this.)

Before re-using an old drive, you may want to zero it out (see SECTION 5, How to Clean Your Hard Drive or Flash Drive Before Reusing It). However, some people simply want to ensure that they have a clean MBR to start with, and therefore want to zero-out the MBR of the drive. Let's do that first.
usb-creator (and other programs): If you run usb-creator on a flash drive that has bootloader code on it (in its MBR), you will have problems (since usb-creator uses Syslinux code in the first partition, and the MBR code you have will confuse it). Thus, before running usb-creator on this flash drive, clear out its MBR.

Zero-out the MBR of drive /dev/sdx
sudo dd if=/dev/zero of=/dev/sdx bs=512 count=1

The block size bs=512 is actually the default block size, but it is nice to write it out so we can see it explicitly here. Then count=1 tells dd to read just one block from the input file (if) and write that block to the output file (of). Thus, the dd statement writes zeros to the first 512 bytes (= the MBR) of sdx.

(CAUTION: Use sudo fdisk -lu to identify the correct drive sdx. Where do you think dd got its nickname of "data destroyer"?)

The first 63 sectors are used for the MBR (the first sector) and other code (e.g., GRUB Legacy embeds Stage_1.5 bootloader code in 16 sectors of that space). Malicious hackers may put code in the space, too. Other programs may dump code there. Thus, you may wish to zero-out the first 63 sectors of a drive.

Zero-out the first 63 sectors of drive /dev/sdx
sudo dd if=/dev/zero of=/dev/sdx bs=512 count=63

------------------------------------------------------
------------------------------------------------------
References

Commands at Konsole: Beginners: 3 parts
http://kubuntuforums.net/forums/inde...opic=3091607.0

Privacy Cleanup 101
http://kubuntuforums.net/forums/inde....msg115011#new

RESCUE Partitions & Data, TestDisk--PhotoRec--Knoppix
http://kubuntuforums.net/forums/inde...opic=3091847.0 )

**Qqmike** · Mar 23, 2008, 11:05 AM

Re: dd Command

dd Application:
What's at the end of YOUR hard drive?

I've read in a few places that there could be anything at the end of your hard drive (phantom MBRs, spies, info plants, and malware). I doubt it. But just for some dd practice, it's easy to have a look:

And what better drive to look at than my XP drive (just in case, of course...

); that would be drive sda.
At Konsole,
sudo fdisk -lu
yields this output for hard drive sda (Seagate 160 GB SATA):

Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x90439043
Device Boot Start End Blocks Id System
/dev/sda1 * 16065 204812684 102398310 7 HPFS/NTFS (Windows XP)
/dev/sda2 204812685 205021529 104422+ 83 Linux
/dev/sda3 205021530 312576704 53777587+ 5 Extended
/dev/sda5 205021593 219351509 7164958+ 83 Linux
/dev/sda6 219351573 233681489 7164958+ 83 Linux
/dev/sda7 233681553 254164364 10241406 83 Linux

The plan -- What to look for:
A partition contains a whole number of cylinders (not any partial cylinder).
=> Find out what's stored in the partial cylinder at the end of the drive.
Fact: 1 sector = 512 bytes
Fact: 1 cylinder = 255*63 = 16065 sectors (= 8,225,280 bytes)

The drive sda has 160,041,885,696 bytes = 312,581,808 sectors
There are 19457 cylinders listed (in the fdisk -lu output above).
19457 cylinders * 16065 sectors/cylinder = 312,576,705 sectors “addressable”
=> 312,581,808 sectors total - 312,576,705 sectors addressable = 5103 sectors leftover
=> 5103 sectors leftover at the end of the disk
(not enough for a cylinder since a cylinder is 16065 sectors; so the 5103 sectors will not be part any partition)

Next, look to see what's in the 5103 leftover sectors.

First, do a quick check to see if there are any non-zero characters in there:
sudo dd if=/dev/sda skip=312576705 | hexdump -C
That gave the following output:
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
5103+0 records in
5103+0 records out
2612736 bytes (2.6 MB) copied, 0.165139 seconds, 15.8 MB/s

=> all zeros in the leftover 5103 sectors.
Note: The dd command does the copy using the default block size of 512 bytes = 1 sector (bs=512 is the default).
Note: skip=312576705 causes the first 312576705 blocks of size 512 bytes each to be skipped before dd reads the sda drive.
Note: 5103 sectors = 5103 sectors * 512 bytes/sector = 2612736 bytes (2.6 MB) copied. 1 record = 1 block size = bs=512 bytes (by default since we didn't specify another value for bs= ); i.e., in our case, 1 record = 512 bytes = 1 sector.
Note: In the dd output, the single * on the second line indicates that the rest of the file consists of zeros.
Note: On the first line of the dd output, the |................| is the ASCII part of the hexadecimal dump and indicates all zeros in the first line.
Note: The dd command skips 312576705 bytes, then copies the rest of sda to the screen (standard output) and that output is piped (using the pipe symbol | ) to hexdump -C which displays its output on the screen.

If there had been nonzero characters in the dd output, then you must investigate what's in there.
You will see some of it in the ASCII part of the hexdump (the part that has “regular” ASCII character conversions (shown on the right between | and |) of the hexadecimal characters).

Another way to investigate more closely is to have dd copy the 5103 sectors to a file, then use Midnight Commander to view that file.
Here's an outline of how to do that:
In Kubuntu, at Konsole Terminal program,
sudo dd if=/dev/sda of=/home/mike/Desktop/myFile skip=312576705
5103+0 records in
5103+0 records out
2612736 bytes (2.6 MB) copied, 0.0996157 seconds, 26.2 MB/s

That put the 2612736 (leftover) bytes into a file called myFile on my Desktop.
Install Midnight Commander using Adept Manager (K > System > Adept Manager).
Indicate this in Adept by typing either Midnight Commander or mc in the search window.
Open Midnight Commander in Konsole by typing mc and press Enter.
In mc, click File, cursor down to View, Enter, type the location of the file (e.g., /home/mike/Desktop/myFile), and it'll come up in mc. You can play with mc using the options at the bottom (e.g., press various function keys Fn for View, hexadecimal, ASCII, etc.).

(Edited for several typos on 3-23-08...)

**Snowhog** · Mar 23, 2008, 03:08 PM

Re: dd Command

Very informative. Thank you. I ran through this on my laptop:

Disk /dev/sda: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders, total 234441648 sectors
Units = sectors of 1 * 512 = 512 bytes

The addressable sectors equals 234,436,545, leaving (interestingly) 5103 'left over' sectors as well. Running the dd command, skipping the first 234436545 bytes, revealed the following:

Code:

00000000 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
00107c00 eb 52 90 4e 54 46 53 20 20 20 20 00 02 08 00 00 |.R.NTFS  .....|
00107c10 00 00 00 00 00 f8 00 00 3f 00 ff 00 00 e8 2e 00 |........?.......|
00107c20 00 00 00 00 80 00 80 00 ff 57 ca 0d 00 00 00 00 |.........W......|
00107c30 00 00 0c 00 00 00 00 00 10 00 00 00 00 00 00 00 |................|
00107c40 f6 00 00 00 01 00 00 00 7a f6 ee 42 05 ef 42 08 |........z..B..B.|
00107c50 00 00 00 00 fa 33 c0 8e d0 bc 00 7c fb 68 c0 07 |.....3.....|.h..|
00107c60 1f 1e 68 66 00 cb 88 16 0e 00 66 81 3e 03 00 4e |..hf......f.>..N|
00107c70 54 46 53 75 15 b4 41 bb aa 55 cd 13 72 0c 81 fb |TFSu..A..U..r...|
00107c80 55 aa 75 06 f7 c1 01 00 75 03 e9 d2 00 1e 83 ec |U.u.....u.......|
00107c90 18 68 1a 00 b4 48 8a 16 0e 00 8b f4 16 1f cd 13 |.h...H..........|
00107ca0 9f 83 c4 18 9e 58 1f 72 e1 3b 06 0b 00 75 db a3 |.....X.r.;...u..|
00107cb0 0f 00 c1 2e 0f 00 04 1e 5a 33 db b9 00 20 2b c8 |........Z3... +.|
00107cc0 66 ff 06 11 00 03 16 0f 00 8e c2 ff 06 16 00 e8 |f...............|
00107cd0 40 00 2b c8 77 ef b8 00 bb cd 1a 66 23 c0 75 2d |@.+.w......f#.u-|
00107ce0 66 81 fb 54 43 50 41 75 24 81 f9 02 01 72 1e 16 |f..TCPAu$....r..|
00107cf0 68 07 bb 16 68 70 0e 16 68 09 00 66 53 66 53 66 |h...hp..h..fSfSf|
00107d00 55 16 16 16 68 b8 01 66 61 0e 07 cd 1a e9 6a 01 |U...h..fa.....j.|
00107d10 90 90 66 60 1e 06 66 a1 11 00 66 03 06 1c 00 1e |..f`..f...f.....|
00107d20 66 68 00 00 00 00 66 50 06 53 68 01 00 68 10 00 |fh....fP.Sh..h..|
00107d30 b4 42 8a 16 0e 00 16 1f 8b f4 cd 13 66 59 5b 5a |.B..........fY[Z|
00107d40 66 59 66 59 1f 0f 82 16 00 66 ff 06 11 00 03 16 |fYfY.....f......|
00107d50 0f 00 8e c2 ff 0e 16 00 75 bc 07 1f 66 61 c3 a0 |........u...fa..|
00107d60 f8 01 e8 08 00 a0 fb 01 e8 02 00 eb fe b4 01 8b |................|
00107d70 f0 ac 3c 00 74 09 b4 0e bb 07 00 cd 10 eb f2 c3 |..<.t...........|
00107d80 0d 0a 41 20 64 69 73 6b 20 72 65 61 64 20 65 72 |..A disk read er|
00107d90 72 6f 72 20 6f 63 63 75 72 72 65 64 00 0d 0a 42 |ror occurred...B|
00107da0 4f 4f 54 4d 47 52 20 69 73 20 6d 69 73 73 69 6e |OOTMGR is missin|
00107db0 67 00 0d 0a 42 4f 4f 54 4d 47 52 20 69 73 20 63 |g...BOOTMGR is c|
00107dc0 6f 6d 70 72 65 73 73 65 64 00 0d 0a 50 72 65 73 |ompressed...Pres|
00107dd0 73 20 43 74 72 6c 2b 41 6c 74 2b 44 65 6c 20 74 |s Ctrl+Alt+Del t|
00107de0 6f 20 72 65 73 74 61 72 74 0d 0a 00 00 00 00 00 |o restart.......|
00107df0 00 00 00 00 00 00 00 00 80 9d b2 ca 00 00 55 aa |..............U.|
00107e00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
*
5103+0 records in
5103+0 records out
2612736 bytes (2.6 MB) copied, 0.182902 seconds, 14.3 MB/s
0027de00

This HD came with Windoze Vista pre-installed, which I eventually dumped in favor of Linux. I used GParted to remove the existing Vista partitions (two of them) and then set up my partitions for use with Linux (eight partitions in all - three primary, one extended, and five logical). I don't believe I formatted the HD before repartitioning with GParted, although, of course, the root and separate home partitions were formatted using GParted.

First question: Does formatting a drive only affect the addressable sectors? Based on my output above, this would seem to be the case.

Second question: As I have only Linux on my HD now (Windoze runs in a VM), is it safe to zero out these bytes in the unaddressable sectors?

**Qqmike** · Mar 23, 2008, 03:48 PM

Re: dd Command

“First question: Does formatting a drive only affect the addressable sectors? Based on my output above, this would seem to be the case.”

I think so. There's a matter of terminology here, which is always confusing unless you program this stuff: “addressable” by the filesystem? But, yes, it certainly seems so.

“Second question: As I have only Linux on my HD now (Windoze runs in a VM), is it safe to zero out these bytes in the unaddressable sectors?”

Yes, you can zero it out. Probably no harm is keeping it, though, in your case – it looks like Vista put a phantom MBR there (note the 55 aa end-of-file signature at the end). Very strange message it put there, too.

I really don't like giving dd recipes, especially after a day of making typos as I have!
But it seems it would be this (>>>> please check the seek= bytes):
dd if=/dev/zero of=/dev/sda bs=512 seek=234436545

(and as you know from the dd stuff, you use seek rather than skip on the output file (“of”) end of the equation; so before writing zeros to sda, the dd command will skip 234436545 bytes on sda; and, as you know, if anything slips here, you will write zeros to that drive ...)

I heard somewhere that Vista plants such a thing at the end of the drive ...

Your post is informative and a good example for others to see and try.

**Snowhog** · Mar 23, 2008, 04:24 PM

Re: dd Command

Originally posted by Qqmike

Your post is informative and a good example for others to see and try.

Thank you. Not one to simply 'leap before looking', I'll leave the data in these last bytes alone (for now).

As you stated, "Probably no harm is keeping it,"

The usefulness of this comparison - that my 'unaddressable sectors' do contain 'data' and yours does not - is that stuff can be written there! While not a Windoze user anymore, outside of keeping Bill trapped within a VM, this might explain why a Windoze user hit with a particularly nasty virus, can't seem to get rid of it, even after reformatting the HD and reinstalling the OS. mbr (Master Boot Record) viri are common enough, but I wonder about eod (End of Disc) varieties??

**Qqmike** · Mar 23, 2008, 04:32 PM

Re: dd Command

S-H: "... but I wonder about eod (End of Disc) varieties??"

Yes, and spyware? trojans? various informational plants? who knows!

It's a good reason, after infection/suspicion, before reformatting and re-installing, to wipe the drive by a zero-fill (or zero-out the drive). If the drive is sdb, that would be, from a live CD:
dd if=/dev/zero of=/dev/sdb
or using the DoD version of dd from Helix Live CD:
dcfldd if=/dev/zero of=/dev/sdb
or, use a zero-fill program like the SeaTools utility.

**txHarleyMan** · Mar 23, 2008, 07:31 PM

Re: dd Command

dd ( data destroyer ) is great as long as you are very careful. Have a backup, right?

**Qqmike** · Mar 23, 2008, 07:50 PM

Re: dd Command

I think by now, that goes without saying, in reference to your anachronism for dd.
We did read the first post, right?
Anyone using these methods is smart enough to take precautions, just as they do when editing system files like fstab, menu.lst, X.org files, sources.list, ...
Or, smart enough to fix things/recover when they don't make those backups, which I'm sure is more often the case.
And, there will always be people who will never get if= and of= straight.

**txHarleyMan** · Mar 24, 2008, 02:38 AM

Re: dd Command

Originally posted by Qqmike

Anyone using these methods is smart enough to take precautions, just as they do when editing system files like fstab, menu.lst, X.org files, sources.list, ...
Or, smart enough to fix things/recover when they don't make those backups, which I'm sure is more often the case.

LOL... Well, I am just not so quick to make assumptions about people.

**Qqmike** · Mar 15, 2009, 07:06 AM

Re: dd Command

Update: dcfldd on Helix CD

dcfldd no longer available on free Helix CD:
The Helix reference for free CD with dcfldd and how-to's:
Helix: http://www.efense.com/helix
No longer available free; by membership only.
Helix3 vs Helix3 Pro -- changes in the program and products
http://www.youtube.com/watch?v=soBNib2ydt0

To get dcfldd:
sudo apt-get install dcfldd
man dcfldd

dcfldd
> dcfldd(1) - Linux man page
http://linux.die.net/man/1/dcfldd
> See also, dev notes re verify capability:
http://www.networksecurityarchive.or.../msg00004.html
> dcfldd - Latest version 1.3.4-1
http://dcfldd.sourceforge.net/

I have modified the reference section of the OP to reflect this change in Helix products.

**Qqmike** · Jan 19, 2010, 02:36 PM

NEW-> The dd Command

The dd Command

Brand new, more tutorial, more complete, tons of examples, much improved, completely rewritten.

How to use the dd command

In three parts:
Part 1: Contains the basics, plenty of discussion and guidance.
Part 2 is in Reply #1 and contains lots of applications/examples.
Part 3 is in Reply #2 and repeats 6 methods from Part 2, methods useful for
my how-to Privacy Cleanup 101, and is expanded and even more tutorial for beginners.

You can scroll up to the first post and read the Table of Contents for all three parts.

This one is loaded for the beginning-to-intermediate level.
16,500 words.

Not to say that I won't improve it and add a bit more here and there, I will. But it may be the most complete guide (I know of) on the Internet, mainly because there isn't much out there. Just a fact. I know, because I searched for stuff to read, and much of it is either incorrect or impossible to read or both. It also clarifies issues that are not correctly reported elsewhere, issues people seem to be confused about. (It no doubt needs some proofing and copyediting for some typos, but all the examples are fully tested.)

Could be fun stuff for folks wanting to learn some more Linux methods.

You don't have to use dd for everything, but almost everyone could find an application or two that would make their work easier.

**#!/bin/bash** · Dec 21, 2011, 06:05 PM

Use dd to create a swap file.

Here is one for your examples section:
Create a 1GB swap file.

This command creates the empty file
$ sudo dd if=/dev/zero of=/boot/swap.img bs=1024 count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB) copied, 3.41333 s, 300 MB/s

This command turns the empty file into a swap file
$ sudo mkswap /boot/swap.img
Setting up swapspace version 1, size = 999996 KiB
no label, UUID=795987b3-d0d5-4c5b-8737-0a29199716b7

This command tells the system to start using the swap file
$ sudo swapon /boot/swap.img

This command verifies that the swap file is being used
$ swapon -s
Filename Type Size Used Priority
/dev/sdb5 partition 975868 16 -1
/boot/swap.img file 999996 0 -2

Lastly you need to edit /etc/fstab and add the swapfile entry
$ sudo nano /etc/fstab

/boot/swap.img none swap sw 0 0

Now I can reclaim my /dev/sdb5 partition by removing the entry from fstab. Or I can leave it as is. If you increase your memory you don't have to fdisk your drive and add more swap space, just create a swap file.

<edit> Corrected count in dd command.

**Qqmike** · Dec 26, 2011, 07:35 PM

Re: The dd Command

Nice example.

Let's just check one thing:
$ sudo dd if=/dev/zero of=/boot/swap.img bs=1024 count=500000

Should that be count=1000000? (To get 1 GB) (Or, use bs=2048?)

Announcement

The dd Command

The dd Command

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment