Best way to move a system to a larger HDD.

[Frank616]

OK, this appears to have worked. I have rebooted the machine after running dd, and it came up as it was before. So far so good.

GParted did not update its information after running dd in the terminal, so I rebooted.

I now have a screenshot taken from GParted, but my command line skills are limited and I do not know how to get it off the machine and on to my laptop, and the Dillo web browser in GParted does not appear to be able to connect to this forum, so I'm going to have to type some of this stuff in.

I have:

/dev/sda1        ntfs                           18.82 GB
/dev/sda2        extended                      130.22 GB
     /dev/sda5   ext 4                         124.90 GB
     /dev/sda6   linux-swap                      5.33  GB
unallocated                                    782.46 GB

It would appear to me that I need to

1) remove the swap partition
2) extend /dev/sda5 to within 5 GB of the end of the disk
3) re-create the 5 GB swap partition

Can anyone let me know if this is correct?

Extending /dev/sda5 will not damage the contents of sda5, correct?

Thanks.

Frank.

[vinnywright]

sounds like a plan
growing /sda5 to the right will/should not damage your files just do not format it.

VINNY

[Snowhog]

Increasing partition sizes (into unused space) from the end of the existing partition (growing from the right) is safe.

[Frank616]

Vinny:

just do not format it.

Will simply extending the partition make it fully usable? How does that extended portion get formatted? Or does ext4 work like NTFS where it gets formatted 'as needed' as the data grows?

Frank.

[SteveRiley]

"Formatting" a modern SATA or SAS hard drive involves nothing more than writing a partition table to the disk and configuring each partition with a particular file system. The partition table defines the starting and ending sectors of each partition. Tools like GParted allow you to change the starting and ending positions of a partition.

Extending a partition into free unused disk space is a safe operation, and is actually the easiest operation for these tools to perform. Other operations are possible, but they require more time because they have to rearrange data within the partition. Consider the following diagram:

 0  1  2  3                   99 100             149 150
+------------------------------------------------------------------+
|     |                         |                   |              |
|  A  |            B            |         C         |  free space  |
|     |                         |                   |              |
+------------------------------------------------------------------+

This disk has three partitions:

* A, occupying sectors 0-1
* B, occupying sectors 2-99
* C, occupying sectors 100-149

From sector 150 to the end is unused space.

Extending C "to the right" is very easy: the partitioning tool simply changes the partition table to indicate the new ending sector. It doesn't matter what might happen to be stored on those sectors from previous use; modern file systems keep track of everything they write. They don't care about contents of sectors they haven't written to. So the file system on partition C will view all those new sectors as fresh usable space and freely write data to them. Of course, once data has been written, the file system index will preserve the contents on those sectors, make them available when you read from them, and allow you to reuse them whenever you "delete" a file (where "delete" really means simply to free up the sectors for yet more reuse).

Shrinking C "to the left" is possible but takes more time, as the tool will have to physically relocate any data in the sectors into free space elsewhere in the partition. Note that a partition can be shrunk only by the amount of unused space: if the partition size is 100 GiB and 15 GiB is used, the partition can be shrunk by a maximum of 85 GiB.

Likewise, shrinking C "to the right" is possible: you can move its starting sector from, say, 100 to 120. Again, this operation will take time, as the tool has to rearrange data to preserve the file system structures. Typically, the reasons you'd want to do such an operation would be because you need to make B larger or you need to create a new partition between B and C.

[dibl]

The last time I moved an installed OS from an old/small drive to a new/big drive, I used GParted from a bootable Parted Magic USB stick. With both hard drives connected, when you're looking at the graphic of the old drive, you can simply "copy" the OS partition, and change the view to the (unpartitioned) new drive, make a new partition table, and then "paste" the partition. If it is smaller than you want, as a second step you can resize it to whatever size you want. For somebody's Dell or HP Windows machine, you can copy and paste the "recovery" partition first, and then copy and paste their OS partition, and they can continue cluttering up the thing for many years into the future. ;-)

[Frank616]

Steve:

Thank you for the tutorial. I mentally had this idea that the various 'nodes' were written across the disk from beginning to end, kind of like mileposts, with the data put in between.

So what, then, is the purpose of formatting a hard disk, and why does it take so long? Is it just writing 'zeroes' to make sure there is no real data there that could be recovered later? (I realize that even than can be recovered in some cases with the right forensic tools.)

Frank.

[Frank616]

dibl:

With both hard drives connected, when you're looking at the graphic of the old drive, you can simply "copy" the OS partition, and change the view to the (unpartitioned) new drive, make a new partition table, and then "paste" the partition.

Cool! I had no idea that one could do that.

It is amazing what the tools are capable of these days. I remember when any changes to a disk were 'black magic'.

Frank.

[Frank616]

OK, I've run into a problem I don't know how to deal with.

In the partition map that I posted a few messages ahead I have:

/dev/sda1        ntfs                           18.82 GB
/dev/sda2        extended                      130.22 GB
     /dev/sda5   ext 4                         124.90 GB
     /dev/sda6   linux-swap                      5.33 GB
unallocated                                    782.46 GB

GParted will let me delete the linux-swap partition, but then I cannot get it to amalgamate the unallocated 5.33 GB from the deleted swap with the remaining 782.46 GB on the disk. When I ask it to expand the 124.90 GB ext4 partition, it will only allow me to expand it to a maximum of the 5.33 GB in the deleted swap area. It doesn't give me the option of expanding it further into the 782.46 GB left on the new disk.

After deleting (and executing the delete operation) it is showing me:

/dev/sda1        ntfs                           18.82 GB
/dev/sda2        extended                      130.22 GB
     /dev/sda5   ext 4                         124.90 GB
     /dev/sda6   unallocated                     5.33 GB
unallocated                                    782.46 GB

What am I doing wrong?

Do I need to format those two unallocated areas as ext4 first?

Frank.

[oshunluvr]

The extended partition needs to be shrunk to exclude the unallocated 5.33 GB. This will automatically add it to the larger unallocated chunk. I don't know what GUI tool you're using to manipulate your hard drive, but usually there's a colored box around each partition showing it's allocation so SDA2 will be a slightly larger box around SDA5 and SDA6.

Basically, shrink SDA2 to the same size as SDA5 and you'll have what you're asking for.

I only skimmed this post so I apologize if this has been asked and answered: What is your final goal with this drive partition wise? I ask because with the extended partition in the middle of the partition table like that it can be a PITA to re-allocate space around later.

For simplicity's sake I would expand the extended partition to include all the unallocated space rather than shrinking it. Then all your subsequent partitions will fall numerically after SDA5.

E.g.:

Shrinking SDA2 and adding a home and swap partition results in:

SDA1 ntfs
SDA2 extended
> SDA5 ext4 (/)
SDA3 ext4 (/home)
SDA4 swap

Expanding SDA2 and adding a home and swap partition results in:

SDA1 ntfs
SDA2 extended
> SDA5 ext4 (/)
> SDA6 ext4 (/home)
> SDA7 swap

Maybe it's just me, but I have an easier time visualizing where my stuff is if it's in some sort of order.

A little late to say but I am in the habit of putting swap as #1 or #2 and putting extended at the end (#4). My usual setup for other people using a single drive dual boot layout is:

SDA1 ntfs
SDA2 swap
SDA3 ext4 (/)
SDA4 extended
> SDA5 (/home)

you could still do this now by moving SDA2 to the right into the 5.33GB and then put swap behind SDA1. You would then need to renumber the partitions (easily done) and you would be left with:

SDA1 ntfs
SDA2 swap
SDA3 extended
> SDA5 ext4 (/)
> SDA6 (/home)

If it's a system for me I also create a couple extra partitions at the far "right" of the partition table so I can install another distro or do a full re-install or upgrade install whenever I'd like without moving stuff around or wiping my current install. In that case my drive looks like this:

SDA1 ntfs
SDA2 swap
SDA3 ext4 (/)
SDA4 extended
> SDA5 (/home)
> SDA6 extra 12-20GB
> SDA7 extra 12-20GB

Just to add one more bit of information to complete the overload of this post if I'm working on a multi-drive system I usually use RAID so I use SDA3 as a separate /boot partition.

[dibl]

It is amazing what the tools are capable of these days. I remember when any changes to a disk were 'black magic'.

It really is about that simple, although upon reflection I should also mention that "pasting" a 60GB OS partition that is relatively full might take quite awhile -- possibly hours -- the processing capacity of the host computer is your bottleneck. So, bring a book to read, or a netbook with which to browse while you're doing this.

[dibl]

Just to add one more bit of information to complete the overload of this post if I'm working on a multi-drive system I usually use RAID so I use SDA3 as a separate /boot partition.

Nahhhh, it's not far enough OT and overloaded yet.

There's one more fairly imaginitive approach, which I had never thought of until a guy on ubuntu forums reported that he uses it all the time. You can partition your new hard drive 100% as an extended partition. Then you will only have logical partitions, and you can disregard the fact that only 4 primary partitions are allowed, and you can forget about the type of enumeration complexities that happen in some of the examples above.

NOWWW it's truly overloaded!

[Frank616]

oshunluvr:

I only skimmed this post so I apologize if this has been asked and answered: What is your final goal with this drive partition wise? I ask because with the extended partition in the middle of the partition table like that it can be a PITA to re-allocate space around later.

This 1 TB disk is what was dd'd from a failing 160 GB drive in my old desktop machine. Whatever is on the 160 GB drive has been there for a long time, and I've never looked at it before this. That layout was copied to this 1 TB drive, which is now the system drive in my desktop machine. I don't maintain a separate /home partition on any of my machines, although that I know a lot of people swear by it.

What the 130.22 GB extended sda2 partition is I have no idea. How would I determine that?

My eventual goal is to keep the WinXP partition, and give the whole remainder of the disk to Kubuntu 12.04, minus a small swap space.

It makes more sense to me to have the swap closer to the 'front' of the disk as well, but the Kubuntu installer does not do that, and I've never bothered to 'argue' with it. The machine has 3 GB of RAM, so I doubt the swap ever gets used anyway.

This machine has a second 1.5 TB sdb drive which I use for /data. It is symlinked into my /home directory. This allows me to take out the system drive (which is in a removable caddy) and pop in another one for testing another distro, and still have all my data safely in the machine. I never install more than one distro on a drive (Windows excluded). I find it better to replace the whole drive in a caddy.

So, how do I determine what sda2 is? If it is irrelevant, I can covert it to swap, or just shrink it out of existence. In Gparted, the 'used' and 'unused' columns beside /dev/sda2 both just show dashes, the same as the unallocated areas of the disk show. I'm guessing that it is not being used at all.

I'd prefer to post a screenshot here, but I don't know how to get that off the desktop machine when running from the GParted live CD. I'm assuming I'd probably have to mount a flash drive, but my command line skills are limited. I can see the screenshot jpeg that GParted created, but I don't see a place that I can move it to so that I can take it off the machine so as to post it here. Alternatively, is there some other app that will give me the report that I am seeing on the GParted Live CD that I can run from within Kubuntu? I can take a screenshot of that, and post that instead.

Frank.

[SteveRiley]

Thank you for the tutorial. I mentally had this idea that the various 'nodes' were written across the disk from beginning to end, kind of like mileposts, with the data put in between.

So what, then, is the purpose of formatting a hard disk, and why does it take so long? Is it just writing 'zeroes' to make sure there is no real data there that could be recovered later? (I realize that even than can be recovered in some cases with the right forensic tools.)

Once upon a time, you could low-level format a hard drive. This was required when drives and controllers were separate entities. Now that controllers are integrated onto drives, low-level formatting isn't necessary, or even possible in many cases.

There's only one kind of physical index on a hard disk: the sector map. Once upon a time physical sectors were 512 bytes; now they're 4096 bytes. These can appear as true 4096-byte sectors, or can be translated into "emulated" 512-byte sectors, depending on whatever the host computer requires.

Old-style formatting often applied some kind of initialization, which varied -- writing zeros, performing read checks, whatever. Now, a "format" pass is nothing more than writing a parition table (if necessary) and laying down the file system. It's been a while since I've had to wait that interminably long time for a disk format.

Oh, and about the ability to perform forensic recovery... a single write to a sector obliterates whatever was there before. The rumors that insist on seven wipes for secure delete are persistent and just wrong.

http://www.h-online.com/security/new...it-739699.html
http://www.anti-forensics.com/disk-w...pass-is-enough
http://www.anti-forensics.com/disk-w...th-screenshots
http://blogs.technet.com/b/steriley/...ntegrator.aspx

[SteveRiley]

I'd prefer to post a screenshot here, but I don't know how to get that off the desktop machine when running from the GParted live CD. I'm assuming I'd probably have to mount a flash drive, but my command line skills are limited. I can see the screenshot jpeg that GParted created, but I don't see a place that I can move it to so that I can take it off the machine so as to post it here. Alternatively, is there some other app that will give me the report that I am seeing on the GParted Live CD that I can run from within Kubuntu? I can take a screenshot of that, and post that instead.

Snap a photo with your mobile phone camera.