check disk for bad clusters on disk

[doctordruidphd]

Re: check disk for bad clusters on disk

Quote from: doctordruidphd on November 06, 2011, 03:56:04 pm
Does the "View Details" option give you a screen with drive information?
no, it says "unsupported"

That may be the last word on trying to use SMART.

You might want to use one of the self-booting cdroms (partedmagic, gparted, systemrescue, etc) to bypass whatever problems gparted is giving you. This has the added advantage that you will be checking your disks in an unmounted state, which always works better.

[lindaonline15]

Re: check disk for bad clusters on disk

Quote from: doctordruidphd on November 06, 2011, 03:56:04 pm
Does the "View Details" option give you a screen with drive information?
no, it says "unsupported"

That may be the last word on trying to use SMART.

You might want to use one of the self-booting cdroms (partedmagic, gparted, systemrescue, etc) to bypass whatever problems gparted is giving you. This has the added advantage that you will be checking your disks in an unmounted state, which always works better.

yup probably...
about gparted, its working... my fault was running as root instead of running with kdesudo.
anyways, back to the topic,
now I detect my external drive with gparted --which by the way has a few MB unallocated space that never was shown with other tools!--
so now... i just want to check some stuff:
I have to delete the partition and create a new one, right?
-its better to be primary or extended?
-Align to: MiB? Cylinder? (I do not understand what this exactly do)
-lastly, I never get this that what file system is the best and fastest for external drives... ntfs?

[doctordruidphd]

Re: check disk for bad clusters on disk

I have to delete the partition and create a new one, right?

If you do that, you lose all the data. If that's what you want to do, then yes. If you just want to check it, then no.

-its better to be primary or extended?

Depends on how many partitions you want. If four or less, then primary. An extended partition is simply a reserved space on the disk for creating "logical partitions", as many as you want.

-Align to: MiB? Cylinder? (I do not understand what this exactly do)

Has to do with whether partitions (which are software entities) are set up along hardware boundaries. Checking yes may result in using a few extra mb, but diagnostic programs will complain less.

-lastly, I never get this that what file system is the best and fastest for external drives... ntfs?

Depends on what you are going to do with it. Linux support for ntfs is not on a par with support for other file systems. If you want the disk readable by both linux and windows, choose FAT, but I think there is a file size limitation of around 4gb. If you have individual files larger than that, you will need ntfs. If you are just using the disk for linux, then ext4 would be the current best choice.

Somewhere there is a FAQ on file systems, google has lots of hits on the topic.

[lindaonline15]

Re: check disk for bad clusters on disk

-Align to: MiB? Cylinder? (I do not understand what this exactly do)

Has to do with whether partitions (which are software entities) are set up along hardware boundaries. Checking yes may result in using a few extra mb, but diagnostic programs will complain less.

you mean... yes for MiB?

[dibl]

Re: check disk for bad clusters on disk

I have to delete the partition and create a new one, right?

Unless you want to change the filesystem type, you can just right-click on the unmounted partition, and choose "check".

Only use ntfs if you need to have Windows see and use that data. Otherwise ext4 is the right answer.

Also, as reminder, on pluggable USB storage devices, you always need to use "safely remove" or "eject" prior to yanking the plug. Failure to do that is a common cause of corrupted data on such devices.

[lindaonline15]

Re: check disk for bad clusters on disk

I have to delete the partition and create a new one, right?

Unless you want to change the filesystem type, you can just right-click on the unmounted partition, and choose "check".

Only use ntfs if you need to have Windows see and use that data. Otherwise ext4 is the right answer.

Also, as reminder, on pluggable USB storage devices, you always need to use "safely remove" or "eject" prior to yanking the plug. Failure to do that is a common cause of corrupted data on such devices.

umount will do, right...?

[dibl]

Re: check disk for bad clusters on disk

Again, just right-click the partition in the Gparted gui, and choose "unmount". (Don't ever try it on your root filesystem, of course!)

[doctordruidphd]

Re: check disk for bad clusters on disk

umount will do, right...?

Yes, but it will need to be sudo from a user konsole.

[lindaonline15]

Re: check disk for bad clusters on disk

ok this drive is giving me headache...
gparted shows its healthy, i've even format and re-checked it (and i've done this a dozen time with different tools) it says its healthy, but when I want to move files to it, its bloody slow (<200KB/s ! ) and it also goes crazy... since it was showing its gonna take a month (!) to copy those files, I had to stop it, but I cant browse into whatever has been copied into the drive and also I cant unmount it.
whenever I want to browse into the drive it takes long time and after all its unsuccessful...

there is one thing I probably should mention... Im not sure if this is the reason, but im pretty sure all this craziness actually started after this incident: previously I was using windows and there was a time that I wanted to get a backup image of my windows. I did this through the backup image utility in control panel (windows 7)... for some reason, in the middle of the process where it was directly getting the image and writing into this drive, I had to stop it. after that incident, i clearly remember that I was not able to browse into the drive, and I thought that i've just damaged the existing data, but after so many formats and checks it still remains and the transfer rate has gone down terribly...

[dibl]

Re: check disk for bad clusters on disk

It might have problems deeper than the partition/filesystem. They can, like all electronics, have a breakdown of a resistor or something like that and start getting flaky and unreliable.

One desperate measure you can take is to dd it, which zeros out every bit of information that was ever on it (well, the first 512 bytes anyway), and then make a new partition table, and then put a new partition on it and a new filesystem. By doing that, you can ensure that nothing that previously got written to it still exists.

Be double-damned certain that you have the right /dev/sdx identification before you do this one!

dd if=/dev/zero of=/dev/sdx bs=512 count=1

(where "x" is the correct letter for your target drive)

After you do this, you can open GParted and the drive should be found, but it will have no partition and no partition table. So do Device > New Partition Table, and select "MS-DOS" as the table type.

Then you can make a new partition, and format it. Then test it and see if the performance is any different than what you have now. If it is not any better, you have an expensive paperweight.

[lindaonline15]

Re: check disk for bad clusters on disk

It might have problems deeper than the partition/filesystem. They can, like all electronics, have a breakdown of a resistor or something like that and start getting flaky and unreliable.

One desperate measure you can take is to dd it, which zeros out every bit of information that was ever on it, and then make a new partition table, and then put a new partition on it and a new filesystem. By doing that, you can ensure that nothing that previously got written to it still exists.

Be double-damned certain that you have the right /dev/sdx identification before you do this one!

dd if=/dev/zero of=/dev/sdx bs=512 count=1

(where "x" is the correct letter for your target drive)

After you do this, you can open GParted and the drive should be found, but it will have no partition and no partition table. So do Device > New Partition Table, and select "MS-DOS" as the table type.

Then you can make a new partition, and format it. Then test it and see if the performance is any different than what you have now. If it is not any better, you have an expensive paperweight.

as great this act is, and I was really happy to see the way to set all bytes to zero, but im not sure yet whats happening here...
this is the result of the action (i was expecting it to take long time since its 1TB, but i immidiately got this):

1+0 records in
1+0 records out
512 bytes (512 B) copied, 2.9708 s, 0.2 kB/s

and when I tried to mount the device simply by opening it in dolphin, got this error : (attachment)

[dibl]

Re: check disk for bad clusters on disk

I wrongly said it zeros "every bit" -- I edited my post but too late. We're really just "lobotomizing" the drive by zeroing the master boot record -- the first 512 bytes. With no partition table, a normal operating system cannot find anything to mount, which is the expected behavior that you saw with Dolphin.

In the terminal, check with fdisk -l and verify that the drive is still seen by Linux.

Then, with the drive still connected, start GParted in super user mode with Alt-F2 "kdesudo gparted", and browse to that drive. It will show no partition present and probably a yellow warning of some kind. Click "Device" and then "New Partition Table", and choose MS-DOS in the dialog at the botton. Apply this change.

Now the "notifier" will probably pop up, as the device now is seen as a storage device. But, there is still no partition and no filesystem. So right-click in the unallocated space, and choose "new", and then let it be formatted to whatever filesystem type you need, and "Apply".

[lindaonline15]

Re: check disk for bad clusters on disk

done all of that... and the drive seems ok,
but again in copying it takes forever for it to copy with speed of <50kb !!
do you think now I should set all the bits to zero..?

[doctordruidphd]

Re: check disk for bad clusters on disk

I'm really wondering at this point if there isn't a hardware problem after all. Is this a drive you can connect with USB as well as esata, and if so, maybe give that a try.

The only other thing I can think of for the slow copying is that you may need to tweak things like block size, etc. But at this stage of the hardware game, that should really not be necessary.

Maybe also try a different cable, if one is available.

You're not running kubuntu in a virtual machine of some kind, are you?

At this point there seems no straightforward explanation for the problem, and inexplicable problems often do come down to hardware, though as I recall you said it's working on windows, so I don't get it. If you made a gparted CD or a partitionmagic CD, you could boot using one of those, and try some copying to the drive and see if it's still slow. The last ditch test would be to try booting a live CD on a different computer and see if the problem still exists.

Other random thoughts:
Is your esata port configurable through your system's BIOS, and if so, you may need to look at that.
Just for the fun of it, you might want to make a live CD of some other linux distribution (opensuse, for example, which is a completely different development branch from ubuntu/debian) and see if it also has the problem. Could be something weird in the kernel setup, which could probably be configured to work if that's what it turns out to be.

[lindaonline15]

Re: check disk for bad clusters on disk

I'm really wondering at this point if there isn't a hardware problem after all. Is this a drive you can connect with USB as well as esata, and if so, maybe give that a try.

yup did, problem exists no matter esata or usb...

The only other thing I can think of for the slow copying is that you may need to tweak things like block size, etc. But at this stage of the hardware game, that should really not be necessary.

if necessary, how to do it?

Maybe also try a different cable, if one is available.

nope unfortunately...

You're not running kubuntu in a virtual machine of some kind, are you?

Nope

At this point there seems no straightforward explanation for the problem, and inexplicable problems often do come down to hardware, though as I recall you said it's working on windows, so I don't get it. If you made a gparted CD or a partitionmagic CD, you could boot using one of those, and try some copying to the drive and see if it's still slow. The last ditch test would be to try booting a live CD on a different computer and see if the problem still exists.

sry i think i misled you... after that incident in windows, its working terribly slow in kubuntu as well as windows actually.

Other random thoughts:
Is your esata port configurable through your system's BIOS, and if so, you may need to look at that.
Just for the fun of it, you might want to make a live CD of some other linux distribution (opensuse, for example, which is a completely different development branch from ubuntu/debian) and see if it also has the problem. Could be something weird in the kernel setup, which could probably be configured to work if that's what it turns out to be.

will do that, but before that, you might wana take a look at this:

sudo hdparm -tT /dev/sdb

/dev/sdb:
 Timing cached reads:  6190 MB in 2.00 seconds = 3095.92 MB/sec
 Timing buffered disk reads: 34 MB in 4.95 seconds =  6.87 MB/sec
linda@linda:~$ sudo hdparm -tT /dev/sdb1

/dev/sdb1:
 Timing cached reads:   2 MB in 4.43 seconds = 462.74 kB/sec
 Timing buffered disk reads: 28 MB in 4.09 seconds =  6.85 MB/sec

--look at the timing cach read rate for sdb1