[solved with recent e2fsck] external usb disk won't mount, error

[Telengard]

Re: external usb disk won't mount, error

I did something bold and dangerous which seems to have worked out. I downloaded and burned grml, which comes with e2fsprogs 1.41.12-4. From the zsh root prompt I issued the following command:

e2fsck -B 4096 -y /dev/sdb1

It ran for a some time, spewing output on the terminal and auto selecting yes as the answer for each prompt. When the zsh prompt returned I mounted /dev/sdb1 and found all my files tucked into a subdirectory of /mnt/sdb1/lost&found.

This doesn't mean my files were successfully recovered. For all I know every one of them is corrupted. I still have to pick through them and verify each one. Good on me for making recursive MD5 files for each major hierarchy on the disk. I'll be checking those tonight.

If any files are lost or corrupted, then I'll be moving on photorec and foremost on raw the disk image I dumped a few days back.

EDIT
In other good news, the new HD is scheduled for delivery tonight

[doctordruidphd]

Re: external usb disk won't mount, error

du -sh /mnt/sdb1/lost&found

should tell you right away if your files have content or not -- if the 90% or so of 160gb (140 or so) is in use, then your files are probably all there.

[Telengard]

Re: external usb disk won't mount, error

The new HD is here. I'll connect it and commence rigorous testing after it has time to rest from the cold.

tmp$ sudo mount /dev/sdb1 /media/Data/ -o ro
tmp$ mount | grep 'Data'
/dev/sdb1 on /media/Data type ext2 (ro)
tmp$ cd /media/Data/
Data$ ls -Ahl
total 4.0K
drwx------ 4 root root 4.0K 2011-12-07 15:43 lost+found
Data$ sudo du -sh lost+found/
138G  lost+found/
Data$ df -h .
Filesystem      Size Used Avail Use% Mounted on
/dev/sdb1       147G 138G 2.1G 99% /media/Data
Data$

The recursive MD5 files all check out fine. I think my data is saved. Thank you very much for the awesome support. I think this was a good experience. We could continuing conversation on this topic if anyone else is interested.

I'm chalking the problems up to the enclosure, which will go to the trash now. It only became flaky after I filled up the drive. It began spuriously disconnecting during heavy activity. Weird.

[doctordruidphd]

Re: [solved with recent e2fsck] external usb disk won't mount, error

whew!

Well, I thought it might be that simple -- just let e2fsck do its thing. But there is a rather large leap of faith required to do that. Such leaps are easier with the safety net of a backup.

FYI, just my personal view -- when a drive gets to about 85% capacity, time to look for a larger drive. No matter what the file system, they all need a certain amount of free space for housekeeping, and I suspect the lack of it contributed to the problems.

Before trashing the old drive, once all your data is off, you might run a thorough check with one of smartctl's checkers (vie gsmartcontrol-root). Might be useable after all.

[Telengard]

Re: [solved with recent e2fsck] external usb disk won't mount, error

But there is a rather large leap of faith required to do that. Such leaps are easier with the safety net of a backup.

(emaphasis mine)
Exactly

FYI, just my personal view -- when a drive gets to about 85% capacity, time to look for a larger drive.

I would amend that to 50%. In my experience drives fill up much more quickly after the 50% mark. Some users never get there, but I am greedy.

No matter what the file system, they all need a certain amount of free space for housekeeping, and I suspect the lack of it contributed to the problems.

Maybe a sound guess, but I don't know. Some space is reserved on ext filesystems by default for root processes. I don't know exactly what that means, but I guess sudo e2fsck might be a root process.

. . .
-m reserved-blocks-percentage
Specify the percentage of the filesystem blocks reserved for the
super-user. This avoids fragmentation, and allows root-owned
daemons, such as syslogd( 8 ), to continue to function correctly
after non-privileged processes are prevented from writing to the
filesystem. The default percentage is 5%.

tmp$ sudo dumpe2fs -h /dev/sdb1
dumpe2fs 1.40.8 (13-Mar-2008)
Filesystem volume name:  <none>
Last mounted on:     <not available>
Filesystem UUID:     aafa5609-5eeb-4d9f-bdad-abbbfa1742bb
Filesystem magic number: 0xEF53
Filesystem revision #:  1 (dynamic)
Filesystem features:   filetype sparse_super
Default mount options:  (none)
Filesystem state:     clean
Errors behavior:     Continue
Filesystem OS type:    Linux
Inode count:       19546112
Block count:       39062039
Reserved block count:   1953101
Free blocks:       2492951
Free inodes:       19545446
First block:       0
Block size:        4096
Fragment size:      4096
Blocks per group:     32768
Fragments per group:   32768
Inodes per group:     16384
Inode blocks per group:  512
Last mount time:     Wed Dec 7 19:56:36 2011
Last write time:     Wed Dec 7 19:59:43 2011
Mount count:       2
Maximum mount count:   30
Last checked:       Wed Dec 7 15:43:19 2011
Check interval:      0 (<none>)
Reserved blocks uid:   0 (user root)
Reserved blocks gid:   0 (group root)
First inode:       11
Inode size:        128

tmp$ block_count=39062039
tmp$ reserved_block_count=1953101
tmp$ percent_reserved=$((100 * reserved_block_count / block_count))
tmp$ echo $percent_reserved "percent of /dev/sdb1 is reserved"
4 percent of /dev/sdb1 is reserved
tmp$ # integer division so fractions truncated means "about 5%"

Before trashing the old drive, once all your data is off, you might run a thorough check with one of smartctl's checkers (vie gsmartcontrol-root). Might be useable after all.

I'm not trashing the disk, just the enclosure. The disk itself seems to work just fine connected directly to my internal IDE cable. The smartctl tool is something worth looking into though. I want to thorougly test the new HD before trusting it with my data.

[dibl]

Re: [solved with recent e2fsck] external usb disk won't mount, error

Some space is reserved on ext filesystems by default for root processes. I don't know exactly what that means, but I guess sudo e2fsck might be a root process.

That's an excellent guess!

Check your reserved blocks:

sudo tune2fs -l /dev/sdxn | grep 'Reserved block count'

Change reserved blocks to zero, for a non-OS partition (best to leave it at default 5% for OS partitions):

sudo tune2fs -m 0 /dev/sdxn

I'm glad you worked through that issue -- I've been lurking to see how you fared, but I don't know enough about recovering data from an image of a broken filesystem to offer any help. But it's been educational!

[Telengard]

Re: [solved with recent e2fsck] external usb disk won't mount, error

sudo tune2fs -l /dev/sdxn | grep 'Reserved block count'

Demonstrates exactly what I wanted to point out in reply #34. FWIW tune2fs -l dumps the superblock information in exactly the same format as dumpe2fs -h; whichever is easiest to remember works fine

test$ sudo dumpe2fs -h /dev/sdb1 > dumpe2fs_output.txt
dumpe2fs 1.40.8 (13-Mar-2008)
test$ sudo tune2fs -l /dev/sdb1 > tune2fs_output.txt
test$ diff tune2fs_output.txt dumpe2fs_output.txt
1d0
< tune2fs 1.40.8 (13-Mar-2008)
33a33
>
test$

Change reserved blocks to zero, for a non-OS partition (best to leave it at default 5% for OS partitions):

sudo tune2fs -m 0 /dev/sdxn

Thanks, but if I'm so tight on space that the 5% matters then I think better advice is this:

when a drive gets to about 85% capacity, time to look for a larger drive.

This is somewhat informative:

-m reserved-blocks-percentage
Set the percentage of the filesystem which may only be allocated by privileged processes. Reserving some number of filesystem blocks for use by privileged processes is done to avoid filesystem fragmentation, and to allow system daemons, such as syslogd( 8 ), to continue to function correctly after non-privileged processes are prevented from writing to the filesystem. Normally, the default percentage of reserved blocks is 5%.

I still don't understand exactly what the indended use of the reserved space is, but it seems important enough that I should probably leave it alone.

I'm glad you worked through that issue -- I've been lurking to see how you fared, but I don't know enough about recovering data from an image of a broken filesystem to offer any help. But it's been educational!

Thanks for being here; just knowing that the friendly members of KFN are reading my woes is very supportive IMHO.

Frankly I'm surprised and gratified that I didn't have to resort to forensics. It isn't every day that an automated tool can recover your mistakes retroactively.

Lessons Learned

• Think your data will be okay without a backup? You're wrong! (Goes double for me!)
• Don't trust any storage media without rigorous testing first.
• When a program fails in ways which can't be explained by a careful reading of the documentation, try a more recent version of the program.

dibl, I'm interested to know what you think I should do to test my new hard disk. I'm reading How to Test Your Hard Drive with Smartmontools right now for some ideas. Any suggestions?

[dibl]

Re: [solved with recent e2fsck] external usb disk won't mount, error

Nice external drive!

SMART won't be of any assistance -- it will only confirm that it is a new drive. If you want to verify its storage integrity, you need something that will do surface scanning. I would suppose one or more of the utilities on Ultimate Boot CD might do that for you. I also found a reference to using e2fsck -c to write/read every block of a hdd*. I guess if that worked, then you would be confident.

FYI, perhaps the most insidious risk to a USB connected storage device is the user's failure to remember to "safely remove" or "eject".


*

~$ man e2fsck

.
.
.

   -c   This option causes e2fsck to use badblocks(8) program to do a read-
       only scan of the device in order to find any bad blocks. If any bad
       blocks are found, they are added to the bad block inode to prevent
       them from being allocated to a file or directory. If this option is
       specified twice, then the bad block scan will be done using a non-
       destructive read-write test.

[doctordruidphd]

Re: [solved with recent e2fsck] external usb disk won't mount, error

smartctl (actually, kdesudo gsmartcontrol-root) has the ability to do several surface tests. I assume it is really acting as a front end to something else -- no matter, seems it can get the job done:

Short self-test consists of a collection of test routines that have the highest chance of detecting drive problems. Its result is reported in the Self-test Log. Note that this test is in no way comprehensive. Its main purpose is to detect totally damaged drives without running the full surface scan.
Note: On some drives this actually runs several consequent tests, which may cause the program to display the test progress incorrectly.

and

Extended self-test examines complete disk surface and performs various test routines built into the drive. Its result is reported in the Self-test Log.

Certainly a good bet to put a new drive through its paces, might also turn up something on an old one.

But I agree completely -- from personal experience -- that the worst thing about a usb drive is how easy it is to unplug at the wrong time.

[Telengard]

Re: [solved with recent e2fsck] external usb disk won't mount, error

I plugged it into a rear USB port on the Visa machine. Windows said "installing device driver", then "could not recognize the device".

Next I switched to a front USB port, and then "New Volume (G" was mounted.

Now I'm trying it on my Hardy 8.04 machine.

[Telengard]

Re: [solved with recent e2fsck] external usb disk won't mount, error

test$ mount | grep sdc1
/dev/sdc1 on /media/New Volume type fuseblk (rw,nosuid,nodev,noatime,allow_other,blksize=4096)

Good so far.

test$ sudo fdisk -lu /dev/sdc1

Disk /dev/sdc1: 2000.3 GB, 2000396746752 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907024896 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x6e697373

This doesn't look like a partition table
Probably you selected the wrong device.

   Device Boot   Start     End   Blocks  Id System
/dev/sdc1p1  ? 1936269394 3772285809  918008208  4f QNX4.x 3rd part
Partition 1 does not end on cylinder boundary.
/dev/sdc1p2  ? 1917848077 2462285169  272218546+ 73 Unknown
Partition 2 does not end on cylinder boundary.
/dev/sdc1p3  ? 1818575915 2362751050  272087568  2b Unknown
Partition 3 does not end on cylinder boundary.
/dev/sdc1p4  ? 2844524554 2844579527    27487  61 SpeedStor
Partition 4 does not end on cylinder boundary.

Partition table entries are not in disk order
test$

What am I to make of this?

EDIT
Don't mind that. I mistakenly addressed the partition instead of the disk (doh!)

test$ sudo fdisk -lu /dev/sdc

Disk /dev/sdc: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x97296785

  Device Boot   Start     End   Blocks  Id System
/dev/sdc1      2048 3907026943 1953512448  7 HPFS/NTFS
test$

Later today I'll begin some write testing.

[Telengard]

Re: [solved with recent e2fsck] external usb disk won't mount, error

test$ sudo smartctl -ia /dev/sdc
smartctl version 5.37 [i686-pc-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is [url]http://smartmontools.sourceforge.net/[/url]

Device: WL2000GS A6472R      Version: 05.0
Device type: disk
Local Time is: Sat Dec 10 21:51:16 2011 EST
Device does not support SMART

Error Counter logging not supported
Device does not support Self Test logging
test$

What does it mean by "does not support SMART"? This is a new 2TB drive, so I imagine it would support smart

EDIT
I found this thread and read the following in the smartctl manpage.

. . .
LINUX:
Use the forms "/dev/hd[a-t]" for IDE/ATA devices, and "/dev/sd[a-z]" for SCSI devices. For SCSI Tape Drives and Changers with TapeAlert support use the devices "/dev/nst*" and "/dev/sg*". For SATA disks accessed with libata, use "/dev/sd[a-z]" and append "-d ata".
. . .

Still no dice

test$ sudo smartctl -ia /dev/sdc -d ata
smartctl version 5.37 [i686-pc-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is [url]http://smartmontools.sourceforge.net/[/url]

Smartctl: Device Read Identity Failed (not an ATA/ATAPI device)

A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.
test$

hdparm tells a different story.

test$ sudo hdparm -I /dev/sdc

/dev/sdc:

ATA device, with non-removable media
    Model Number:    WL2000GSA6472R
    Serial Number:   WOL240177338
    Firmware Revision: 05.01D05
    Transport:     Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5
Standards:
    Supported: 8 7 6 5
    Likely used: 8
Configuration:
    Logical     max   current
    cylinders    16383  16383
    heads      16   16
    sectors/track  63   63
    --
    CHS current addressable sectors:  16514064
    LBA  user addressable sectors: 268435455
    LBA48 user addressable sectors: 3907029168
    device size with M = 1024*1024:   1907729 MBytes
    device size with M = 1000*1000:   2000398 MBytes (2000 GB)
Capabilities:
    LBA, IORDY(can be disabled)
    Queue depth: 32
    Standby timer values: spec'd by Standard, with device specific minimum
    R/W multiple sector transfer: Max = 16 Current = 0
    Recommended acoustic management value: 128, current value: 254
    DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
       Cycle time: min=120ns recommended=120ns
    PIO: pio0 pio1 pio2 pio3 pio4
       Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
    Enabled Supported:
      *  SMART feature set
        Security Mode feature set
      *  Power Management feature set
      *  Write cache
      *  Look-ahead
      *  Host Protected Area feature set
      *  WRITE_BUFFER command
      *  READ_BUFFER command
      *  NOP cmd
      *  DOWNLOAD_MICROCODE
        Power-Up In Standby feature set
      *  SET_FEATURES required to spinup after power up
        SET_MAX security extension
        Automatic Acoustic Management feature set
      *  48-bit Address feature set
      *  Device Configuration Overlay feature set
      *  Mandatory FLUSH_CACHE
      *  FLUSH_CACHE_EXT
      *  SMART error logging
      *  SMART self-test
      *  General Purpose Logging feature set
      *  64-bit World wide name
      *  WRITE_UNCORRECTABLE_EXT command
      *  {READ,WRITE}_DMA_EXT_GPL commands
      *  Segmented DOWNLOAD_MICROCODE
      *  SATA-I signaling speed (1.5Gb/s)
      *  SATA-II signaling speed (3.0Gb/s)
      *  unknown 76[3]
      *  Native Command Queueing (NCQ)
      *  Host-initiated interface power management
      *  Phy event counters
      *  unknown 76[12]
        DMA Setup Auto-Activate optimization
      *  Software settings preservation
      *  SMART Command Transport (SCT) feature set
      *  SCT Long Sector Access (AC1)
      *  SCT LBA Segment Access (AC2)
      *  SCT Features Control (AC4)
      *  SCT Data Tables (AC5)
        unknown 206[12] (vendor specific)
        unknown 206[13] (vendor specific)
Security:
    Master password revision code = 65534
        supported
    not   enabled
    not   locked
    not   frozen
    not   expired: security count
        supported: enhanced erase
    302min for SECURITY ERASE UNIT. 302min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50014ee6ab9e2501
    NAA       : 5
    IEEE OUI    : 14ee
    Unique ID    : 6ab9e2501
Checksum: correct
test$

[Snowhog]

Re: [solved with recent e2fsck] external usb disk won't mount, error

Well, according the Eagle Tech's own specs on the drive, it doesn't specify S.M.A.R.T., so I'd be inclined to believe what is being reported.

[doctordruidphd]

Re: [solved with recent e2fsck] external usb disk won't mount, error

I have seen this before with usb drives. Apparently the problem is with the usb passthrough.

take a look at the smartctl man page. you should also look at lsusb. Might need a command like:

sudo smartctl -a -d usbcypress /dev/sdc

edit: also, as was stated, if the drive itself doesn't support smart, it won't play with smartctl. I'm a little surprised that a current manufacture drive won't do that, but maybe.

[Telengard]

Re: [solved with recent e2fsck] external usb disk won't mount, error

So I still don't understand why hdparm thinks my drive does support, but smartctl doesn't. I'm inclined to believe this though:

I have seen this before with usb drives. Apparently the problem is with the usb passthrough.

Moving on from that ...

This drive came with a single pre-formatted NTFS partition. As a simple test, I made a recursive MD5 file of my /usr/ hierarchy. Then I copied the /usr/ hierarchy to the NTFS partition and checked the copy against the recursive MD5 file. It passed.

TestingStorageMedia - Community Ubuntu Documentation led me to Testing your hard drive in Linux. fsck isn't (yet) capable of testing or repairing NTFS filesystems.

I also found a reference to using e2fsck -c to write/read every block of a hdd*. I guess if that worked, then you would be confident.

What e2fsck -c does is to invoke badblocks to test blocks in the partition. badblocks is filesystem agnostic. All it needs to know is the block size and which blocks to test. It is capable of performing read-only tests, destructive-write tests, and non-destructive write tests.

Yesterday evening I issued the following command:

$ sudo badblocks -b 4096 -p 2 -c 16384 -s /dev/sdc1

I intended to post the results here, but it is still running. Perhaps a higher block count would have made it faster, but for now I intend to allow it to finish.

Maybe I'll try a destructive write test later with a larger block count to speed it up. After that I can reformat and make an EXT3 or EXT4 partition. What's the status of EXT4 on 8.04 Hardy?