Can't remove Western Digital "SmartWare"

[claydoh]

Hmmm I recently bought a WD MyBook Essential 1tb external drive, but I for the life of me can't recall how I got rid of the Smartware partition. I know I did not use a livecd - it is an external drive, why need to do that? I could have used dd to zero out the entire drive or nuke the partition table, or simply used partitionmanager or gparted to ditch everything. I am assuming it isn't really gone, the drive shows as 917gb, my guess is that it should be 930 or so.

I recall somewhere that some recent updates to the firmware or software make it possible to delete it.

Still racking my head to remember what I did.

[Snowhog]

Commentary on "how to" do it from 2010, by Dedoimedo, which sheds light on the issue, and solutions.

How to remove hidden Virtual CD (VCD) partitions on your Western Digital external disks

[Feathers McGraw]

That's the solution I said I tried (post 13), unfortunately it didn't work for me .

Feathers

[SteveRiley]

WD Smartware still appears.

Managed to hide it using the Windows tool, the "SmartWare" virtual CD doesn't show up any more.

I have a suspicion that in your earlier post, the image was simply showing a left-over label of some kind. I have seen this when I've dded an ISO to a USB and then subsequently wiped out the partitions and the partition table -- the ISO's label remains.

Can you "unhide" that stuff? If so, it might be interesting to investigate it a bit more by showing the output of these two commands:

lsblk

sudo blkid

both before and after attaching the external drive.

[Feathers McGraw]

Yeah, will try that as soon as I can (staying away from home atm, so don't have access to the drive).

Just out of interest, can anyone explain how you can have a virtual CD in the firmware that doesn't take up any space on the drive? (can you)? If so, where is the info stored? Some kind of secondary storage that can't be accessed when you plug it into a computer?

Feathers

[Feathers McGraw]

OK I unhid the virtual CD again. Here is the output before the drive is attached:

feathers-mcgraw@62-West-Wallaby-Street:~$ lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda       8:0    0 465.8G  0 disk 
├─sda1    8:1    0   100M  0 part 
├─sda2    8:2    0 139.9G  0 part 
├─sda3    8:3    0     1K  0 part 
├─sda5    8:5    0  11.2G  0 part [SWAP]
├─sda6    8:6    0 139.7G  0 part /data
├─sda7    8:7    0    28G  0 part /home
├─sda8    8:8    0  23.3G  0 part 
├─sda9    8:9    0  24.4G  0 part /
└─sda10   8:10   0  24.4G  0 part 
sr0      11:0    1  1024M  0 rom  
feathers-mcgraw@62-West-Wallaby-Street:~$ sudo blkid
[sudo] password for feathers-mcgraw: 
/dev/sda1: LABEL="System Reserved" UUID="6C209CC7209C99A2" TYPE="ntfs" 
/dev/sda2: UUID="44329FBA329FB002" TYPE="ntfs" 
/dev/sda5: UUID="e68dea6c-f7d3-4414-9382-6c2eb0db0c29" TYPE="swap" 
/dev/sda6: LABEL="Data" UUID="f052e620-9e88-4b19-9b9a-d6dce4d6603b" TYPE="ext4" 
/dev/sda7: LABEL="Home" UUID="83c84ef2-3402-40bd-b8ac-f50c4891f8b3" TYPE="ext4" 
/dev/sda8: UUID="0de58072-6c77-404a-8c1f-6acd4d8fb36d" TYPE="ext4" 
/dev/sda9: LABEL="Kubuntu_13_04_R" UUID="910b7db8-b3c8-4a90-b3da-ac1eba45c148" TYPE="ext4" 
/dev/sda10: LABEL="MINT" UUID="911768e5-9188-41d3-8e38-00dfca00145c" TYPE="ext4"

And here is the output afterwards:

feathers-mcgraw@62-West-Wallaby-Street:~$ lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda       8:0    0 465.8G  0 disk 
├─sda1    8:1    0   100M  0 part 
├─sda2    8:2    0 139.9G  0 part 
├─sda3    8:3    0     1K  0 part 
├─sda5    8:5    0  11.2G  0 part [SWAP]
├─sda6    8:6    0 139.7G  0 part /data
├─sda7    8:7    0    28G  0 part /home
├─sda8    8:8    0  23.3G  0 part 
├─sda9    8:9    0  24.4G  0 part /
└─sda10   8:10   0  24.4G  0 part 
sdb       8:16   0 465.1G  0 disk 
└─sdb1    8:17   0 465.1G  0 part 
sr0      11:0    1  1024M  0 rom  
sr1      11:1    1   668M  0 rom  
feathers-mcgraw@62-West-Wallaby-Street:~$ sudo blkid
/dev/sda1: LABEL="System Reserved" UUID="6C209CC7209C99A2" TYPE="ntfs"                                                                                                                          
/dev/sda2: UUID="44329FBA329FB002" TYPE="ntfs"                                                                                                                                                  
/dev/sda5: UUID="e68dea6c-f7d3-4414-9382-6c2eb0db0c29" TYPE="swap"                                                                                                                              
/dev/sda6: LABEL="Data" UUID="f052e620-9e88-4b19-9b9a-d6dce4d6603b" TYPE="ext4"                                                                                                                 
/dev/sda7: LABEL="Home" UUID="83c84ef2-3402-40bd-b8ac-f50c4891f8b3" TYPE="ext4" 
/dev/sda8: UUID="0de58072-6c77-404a-8c1f-6acd4d8fb36d" TYPE="ext4" 
/dev/sda9: LABEL="Kubuntu_13_04_R" UUID="910b7db8-b3c8-4a90-b3da-ac1eba45c148" TYPE="ext4" 
/dev/sda10: LABEL="MINT" UUID="911768e5-9188-41d3-8e38-00dfca00145c" TYPE="ext4" 
/dev/sr1: LABEL="WD SmartWare" TYPE="udf" 
/dev/sdb1: UUID="38B65955B659152A" TYPE="ntfs"

Thanks for your interest, would love to get to the bottom of this!

Feathers

[Buddlespit]

From what I read on the WD forums, Smartware can be a) uninstalled, but no directions on how to do that or 2) wipe the space using the HP USB tool, which I have no ides if that'll work on Linux. Someone else suggested formatting the drive as a primary partition instead of an extended partition, which will force Smartware to ignore the drive and not install itself on the machine it's plugged into.

[Feathers McGraw]

Thanks for your thoughts,

2) wipe the space using the HP USB tool, which I have no ides if that'll work on Linux.

This keeps coming up. It's in the solution I mentioned and linked in post 13 (http://www.dedoimedo.com/computers/passport-vcd.html) that I tried and unfortunately didn't work. When I say I tried it, I mean I used the HP USB tool on Windows (so it's not a WINE problem). The rest of the link is a good read though.

Someone else suggested formatting the drive as a primary partition instead of an extended partition, which will force Smartware to ignore the drive and not install itself on the machine it's plugged into.

Were they talking about linux or windows? The smartware virtual CD contains some windows executables and autorun stuff that would be a pain in Windows, but doesn't have any effect on linux because it won't run anyway.

Feathers

[Buddlespit]

I think they were talking about windows, but I don't see why you couldn't format it ntfs thru kparted

[Feathers McGraw]

My point was that although windows users may be trying to avoid auto-run features, they don't affect Linux because they won't run (which is good).

Formatting the drive NTFS would still leave the virtual CD there

Feathers

[SteveRiley]

We know that the drive, /dev/sdb, already contains a single primary partition, /dev/sdb1. If it were extended, we'd see two: /dev/sdbX for the extended partition container and /dev/sdbY for the virtual partition within.

Here's what I think: Smartware takes up zero space on the actual disk. Note that the sizes of the disk and its partition are the same. In most cases, we can suss out this stuff using the tools we've had you try here. But nothing's working. My supposition is that Smartware lives entirely within firmware. This firmware can be activated ("unhidden") or deactivated ("hidden") using that Windows utility. When activated, it advertises a CD-ROM -- here, /dev/sr1. When deactivated, that CD-ROM doesn't appear.

If you're feeling adventurous, try removing the drive from its housing and placing it in some other USB enclosure. If Smartware disappears completely, then my suspicion is confirmed: Smartware lives in firmware contained on a circuit board in the original enclosure. If you still see Smartware, then we haven't confirmed my suspicion -- Smartware could still be in firmware, but this would be on the controller board attached to the drive.

[Feathers McGraw]

Thanks for that, it's nice to get a better understanding of what's happening. I wonder how (or if) that HP tool really did ever have an effect on other drives if the virtual CD is in the firmware?

If you're feeling adventurous, try removing the drive from its housing and placing it in some other USB enclosure. If Smartware disappears completely, then my suspicion is confirmed: Smartware lives in firmware contained on a circuit board in the original enclosure. If you still see Smartware, then we haven't confirmed my suspicion -- Smartware could still be in firmware, but this would be on the controller board attached to the drive.

The disk is a passport drive, not sure if I could take it apart without damaging it. What exactly do you mean by putting it in another USB enclosure?

Feathers

[SteveRiley]

I wonder how (or if) that HP tool really did ever have an effect on other drives if the virtual CD is in the firmware?

The HP tool works if the virtual CD is actually taking space on the disk. Since the tool isn't working for you, I consider this to be a clue that the Smartware virtual CD lies elsewhere.

The disk is a passport drive, not sure if I could take it apart without damaging it. What exactly do you mean by putting it in another USB enclosure?

I mean exactly that. Get out your small screwdriver kit (you do have one, right? lol) and dismantle the Passport. Inside you will find a standard 2.5" laptop drive attached to a small circuit board. Here's a representative sample:



Search YouTube for recordings of how other people have pried open the Passport. It's possible, but it'll feel like you're going to demolish it. With care, you can open it without damage.

External enclosures are readily available; see NewEgg's list.

[Feathers McGraw]

Yeah, don't worry I have several mini screwdriver sets!

Screw it, I've bought a new enclosure (from ebay, newegg is a US-only thing unfortunately). Only cost a few nug.

Will report back, even if I fail miserably!

More generally, what does the firmware actually do? Was it only included to give the "extra" functionality, or do all hard drives have some firmware to enable the drive to be read etc.? Is there a "standard" firmware that everyone uses and adds to?

How do the manufacturers make sure it's always compatible with Windows, Mac and Linux - are there three sets of firmware or does it not matter in the same way that a browser on any platform can display a web page regardless of what software the web server is running?

Feathers

[SteveRiley]

The firmware on the drive's controller mediates between the operating system's command to "Read file foo" and the necessary mechanical activity to perform that function. The firmware defines the interface (IDE, SCSI, SATA, SAS, Fibre Channel, others) and the transfer rate. Interface types are standard and are documented, and require drivers. The driver sits between the firmware and the operating system; the OS issues generic commands that the driver translates into specific commands appropriate for the interface type.

Firmware is software on a chip -- usually an EEPROM. The only way device manufacturers can differentiate themselves is through software. Smartware is one such example. Their firmware, in addition to performing the required functions for the drive, also has additional code that causes the operating system to detect a CD-ROM. Now, how can this be possible, since there really is no CD-ROM drive in the case? That's the beauty of software abstractions. From the perspective of the OS, a CD-ROM drive is anything that appears on the system bus and exhibits the necessary characteristics to be detected as such. Smartware is software (in the firmware, remember) that advertises an IDE controller attached to a CD-ROM drive containing an ISO 9660 file system.

There is only one firmware. But since everything is standards-based, all operating systems can support it (so long as drivers are present, of course -- it's unlikely that you'll find Windows NT drivers for SAS disks, I suspect.)