Announcement

Collapse
No announcement yet.

Major NTFS corruption issue

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Major NTFS corruption issue

    Kubuntu release: 19.04
    How is Kubuntu installed: Dual-booted
    KDE version: KDE Plasma 5.15.4
    Grub version: Grub2
    Other OS: Windows 10 Professional

    PC Info:
    Laptop or Desktop: Laptop
    BIOS setting: UEFI, Secure Boot disabled
    CPU: Intel Core i3-5015U @ 2.10GHz (4 cores)
    GPU: Intel Corporation HD Graphics 5500 (rev 09)
    RAM: 8GB (7.7GB available, 330MB Shared)
    Optical drvies: 1 HITACHI LG DATA STORAGE DVD+/-RW, 8X
    HD:
    Internal: 1TB Internal SATA HDD
    External (In this case): 1 1TB WD Internal HDD (NTFS) attached via USB3 SATA Drive enclosure

    Though I've used Linux before, my experience with it is still VERY NEW, especially when it comes to this...

    I've used this external drive several times on a desktop that runs Windows 10 almost exclusively.

    Any files written to the disk via Windows 10 on any device were easily able to be copied back off the disk with no recovery effort of any kind.

    Any files written to the disk using the linux laptop were corrupted, and required me to run "chkdsk x: /r" to run recovery.

    On stage 4 (Looking for bad clusters in user file data...) literally every single file written by linux has resulted in "Windows replaced bad clusters in file (file number) of name (file name)"

    I've run it multiple times, recovering several files, then interrupting the process to copy those recovered files.

    This is a VERY tedious process, and I'm on the last legs, with some rather large files left on the disk. Currently chkdsk claims it's 1% of the way through current stage 4 (having been running for DAYS), and current ETA is over 444 hours (not a typo) for maybe 100-200GB worth of data at most.

    When I attached the drive, Linux mounted it automatically, and gave zero errors or troubles, when writing the files to the drive.

    When copying files off the drive in Windows, the only files that did not require chkdsk stage 4 recovery were those written to the disk by Windows. All other files are encountering logical cluster corruption (eg FS corruption that is not physical damage)

    Is there a specific way I'm supposed to mount NTFS partitions to ensure that writing to the disk in Linux does not corrupt those files during writing? I'd really like to avoid this problem in the future, as waiting more than a week to recover from this kind of corruption is something no one has time for.

    Thanks in advance for any help.

    #2
    Hi
    thank you for posing such an interesting question.
    I do not have "an answer" but can offer an anecdote which I have often posted here and at other fora.

    I run Kubuntu exclusively but have to take files to my colloege to use on their latest and greates and continually updated Windows(tm) machines.

    the files are all "presentations" which are made with LibreOffice, formerly Open Office. I DO HAVE a few presentations which were made with Powerpoint(tm) which I purchased specifically to use in my FIRST LECTURE... it is a very ...Tech centric campus" Biotechnology mostly, and I teach introductory courses. ( I will come back to this later).

    SEVERAL YEARS ago... when the whole ".docx" thing appeared the presentations which had FOR SEVERAL YEARS opened JUST FINE were demed by Microsoft(tm) to be "corrupted".

    This gets complicated but bear with me.

    The files were being kept ON MY "drive" on the college's learning management system.

    They were in the ".doc" format.

    I opened the presentation for the first lecture by clicking on it and a message appeared to the effect of:

    "Windows has determined that this is a corrupted file and has replaced some of the slides with a blank page".

    Okaaayyyyy...

    So I am going down through the file and maybe a third of the slides were "WHITE". ...well that kind of made ME FREAKING MAD!!!

    So, since I was familiar with what was on the slides, generally, it was introductory stuff about metric system (SI) and scientific method etc. and

    later went tot he office and BLEW UP and got a contact person at IT and told them what happened and they said something like "sorry, this is new to us, maybe use Windows(tm) you can get it for a reduced educator price ...whereupon I said that I DO have windows(tm) and Powerpoint(tm) thank you very much and hung up.

    I then borrowed a cd from the secretary, that kind of says how long ago this was and copied the file to it and took it home.

    And...what was it YOU POSTED? IT RAN FINE...

    So...I actually LOOKED at the file propertiers and in the properties there was a script... an .xml script...hmmm

    I got the SAME FILE that was ON MY HOME MACHINE...and looked at the file properties and the HOME FILE was SMALLER than the file that was on the computer at the college.

    MS HAD LOADED A SCRIPT INTO THE file...

    So...that was when I really started RESEARCHING ".xml".

    The script is just a few lines. not much at all.

    Windows(tm) has not "replaced the corrupted slides" with "a blank slide"...the script PLACES A WHITE SQUARE ...over the slide...

    Lets pause... when you get time please look around on the net for "how can I repair a Powerpoint(tm) file?"

    There are a LOT of sites that will offer to have you upload your file and, for a fee will very kindly remove the "white slides" and somehow MAGICALLY... have the original slide appear...

    imagine that...

    they just remove the script, and charge you money for it.

    So... I then went about removing the script...since I am the "owner" I can do that and actually drove back to the college with it on a usb stick and popped into the computer and opened it up and WHAT!!! THE...fudge..." same message only different slides were corrupted ... hmmm

    so...back home I go...i got the original file and burned it to a cd...

    Drove BACK to the college popped it into a cd tray, opened it and there it was in all of it's PRISTINE glory...

    Moral to the story... if MS cannot write to the file it cannot install the .xml script and will run it.

    The same thing ALSO happened with a music file only in a different way...the first semester I would take in a usb stick of tracks from a cd that I had copied to play before class. MS Media player would not play it AT ALL, would not even say that it saw the file.

    Okaaayyyyy... I burned the file to a cd and took it in...same thing would not even recognize it...

    okaaayyyyy...i took in the original cd and it played fine...

    moral to the story...at that stage MS was doing everything in it's power to make money by ONLY playing something that was on a copy protected medium.

    Nowadays...VLC will play any file that I take in and the college puts VLC onto all the machines of instructors who ask for it.

    So...to finish this very sad tale of woe...

    MS is probably just trying to bugger with NTFS... and how you would "remove" the offending scripts without having ANOTHER script to remove the scripts I don't know...

    This has not been any help I am sure but at least you know that someone else has encountered the problem ... and the comments from the Linux community have always been a thunderous SILENCE...

    BUT...there IS a P.S.

    And I think that this has to do with "non published" agreements that ensued from the settlement with the DOJ back under...Clinton? The whole thing about putting RealPlayer(tm) on a Windows(tm desktop...

    About two semesters ago...BECAUSE I actually TEST FOR ALL OF THIs...I know...the Linux people think that i'm just stupid...

    MS will now EASILY OPEN a .odp file...with NO complaints... because...,..it MAY be...that MS had to agree to GRADUALLY start running open documents...could not CORRUPT THEM...but ...nothing wrong with Open Document fonts not running on a MS system...PROPERLY...

    I mean the text is there but just way honking TOO BIG and the images are moved...

    It (the Powerpoint {tm) on the college computer "tries to render" the fonts" and the "spacing" of an image in a slide in a way that makes the font way to BIG and slides the images over...

    So... lol...I know...THIS IS WAAAY TOO lONG...

    I now do only ARIAL for font... and keep the "sentences' very short... so that they do not bleed out of a frame on the slide and RESIZE all images to a certain size and then lock the size and position of each and every image...

    AND THEN...I lock the file itself to READ ONLY and lol...just...lol... .odp works just fine!!!

    PPS..

    this is the "later" part. I tested all of this with AN ORIGINAL FILE made with ppt 2012 and guess what...the Windblows computers at the college say that it is a corrupt file...lol...because they want everybody to be running ONLY the newest Windblows...

    And... PPPPSSS they are moving to a subscription only system...pay now...pay later...and later and later... lol

    Again, I apologize for not having an answer.

    woodsmoke
    Last edited by woodsmoke; Aug 11, 2019, 08:45 PM.

    Comment


      #3
      Firstly, are you familiar with "fast boot" on Windows? (These days by default Windows does not fully shut down, and leaves NTFS in a not so good state as seen by Linux; this can result in Windows seeing changes made by Linux as corruption.)

      I've never had the trouble you refer too, when using an NTFS formatted external drive. But I suggest you have a read of
      this reddit thread, particularly the last post.
      ...444 hours...
      (That sounds like the drive is getting USB 1 speeds.) If you've got anything important on that drive, I'd be worried about it, and I suggest ensuring you've got another copy somewhere else. I suspect that you'll have grief with that drive until you reformat it.
      Regards, John Little

      Comment


        #4
        oops! I forgot to mention that when the drive is attached to the Widows Desktop, it is attached directly as an internal, through a hot swap bay, plugged into the motherboard's SATA port. When copying repaired files, or files that were not corrupted, it gets normal access for being an internal disk.

        So the chkdsk ETA is being generated in that environment.

        File copies would fail with "Cannot read file" in Windows, or CRC errors in a dos prompt, until corrected with chkdsk.

        I also don't ever actually shutdown windows. I restart it

        EDIT: Although my actual question wasn't answered, i found a solution that will work instead. Since Linux doesn't have support for journaling on NTFS (which appears to be the cause of this problem), I'm switching to fat32, which doesn't even use journaling. Since I'm not dealing with any files over 4GB, this will be enough. I do hope someone will resolve the issues with Linux, and NTFS though, since most modern versions of Windows use it.
        Last edited by jasoncollege24; Aug 12, 2019, 07:32 AM.

        Comment


          #5
          NTFS is a horrible file system so I wouldn't expect anyone in the Linux developer world to rush to support it. Personally, I've not had any corruptions when forced to use NTFS with my Linux installs so maybe there's another cause on your end or there's a larger problem like jlittle suggests.

          Do you have NTFS-3G installed? NTFS-3G supports partial NTFS journaling.

          A better solution might be to run Windows in virtual machine and avoid most of the Windows related deficiencies.

          Please Read Me

          Comment


            #6
            The laptop isn't good enough to run windows in a VM, though that is a good suggestion. I didn't even know about NTFS-3G, until after I had this problem. I was using the built-in NTFS support. Even NTFS-3G runs the same risk, and I'm not risking my files anymore. If I have to move files from one OS to the other, I'll either use the network (which crashes on the laptop pretty consistently using SMB), or externals formatted as FAT32, which have no journalling to begin with.

            The drive itself is perfectly fine. if it weren't, chkdsk (or some other utility) would tell me there were bad sectors, and any files written by windows were readable with zero issues. I think this was the kernal's NTFS support that broke it. After I chkdsk eventually finishes with stage 4, I'll get my files, and format as FAT32.

            Comment

            Working...
            X