Announcement

Collapse
No announcement yet.

Alternatives to the Cloud

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Alternatives to the Cloud

    Many people and corporations are putting their data into servers hosted on the the Internet. Some, like Amazon and Google, have opted to use Linux servers. Others, like GoDaddy moved their parked domains from Linux to Windows, a dubious moved designed to bolster Microsoft's Internet server share at a time when Linux was moving up. Currently, UNIX is running on 67% of all Internet hosts. Of the Unix type OS's, 57% is running Linux and 2/3rds are running Ubuntu & Debian, and the remainder CentOS and others. That means that Linux runs 37% of all Internet servers and Windows powers 33%.

    Symantec reports that "... more than 430 million new unique pieces of malware in2015, up 36 percent from the year before." Zero day vulnerabilities, discovered at more than one per week, not counting what the NSA or CIA have, is becoming a commodity. "...Vulnerabilities can appear in almost any type of software,but the most attractive to targeted attackers is softwarethat is widely used. Again and again, the majority of thesevulnerabilities are discovered in software such as InternetExplorer ...". Translation: Windows. It is, however, no secret that the vast majority of all malware is launched from Internet websites running Windows. "More than 75 percent of alllegitimate websites have unpatched vulnerabilities. Fifteenpercent of legitimate websites have vulnerabilities deemed‘critical,’ which means it takes trivial effort for cybercriminalsto gain access and manipulate these sites for theirown purposes."

    With the release of Valut 7 it is only going to get worse for systems running Windows or using Windows compatibility layers. It is such layers that account of most of the vulnerabilities found in the Mac OSX.

    I used to use Google cloud to store my data. For other reasons I recovered my data and deleted my Google account.

    Two years ago I had only 750Gb along with two WD 350Gb USB drives, onto which I regularly dragged and dropped my home account to to backup my data. This was not entirely satisfactory because sockets, links and devices don't transfer well in a Dolphin d&d operation. It is also a pain, even when being retired and with plenty of time on my hands , to pull out the WD USB drives and figure out which one I should drop my data on to. Sometimes I had to delete previous copies to make room. I often wondered if I had deleted something, deliberately or inadvertently, that I might want later.

    I looked at software which automatically synced files to 2nd drives and/or remote locations, like a server on my LAN. An example is Syncthing. This statement didn't set well with me: "Each device scans for changes every 60 seconds...". I didn't need any more file indexing taking place.

    I can't remember if I first heard about Btrfs from one of Oshunluver's posts, or from an Internet source, but the idea of COW (copy of write) beats file scanning looking for changes to backup. When the file system writes to the disk it automatically updates any snapshots, making file scanning unnecessary. A snapshot of @ or @home copies everything, files, links, sockets, devices, etc. Using send & receive to move snapshots to storage devices completed the process.

    I first formatted the USB drives as Btrfs devices and mounted them as /backup. Then I'd take ro snapshots and then send them to the /backup HD. That went on for a couple years. Then my sda device started throwing sector errors so I bought two 750 Gb HDs and a CDROM caddy in which I could mount the 2nd HD. When I attempted that I discovered that a) the caddy wasn't working and b) there was a second bay in my laptop! So, I mounted the new drive as sda and the old drive as sdb and re-installed KDE Neon User Edition. Recovering my data from the 320Gb HD was easy because my entire data stash was 90Gb, Then I read a post by Oshunluver about his RAID setup and I set up my two HDs as a RAID1 pair. Perfect! Mirroring data, metadata and system data as RAID1. I got help from the ORINCO caddy support about a switch which allowed the caddy to work, so I put the other new drive into it and formatted it as Btrfs but did not add it to my pool. Strangely, when I plugged it in and turned on my laptop what was sdb became sdc and the 3rd drive became sdb. So, before adding the 3rd HD the sda was the main HD and sdb was the mirror HD. After adding the 3rd drive sda was still the main drive but sdc was now the mirror. The 3rd HD was sdb. The UUIDs kept /etc/fstab from getting confused. Now I can keep several copies of @ & @home snapshots on /backup (sdb) but most of the time it remains unmounted. If the old drive (now sdb) begins to fail I can use Btrfs procedures for replacing it without losing any data.

    My Btrfs pool is now my cloud.
    Last edited by GreyGeek; May 18, 2017, 12:07 PM.
    "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
    – John F. Kennedy, February 26, 1962.

    #2
    Some time ago I read this blog about Butter FS. All the pros didn't impress upon me any need to jump from Extended 4 to Btrfs. I don't cloud at all. I always have start a new Linux PC / Laptop with a partition of my HDD. With the increased size of these drives, I have a minimum of 40 GB for a linux OS used for recovery of the main system should something fail. As recent as a month ago on my laptop I messed with something and boned the boot of my 16.04 into the X session. I rebooted and went to my 40 GB partition (4% of my total drive space) that boots 14.04 and I started searching my lost X Session issue. Took me 2 hours but I recovered without data loss. Needless to say, the biggest threat to my laptop is me.

    Don't read me wrong, I encourage backups, I normally backup very important documents to DVD/Blue Ray disks. My issue with clouds (call me paranoid) you are handing your data off to a stranger. I don't care if you told me any of the following;
    • I paid them not to look.
    • I encrypted my files.
    • I trust the company.


    I would sooner hand my set of DVD backups to the homeless men down at the local shelter.
    Free storage sounds nice on the surface but not for me. My issues are just that, my issues. I really don't need to pass my issues off to total strangers LOL.

    Comment


      #3
      Wow, in terms of discussing Btrfs's pros & cons that article is OLD (July 28, 2012) !!!
      There have been huge changes, additions and improvements since then.

      One feature I like is being able to mount (again) a live Btrfs system and work on it. I.E., you have installed your system on /dev/sda1, for example, and your are currently running it. You open a konsole and issue
      sudo mount /dev/sda1 /mnt
      and that reveals the ROOTFS of your Btrfs system. You can make a snapshot (copy) of @ a/o @home while running it and send that copy to an external drive (external to your pool) as either a binary file or an ascii file. The snapshot is created almost instantly. Because it is COW (Copy On Write) as you subsequently modify a file the fs writes a copy of the original file to the snapshot. If an update or something you do messes up your system you but you can still log into it you can rename @ to @old, @home to @homeold, and mv the @copy to @ and the @homecopy to @home, umount /mnt and reboot. You have your old system back. You can also do incremental snapshots and restore from them.

      I discovered that my laptop had two HD bays so I put an identical HD in the second bay and while running live in my system I converted my singleton on /dev/sda1 to RAID1 on a pool composed of sda1 and sdb1, with data, meta data and system data all as RAID1. Instant backup! If either sda1 or sdb1 begins to fail I can tell Btrfs to degrade the failing drive, add a replacement drive to the Btrfs pool, and rebalance the drives. Btrfs will automatically replace the failing drive with the new drive and do what is necessary. Then I can shut down, pull the bad drive and put the replacement in its bay and power back up. The Btrfs uses UUIDs to know which drives are the primary and the mirror. If an update brought down my xserver or the printer I could copy the affected files/directories from the @bkup or @homebkup from /mnt and then log out and back in and that has always fixed stuff. In two years I have never had to actually roll back to a previous @bkup or @homebkup snapshot in their entirety to recover my system. It has been very stable.

      IMO, while EXT4 has been an excellent filesystem, Btrfs is so far above it in capabilities that I won't consider returning to EXT4 or using a distro that doesn't offer Btrfs as a normal option during the install. With EXT4 I have to use some sort of file syncing system. I usually mounted one of my WD USB HD's to /backup and then used Dolphin as root to drag & drop my home account to the backup drive. Links, sockets and some other files never copied, but I was only interested in my documents, pdfs, games and such.
      Last edited by GreyGeek; May 18, 2017, 06:57 PM.
      "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
      – John F. Kennedy, February 26, 1962.

      Comment


        #4
        btrfs needs some friendly gui for file system management. backup/restore optiosn should be easy to setup perhaps at default install using some sane and esilly changeable defaults?

        Comment


          #5
          There are some GUI's available for apps like Btrbk and I've even thought about writing my own GUI but then the process is so repetitive that a single script would work. Snapper and other packages even automate the process so no GUI is needed.

          However, I find those apps to be overkill. The process is easy:
          sudo. -i
          mount /dev/sda1 /mnt
          mount /dev/sdb1 /backup
          btrfs su snapshot. -ro /mnt/@ /mnt/snapshots/@.yyyymmdd
          btrfs su snapshot -ro /mnt/@home /mnt/snapshots/@home.yyyymmdd
          btrfs fi send /mnt/snapshots/@.yyyymmdd | btrfs receive /backup
          btrfs fi send /mnt/snapshots/@home.yyyymmdd | btrfs receive /backup
          umount /backup
          umount /mnt
          exit
          exit

          So, for me a script fits much better for backing up than a GUI. It's why most still use a konsole to do software updates instead of muon, or worse, Discover. It's like driving your car to your mailbox at the end of your driveway. Doing an incremental backup involves adding only a "-p" switch. Rolling back to a snapshot is drop dead easy as well.


          Sent from my iPhone using Tapatalk
          "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
          – John F. Kennedy, February 26, 1962.

          Comment


            #6
            btrfs is great for when you just want to back up data on a single system.

            Just wanted to mention Nextcloud, which open source server software for sharing calendars, contacts and files across multiple devices running different operating systems.
            samhobbs.co.uk

            Comment


              #7
              This is tangential to the main conversation, but thought I'd mention it.

              The college whereat I instruct has moved the "e-mail" function to where one goes "through the Microsoft Cloud" BEFORE one gets to the supposed, I guess, Outlook function in the college servers themselves. It all now has the "new and improved - blank / neutered" look of all of the MS stuff.

              However two situations occurred:

              a) it is SLOW...perceptibly slow to get anything done. Whether working at the college OR from home. This may be the college's backend or the cloud do not know.

              BUT

              b) here is the "worrisome" thing.

              Working from home on Kubuntu and cable, normally not a problem.

              I just finished trying to submit end of semester documents, as an attachment to an e-mail within the college's system.

              One .doc and three .pdf.

              When trying to attacch I could only see ONE of the .pdf that was in a "folder". Curiously it was a .pdf from LAST SEMESTER which I had merely retitled, ( faculty info) for the spring.

              However, WHEN I MOVED the files out of the folder into HOME, I could see all of them. There WAS a "lag" in displaying symbols, such as the .pdf red symbol.

              I was, indeed, able to attach them and send the e-mail with verification of delivery.

              So, this "could be" Kubuntu.

              Or it could be "Kubuntu interacting with MS Cloud".

              Do NOT know...just thought I'd mention it.

              woodsmoke
              Last edited by woodsmoke; May 19, 2017, 04:29 PM.

              Comment


                #8
                Originally posted by Feathers McGraw View Post
                btrfs is great for when you just want to back up data on a single system.

                ....
                Indeed, but running on a system used by multiple users, like a remote server, it is even better.
                https://seravo.fi/2016/perfect-btrfs-setup-for-a-server
                Individual home accounts can be put onto subvolumes and snapshots taken of them independently. A rollback of one user's account without touching the other users, or requiring them to logout, is as easy as rolling back the @home account of a single user system.
                Last edited by GreyGeek; May 19, 2017, 07:58 PM.
                "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                – John F. Kennedy, February 26, 1962.

                Comment


                  #9
                  Originally posted by GreyGeek View Post
                  Wow, in terms of discussing Btrfs's pros & cons that article is OLD (July 28, 2012) !!!
                  There have been huge changes, additions and improvements since then.
                  ...
                  Like I said, I had read about this file system "long ago", but still I use the ext4. Btrfs will always be there, if the need arises. I know, I get stuck in old methods. I am just cautious about moving too far too quick. I will always be the caboose when it comes to upgrading.

                  Comment


                    #10
                    Originally posted by Simon View Post
                    ... I will always be the caboose when it comes to upgrading.

                    Be not the first by whom the new are tried,
                    Nor yet the last to lay the old aside.Alexander Pope, An Essay on Criticism, 1711
                    English poet & satirist (1688 - 1744)

                    "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                    – John F. Kennedy, February 26, 1962.

                    Comment


                      #11
                      Originally posted by GreyGeek View Post
                      It's why most still use a konsole to do software updates instead of muon, or worse, Discover.
                      they probably use it because they don't always work so well.

                      average user would still prefer a GUI. many people use smart phones these days, few know about terminal/console access on them. i doubt so many people would buy them if most things were done using terminal.

                      Comment


                        #12
                        Originally posted by mastablasta View Post
                        they probably use it because they don't always work so well.

                        average user would still prefer a GUI. many people use smart phones these days, few know about terminal/console access on them. i doubt so many people would buy them if most things were done using terminal.
                        True! For me, anyway. The older I get the more I prefer GUI's.
                        "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                        – John F. Kennedy, February 26, 1962.

                        Comment


                          #13
                          I can't pass this thread over without making a couple comments: EXT4 is a stop-gap temporary file system - this is the opinion of the EXT4 developer, not mine. BTRFS does far more than just provide easy backups. IMO, that's not even the top benefit.

                          I totally agree we need some solid GUI tools to introduce more people to BTRFS. The biggest problem I have with any GUI is your are hamstrung to the developer's idea of how it should work and what feedback you get - case in point: Muon Discover.

                          Simon, I know for a fact you are both Linux savvy enough to convert to BTRFS if you wanted to - without a GUI, and frankly I can't believe you would rest your opinion of an emerging new technology like BTRFS on a 5 year old blog!

                          To the point: BTRFS is "unstable" in the Linux sense - meaning development is ongoing. This means it's improving - tools mostly, but even more. RAID 5/6 will be available with kernel 4.12 for example. The file system format is "stable" meaning future changes will not require a wipe-and-re-format. However, EXT4 is stable - as in no more development. Being cautious is one thing, but missing out of ease of use and many other advantages are another. The 2.6 series kernels are great, but I wouldn't recommend using it just because it's stable

                          Finally, Jerry - I'm stealing that quote from you and putting it in my signature!

                          Please Read Me

                          Comment


                            #14
                            One more comment; there's another thread on here where the author complained that mdadm is not included by default in the ISO. mdadm is another example of holding onto an old technology that simply isn't needed any more - in most cases at least. BTRFS does most forms of RAID pretty well and a lot easier than any mdadm setup I've ever dealt with. However, there are use-cases where BTRFS doesn't quite fill the bill yet, but that's why we have so many file system choices.

                            Please Read Me

                            Comment


                              #15
                              I'll freely admit that I still use ext4 on my installs. I know that it's 'long in the tooth', but it works and I know how to use it. Frankly, contemplating a switch to btrfs makes me nervous. It's an unfounded nervousness I know, but...

                              When I back up my two laptops, I do so using Clonezilla and do device-partition to device-partition backups to external HDDs. This works for me.
                              Windows no longer obstructs my view.
                              Using Kubuntu Linux since March 23, 2007.
                              "It is a capital mistake to theorize before one has data." - Sherlock Holmes

                              Comment

                              Working...
                              X