Announcement

Collapse
No announcement yet.

wget or rsync

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    wget or rsync

    I am writing a bash script/cronjob that will check a remote file url once a day and grab the file if it differs from the one currently in my dropbox folder. It seems wget or rsync would both work for this task, but I am curious to see which tool is better for the job in my specific case(and generally speaking as well) and why? The remote file is under 10 MB and considering it is an android app, I do not see it growing to over 10 MB any time soon unless some crazy radical new feature gets added. Even then, again, we are talking about an android app, so the size should remain relatively small in any case.
    Last edited by Xplorer4x4; May 11, 2013, 06:15 PM.
    OS: Kubuntu 12.10/Windows 8
    CPU: Intel Core i7 2600K
    Motherboard: Gigabyte GA-Z77X-UD5H
    Memory: 2x4GB Corsair Dominator
    Graphics Card: MSI R7770
    Monitor: Dell 2208WFP
    Mouse: Mionix NAOS 5000
    PSU: Corsair 520HX
    Case: Thermaltake Mozart TX
    Cooling: Thermalright TRUE Black Ultra-120 eXtreme CPU Heatsink Rev C
    Hard Drives: 1x180 GB Intel 330 SSD - 1xWD 1 TB Caviar Black - 1xWD 2 TB Caviar Green - 2xWD 3 TB Caviar Green

    #2
    I'm curious...wouldn't Dropbox's own sync feature take care of that for you?

    Comment


      #3
      Not that I am aware of. Dropbox can grab the file via an http url and upload it to dropbox?

      On further examination, it doesn’t seem rsync can retrieve a file via http can it? Did I miss something here as well?
      OS: Kubuntu 12.10/Windows 8
      CPU: Intel Core i7 2600K
      Motherboard: Gigabyte GA-Z77X-UD5H
      Memory: 2x4GB Corsair Dominator
      Graphics Card: MSI R7770
      Monitor: Dell 2208WFP
      Mouse: Mionix NAOS 5000
      PSU: Corsair 520HX
      Case: Thermaltake Mozart TX
      Cooling: Thermalright TRUE Black Ultra-120 eXtreme CPU Heatsink Rev C
      Hard Drives: 1x180 GB Intel 330 SSD - 1xWD 1 TB Caviar Black - 1xWD 2 TB Caviar Green - 2xWD 3 TB Caviar Green

      Comment


        #4
        Sorry -- I fell victim of reading your post too quickly. I see now that you're fetching from someplace that is not Dropbox-aware.

        rsync, on its own, can't use HTTP. However, zsync can. I suspect this will work for you. Its advantage over wget is that it implements the rsync protocol inside HTTP, meaning it downloads only the changed portions of files. wget can't do that. Plus, it's in the *buntu repository.

        Code:
        steve@t520:~$ [B]apt-cache show zsync[/B]
        Package: zsync
        Priority: optional
        Section: universe/net
        Installed-Size: 245
        Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
        Original-Maintainer: Robert Lemmen <robertle@semistable.com>
        Architecture: amd64
        Version: 0.6.2-1ubuntu1
        Depends: libc6 (>= 2.15)
        Filename: pool/universe/z/zsync/zsync_0.6.2-1ubuntu1_amd64.deb
        Size: 115036
        MD5sum: d3eb1d12ee24ab25080c7741e3e52be3
        SHA1: f07682cccbec4a2f13e7d1ce43f9c6437c3ef94f
        SHA256: d68fa56816e78028e9c383d39979664b86278c61e64f99096454e3c974a83e3c
        Description-en: client-side implementation of the rsync algorithm
         zsync is a file transfer program to download files from
         remote web servers. If a previous version of a file is available
         locally, zsync will only download changed parts and hereby
         minimise the download volume. The algorithm is the same as used
         by rsync(1), but zsync does not require any server software
         (apart from a web server), nor does it need shell access.
         Instead, it uses a control file (.zsync file) that describes the
         file to be downloaded, which it uses to determine the blocks to
         fetch. This file is created once on the server (and not for each
         request) and sits next to actual file to download
        Homepage: http://zsync.moria.org.uk/                                                                                                                                                     
        Description-md5: 55c7895466b4c20f2f56e9be9a5d33c4                                                                                                                                        
        Bugs: https://bugs.launchpad.net/ubuntu/+filebug                                                                                                                                         
        Origin: Ubuntu

        Comment


          #5
          Well I figure it is probably a good idea to keep the last 3 versions of apk in case the latest one or two is buggy. FYI, I am referring to Titanium Backup as the android app. Thanks for the tip on zsync though. Looking in to it now.
          OS: Kubuntu 12.10/Windows 8
          CPU: Intel Core i7 2600K
          Motherboard: Gigabyte GA-Z77X-UD5H
          Memory: 2x4GB Corsair Dominator
          Graphics Card: MSI R7770
          Monitor: Dell 2208WFP
          Mouse: Mionix NAOS 5000
          PSU: Corsair 520HX
          Case: Thermaltake Mozart TX
          Cooling: Thermalright TRUE Black Ultra-120 eXtreme CPU Heatsink Rev C
          Hard Drives: 1x180 GB Intel 330 SSD - 1xWD 1 TB Caviar Black - 1xWD 2 TB Caviar Green - 2xWD 3 TB Caviar Green

          Comment

          Working...
          X