Issues with sync when updating existing files

FTP client for Android handsets.

Issues with sync when updating existing files

Postby Achillion » Sat Nov 29, 2014 4:10 pm

Greetings.

I'm a new AndFTP user and I'm loving it. Perfect solution for managing server storage via sftp. I especially love the key auth feature, which is either missing on other apps, or is buggy and unusable.

I noticed that when I use the sync feature to synchronise two directories (from server to phone), the synchronisation doesn't happen properly when a file that already exists on the phone needs to be updated. There are two issues here:

(a) New version of file is downloaded incorrectly.
(b) Changes to a file that keep the filesize the same are not detected.

Allow me to elaborate on how these issues presented and how I investigated, so you can tell me if I understood something incorrectly or if there's another, underlying issue.

(a)
1 - Set up a connection to my server (tested and working) and in the SYNC tab, I set it up as follows:
Code: Select all
Sync type:  Mirror remote
Local dir:  /storage/emulated/0/tmp
Remote dir: /home/achilleas/tmp

2 - Create two files on the server, in $HOME/tmp. One plain text file (easy to check with an editor) and a bigger random file (for testing larger files):
Code: Select all
$ echo "foo" > ~/tmp/textfile
$ dd if=/dev/urandom of=$HOME/tmp/randfile
5+0 records in
5+0 records out
5242880 bytes (5.2 MB) copied, 0.295744 s, 17.7 MB/s

3 - Check md5 hashes:
Code: Select all
$ md5sum ~/tmp/*
7b23e08d9489c452351a6cebfdb8c76b  /home/achilleas/tmp/randfile
d3b07384d113edec49eaa6238ad5ff00  /home/achilleas/tmp/textfile

4 - Run sync on phone in AndFTP (long press connection and select sync):
5 - Check local files on phone (using adb shell and builtin md5 binary):
Code: Select all
$ adb shell cat /sdcard/tmp/textfile
foo
$ adb shell md5 /sdcard/tmp/randfile
7b23e08d9489c452351a6cebfdb8c76b  /sdcard/tmp/randfile
$ adb shell md5 /sdcard/tmp/textfile
d3b07384d113edec49eaa6238ad5ff00  /sdcard/tmp/textfile

All good so far.
6 - Now let's change the files on the server
Code: Select all
$ echo "editing textfile" > ~/tmp/textfile
$ dd if=/dev/urandom of=$HOME/tmp/randfile bs=1M count=2
7+0 records in
7+0 records out
7340032 bytes (7.3 MB) copied, 0.418478 s, 17.5 MB/s

7 - Get the hashes
Code: Select all
$ md5sum ~/tmp/*
f9c2ad2151d115bdd2a9d467c5ba0eba  /home/achilleas/tmp/randfile
b616a369c332fb5feaebf8bda45b6ff3  /home/achilleas/tmp/textfile

8 - Run the sync operation on the phone again: 2 file changes detected and both are downloaded.
9 - Check the files on the phone:
Code: Select all
$ adb shell cat /sdcard/tmp/textfile
foo
ing textfile
$ adb shell md5 /sdcard/tmp/randfile
8956672ec49b5778e5e40fbc91392831  /sdcard/tmp/randfile
$ adb shell md5 /sdcard/tmp/textfile
0326c6a90dd344d11659fbe212ddcbb9  /sdcard/tmp/textfile


(b)
1 - For the second issue, we simply change the files on the server, but keep the same exact file size.
Code: Select all
$ echo "3diting_t3xtfil3" > ~/tmp/textfile
$ dd if=/dev/urandom of=$HOME/tmp/randfile bs=1M count=7
7+0 records in
7+0 records out
7340032 bytes (7.3 MB) copied, 0.523481 s, 14.0 MB/s

2 - Running the sync operation on the phone claims that there are no differences between the local folder and the remote folder.

It's pretty clear that the sync operation tests for file sizes alone to see whether files have changed. I don't think this is good practice and it should use, perhaps, timestamps in addition to the file size.
As for the first issue, it appears that the sync operation only downloads the extra bytes that were created since the last sync operation. This is bad since binary files get corrupted and even for plain text files, there's no guarantee that no other bytes before the end of the file were changed.

Sorry for the (very) long post; I like being thorough :)
Thanks for a great app and I hope this gets resolved, or at least helps identify some issues with the sync function.
Achillion
 
Posts: 2
Joined: Sat Nov 29, 2014 3:29 pm

Re: Issues with sync when updating existing files

Postby support » Wed Dec 03, 2014 7:36 pm

Did you select filename + filesize + newest timestamp in Sync option? Default is filename + filesize only.
support
 
Posts: 1010
Joined: Sun Apr 20, 2008 4:40 pm

Re: Issues with sync when updating existing files

Postby Achillion » Wed Dec 03, 2014 7:45 pm

support wrote:Did you select filename + filesize + newest timestamp in Sync option? Default is filename + filesize only.


Well, turns out I'm not as thorough as I claim to be. I hadn't noticed the option in the general settings. That was a good waste of time on my part. Thanks! :)

As for the other issue, it turns out it's due to "resume support", so it works fine when disabled.

Everything is lovely now.
Achillion
 
Posts: 2
Joined: Sat Nov 29, 2014 3:29 pm

Re: Issues with sync when updating existing files

Postby support » Thu Dec 18, 2014 12:40 pm

Great!
support
 
Posts: 1010
Joined: Sun Apr 20, 2008 4:40 pm


Return to AndFTP



cron

Sitemap | Privacy Statement | Java and all Java-based marks are trademarks or registered trademarks of Sun Microsystems, Inc.
in the U.S. and other countries. All other company and/or product names are the property of their respective owners.