Mdthbs

Why does smartdiagram replace the Greek letter xi by a number?

Getting UPS Power from One Room to Another

Do you have to have figures when playing D&D?

Was Self-modifying-code possible just using BASIC?

Did Apple bundle a specific monitor with the Apple II+ for schools?

Possible runaway argument using circuitikz

Electricity free spaceship

How to befriend someone who doesn't like to talk?

How to avoid typing 'git' at the begining of every Git command

How to safely destroy (a large quantity of) valid checks?

Why Does Mama Coco Look Old After Going to the Other World?

Can I utilise a baking stone to make crepes?

Should I refuse to be named as co-author of a low quality paper?

Why was this person allowed to become Grand Maester?

Why do radiation hardened IC packages often have long leads?

Is there a DSLR/mirorless camera with minimal options like a classic, simple SLR?

Can we completely replace inheritance using strategy pattern and dependency injection?

Increase speed altering column on large table to NON NULL

Do you need to let the DM know when you are multiclassing?

Why did Intel abandon unified CPU cache?

How can I remove material from this wood beam?

Is the use of umgeben in the passive unusual?

Proving that a Russian cryptographic standard is too structured

Is using 'echo' to display attacker-controlled data on the terminal dangerous?

rsync vs mtime and ctime

How to prevent `atime` in Linux from overwriting `Date created` in Windows on NTFS?Does rsync verify files copied between two local drives?Rsync keeps writingrsync redoing files?rsync and backup and changing timezonersync partial LVM volume to Remote DirectoryTo Rsync files where permission deniedDifferent hash value of large rsynced file on centos and ubuntu?Can rsync detect changes in owner/group or perms?What differences are between using `-u` and not with rsync?Why is rsync taking a long time on large files that already exist?

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}

1

I've been using rsync for Android to backup my phone to a remote NTFS filesystem on a Linux system for a while.

Recently, the HDD containing the NTFS filesystem has started to fail (or throw "I/O Errors") so I took the opportunity to copy all the files onto a new HDD and new NTFS filesystem. In this instance I used the "FastCopy v2.11" tool for Windows.

My problem is that when I do an rsync "dry run" I can see that it wants to recopy files which already exist on the remote rsync folder. For example, when I run with "-iv" I get this kind of output:

Which, as I understand it means that rsync wants to copy this file to the remote rsync because of a timestamp difference.

The strange thing is that if I use "Astro" for Android to look at the local file properties, I can see that the file's size, modified time, and MD5 checksum are exactly the same as that of the remote file (using ls -l to check the modified time).

Given that I recently copied the remote rsync files from an old NTFS filesystem, the remote file's ctime is different (using ls -lc).

Does rsync look at the remote ctime, and if so is there any way I can use rsync, or ntfs-3g to get around this problem?

rsync android ntfs

share|improve this question

edited Dec 11 '13 at 15:35

Anthon

62.7k17110173

asked Nov 10 '13 at 23:18

wombatwombat

62

bumped to the homepage by Community♦ 1 hour ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

add a comment | 

1

I've been using rsync for Android to backup my phone to a remote NTFS filesystem on a Linux system for a while.

Recently, the HDD containing the NTFS filesystem has started to fail (or throw "I/O Errors") so I took the opportunity to copy all the files onto a new HDD and new NTFS filesystem. In this instance I used the "FastCopy v2.11" tool for Windows.

My problem is that when I do an rsync "dry run" I can see that it wants to recopy files which already exist on the remote rsync folder. For example, when I run with "-iv" I get this kind of output:

Which, as I understand it means that rsync wants to copy this file to the remote rsync because of a timestamp difference.

The strange thing is that if I use "Astro" for Android to look at the local file properties, I can see that the file's size, modified time, and MD5 checksum are exactly the same as that of the remote file (using ls -l to check the modified time).

Given that I recently copied the remote rsync files from an old NTFS filesystem, the remote file's ctime is different (using ls -lc).

Does rsync look at the remote ctime, and if so is there any way I can use rsync, or ntfs-3g to get around this problem?

rsync android ntfs

share|improve this question

edited Dec 11 '13 at 15:35

Anthon

62.7k17110173

asked Nov 10 '13 at 23:18

wombatwombat

62

bumped to the homepage by Community♦ 1 hour ago

This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.

add a comment | 

I've been using rsync for Android to backup my phone to a remote NTFS filesystem on a Linux system for a while.

Recently, the HDD containing the NTFS filesystem has started to fail (or throw "I/O Errors") so I took the opportunity to copy all the files onto a new HDD and new NTFS filesystem. In this instance I used the "FastCopy v2.11" tool for Windows.

My problem is that when I do an rsync "dry run" I can see that it wants to recopy files which already exist on the remote rsync folder. For example, when I run with "-iv" I get this kind of output:

Which, as I understand it means that rsync wants to copy this file to the remote rsync because of a timestamp difference.

The strange thing is that if I use "Astro" for Android to look at the local file properties, I can see that the file's size, modified time, and MD5 checksum are exactly the same as that of the remote file (using ls -l to check the modified time).

Given that I recently copied the remote rsync files from an old NTFS filesystem, the remote file's ctime is different (using ls -lc).

Does rsync look at the remote ctime, and if so is there any way I can use rsync, or ntfs-3g to get around this problem?

rsync android ntfs

share|improve this question

edited Dec 11 '13 at 15:35

Anthon

62.7k17110173

asked Nov 10 '13 at 23:18

wombatwombat

62

share|improve this question

edited Dec 11 '13 at 15:35

Anthon

62.7k17110173

asked Nov 10 '13 at 23:18

wombatwombat

62

share|improve this question

edited Dec 11 '13 at 15:35

Anthon

62.7k17110173

asked Nov 10 '13 at 23:18

wombatwombat

62

edited Dec 11 '13 at 15:35

Anthon

62.7k17110173

edited Dec 11 '13 at 15:35

Anthon

62.7k17110173

asked Nov 10 '13 at 23:18

wombatwombat

62

asked Nov 10 '13 at 23:18

wombatwombat

62

2 Answers
2

active

oldest

votes

0

By default rsync relies on the mtime/ctime and size for file comparison, but if you use the -c flag it would ignore them and use content checksums.

The problem is this way can be much slower(those checksums might be really expensive to calculate on your mobile device) and you might need to always run it like this from now on, so it might make sense to just let it run once without checksums, let it do its thing so it copies everything once again based on the mtime/ctime/size comparison method it uses by default, but at least next times you won't spend time calculating checksums.

share|improve this answer

answered Nov 10 '13 at 23:29

Cristian Măgherușan-StanciuCristian Măgherușan-Stanciu

63946

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks Cristian. I had looked at the checksum option but wanted to explore other possibilities for the reason you mention. Especially given that the remote linux rsync host is a Raspberry Pi at the moment! Maybe I should temporarily mount both filesystems on a faster linux system (laptop/desktop) and then do a local rsync based on the checksum algorithm, and then return the filesystems to their respective places from there on...
  
  – wombat
  Nov 12 '13 at 10:37
  
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I guess the other option is to find or write a utility which can set my file ctime to the same value as the file mtime. Or maybe get the rsync source code and build my own version which can be instructed to only consider "size and mtime" in its quick check algorithm?
  
  – wombat
  Nov 12 '13 at 10:42
  

  
  
  
  

add a comment | 

0

I just did a quick experiment; rsync compares mtime and ignores ctime (on Mac, at least.) Unfortunately, Windows file systems only have ctime, which they return for atime and mtime as well.

This is why rsync is so aggressive about copying files from Windows file systems that don't need to be copied — the mtime on the Unix file is being compared to the "ctime" on the Windows file.

share|improve this answer

answered Jan 10 '17 at 22:03

Edward FalkEdward Falk

1,056712

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I am curious about that. What do you mean by "Windows file systems only have ctime, which they return for atime and mtime as well"? If one right-click on a file and go to properties, one will get all 3 different values, right? Or are there other situations where only one and the same value is returned? I am wondering if I have a related problem here: unix.stackexchange.com/questions/519547/…
  
  – The Quark
  May 21 at 21:03
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f100707%2frsync-vs-mtime-and-ctime%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

0

By default rsync relies on the mtime/ctime and size for file comparison, but if you use the -c flag it would ignore them and use content checksums.

The problem is this way can be much slower(those checksums might be really expensive to calculate on your mobile device) and you might need to always run it like this from now on, so it might make sense to just let it run once without checksums, let it do its thing so it copies everything once again based on the mtime/ctime/size comparison method it uses by default, but at least next times you won't spend time calculating checksums.

share|improve this answer

answered Nov 10 '13 at 23:29

Cristian Măgherușan-StanciuCristian Măgherușan-Stanciu

63946

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks Cristian. I had looked at the checksum option but wanted to explore other possibilities for the reason you mention. Especially given that the remote linux rsync host is a Raspberry Pi at the moment! Maybe I should temporarily mount both filesystems on a faster linux system (laptop/desktop) and then do a local rsync based on the checksum algorithm, and then return the filesystems to their respective places from there on...
  
  – wombat
  Nov 12 '13 at 10:37
  
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I guess the other option is to find or write a utility which can set my file ctime to the same value as the file mtime. Or maybe get the rsync source code and build my own version which can be instructed to only consider "size and mtime" in its quick check algorithm?
  
  – wombat
  Nov 12 '13 at 10:42
  

  
  
  
  

add a comment | 

0

By default rsync relies on the mtime/ctime and size for file comparison, but if you use the -c flag it would ignore them and use content checksums.

The problem is this way can be much slower(those checksums might be really expensive to calculate on your mobile device) and you might need to always run it like this from now on, so it might make sense to just let it run once without checksums, let it do its thing so it copies everything once again based on the mtime/ctime/size comparison method it uses by default, but at least next times you won't spend time calculating checksums.

share|improve this answer

answered Nov 10 '13 at 23:29

Cristian Măgherușan-StanciuCristian Măgherușan-Stanciu

63946

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks Cristian. I had looked at the checksum option but wanted to explore other possibilities for the reason you mention. Especially given that the remote linux rsync host is a Raspberry Pi at the moment! Maybe I should temporarily mount both filesystems on a faster linux system (laptop/desktop) and then do a local rsync based on the checksum algorithm, and then return the filesystems to their respective places from there on...
  
  – wombat
  Nov 12 '13 at 10:37
  
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I guess the other option is to find or write a utility which can set my file ctime to the same value as the file mtime. Or maybe get the rsync source code and build my own version which can be instructed to only consider "size and mtime" in its quick check algorithm?
  
  – wombat
  Nov 12 '13 at 10:42
  

  
  
  
  

add a comment | 

By default rsync relies on the mtime/ctime and size for file comparison, but if you use the -c flag it would ignore them and use content checksums.

The problem is this way can be much slower(those checksums might be really expensive to calculate on your mobile device) and you might need to always run it like this from now on, so it might make sense to just let it run once without checksums, let it do its thing so it copies everything once again based on the mtime/ctime/size comparison method it uses by default, but at least next times you won't spend time calculating checksums.

share|improve this answer

answered Nov 10 '13 at 23:29

Cristian Măgherușan-StanciuCristian Măgherușan-Stanciu

63946

share|improve this answer

answered Nov 10 '13 at 23:29

Cristian Măgherușan-StanciuCristian Măgherușan-Stanciu

63946

answered Nov 10 '13 at 23:29

Cristian Măgherușan-StanciuCristian Măgherușan-Stanciu

63946

answered Nov 10 '13 at 23:29

Cristian Măgherușan-StanciuCristian Măgherușan-Stanciu

63946

0

I just did a quick experiment; rsync compares mtime and ignores ctime (on Mac, at least.) Unfortunately, Windows file systems only have ctime, which they return for atime and mtime as well.

This is why rsync is so aggressive about copying files from Windows file systems that don't need to be copied — the mtime on the Unix file is being compared to the "ctime" on the Windows file.

share|improve this answer

answered Jan 10 '17 at 22:03

Edward FalkEdward Falk

1,056712

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I am curious about that. What do you mean by "Windows file systems only have ctime, which they return for atime and mtime as well"? If one right-click on a file and go to properties, one will get all 3 different values, right? Or are there other situations where only one and the same value is returned? I am wondering if I have a related problem here: unix.stackexchange.com/questions/519547/…
  
  – The Quark
  May 21 at 21:03
  

  
  
  
  

add a comment | 

0

I just did a quick experiment; rsync compares mtime and ignores ctime (on Mac, at least.) Unfortunately, Windows file systems only have ctime, which they return for atime and mtime as well.

This is why rsync is so aggressive about copying files from Windows file systems that don't need to be copied — the mtime on the Unix file is being compared to the "ctime" on the Windows file.

share|improve this answer

answered Jan 10 '17 at 22:03

Edward FalkEdward Falk

1,056712

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I am curious about that. What do you mean by "Windows file systems only have ctime, which they return for atime and mtime as well"? If one right-click on a file and go to properties, one will get all 3 different values, right? Or are there other situations where only one and the same value is returned? I am wondering if I have a related problem here: unix.stackexchange.com/questions/519547/…
  
  – The Quark
  May 21 at 21:03
  

  
  
  
  

add a comment | 

I just did a quick experiment; rsync compares mtime and ignores ctime (on Mac, at least.) Unfortunately, Windows file systems only have ctime, which they return for atime and mtime as well.

This is why rsync is so aggressive about copying files from Windows file systems that don't need to be copied — the mtime on the Unix file is being compared to the "ctime" on the Windows file.

share|improve this answer

answered Jan 10 '17 at 22:03

Edward FalkEdward Falk

1,056712

share|improve this answer

answered Jan 10 '17 at 22:03

Edward FalkEdward Falk

1,056712

answered Jan 10 '17 at 22:03

Edward FalkEdward Falk

1,056712

answered Jan 10 '17 at 22:03

Edward FalkEdward Falk

1,056712