Re: [imapsync] usecache option => different results?

[Gilles LAMIRAL <gilles dot lamiral at laposte dot net>]

Hi Michele,

> sorry for my late answer, but since the migration I had few time to dedicate to your project.

No problem. Users and developers spending time after a job is done are 
really really rare. It's a pity since our brain is full of really good 
things at this time of the project. No stress, best knowledge on the job 
and on tools. The best time to improve and consolidate tools, 
documentation and knowledge. The bonus is without too much extra time.

> Today I could test successfully the *new* cache feature that you corrected in 1.404 ;-)

Thanks.

> > The --usecache should work and be efficient on the first run on Exchange now with 1.404 release.
> Here you say that the imapsync "should be efficient on the first run"... you maybe meant the *second* run. 

Yes you're right. Before it became efficient on the third run.
In fact I meant the cache wasn't updated on the first run, unless 
--useuid option be used.

> Here my results:
> 1st run:
> Transfer time                     : 2885 sec
> Messages transferred              : 27346 
> Messages skipped                  : 2481
> Messages found duplicate on host1 : 234
> Messages found duplicate on host2 : 0
> Messages deleted on host2         : 0
> Detected 1 errors

It takes time because messages are transfered.

> 2nd run:
> Transfer time                     : 60 sec
> Messages transferred              : 243 
> Messages skipped                  : 29584
> Messages found duplicate on host1 : 7
> Messages found duplicate on host2 : 0
> Messages deleted on host2         : 0
> Detected 1 errors
>
> 3rd run:
> Transfer time                     : 45 sec
> Messages transferred              : 5 
> Messages skipped                  : 29822
> Messages found duplicate on host1 : 2
> Messages found duplicate on host2 : 0
> Messages deleted on host2         : 0
> Detected 1 errors
>
> 4th run:
> Transfer time                     : 42 sec
> Messages transferred              : 2 
> Messages skipped                  : 29825
> Messages found duplicate on host1 : 0
> Messages found duplicate on host2 : 0
> Messages deleted on host2         : 0
> Detected 1 errors
>
> By the way: Host1 is now "offline" and it doesn't get any new mails...
> so I'm wondering why almost all values (ignoring "Transfer time")
> are different for the run 2 to 4 ;-)

I think it comes from the duplicates on host1, they ended to be 
transfered. The cache is used to avoid getting the header again and 
again. Without header for messages in the cache (the cache caches uids, 
not header) imapsync only detects the "triplicates" on second run and 
the "quadriplicates" on third run etc. So imapsync transfers duplicate 
messages on second run because it can't guess (for now, we may consider 
this a bug) that a twin message is already transfered (with only its uid 
in the cache).

The --useuid option might solve this issue. --useuid ignore what is a 
duplicate message, it transfers every message and then record that 
transfer. It may be the default behavior one day.

Maintaining a good cache is really a tough job, I knew that and it's why 
imapsync was stateless by design.

The good thing of the current cache code is that the "backup and restore 
on files" is less far now. imapsync can cache message content with just 
one additional line of code. The issues come from flags cache and restore.

Thanks again for your feedback.

--
Au revoir,                             09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06