[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [imapsync] usecache option => different results?


From Ameir Abdeldayem <ameirh at gmail dot com>
Subject Re: [imapsync] usecache option => different results?
Date Thu, 31 Mar 2011 13:38:27 -0400

I can confirm that I receive the same errors when I use --useuid.  I narrowed it down to that parameter having issues.  The errors seem to be more prominent when the source mailbox is large.  This was tested from Gmail and AOL.

-Ameir

On Wed, Mar 30, 2011 at 9:57 AM, Michele Marcionelli <michele dot marcionelli at math dot ethz dot ch> wrote:
Hi Gilles

I tried with the option --useuid and without --useheader (if I understood correctly, this option is obsolete if one uses the --useuid option) but I've go a tons of error messages like this one:

Use of uninitialized value $h1_flags in substitution (s///) at /usr/local64.hg/app/imapsync/current/bin/imapsync line 2374.
Use of uninitialized value $flags in split at /usr/local64.hg/app/imapsync/current/bin/imapsync line 2256.
Use of uninitialized value $_[0] in sort at /usr/local64.hg/app/imapsync/current/bin/imapsync line 1300.
Use of uninitialized value $_[0] in sort at /usr/local64.hg/app/imapsync/current/bin/imapsync line 1300.
- msg INBOX/333 couldn't append  (Subject:[]) to folder INBOX: Error sending '523 APPEND INBOX () {0}': 523 BAD Command Argument Error. 12

and even after 14227 seconds the sync wasn't finish.

Any idea?

Michele


On Mar 29, 2011, at 20:48 , Gilles LAMIRAL wrote:

> Hi Michele,
>
>> sorry for my late answer, but since the migration I had few time to dedicate to your project.
>
> No problem. Users and developers spending time after a job is done are really really rare. It's a pity since our brain is full of really good things at this time of the project. No stress, best knowledge on the job and on tools. The best time to improve and consolidate tools, documentation and knowledge. The bonus is without too much extra time.
>
>> Today I could test successfully the *new* cache feature that you corrected in 1.404 ;-)
>
> Thanks.
>
>>> The --usecache should work and be efficient on the first run on Exchange now with 1.404 release.
>> Here you say that the imapsync "should be efficient on the first run"... you maybe meant the *second* run.
>
> Yes you're right. Before it became efficient on the third run.
> In fact I meant the cache wasn't updated on the first run, unless --useuid option be used.
>
>> Here my results:
>> 1st run:
>> Transfer time                     : 2885 sec
>> Messages transferred              : 27346 Messages skipped                  : 2481
>> Messages found duplicate on host1 : 234
>> Messages found duplicate on host2 : 0
>> Messages deleted on host2         : 0
>> Detected 1 errors
>
> It takes time because messages are transfered.
>
>> 2nd run:
>> Transfer time                     : 60 sec
>> Messages transferred              : 243 Messages skipped                  : 29584
>> Messages found duplicate on host1 : 7
>> Messages found duplicate on host2 : 0
>> Messages deleted on host2         : 0
>> Detected 1 errors
>> 3rd run:
>> Transfer time                     : 45 sec
>> Messages transferred              : 5 Messages skipped                  : 29822
>> Messages found duplicate on host1 : 2
>> Messages found duplicate on host2 : 0
>> Messages deleted on host2         : 0
>> Detected 1 errors
>> 4th run:
>> Transfer time                     : 42 sec
>> Messages transferred              : 2 Messages skipped                  : 29825
>> Messages found duplicate on host1 : 0
>> Messages found duplicate on host2 : 0
>> Messages deleted on host2         : 0
>> Detected 1 errors
>> By the way: Host1 is now "offline" and it doesn't get any new mails...
> > so I'm wondering why almost all values (ignoring "Transfer time")
> > are different for the run 2 to 4 ;-)
>
> I think it comes from the duplicates on host1, they ended to be transfered. The cache is used to avoid getting the header again and again. Without header for messages in the cache (the cache caches uids, not header) imapsync only detects the "triplicates" on second run and the "quadriplicates" on third run etc. So imapsync transfers duplicate messages on second run because it can't guess (for now, we may consider this a bug) that a twin message is already transfered (with only its uid in the cache).
>
> The --useuid option might solve this issue. --useuid ignore what is a duplicate message, it transfers every message and then record that transfer. It may be the default behavior one day.
>
> Maintaining a good cache is really a tough job, I knew that and it's why imapsync was stateless by design.
>
> The good thing of the current cache code is that the "backup and restore on files" is less far now. imapsync can cache message content with just one additional line of code. The issues come from flags cache and restore.
>
> Thanks again for your feedback.
>
> --
> Au revoir,                             09 51 84 42 42
> Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06

--
Michele Marcionelli · mm at ethz dot ch · +41 44 631 6193



---- imapsync mailing-list ----
unsubscribe, mailto:imapsync-unsubscribe at listes dot linux-france dot org
imapsync,    http://linux-france.org/prj/imapsync/