[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [imapsync] Memory Usage


From Gilles LAMIRAL <gilles dot lamiral at laposte dot net>
Subject Re: [imapsync] Memory Usage
Date Thu, 11 Nov 2010 00:53:29 +0100

Hello Ameir,

> From my understanding, imapsync stores all the message headers in memory
> when iterating through folders before doing the actual sync.

No, just a md5 header signature.

> Is it possible to break this down to a user-specified set (e.g. store
> 500 emails' headers at a time)?  

No. 
It would be useless anyway.

> I understand that might slow things a
> tad bit, but memory can be a big problem on some systems for some people.  

I studied very close the memory consumption of imapsync.
I though like you think,  headers crunch memory or something else.
The memory consumption of imapsync is mostly taken by Mail-IMAPClient.
Where ? with the messages themselves.
Take the biggest message size of all folders, for example 15Mo,
then imapsync, ie Mail-IMAPClient, will crunch 15 times this 
size into memory about 225 Mo if imapsync has to transfer the message.
If this message is already transfered then imapsync will crunch 
only 15 Mo.

> I think the tricky part is comparing those 500 emails on the source to
> all the email on the target to avoid duplicates. 

No trick. It is simply impossible without a file cache.

> Thanks for the wonderful script!  I can see that it's definitely a
> lifesaver for many people.

Thanks.

-- 
Au revoir,                             09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06