Re: [imapsync] Reducing the resident memory per imapsync process

[Gilles LAMIRAL <gilles dot lamiral at laposte dot net>]

Hello David,

> I'm using imapsync 1.255 to move 238,000 users accounts.

Nice job !

I won't support 1.255 with patches.
1.267 is the current public release.
1.267 won't stay current for the next week.
So keep 1.255 if it works for you.
I don't think 1.267 is better than 1.255 for memory management.

> I started using imapsync on a maildrop with 35 simultaneous 
> processes but this took the maildrop down after an hour or two.

Are you from localhost to localhost ?

> I've moved imapsync to it own machine and testing with 15 connections. 
> It's been running for 6 hours without a glitch.

What is imapsync best bandwith use ?

> The reason the maildrop goes down is because with users with large 
> mailboxes, the resident memory for the process can be in the 
> neighborhood of 500 meg to 1 Gig.

Take a big mailbox.
How many messages in a big mailbox ?
What is the total size of a big mailbox ?
What is the average message size in a big mailbox ?
Are you on FreeBSD ?

I ask thoses questions you to start big memory tests,
compare, make statistics and find where is the problem.

> I'm wondering if changing the Clear (command history) in
> the IMAP::Client Module from 20 to 0 will break anything
> with imapsync?

I think you can.

> It seems to reduce the resident memory at least 100 Meg.

Good. Many users complain about memory and a TODO item is to
think about reducing memory consomption.  May be I should
set Clear(0) by default.  In my memory I played with that a
long time ago but something like the initial banner broke, I
don't remember very well.

If it works it works so go on.

> Is there anything else I can do to lower the memory overhead per process?

What is your command line ?
--useheader is nice to reduce memory and time.

> I'd like to use imapsync with 50 to 100 connections.

imapsync was designed to work with a bad slow connection in
a linear migration, and without memory or speed
consideration, I've only used it with one process, really.
You're on a freeride land.

-- 
Au revoir,                               02 99 64 31 77
Gilles Lamiral. France, Chavagne (35310) 06 20 79 76 06