[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Gmail to Dovecot failing due to disruption of connection


From abhi <abhi at littlewiki dot in>
Subject Gmail to Dovecot failing due to disruption of connection
Date Sat, 24 Apr 2010 15:03:43 +0530

Hi All,

I am running imapsync to sync my client's google apps email to our dovecot server. Here is the log one such server:

$RCSfile: imapsync,v $ $Revision: 1.286 $ $Date: 2009/07/24 15:53:04 $
Here is a [linux] system (Linux scs 2.6.26-2-686 #1 SMP Wed Feb 10 08:59:21 UTC 2010 i686)
with perl 5.10.0
Mail::IMAPClient 3.23
IO::Socket 1.30_01
IO::Socket::SSL
Digest::MD5 2.36_01
Digest::HMAC_MD5
Term::ReadKey
Date::Manip
and the module Mail::IMAPClient version used here is 3.23
Command line used:
/usr/bin/imapsync --host1 imap.gmail.com --ssl1 --user1 someuser at somegmail dot com --authmech1 LOGIN --password1 PWCHANGED --reconnectretry1 1 --host2 127.0.0.1 --user2 someuser --authuser2 someuser --authmech2 PLAIN --password2 PWCHANGED --reconnectretry2 1 --useheader Message-Id --skipsize --allowsizemismatch --subscribe --regextrans2 s/\[Gmail\]\.// --regextrans2 s/Sent Mail/Sent/ --regextrans2 s/Spam/Junk/
Turned ON syncinternaldates, will set the internal dates (arrival dates) on host2 same as host1.
TimeZone:[EST]
Will try to use LOGIN authentication on host1
Will try to use PLAIN authentication on host2
From imap server [imap.gmail.com] port [993] user [someuser at somegmail dot com]
To imap server [127.0.0.1] port [143] user [someuser]
Banner: * OK Gimap ready for requests from 12.199.191.194 26if7194711qwa.26
Host imap.gmail.com says it has NO CAPABILITY for AUTHENTICATE LOGIN
Success login on [imap.gmail.com] with user [someuser at somegmail dot com] auth [LOGIN]
Banner: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN AUTH=GSSAPI] Dovecot ready.
Host 127.0.0.1 says it has CAPABILITY for AUTHENTICATE PLAIN
Success login on [127.0.0.1] with user [someuser] auth [PLAIN]
host1: state Authenticated
host2: state Authenticated
Warning: ignoring folder [Gmail] because it is not selectable
From separator and prefix: [/][]
To separator and prefix: [.][]
++++ Calculating sizes ++++
From Folder [INBOX] Size: 55968095 Messages: 83
From Folder [[Gmail]/All Mail] Size: 55968095 Messages: 83
From Folder [[Gmail]/Drafts] Size: 0 Messages: 0
From Folder [[Gmail]/Sent Mail] Size: 0 Messages: 0
From Folder [[Gmail]/Spam] Size: 0 Messages: 0
From Folder [[Gmail]/Starred] Size: 0 Messages: 0
From Folder [[Gmail]/Trash] Size: 0 Messages: 0
Total size: 111936190
Total messages: 166
Time: 6 s
++++ Calculating sizes ++++
To Folder [All Mail] Size: 6888337 Messages: 81
To Folder [Drafts] Size: 0 Messages: 0
To Folder [INBOX] Size: 6895089 Messages: 83
To Folder [Junk] Size: 0 Messages: 0
To Folder [Sent] Size: 0 Messages: 0
To Folder [Starred] Size: 0 Messages: 0
To Folder [Trash] Size: 0 Messages: 0
Total size: 13783426
Total messages: 164
Time: 0 s
++++ Listing folders ++++
From folders list: [INBOX] [[Gmail]/All Mail] [[Gmail]/Drafts] [[Gmail]/Sent Mail] [[Gmail]/Spam] [[Gmail]/Starred] [[Gmail]/Trash]
To folders list: [All Mail] [Drafts] [INBOX] [Junk] [Sent] [Starred] [Trash]
++++ Looping on each folder ++++
From Folder [INBOX]
To Folder [INBOX]
Subscribing to folder INBOX on destination server
++++ From [INBOX] Parse 1 ++++
++++ To [INBOX] Parse 1 ++++
++++ Verifying [INBOX] -> [INBOX] ++++
+ NO msg #31 [0UtE8SLaHTupGXE19oQEPA] in INBOX
+ Copying msg #31:22752996 to folder INBOX
flags from: []["09-Jun-2009 14:15:28 -0500"]
Copied msg id [31] to folder INBOX msg id [85]
+ NO msg #37 [4Dsa71EC8TMvKOtMmUfa/g] in INBOX
+ Copying msg #37:26422318 to folder INBOX
Could not fetch message #37 from INBOX: socket closed while reading data from server (2x)
Time: 1572 s
From Folder [[Gmail]/All Mail]
To Folder [All Mail]
Subscribing to folder All Mail on destination server
++++ From [[Gmail]/All Mail] Parse 1 ++++
++++ To [All Mail] Parse 1 ++++
++++ Verifying [[Gmail]/All Mail] -> [All Mail] ++++
+ NO msg #31 [0UtE8SLaHTupGXE19oQEPA] in All Mail
+ Copying msg #31:22752996 to folder All Mail
Could not fetch message #31 from [Gmail]/All Mail: socket closed while reading data from server (2x)
+ NO msg #37 [4Dsa71EC8TMvKOtMmUfa/g] in All Mail
+ Copying msg #37:26422318 to folder All Mail
Could not fetch message #37 from [Gmail]/All Mail: socket closed while reading data from server (2x)
Time: 1782 s
From Folder [[Gmail]/Drafts]
To Folder [Drafts]
Subscribing to folder Drafts on destination server
++++ From [[Gmail]/Drafts] Parse 1 ++++
++++ To [Drafts] Parse 1 ++++
++++ Verifying [[Gmail]/Drafts] -> [Drafts] ++++
Time: 0 s
From Folder [[Gmail]/Sent Mail]
To Folder [Sent]
Subscribing to folder Sent on destination server
++++ From [[Gmail]/Sent Mail] Parse 1 ++++
++++ To [Sent] Parse 1 ++++
++++ Verifying [[Gmail]/Sent Mail] -> [Sent] ++++
Time: 0 s
From Folder [[Gmail]/Spam]
To Folder [Junk]
Subscribing to folder Junk on destination server
++++ From [[Gmail]/Spam] Parse 1 ++++
++++ To [Junk] Parse 1 ++++
++++ Verifying [[Gmail]/Spam] -> [Junk] ++++
Time: 1 s
From Folder [[Gmail]/Starred]
To Folder [Starred]
Subscribing to folder Starred on destination server
++++ From [[Gmail]/Starred] Parse 1 ++++
++++ To [Starred] Parse 1 ++++
++++ Verifying [[Gmail]/Starred] -> [Starred] ++++
Time: 0 s
From Folder [[Gmail]/Trash]
To Folder [Trash]
Subscribing to folder Trash on destination server
++++ From [[Gmail]/Trash] Parse 1 ++++
++++ To [Trash] Parse 1 ++++
++++ Verifying [[Gmail]/Trash] -> [Trash] ++++
Time: 0 s
++++ End looping on each folder ++++
++++ Statistics ++++
Time: 3362 sec
Messages transferred : 1
Messages skipped : 162
Total bytes transferred: 22752996
Total bytes skipped : 13585562
Total bytes error : 75597632
Detected 3 errors


Happy with this free, open and gratis GPL software?
Please, thank the author (Gilles LAMIRAL) by giving him a book:
http://www.amazon.com/gp/registry/wishlist/1C9UNDIH3P7R7/
or rate imapsync at http://freshmeat.net/projects/imapsync/


As you can see, messages with UID 31 and 37 failed with a connection error. So I tried manually downloading them using python. This is what I did:


import imaplib2
[...]
M.lsub()
M.select("[Gmail]/All Mail")
M.uid('FETCH', '30', '(RFC822 FLAGS INTERNALDATE)')

The messages appears to be downloading and then after some time(5 mins) it just stops and the library keeps on polling for data. Can anyone confirm if this is a bug with Gmail's IMAP and if there is something that I could do to download these mails without running an entire sync.

I can post the debug log of imaplib2 if needed.

Regards,
Abhi