Re: [imapsync] Re: Mail-IMAPClient-3.22 just released on CPAN

[Gilles LAMIRAL <gilles dot lamiral at laposte dot net>]

Hello Phil,

> I've still got a few big ideas for imapsync if you're
> up for a few 

Yes. No problem when they pass all the no-regression tests.

> (handling case insensitivity of folders on the destination for example).

 --regextrans2 's,(.*),\U$1,'

or other regex can do nice things with case, you're better at regex than
me, don't hesitate to give them us.

>>>> Just an unbelievable decision about Socket() function.

> Agreed, changing the public API was not a good choice and I would
> have preferred that Socket remain as it was.

Good.

> > The solutions are so easy to avoid such breaks:
> ...
> > - Name the new (ugly) Socket with any other name than Socket,
> >    there is an infinite number of names to pick up. Better.
> >    And fully document it and respect what is documented.
>
> At this point, I not convinced it would be beneficial to go back to
> the old name.  What do you think? 

I am convinced it would be beneficial. Why?
Who can now use the Socket() function in several contexts by just reading
the documentation? No one. Who can now use it in several contexts
by reading the code without a headache? No one.

Just two good ways for Socket(), a) or b):

a) Back with old and good code (no code at all in fact) with old and
good documentation.

b) Keep the code as is. Never change anything from now.
Remove the Socket() from the API documentation
by saying "Obfuscated code since 3.xx you're on your own to read
and understand the code by yourself. Good luck."

Don't worry, a soon patch to imapsync will be to remove all calls to
Socket. So you can decide what to do with Socket by yourself.
Now I've understood the issue (two years later) I won't accept
any imapsync patch with Socket calls.

> Changing things again would break
> things for anyone now using the 3.x behavior.

No one can use Socket() properly in various situations.
Don't make me think again 3.xx is still in beta stage
and want to stay there.

> I am happy to discuss
> but it doesn't seem like much of a win now that imapsync has already
> worked around it.

I want to remove ugly code for clarity.

> > > The biggest issue I see is that the docs are *so* wordy that it's
> > > hard to follow the fact that new can call connect which...

I want to remove ugly documentation too (for clarity).


> My goal is to respect the API, I'm willing to discuss reverting to
> the 2.x documented behaviour but that would probably mean we'd have
> to bump the version to 4.x I think and we'd be causing pain for anyone
> that wrote to the documented 3.x behaviour (in particular Socket -- not
> sure if there are any other major changes off the top of my head).

Who?
Who wants more headache?

Maybe for bad laziness the previous Mail-IMAPClient maintainer didn't
add any regression test before refactoring. Ok. So imapsync could be the
regression test. I told him that not all my regression passed,
where and why. He ignored (arguing it was my development environment)
and promoted 3.xx official robust successor. It wasn't robust.
It wasn't during more than two years.
I'm still more confident with 2.xx than 3.xx so I'll keep 2.xx during
more than two years from now, and also because I can run all the
62 functional tests successfully with both.

Just have a look at the stable distributions running everywhere.
What are the Mail-IMAPClient releases?

Ubuntu
hardy    3.04 (Long Time Support) no comment.
intrepid 3.08
jaunty   3.12
karmic   3.19

Debian Lenny 3.08 (no comment)

OpenBSD 2.2.9 (they contacted me, they listened my beta stage argument)

FreeBSD 3.23 but you're lucky it was 3.08 for 20 months 5 days ago :-)
http://www.freebsd.org/cgi/cvsweb.cgi/ports/mail/p5-Mail-IMAPClient/

I can continue with any stable Unix used in production environment
by thousand users and sysadmins.

The development and distribution cycle is such that when one lunches
a  crap perl Mail-IMAPClient library in 2007 then his code is still
alive in stable distributions 3 years later (the only way to reduce
this time is to release with security bugs). It's one of the reasons
I use regression tests.

When we release a program now it'll be used too years later in production
work for several years. I have this in mind every time I publish code.
The imapsync Makefile enforces that all regression tests pass before
I publish. I never cross my fingers when I publish,
I cross my fingers when I run imapsync tests with new code.

So in 2012, when all stable Unix distributions will have a robust
3.xx >=3.23, I'll get rid of the good old 2.2.9.

> > Of course, you can add any new functions and document them,
> > I will respect that.
>
> Great, my hope is that eventually we can git rid of all the copied code
> you have in imapsync from Mail::IMAPClient.  And ideally you (and others)
> can use the library without having to work around any issues.  

I disagree for two reasons:

1) few weeks ago we still found a functional bug in 3.xx that
wasn't in 2.xx (expunge).

2) Not a single line of the copied code is used when imapsync is called
   with 3.xx

3) I had so much problem with the underlying library that I even
envisaged to change:

http://search.cpan.org/dist/Net-IMAP-Client/
http://search.cpan.org/~robm/Mail-IMAPTalk/

> I think
> we've come a long way, but I know you won't be able to do that completely
> without dropping support for Mail::IMAPClient 2.x altogether due to
> numerous outstanding bugs in that code.

No known bug for imapsync with 2.2.x I can't fix easily.

> > In fact, from a user point of view, the good way would be:
> >
> > $imap->SSL(1); # The only thing to write to be SSL
> > $imap->TLS(1), # The only thing to write to be TLS
>
> With the latest (3.22) release, if you specify Ssl => 1 OR Starttls => 1
> to new() then it should do the right thing.  Also true if you do
> $imap->Ssl(1) OR $imap->Starttls(1) before connect/login so we are
> close to what you want I think.  I believe Starttls is a better name
> due to the fact that it maps to actual IMAP command that we want to
> use (the method that implements this is starttls() right now).

Yes Starttls is a good name.

> > May be we have to think more about the TLS API since
> > it's a "can do?" thing.
>
> Is there something more/missing from this discussion?  Are there
> technical concerns?  Or is this more an issue of clearly naming the
> attrs/methods?

Starttls can start at any time in the imap protocol.

> I would like you to be able to take advantage of starttls in
> Mail::IMAPClient.  FWIW, I think the starttls implementation in
> the latest imapsync is slightly broken.  Specifically, this:
>
>   unless($txt =~ /^STARTTLS OK/){
>     die "Invalid response for STARTTLS: $txt\n";
>   }
>
> The regex on the response should probably similar to just /^OK/
> per rfc2595 section 3.1.

Ok. Thanks.

  print $socket "STARTTLS\015\012";
  my $txt = $socket->getline();
  $debug and print "Read: $txt";
  unless($txt =~ /^STARTTLS OK/){
    die "Invalid response for STARTTLS: $txt\n";
  }

The issue is that the $txt gets back the whole
"STARTTLS OK Begin SSL/TLS negotiation now."

A direct connect gives the same thing:

$ telnet l 143
Trying 192.168.222.1...
Connected to l.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA AUTH=PLAIN CRAM-MD5 CRAM-SHA1 IDLE STARTTLS] Courier-IMAP ready. Copyright 1998-2005 Double Precision, Inc.  See COPYING for distribution information.
STARTTLS
STARTTLS OK Begin SSL/TLS negotiation now.

The error is above:

  print $socket "STARTTLS\015\012";

I don't know if it is RFC compliant but it is real world compliant
with the imap servers I use (and certainly with the ones of the user
who gave the copied patch).

Fixed with:

  print $socket "z00 STARTTLS\015\012";


--
Au revoir,                               02 99 64 31 77
Gilles Lamiral. France, Chavagne (35310) 06 20 79 76 06