Mail problems

[DoYouKubuntu]

Let me just address this for right now:

For the sake of correctness and security, change your outgoing server's port to 25/tcp, using STARTTLS connection security. I can't seem to force the server to tell me which authentication method it wants, so you'll need to experiment here.

I can, and did just now, change the outgoing server's port to 25, but I have no TCP choice; I set connection security to STARTTLS and then attempted to send mail. Nope:

Sending of message failed.
An error occurred sending mail: Unable to establish a secure link with SMTP server smtpauth.earthlink.net using STARTTLS since it doesn't advertise that feature. Switch off STARTTLS for that server or contact your service provider.

Leaving its port set at 25 but changing its connection security back to none, sending mail succeeds.

I changed "Authentication method" to "Encrypted password," left everything else as above, and sending mail succeeded. PLEASE NOTE that years ago I set SM's settings per Earthlink's instructions at that time, and using "Encrypted password" did not work, hence the reason I'd chosen "Password, transmitted insecurely." Really.

[SteveRiley]

You'll definitely want to use smtpauth.earthlink.net for outgoing mail:

steve@t520:~$ [B]telnet smtpauth.earthlink.net 587[/B]
Trying 207.69.189.208...
Connected to smtpauth.earthlink.net.
Escape character is '^]'.
220-elasmtp-kukur.atl.sa.earthlink.net ESMTP Exim 4.67 #1 Thu, 19 Feb 2015 17:02:19 -0500
220-NO UCE.  EarthLink does not authorize the use of its computers or network
220 equipment to accept, transmit, or distribute unsolicited e-mail.
[B]ehlo example.com[/B]
250-elasmtp-kukur.atl.sa.earthlink.net Hello example.com [63.224.42.106]
250-SIZE 29360128
250-PIPELINING
250-AUTH PLAIN LOGIN CRAM-MD5
250-STARTTLS
250 HELP
[B]^][/B]
telnet> [B]quit[/B]
Connection closed.

On port 587/tcp, using STARTTLS, with PLAIN authentication. This will protect your password.

BTW, proper port number specifications follow the format number/transport. transport is either tcp or udp depending on the protocol. That's why I almost always use this format, even though in configuration dialogs you typically see only the numbers in isolation.

[Feathers McGraw]

BTW, proper port number specifications follow the format number/transport. transport is either tcp or udp depending on the protocol. That's why I almost always use this format, even though in configuration dialogs you typically see only the numbers in isolation.

I've noticed you always do that. I didn't appreciate the reason why until I started experimenting with SIP and XMPP (basically trying to replace Hangouts with something free and federated) - some of those things can be configured to use TCP or UDP on the same port, which is something I hadn't come across before. SIP in particular seems to be more workable over UDP but of course you can't use TLS which is a bummer.

[SteveRiley]

over UDP but of course you can't use TLS which is a bummer.

Sure you can: Datagram Transport Layer Security, RFC 6347.

[Feathers McGraw]

@DYK, I'm sorry for threadjacking... it looks like you have your answer now but if not I'm sure we can move this discussion to another thread

@Steve...
Neat! I'll have to remember that, I don't remember seeing it as an option when I was looking into SIP. There's so much to learn, I was a bit overwhelmed by the plethora of options available for SIP at the time to be honest - I got as far as "you have to choose which method to use: TCP or UDP" and looking up the differences between the two. I didn't want to send the PW in plaintext without TLS, so that left me with TCP (or so I thought). Maybe I should have another go - I got quite far last time...

What I found with SIP is that it's easy to get diverted by guides that are setting up systems with some kind of video transcoding, whereas what I really wanted was a server-side program that could just route everything for me and let the clients decide on an audio/video format they could both use. GNU SIP Witch seems to be the most suitable for this, and I managed to set up a one-way video two-way audio session between two Android devices running CSipSimple... I couldn't get two-way video, which was a shame.

I see that routing everything through a central server isn't ideal for big installations because of the server load, but for a small home server that won't have more than a couple of calls at any one time I don't see the problem and it cuts out a whole world of pain with STUN. I actually looked into how much it would cost me to have 2 IP addresses so that I could run a STUN server - I think it would have taught me lots about NAT. Turns out you can only do that if you're a business, and it would be more than double what I pay now, so no dice.

Do you have any experience with videocalling with SIP? If so, I'm curious to see which client and server programs you used, and why.

XMPP was quite simple by comparison. Ejabberd seems to be the only free option that is actively developed and supports a good number of extensions, even if it does feel a bit odd due to erlang. One thing that bothered me was not being able to use the current stable version of fail2ban (0.8) to check failed password attempts (entries in the log file are split over multiple lines!). The new version of fail2ban (0.9) can do multiline regex, so I'm hoping it will make its way into the ubuntu repos soon.

It's a real shame that Google abandoned XMPP/Google Talk and switched to a proprietary protocol with Hangouts. Federated XMPP is something I think is pretty cool for the same reasons that federated SMTP is great.

[SteveRiley]

Neat! I'll have to remember that, I don't remember seeing it as an option when I was looking into SIP. There's so much to learn, I was a bit overwhelmed by the plethora of options available for SIP at the time to be honest - I got as far as "you have to choose which method to use: TCP or UDP" and looking up the differences between the two. I didn't want to send the PW in plaintext without TLS, so that left me with TCP (or so I thought).

I'm not aware of any SIP server that uses DTLS, so I'm not surprised you didn't see it as an option.

But you do realize that SIP is only used for call setup and signaling, right? The actual audio and video is carried in RTP, which almost always uses UDP. RTP is the bulk of the call; the SIP signaling is minimal. I can't think of any efficiency gains one might realize by putting SIP in UDP.

Do you have any experience with videocalling with SIP? If so, I'm curious to see which client and server programs you used, and why.

Very minimal. My experience is limited to the continual frustration I encounter trying to get $RANDOM_SOFT_PHONE working with my employer's Avaya phone switch. I'm convinced its SIP implementation is from the devil; various clients exhibit weird, unexplained behavior. The only one that worked consistently well was that ancient Twinkle -- probably because it cared little for "proper" protocol behavior and brute-forced every call setup. Alas, it no longer runs on modern Qt and is unmaintained.

It's a real shame that Google abandoned XMPP/Google Talk and switched to a proprietary protocol with Hangouts. Federated XMPP is something I think is pretty cool for the same reasons that federated SMTP is great.

LWN's recent writeup about Matrix is illuminating. Particularly the comments.

[Feathers McGraw]

I'm not aware of any SIP server that uses DTLS, so I'm not surprised you didn't see it as an option.

But you do realize that SIP is only used for call setup and signaling, right? The actual audio and video is carried in RTP, which almost always uses UDP. RTP is the bulk of the call; the SIP signaling is minimal. I can't think of any efficiency gains one might realize by putting SIP in UDP.

That makes sense, I don't think I had put two and two together.

Very minimal. My experience is limited to the continual frustration I encounter trying to get $RANDOM_SOFT_PHONE working with my employer's Avaya phone switch. I'm convinced its SIP implementation is from the devil; various clients exhibit weird, unexplained behavior. The only one that worked consistently well was that ancient Twinkle -- probably because it cared little for "proper" protocol behavior and brute-forced every call setup. Alas, it no longer runs on modern Qt and is unmaintained.

That seems to be a common complaint (certain pieces of software being not-quite-standards-compliant, other software changes to work with it until none of it is actually following the standard). One of the comments in that article was about Google doing that with Jingle. Why do you think that happens so often? If you're the big player and you were planning on doing things differently, why not influence the standards committee to write your methods into the standard? And if it's your own standard, then there really is no excuse for not complying with it!

LWN's recent writeup about Matrix is illuminating. Particularly the comments.

That's a great article. Pretty sharp comments too like you said, they certainly aren't pulling punches.

So... the consensus seems to be that everything is a complete mess. What do you think is the way forward? Stick with XMPP or something new? Setting up an XMPP server for real is on my to-do list, but if it's on the way out it's a bit of a waste of time. I can't help but think XMPP has enough inertia to live on, though... there are so many nice clients out there on every platform.

[DoYouKubuntu]

@DYK, I'm sorry for threadjacking...

No, no, no! I LOVE thread drift! It's what makes talking online more like real life, you know?

it looks like you have your answer now

Yep, that's why it dawned on me that I should post an update. Since changing a few of the outgoing server's settings, that 'sending failure issue' hasn't happened again...at all! Coincidence? I don't know. At any rate, here are the outgoing server's settings now:

Server name: smtpauth.earthlink.net
Connection security: none (as above)
Authentication method: Encrypted password
Username: my_earthlink_address@earthlink.net
Port: 25 Default: 587

Thanks again, all around, for the great input and insight.

[SteveRiley]

That seems to be a common complaint (certain pieces of software being not-quite-standards-compliant, other software changes to work with it until none of it is actually following the standard). One of the comments in that article was about Google doing that with Jingle. Why do you think that happens so often? If you're the big player and you were planning on doing things differently, why not influence the standards committee to write your methods into the standard? And if it's your own standard, then there really is no excuse for not complying with it!

Standards create level playing fields and give consumers choice. If you're a profit seeking company, you don't like that -- you want to monopolize as much of your market as you can. If Google used only standard XMPP, then people could connect with whatever client they wish. The result is overall loss of control for Google and certainly an inability to maximize ad placement. So instead they bastardize the standards just enough to make using a non-Google client very painful. So painful for most people that they'll instead capitulate and use Google's client. Now Google is in control, they can track your activities better, and shove more ads at you.

So... the consensus seems to be that everything is a complete mess. What do you think is the way forward? Stick with XMPP or something new? Setting up an XMPP server for real is on my to-do list, but if it's on the way out it's a bit of a waste of time. I can't help but think XMPP has enough inertia to live on, though... there are so many nice clients out there on every platform.

The tale of standards-based presence and instant messaging is a rather sad one. In 1999 an IETF working group was chartered to begin defining the standards. The results were two informational RFCs: 2778, "A model for presence and instant messaging" and 2779, "Instant messaging / presence protocol requirements". Incidentally, both were created in part by my former manager at Riverbed. IMPP concerned itself with defining the architecture and gateway semantics for connecting messaging and presence systems together. Some individuals in the WG wanted to go further and define transport protocols, too; various factions couldn't come to an agreement and IMPP kind of imploded. Acting outside the IMPP WG, two other standards-based transport protocols emerged: SIMPLE and XMPP are implementations of IMPP concepts.

Alas, SIMPLE (given its reliance on SIP) and XMPP suffer from two problems: (1) they're late to the game, and (2) they're bloated.

The only reason we have standards-based email (SMTP) today is because it emerged long before the commercialization of the Internet. Interoperability was paramount -- not walled gardens. Instant messaging's roots are in consumer grade online networks -- think AOL Messenger, MSN Messenger; there's also ICQ, which had its own protocol. The incentive here is not to interoperate but to capture as many users as possible. Enterprise IT was late in adding presence and messaging to its stable of services; if presence and messaging had emerged as a business function in the late 1990s, I suspect we'd see a very different landscape today. Enterprises, after all, need interoperability over anything else.

SIP/SIMPLE and XMPP are monstrous and unwieldy. As you've seen, there's just a lot of stuff in SIP and its hangers on. Sure, real time audio and video are harder than text. But not that much. There's been a lot of bickering in the standards groups; protocol-by-committee isn't pretty. XMPP's albatross is XML. Anything XML touches turns to dust. And dust, unfortunately, is probably where XMPP is headed. Google is nudging people away; Facebook will drop it later this year.

Good reads:
http://jeff.ecchi.ca/blog/2013/12/20...aging-in-2013/
https://news.ycombinator.com/item?id=7151857
https://bloggeek.me/death-signaling/

Oh, and jabber.org's X.509 certificate expired today: https://xmpp.net/result.php?domain=j...rg&type=server