Repository navigation
Deprecate smtpd (based on deprecated asyncore/asynchat) #69196
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/warsaw'
closed_at = <Date 2017-02-24.20:32:41.162>
created_at = <Date 2015-09-05.17:08:13.926>
labels = ['3.7', 'docs']
title = 'Deprecate smtpd (based on deprecated asyncore/asynchat)'
updated_at = <Date 2019-05-10.18:11:02.809>
user = 'https://github.com/brettcannon'bugs.python.org fields:
activity = <Date 2019-05-10.18:11:02.809>
actor = 'ned.deily'
assignee = 'barry'
closed = True
closed_date = <Date 2017-02-24.20:32:41.162>
closer = 'barry'
components = ['Documentation']
creation = <Date 2015-09-05.17:08:13.926>
creator = 'brett.cannon'
dependencies = []
files = ['40605']
hgrepos = []
issue_num = 25008
keywords = ['patch']
message_count = 18.0
messages = ['249911', '249913', '249935', '249942', '249953', '250019', '250110', '250111', '250149', '250510', '251736', '252285', '253097', '288391', '290408', '290409', '290410', '290411']
nosy_count = 10.0
nosy_names = ['barry', 'brett.cannon', 'rhettinger', 'terry.reedy', 'vstinner', 'eric.smith', 'giampaolo.rodola', 'ned.deily', 'r.david.murray', 'neic']
pr_nums = ['274', '278', '279', '280', '703', '552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue25008'
versions = ['Python 3.7']
Activity
brettcannon commentedon Sep 5, 2015
As mentioned in passing in issue bpo-25002, perhaps smtpd should be deprecated or at least be updated to use asyncio.
brettcannon commentedon Sep 5, 2015
I should mention one of the motivations behind deprecating smtpd is that no one should be using that module to run an SMTP server and it doesn't provide utility like http.server does and so we probably should drop the code to prevent anyone accidentally using it and not have to maintain it.
rhettinger commentedon Sep 5, 2015
Bringing in Eric V Smith's comment in
http://bugs.python.org/issue25002#msg249873 :
'''
I use smtpd.py for testing my code. But it's not such a big deal that I couldn't live without it. If I have some time to burn, I might convert it to asyncio so I can continue to use it.
'''
Bringing in R David Murray's comment in http://bugs.python.org/issue25002#msg249882 :
'''smtpd is used for testing smtplib. It is also used in test_logging. I would object to having it removed, but I suspect we can manage to rewrite it by 3.8. Maybe Eric and I can collaborate on it.
'''
rhettinger commentedon Sep 5, 2015
FWIW, I would like to see the module live on (after conversion to asyncio). Doug Hellman's example at https://pymotw.com/2/smtpd/ makes for a nice demonstration of Python's capabilities.
warsaw commentedon Sep 6, 2015
Removing smtpd would definitely be a hardship for me right now, probably for obvious reasons. I use it in testing frameworks, and even wrote a library called lazr.smtptest that is built around smtpd. In Mailman, we have an LMTP server built on smtpd that implements *the* official way of getting mail into the system from an upstream mail server. Mailman 3 core FWIW is Python 3.4 and 3.5 compatible.
That all being said, I've also been thinking lately about an asyncio-based reimplementation of SMTP and LMTP. asyncore/chat and smtpd are not the easiest modules to work with, extend, understand, or debug. I'd be very happy to collaborate with folks on an asyncio-based version. Maybe we can put that out as a third party module, with a long term plan on pulling it into the stdlib and promoting it instead of smtpd.
bitdancer commentedon Sep 6, 2015
I believe that smtpd is used for a *lot* of testing code (anyone who is writing smtp clients in python, pretty much, and probably some writing in other languages). I really don't see the rationale for removing it, assuming we can rewrite it "in time".
On the other hand, are we really going to *remove* asyncore/asynchat? Or is this a "deprecation in place" like optparse? If the latter then I don't really care if the same is done to smtpd :) If the former, why, if we've kept optparse?
vstinner commentedon Sep 7, 2015
Even if it may be possible to support asyncore/asynchat and asyncio at the same time, I would prefer to drop asyncore/asynchat support to simplify the code.
The problem is that the API will be very different.
asyncio has a different design. You don't instanciate a class to open a network connection, but you have to call "yield from loop.create_connection()" in a coroutine.
We might provide helper blocking functions starting the server for us and then running the event loop until CTRL+c is pressed (or another event to stop to stop the loop).
Rewriting smtpd with asyncio would allow to use it in an asyncio application. In 2015, I expect to see more and more applications using asyncio than asyncore/asynchat.
Can we modify the issue title to "Rewrite smtpd with asyncio"?
warsaw commentedon Sep 7, 2015
On Sep 07, 2015, at 02:26 PM, STINNER Victor wrote:
Sure, although I'm currently thinking it's best to go third party until the
API and implementation settle down.
rhettinger commentedon Sep 8, 2015
Unless there is a drop-in replacement, I don't think this should be deprecated or removed. Our biggest problem in the Python world is getting organizations using Python 2 to switch to Python 3. Deprecating modules that have been a part of the firmament for a long time works against that goal.
Side note: a quick search shows that people are still writing SMTPD modules in other languages (i.e. Go now has one)
terryjreedy commentedon Sep 12, 2015
http://nullege.com/codes/search?cq=import+smtpd
shows 135 hits. FWIW, import+asyncore get 972.
[-]Deprecate smtpd[/-][+]Deprecate smtpd (based on deprecated asyncore/asynchat): write a new smtp server with asyncio[/+]bitdancer commentedon Sep 28, 2015
Here is a proof of concept port of smtpd to asyncio. Obviously I'm not suggesting we commit this patch (it isn't complete...the tests don't run). Instead I'm posting it as an outgrowth the of the sprint that was conducted today. The DC team took a different tack, so I'm not submitting this as a patch to their repo, and since it is useful to look at the diff against default, it seems appropriate to post it here.
The advantage of this port is that it uses the existing smtpd protocol code, which is the "field tested" code. Of course, I did have to change how the data is turned from bytes into lines, so I may have introduced some bugs.
As discussed, *using* an asyncio based smtpd has a different calling API than using the asynchat one. However, the *pattern* is similar, so it may not be crazy to replace smtpd with the cleaned up version of this patch, exactly because the number of places this is used is relatively small and mostly in test code. IMO it's a better option than deleting smtpd from the stdlib.
The code as is can be run using its CLI. I tested it via telnet using the DebuggingServer and hand coding an email...for some reason I can't currently import smtplib (Enum errors in socket), so I didn't test it with smtplib, but it should work.
warsaw commentedon Oct 4, 2015
I'm liking RDM's port of smtpd.py. I've done some clean ups, bug fixes, and added some debugging. See this branch over on gitlab:
https://gitlab.com/python-smtpd-hackers/aiosmtpd/tree/issue25508
Try
python3 server.pythen run client.py in another terminal (it uses smtplib) or just telnet.I plan to continue pursuing this approach; please comment over there.
warsaw commentedon Oct 16, 2015
This has come along pretty well. The project is managed here:
https://gitlab.com/python-smtpd-hackers/aiosmtpd
and documentation here:
http://aiosmtpd.readthedocs.org/en/latest/
warsaw commentedon Feb 22, 2017
aiosmtpd is coming along nicely:
http://aiosmtpd.readthedocs.io/en/latest/
We'll soon have a 1.0 release. Still, I don't think it's worth pulling this into the stdlib. But we could silently deprecate it and point to aiosmtpd in the docs.
[-]Deprecate smtpd (based on deprecated asyncore/asynchat): write a new smtp server with asyncio[/-][+]Deprecate smtpd (based on deprecated asyncore/asynchat)[/+]warsaw commentedon Mar 24, 2017
New changeset 66b5092 by Barry Warsaw in branch '3.5':
bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#279)
66b5092
warsaw commentedon Mar 24, 2017
New changeset 0a1b656 by Barry Warsaw in branch '3.6':
bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#278)
0a1b656
warsaw commentedon Mar 24, 2017
New changeset f37b0cb by Barry Warsaw in branch '3.4':
bpo-25008: Deprecate smtpd and point to aiosmtpd (#274) (#280)
f37b0cb
vstinner commentedon Mar 24, 2017
New changeset fba79dc by Victor Stinner (Barry Warsaw) in branch 'master':
bpo-25008: Deprecate smtpd and point to aiosmtpd (#274)
fba79dc