Twisted Trial Ported to Python 3!

Twisted, as many of you know, is an asynchronous, or event driven networking framework written in Python (https://twistedmatrix.com/trac/wiki). Twisted has been around for about a decade, offers many features, including low-level primitives and high-level interfaces, and works with many protocols (including HTTP, XMPP, NNTP, IMAP, SSH, IRC, FTP). 

Twisted Logo

Due to its maturity and complexity, Twisted requires a lot of time and effort to be completely ported to Python 3. Fortunately, the PSF was able to help fund some of this work; one recent result is the release of Twisted 15.4, which includes Twisted’s standard test-runner, Trial (codenamed "Trial by Fire").

The PSF Grant allowed core developer and Twisted release manager,
Amber (HawkOwl) Brown, to port Trial to Python 3. She recently sent the PSF this announcement:

“Just wanting to let you all know that a Twisted with the PSF-funded Trial Py3 port is now released. And a little example of it in action:
https://asciinema.org/a/cthr9xezlt8mxg5dp0n73fzc9. Again, many thanks for accepting the grant proposal – the ability to dedicate a significant chunk of time to this work has meant it was completed well sooner than if the grant had not been accepted.”

Due to certain differences between Python 3 and Python 2 (e.g., removal of ClassType and unbound methods), Amber tells us that the porting of Trial required a rewriting and retesting of the test suite loader. The work is mostly done and the current port duplicates most of Trial’s previous functionality with the exception of its distributed test runner (DistTrialRunner).

Specifically, the PSF grant allowed Amber to perform the following steps:

- Complete and test the Trial unittest loader 

- Fix the remaining failing Trial tests  

- Create a tool which runs only the portions of Twisted that have been ported to Python 3 for use in Twisted development  

- Break up the port into smaller pieces, put them up for review, and address the review comments  

- Merge the reviewed portions

Trial’s features–a front-end, the ability to handle Deferreds and asynchronous tests, and the capacity to build testcase-duration reactors, make testing much easier. The Twisted team will now be able to use Trial for continued Python 3 porting, while users of Twisted will be able to test their codebases more easily as they port them to Python 3. Because of Trial, we can look forward to Twisted 15.5 in the near future (and hope to see more users' code ported to Python 3, as well). As Amber tells us,

"15.5, coming soon, will come with another handful of ported modules, and the twistd application (a daemoniser + plugin runner, the recommended way of spawning long-running Twisted services)."

The PSF sends its gratitude and congratulations to Amber Brown and the Twisted team on this important accomplishment.

To learn more about Twisted, the following websites, video talks, and tutorials are available:

http://labs.twistedmatrix.com/

http://krondo.com/wp-content/uploads/2009/08/twisted-intro.html

http://www.ibm.com/developerworks/library/l-twist1/

http://pyvideo.org/video/2597/twisted-mixing

https://www.youtube.com/watch?v=_HZR7_ZBkYY

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

Grants to Assist Kivy, NLTK in Porting to Python 3

From time to time, the Python Software Foundation receives grant requests for projects wishing to receive assistance to complete some task. In the past, grants have gone towards aspects of the PyPy project and the email6 redesign, and have even financed the creation of a new developer guide. Whether it’s a project central to the Python language or any of the great projects this community has created, the PSF is there for all and accepts requests from any projects that provide benefit to the Python community.

Requestors are asked to provide answers to questions such as a project background, what work will be completed, who will be involved, who will benefit, and costs associated with the work. The requests then go before the board of directors, who then debate various aspects of each request and render a decision.

The last two months were very interesting for the board, as we received two requests to help finance the porting of projects to Python 3.

Kivy

In October, the Kivy project reached out to us with a request to fund their porting efforts, where they requested a grant of $5,000 USD. Their plan includes the porting a number of Kivy project dependencies (their Android and iOS tools), the porting of a number of third-party dependencies (e.g., PIL, gstreamer, opencv), some work on their OSX and Windows packaging, and of course, the core Kivy codebase.

On October 17, the board voted to approve the Kivy request. Their porting effort hopes to be a great story for Kivy and Python 3 on a number of platforms and interfaces, as Kivy supports development of applications for desktops, tablets, and phones, and includes support for multi-touch. We’re really looking forward to sharing more details as the Kivy port moves along.

NLTK

Just a few weeks ago, a request came before the board to fund porting efforts for the NLTK project. NLTK is the Natural Language Toolkit, a project which exposes very powerful linguistics tools to Python. For many, NLTK is one of the major remaining roadblocks to Python 3 adoption. As many projects have been ported and many more are working on it, getting NLTK on Python 3 will be huge for the community.

On November 21, Mikhail Korborov was granted $1,000 USD to finish the in-progress port of NLTK under the watchful eye of project lead Steven Bird. Mikhail was recently added as a committer to the project after some time as a contributor, and he’ll be following their plan as laid out on their GitHub repo. Not only will the NTLK port be a boon to wider Python 3 adoption, but it should provide a good story for others to lean on when porting large codebases, especially when it comes to working with Python 3’s Unicode implementation.

---

We hope you’re as excited as we are to see Kivy and NLTK making moves towards Python 3! We’ll follow up with progress reports as we receive them.