Welcome to the Warehouse!

Warehouse is the new codebase being developed to power the Python Packaging Index (PyPI). Python developers and users already know that PyPI is the official comprehensive repository of third-party open source Python packages (see Wikipedia). PyPI, maintained by the PSF, is where developers publish their software modules and from which package managers, such as pip, download packages. Given the important role played by PyPI, the Warehouse project is bound to have a huge impact on the continued use and growth of Python. 

While many volunteers have been working on the project, the biggest contributions have come from lead developer Donald Stufft and web-design specialist, Nicole Harris. Donald, based in the US, is a core contributor to PyPI (as well as to CPython, pip, virtualenv, Django & Cryptography), while UK-based Nicole runs a web development business, Kabu Creative.

The design goals, as stated on Nicole's website, were:

• To update the visual identity 
• To make packages more discoverable
• To accommodate the needs of both users and package maintainers 
• To give the project the same level of professionalism as a commercial project of the same scale 
• To ensure that the user experience reflects the Python community–a community that is welcoming, helpful and inclusive

Donald and Nicole have recently released a first look at the new design on the demo sites, Warehouse and Warehouse staging.

Looking great, thanks to Donald and Nicole's hard work, but there is plenty more to be done: writing code, writing the user guide (PyPUG), usability testing, and giving feedback. Details on how to volunteer and more info can be found on Nicole's site.

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

First-Ever PyCaribbean Coming This February!

At the October 28th meeting of the Board of Directors of the PSF, the following resolution was passed:

 

"RESOLVED, that the Python Software Foundation sponsor PyCaribbean on February 20-21, 2016 in Santo Domingo, Dominican Republic, in the amount of $3000 USD."

This will be the first-ever PyCaribbean, and the PSF is proud to be a Platinum-level sponsor. The venue will be the headquarters of the Fundación Global Democracia y Desarrollo (FUNGLODE) located at Calle Capitán Eugenio de Marchena 26, Santo Domingo, Dominican Republic.

According to organizer, Leonardo Jiminez, they are expecting approximately 300 - 350 people from all over the Americas and the Caribbean. This is a great size for a conference, allowing for both intimacy and breadth of content. As Leornardo puts it:

"I think we have the spirit you can find in PyCon but in a more concentrated way and with better weather."

Better weather and gorgeous beaches!
Photo Credit: CC License

Based on proposals received so far, it appears that the talks will be quite diverse. Along with such expected topics as strategies for improving data analysis, Leonardo particularly mentioned a talk about the adventures of someone teaching Python in Latin America in his car.

Of course, that adventurer is none other than Manuel Kaufmann! Some of you may recall that last year the PSF funded his project, and it was featured in a couple of posts to this blog: Python in Argentina and Highly Contagious. And in addition to what is sure to be a fascinating talk by Manuel Kaufmann, there will also be a keynote by the PSF’s very own Brandon Rhodes, who is also the organizer of the upcoming PyCon 2016 in Portland.

I asked Leonardo to tell us a little about his local Python community. Here’s what he said:

Santa Domingo
Photo Credit:  CC License

"The Python Dominicana Usergroup was the first usergroup formed in Santo Domingo and after that a lot of progress happened. We have more than three years meeting consistently every month. The experience has been really transformative for the city. We have done road trips, hackathons, and a lot of events to promote the language. . .[This work] is paying off with all the growth we see in Software Development locally."  

And when I asked about his own interest in Python, he gave a great answer:

"I fell in love with Python in High School while reading How to Think Like a Computer Scientist in Spanish. I really enjoy being part of this community, which I think is the killer feature of Python."

I have to agree that the community is our killer feature, so I’m hoping that some of you will be able to participate in PyCaribbean by giving a talk or by attending. You have until November 20th to submit a proposal, so please give it some thought. Of course, if you don’t have a talk prepared, the organizers would still love to see you there. And could there be any better vacation than to visit the Dominican Republic in February? In fact, this conference looks so good to me that I may see you there myself!

For further information, you can contact the organizers (at leonardo@pycaribbean.com or vivian@pycaribbean.com). Also you can follow them on Twitter (at https://twitter.com/pycaribbean).

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

PyCon Singapore

The PSF is happy to report that the third annual PyCon Singapore took place June 17 to 19, 2015. This event, organized by the Python User Group Singapore, is a testament to the robust presence of the Python community in the Asia-Pacific region. In addition to three PyCons Singapore, PUGS has held three PyCons Asia-Pacific. Congratulations to Ivan Zimine and the organizing committee for a successful conference. According to Ivan, “Feedback from the participants was mostly positive. Out of 18 responses, 7 people gave 4 out of 5 stars, and 6 people gave 5 out of 5 stars for the “How did you like PyConSG?” question."

Photo Credit Martin Brochhaus

CC 2.0

The PSF contributed to the event as a Silver Level Sponsor, and we were proud that keynote speakers included our own PSF Director Lynn Root and former PSF Director Jessica McKellar. Django core developer Andrew Godwin also gave a keynote.

The event was attended by 181 people and was held on the campus of Singapore Polytechnic. It consisted of one day of tutorials and two days of talks. Tutorials covered topics such topics as unit testing, data stores, and beginning programming with Python and Django, and were attended by 96 people, of whom 64 were students. Lynn Root’s tutorial, “How to Spy with Python,” explained how the NSA and the UK’s Tempora programs can collect data on citizens' search histories, emails, IRC conversations, PGP usage, etc. As Lynn was clear to point out, however, the talk was not an endorsement of spying or a how-to, but rather a “… way of understanding the current political environment, as well as indirectly understanding how to protect one’s privacy” (see How to Spy).

127 people (of whom 41 were students) attended the two days of conference talks. Featured speakers included Anand Chitipothu, Kristin Nguyen, Ricky Setyawan, Sacha Goedegebure, Colm O'Connor, and others covering a wide range of topics, including interpreters, data processing, educational games, data processing, machine learning, multicore processing, and film production.

A first-time feature of the conference was its edu-summit, which was attended by approximately 40 Computer Science teachers. 

The summit included a talk by Praveen Patil titled Python in my Physics classroomabout how to incorporate computer science into the science curriculum using ExpEYES, an Open Source Pocket Science Lab (https://pycon.sg/schedule/presentation/59/).

Here are some links and pictures of the event: PyCon Singapore, Blog Post, Group Photo, Welcome.

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

PSF funds development: Armin Rigo's CFFI 1.0

In looking back over the PSF newsblog posts, it appears that most of the PSF funded projects I’ve written about were conferences, workshops, and education/outreach efforts. These are, of course, truly important. However, it’s also important to get the word out about several development projects that the PSF has sponsored in 2015. One such project is Armin Rigo’s work on CFFI 1.0. 

RESOLVED, that the Python Software Foundation grant Armin Rigo $2500 towards cffi development aimed at making cffi generated extension modules importable without runtime dependencies on an extension module build toolchain.

CFFI or C Foreign Function Interface for Python provides a way to call compiled C code, i.e., external C libraries, from Python using interface declarations written in C. This eliminates the need to use a programming language other than C and Python. At the same time, CFFI minimizes the amount of C code that needs to be written, so it really is a boon for Python developers. It works with Python 2.6 and up and with PyPy 2x and 3x. See CFFI Documentation.

CFFI has already had approximately 7 million downloads, so it is clearly of use, but its creator, Armin Rigo (who is also one of the creators of PyPy) saw room for improvement. Specifically, according to Armin, there were two main problems:

1. The difficulties of installation [which] can be seen from outside by looking at various workarounds and 3rd-party documentation that have grown into existence. For example, the setup.py of projects like cryptography, PyNaCl and bcrypt deploys workarounds that are explicitly documented inhttps://caremad.io/2014/11/distributing-a-cffi-project/.
2. The time taken at import is excessive in some cases. For example, importing pygame-cffi on a Raspberry Pi ARM board takes on the order of 10 to 20 seconds (and this is the fast case where the compiler doesn’t need to be invoked any more)

Due to the PSF grant, Armin was able to fix both problems. The 1.0 version, now in beta, is available at CFFI 1.0.

Our thanks to the amazing Armin Rigo for this very welcome tool!

Stay tuned for my next post about the PSF’s recent award of $8,000 USD for Read the Docs. 

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

South African Brothers and Sisters for Python

Today’s post is a follow-up to a previous post about a PSF funded project by Hyperion Development in South Africa. 

The PSF has recently heard from Riaz Moola about his latest project–working with Brothers For All to deliver Python courses in 42 prisons–including eight female corrections facilities–in the Western Cape.

Brothers is run by an ex-con, Sihle Tshabalala, who taught himself to code online. When he got out of prison, he wanted to do something to slow the recidivism rate of South African ex-convicts. Not only does South Africa have the world’s eighth highest prison population, but the rate of re-offenders is over 80%. In addition, an extremely high unemployment rate of 25% adds to the hopelessness felt by those with prison records and without marketable skills.

Brothers began last September by opening a center in a Western Cape township area. There, ex-cons and at-risk youth come to learn coding skills, entrepreneurship, and marketable crafts.

According to Tshabalala,

“We chose coding because it leapfrogs over the need for more conventional, expensive and time consuming job skills training… Plus there currently is a high demand in the market for such skilled labour.”

They have recently been given approval by the provincial government to take the program into 42 prisons in the Western Cape. This way, prisoners can gain skills while incarcerated so that they are ready for employment immediately upon release. 

The ambitious scope of this program caused Brothers to partner up with Hyperion. While Brothers will focus on getting the programs into the prisons and more township areas, Hyperion will provide course content.

Hyperion’s main concerns at this point are logistical and for long-term sustainability. As Riaz told the PSF, there are many challenges specific to South Africa; for example, for four hours every day there is a power blackout, which obviously poses a problem for internet connectivity. In addition, South Africa has the highest data costs of any country in the world. Travel for tutors and volunteers is also very expensive. And then, of course, there’s the poverty and its inevitable expression in crime. Sadly, one of the project’s computer labs was recently burglarized, resulting in the loss of equipment. But Brothers also has support streams, so they are currently receiving donations to make up for the loss. In future, they plan to purchase laptops, which can be more easily secured.

Hyperion has been able to achieve a solid team of Python instructors. With the help of the PSF and other institutions, Hyperion is able to pay the travel and data costs for their volunteers. This has allowed them to be very selective; their instructors must pass tests of their technological and teamwork skills. Even so, Riaz tells us, they are never at a loss for instructor applicants.

The Cape Town team, led by Hyperion manager Sobane Motlomelo (a Master’s degree student at University of Cape Town), will be primarily responsible for handling the prison project. Sobane's team has recently accepted four new members, and Riaz is confident of their ability. This is good news for Riaz, who began Hyperion as an experiment when he was an undergraduate, then saw it spiral into something very large, to which he has been giving all his time. Currently he’d like to step back just a bit. Our congratulations to him on being offered a Google internship! 

More info on this remarkable program can be found at its twitter page and htxt.africa.

I would love to hear from readers. Please send feedback, comments, or blog ideas to me at msushi@gnosis.cx.

PyLadies: announcing paid summer internships with CPython

The Python Software Foundation is proud to announce that it is sponsoring CPython internships for women this summer through the GNOME Outreach Program for Women.

Please visit our project page for the full program details and how to apply!

• What: Earn a $5500 USD stipend while contributing to the CPython interpreter and standard library.
• When: This is a full-time summer internship lasting from May 19 through August 18.
• Where: Anywhere! This is remote internship, with most communication happening on mailing lists, bug trackers, and IRC.
• Who: This internship is open to anyone who identifies as a woman or is genderqueer, genderfluid, or genderfree. Note that unlike Google Summer of Code, you do not need to be a student.

Applications are due by March 19th.

Python in Google Summer of Code 2014

Python project contributors and student enthusiasts, mark your calendars: Google Summer of Code applications open soon!

Google Summer of Code is an annual, global program pairing student developers with mentors in open source projects for paid summer internships.

You can learn more about this year's Google Summer of Code here.

Python projects

Python serves as an umbrella organization for around a dozen open source Python projects each year. This year, the following projects are participating:

• CPython: core Python and the standard library
• GNU Mailman: the ubiquitous mailing list package
• Mercurial: a distributed source control management tool 
• BinPy: a platform for building circuit-based applications or logical games
• Vispy: high-performance interactive visualizations
• TARDIS-SN: supernova radiative transfer in Python
• SunPy: Python for solar physics
• Scrapy: a fast, high-level screen scraping and web crawling framework
• Theano: an optimizing compiler for numpy.ndarray and scipy.sparse matrix
• Kivy: a library for making cross-platform, multi-touch apps
• MNE-Python: a software package for processing MEG and EEG data
• scikit-image: a collection of algorithms for image processing
• scikit-learn: a Python module for machine learning
• PyDy: a package for studying multibody dynamics with Python
• SciPy and NumPy: open-source software for mathematics, science, and engineering
• AstroPy: a community Python library for astronomy

Students

Google Summer of Code is a paid summer internship program for college/university students who will be 18 years of age or older on April 21, 2014. Participating in Google Summer of Code is a great way to develop real-world software engineering skills while giving back to an open source Python project you love.

Read more about eligibility in the FAQ.

If you are interested in participating in Google Summer of Code under the Python umbrella, it's time to start exploring potential projects and practicing the tools of open source development:

1. Read the Python Google Summer of Code guidelines.
2. Review this year's projects and their idea pages.
3. Start practicing the tools of open source development, including:
   
   • IRC
   • a revision control system like git or svn
   • the diff and patch utilities
   • bug trackers

If you've never used some of these tools before, don't worry! You have plenty of time to practice. A good resource for getting familiar with these tools is the OpenHatch training missions.

Important deadlines

• March 10: Student application period opens.
• March 21: Student application deadline.
• April 21: Accepted student proposals announced.

Note that the best way to boost your chances of being accepted for Google Summer of Code is to start contributing to a project before you apply. If you have questions about how to get started or just want some friendly encouragement, visit the OpenHatch project and say hello.

PSF Python Marketing Brochure - Last call for Ad Sponsors

Please support the PSF in providing the Python community with
free high quality marketing material to
promote Python

Introduction

Over the last few years, the Python brochure team has worked on and created a high-quality brochure to help user groups, conferences and companies using Python to promote and spread the word about Python.

The brochure will be printed in a first edition of 10,000 copies which the PSF will then distribute to user groups, Python conferences and educational institutions on request and free of charge.

With the Python brochure, we hope to reach out to an audience which is not easy to address and convince using electronic and mostly developer oriented media.

Preview

Please take a look at our preview PDF version of the brochure to see for yourself:

http://brochure.getpython.info/preview.pdf

Seeking your help

The team set out to create and print the brochure without introducing extra costs for the PSF. Our aim is to fully finance the brochure production, printing and shipment to interested parties using money from sponsors.

To make this happen, we are seeking your help !

• We have already signed up sponsors for 6 half page ads, but still need another 5 half page ad sponsors to sign up.
  
  
• There are also 6 smaller reference entry sponsorships left to be sold.

If you are affiliated with or know a company investing into Python and looking for ways to reach out to a large audience of interested Python users, students, developers - and people in key decision making positions, please reach out to us and help make the project a success.

The deadline for ad and reference entry signups is Feb 28 2014 - in just under three weeks.

You can find all the details about the available sponsorship options on this page:

• Brochure Sponsoring Options

Orders can be placed directly with the production company, Evenios Publishing on the website. All sponsors will receive a box of about 120 free copies of the brochure as Thank You gift.

Ordering extra copies

Companies who are interested in receiving extra copies can pre-order additional boxes which will then be printed in addition to the initial 10.000 copy batch:

• Subscription Sponsorships

It is also possible to donate such extra boxes to educational institutions:

• Educational Sponsorships 

If you have special requirements, please contact the team at brochure@getpython.info for more information. We're very flexible in addressing your needs.

More information

More information on the brochure, the idea behind it, media data and ordering links are available on our project page:

• PSF Python Brochure Project Website

Thanks for your help !

Python Surpasses Standards, Reaches New Levels of Quality

Throughout Python's 20-plus year history, its quality has been in the hands of the volunteers around the world openly contributing to it. Thanks to Coverity, those volunteers have been pointed to many quality and security issues via Coverity Scan, a service which finds defects in C/C++ and Java projects at no cost.


As the CPython project includes over 370,000 lines of C code*, accounting for 42% of the codebase, a lot of it lies outside of the analysis tools our community has created to work with Python code. Since 2006, Coverity's scans of that code have found nearly 1,000 defects, 860 of which our contributors have fixed.

In an industry where the standard defect density is a rate of 1 per 1,000 lines of code, CPython has attained a rate of 0.005 defects per 1,000 lines, and "introduces a new level of quality for open source software," said Coverity.

“Python is the model citizen of good code quality practices, and we applaud their contributors and maintainers for their commitment to quality,” said Jennifer Johnson, chief marketing officer for Coverity.

The PSF and the rest of the community join Coverity in applauding all of those who have contributed their time and effort to make CPython a better project, along with the countless others who contribute to a powerful landscape of Python interpreters.

---

For more information, read Coverity's "Coverity Finds Python Sets New Level of Quality for Open Source Software" press release.


* generated using David A. Wheeler's 'SLOCCount'.

PyCon DE 2011 Funding

The PSF Board of Directors has awarded a $1500 USD grant to sponsor PyCon DE 2011.

Date and Location

PyCon DE 2011 will be held from October 4th to 9th in Leipzig, Germany. The conference will be held at the Leipziger Kubus.

PyCon DE

PyCon DE 2011 is the first PyCon in the German-speaking countries. It consist of one day of 12 high-quality tutorials, three days of more than 60 talks, and two days of sprints. The topics reflect the wide usage of Python, including web development, science and engineering, testing, Python in business, databases, financial engineering, code quality and much more. The organizers are expecting between 250 and 300 attendees from the excited German-speaking Python community.

More Information

For more details, refer to the PyCon DE 2011 website.

The Python Core Mentorship Program

Jesse Noller recently announced the formation of the Python Core Mentorship program. The idea behind the program is to help programmers, including students and developers from other projects, connect with experienced contributors who serve as mentors to ease them into Python Core development.

Contributors Wanted

The mentors will help people regardless of experience level by bringing them up to speed, answering questions, and giving guidance as needed in a non-confrontational and welcoming way. The contributors will receive guidance through the entire contribution process, including discussions on the related mailing lists, the bug tracker, Mercurial, code reviews, and much more.

Early Success

The program already has been successful, and the participants have actively committed a number of patches. There have also been several constructive discussions on the mailing list, helping guide people in the right direction for a variety of issues.

Code of Conduct

The program has a code of conduct explained on the website that aims to assuage concerns many new contributors have when interacting with experienced developers and mailing lists on contribution in general. Jesse and the other mentors hope that this program can act as a model for other projects long-term, not just benefiting Python-Core. They also want the program to help increase the overall diversity of the contributors to Python.

Signing Up

The program is run via the mailing list and has a clear, concise website devoted to it. If you would like to join to ask questions and begin on the path of core contribution, or even if you are an experienced developer (even experienced in Python-Core) looking to ask questions you're worried about asking on other lists, this is an excellent opportunity to jump in, ask and get your feet wet!

PSF Sponsored Sprints

At their May 10 meeting, the PSF Board create a new Sprint Committee to organize a series of sprints for working on Python. Jesse Noller is the committee chair.

This new "focused sprint" initiative is part of a concerted effort to attract more contributors to Python's development team and to speed the adoption of Python 3. The PSF has agreed to provide funding to groups interested in organizing sprints related to:

• Python core tasks such as bug triage and evaluating patches
• Python core documentation work
• Porting existing third-party libraries to run under Python 3
• Enhancing http://python.org/ with new content, organization, or design work
• Development of PyPI

Over time, other projects may be added as the committee identifies them. Complete instructions for proposing a sprint for your group will be posted once they are worked out.

In the mean time, the Sprint Committee needs volunteers to help launch this project. The first steps will be to set up some communication channels, and then start developing several guides for new contributors. If you want to help out, contact Jesse Noller via sprints@python.org.

See the announcement on Jesse's blog for more details.

Have You Signed a Python Developer Contributor Agreement Yet?

A key aspect of the Python Software Foundation's mission is managing the intellectual property for the Python language. This includes the source code for the C implementation of the Python interpreter, the Python standard library, and any related tools and documentation. In order to share the source code and documentation with users of Python, the PSF needs to hold a valid license that gives us permission to do so. The licensing requirement applies to every contribution, so the PSF board is contacting all contributors to make sure they have filled out a contributor agreement.

In June 2009, Andrew Kuchling analyzed the svn repository for CPython and compared the actual contributor names with the list of people who had signed an agreement. 39% of the lines in the Python 2.7 portion of the tree were committed by developers without signed agreements. Around 31% of the lines for Python 3.1 came from unsigned contributors. Through the efforts of Andrew and other PSF members, those statistics improved to 8.5% for Python 2.7 and 9.9% for Python 3.1 by February of 2010.

Our goal is 100% coverage, so if you have contributed source code or documentation, either directly or via a patch in the issue tracker, please make sure you have submitted a form. If you are unsure of your status, you can check your issue tracker account by logging in and then clicking the "Your Details" link in the left sidebar. The account details page includes a "Contributor Form Received" field, which will show the date the form was filed. If the field says no agreement was received, please take a few minutes to fill out the contributor agreement and send it in. If you think you have already signed an agreement but the tracker does not show it, contact psf@python.org so we can check our records.

How Useful is the Python Job Board?

A service that the PSF has run for a very long time now is the Python Job Board. This allows any potential employer with a requirement for Python skills to post a free advertisement. The service is free because it is primarily intended to benefit Python community members. Employers are encouraged, but not required, to contribute to the Foundation.

So what I'd like to know is this: how useful is that service? If you have ever found the Job Board helpful, as either employee or employer, or even just to keep an eye on the Python employment market, please post a brief comment here letting us know what the Board did for you.

Employers and agents: please read this page before submitting jobs: the Board is run by volunteers, and you can make their task easier.