Python 2 to 2&3 porting cheat-sheet

Instructions and notes on porting code from Python 2 to both Python 3 and 2 using future:

Step 0: setup

Step 0 goal: set up and see the tests passing on Python 2 and failing on Python 3.

1. Clone the package from github/bitbucket. Rename your repo to package-future. Examples: reportlab-future, paramiko-future, mezzanine-future.

2. Create and activate a Python 2 virtualenv. Install the package with python setup.py install and run its test suite on Py2.7 or Py2.6 (e.g. python setup.py test or py.test or nosetests)

3. Optionally: if there’s a .travis.yml file, add Python version 3.3 and remove any versions < 2.6.

4. Install Python 3.3 with e.g. sudo apt-get install python3. On other platforms, an easy way is to use Miniconda3. See Miniconda3. Then e.g.:

   conda create -n py33 python=3.3
   

Step 1: modern Py2 code

The goal for this step is to modernize the Python 2 code without introducing any dependencies (on future or e.g. six) at this stage.

1a. Install future into the virtualenv using:

pip install future

1b. Run futurize --stage1 -w *.py subdir1/*.py subdir2/*.py

1c. Commit all changes

1d. Re-run the test suite and fix any errors.

See :ref:`forwards-conversion-stage1` for more info.

Example error

One relatively common error after conversion is:

Traceback (most recent call last):
  ...
  File "/home/user/Install/BleedingEdge/reportlab/tests/test_encrypt.py", line 19, in <module>
    from .test_pdfencryption import parsedoc
ValueError: Attempted relative import in non-package

If you get this error, try adding an empty __init__.py file in the package directory. (In this example, in the tests/ directory.) If this doesn’t help, and if this message appears for all tests, they must be invoked differently (from the cmd line or e.g. setup.py). The way to run a module inside a package on Python 3, or on Python 2 with absolute_import in effect, is:

python -m tests.test_platypus_xref

(For more info, see PEP 328 and the PEP 8 section on absolute imports.)

Step 2: working Py3 code that still supports Py2

The goal for this step is to get the tests passing first on Py3 and then on Py2 again with the help of the future package.

2a. Run:

futurize —-stage2 myfolder/*.py

This adds this further import to each module:

from __future__ import unicode_literals

All strings are then unicode (on Py2 as on Py3) unless explicitly marked with a b'' prefix.

It also makes other conversions needed to support both Python 2 and 3. These will likely require additional imports from future, such as:

from future import standard_library
from future.builtins import bytes
from future.builtins import open

If you would like futurize to import all the changed builtins to have their Python 3 semantics on Python 2, invoke it like this:

futurize --stage2 --all-imports myfolder/*.py

2b. Re-run your tests on Py3 now. Make changes until your tests pass on Python 3.

2c. Commit your changes! :)

2d. Now run your tests on Python 2 and notice the errors. Add wrappers from future to re-enable Python 2 compatibility:

• :func:`utils.reraise()` function for raising exceptions compatibly
• bytes(b'blah') instead of b'blah'
• str('my string') instead of 'my string' if you need to enforce Py3’s strict type-checking on Py2
• int(1234) instead of 1234 if you want to enforce a Py3-like long integer
• :func:`@utils.implements_iterator` decorator for any custom iterator class with a .__next__() method (which used to be .next())
• :func:`@utils.python_2_unicode_compatible` decorator for any class with a __str__ method (which used to be __unicode__).
• :func:`utils.with_metaclass` to define any metaclasses.

See :ref:`what-else` for more info.

After each change, re-run the tests on Py3 and Py2 to ensure they pass on both.

2e. You’re done! Celebrate! Push your code and announce to the world! Hashtag #python-future