Improved exception diagnostics in Python 3.11

Sat 18 June 2022 by Vladimir Obrizan

In large programs, it is difficult to understand from the printout of the call stack where exactly the error occurred in the code. Especially if complex expressions are used. Python 3.11 introduced detailed diagnostic information in the call stack printout. Let's look at a few examples.

At the time of writing, Python 3.11 has not yet been officially released. You can install version 3.11 separately from the prerelease page.

You can check the Python version with the --version option in the console:

% python --version
Python 3.11.0b3

Consider a small example (the main.py file) where a ZeroDivisionError exception could potentially occur in the f1 function:

# main.py
import sys


def f1(a: float, b: float, c: float) -> float:
    return a / b / c


a1 = float(sys.argv[1])
b1 = float(sys.argv[2])
c1 = float(sys.argv[3])
print(f"Result: {f1(a1, b1, c1) * sum([a1, b1, c1])}")

Let's run this example from the command line. Here we deliberately pass zero as one of the arguments to get a ZeroDivisionError exception:

python main.py 1 0 2

What the call stack printout looks like in Python 3.10:

Traceback (most recent call last):
  File "main.py", line 12, in <module>
    print(f"Result: {f1(a1, b1, c1) * sum([a1, b1, c1])}")
  File "main.py", line 6, in f1
    return a / b / c
ZeroDivisionError: float division by zero

Note that the interpreter reports a ZeroDivisionError exception on line 6, but doesn't specify which division operator to use: first or second?

Let's run the same code with the same arguments in the Python 3.11 interpreter. What the call stack printout looks like in Python 3.11:

Traceback (most recent call last):
  File "main.py", line 12, in <module>
    print(f"Result: {f1(a1, b1, c1) * sum([a1, b1, c1])}")
                     ^^^^^^^^^^^^^^
  File "main.py", line 6, in f1
    return a / b / c
           ~~^~~
ZeroDivisionError: float division by zero

What to look for in this printout:

• the symbol ~~^~~ underlines that it is in the first operator that division by zero occurs: a / b;
• the symbol ^^^^^^^^^^^^^^^ underlines that part of the complex expression, during the evaluation of which an exception occurred: f1(a1, b1, c1).

Let's try to run the example in the Python 3.11 interpreter with different arguments:

python main.py 1 2 0

The interpreter will now point to the second division operator:

  File "main.py", line 6, in f1
    return a / b / c
           ~~~~~~^~~
ZeroDivisionError: float division by zero

Working with complex dictionaries is another example where improved diagnostics can help you find errors faster. Consider the JSON document data.json and the code for the file account.py that sums up the balances of accounts account1 and account2. Note that in this example there are intentionally no transactions in account2.

{
  "account1": {
    "transactions": [
      {"id": "1", "title": "Покупка", "sum": 123, "balance": 1000},
      {"id": "2", "title": "Покупка", "sum": 800, "balance": 200},
      {"id": "3", "title": "Покупка", "sum": 15, "balance": null}
    ]
  },
  "account2": {
    "transactions": null
  }
}

# account.py
import json

with open('data.json') as f:
    data = json.load(f)

summa = data['account1']['transactions'][0]['balance'] + data['account2']['transactions'][0]['balance']

print(f"Сумма счетов: {summa}")

If we run this script with a Python 3.10 interpreter, we get the following listing of the call stack:

Traceback (most recent call last):
  File "account.py", line 7, in <module>
    summa = data['account1']['transactions'][0]['balance'] + data['account2']['transactions'][0]['balance']
TypeError: 'NoneType' object is not subscriptable

Here it is clear that the exception occurs on line 7, but it is not clear what kind of object is it? After all, the expression is complex. It is required to restart the program under the debugger, set a breakpoint and watch the state of internal variables.

If we run this script with a Python 3.11 interpreter, we get the following listing of the call stack:

  File "account.py", line 7, in <module>
    summa = data['account1']['transactions'][0]['balance'] + data['account2']['transactions'][0]['balance']
                                                             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable

Here, using the symbol ^^^, it shows which operator caused the exception: the result of the expression data['account2']['transactions'] is None and cannot be applied to [0].

Full details of the fine-grained error diagnostic capabilities in exceptions can be found in PEP 657 – Include Fine Grained Error Locations in Tracebacks.