Document how to print an exceptiongroup traceback when the monkey patching doesn't work / is not applied.

[matmel]

First, thanks for working on this project.

I updated to the latest cattr 22.1.0 from @Tinche who started to use the exceptiongroup backport to report various errors occurring while "structuring" objects. All fine.

Unfortunately, I think I have an installed package that messes with the monkeypatching of the traceback formatting so when I run my buggy object structuring which fails, I get this console output:

Traceback (most recent call last):
  File "debug.py", line 40, in <module>
    main()
  File "debug.py", line 34, in main
    pressure_rule_structured = DATA_QUALITY_CONVERTER.structure(
  File "/opt/python3toolbox/.venv/lib/python3.8/site-packages/cattrs/converters.py", line 281, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "<cattrs generated structure DataQuality.rule_objects.RangeConfig>", line 44, in structure_RangeConfig
cattrs.errors.ClassValidationError: While structuring RangeConfig (4 sub-exceptions)

When I run this same code in an virtualenv with less packages, I get the traceback "trees" akin to the ones shown here: python-attrs/cattrs#258 (comment)

I tried for some time to try to force the exceptiongroup exception formatting and it resulted in different form attribute errors or the likes deep in the exceptiongroup code which makes me think there are 2 monkey patching at play.

My question: is there anyway to force the display of this exceptiongroup formatting even if someone else elsewhere monkeypatched traceback.TracebackException? All my incantations involving traceback stdlib module and exceptiongroup._formatting failed miserably.

[agronholm]

You would have to patch the library that overrode the exception hook or TracebackException.format().

[agronholm]

I'm thinking this could be done by subclassing TracebackException and patching that, in cases where the original can't be patched (in cases where we detect that someone else has already patched sys.excepthook).

[agronholm]

Would the linked PR solve the problem for you?

[matmel]

Sorry for the wall of text. The TLDR is no, I don't think it solves the problem if I understood correctly how to use format_exception.

Here is the long answer

Minimal reproducible example

I've found a minimal reproducible example for the original issue I had originally. You'll need to install the following:

pip install exceptiongroup attrs cattrs trio

Notes

I am sure my original issue wasn't due to trio because it was not in pip list but some other yet unidentified package. The good news is that the two kind of traceback I get now are the same I had the first time if my memory serves well. And I guess it will be easier for you to understand the intricacies with trio.

Case 1: Working example

So here it is. First the "working" example (i.e. the traceback as it is supposed to be). We define an attrs class and we try to unserialize it with cattrs. It fails because we try to convert string values "foo" and "bar" to int attributes. cattrs reports both errors instead of failing on the first error as its ClassValidationError is a subclass of ExceptionGroup. All nice and cool.

from attrs import define
from cattrs.preconf.json import make_converter as json_converter


@define
class Test:
    a: int
    b: int

converter = json_converter()

def main():
    s = '{"a": "foo", "b": "bar"}'

    print(converter.loads(s, Test))

if __name__ == "__main__":
    main()

Executing the above script saved in test.py produces the expected output:

python test.py
  + Exception Group Traceback (most recent call last):
  |   File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 18, in <module>
  |     main()
  |   File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 15, in main
  |     print(converter.loads(s, Test))
  |   File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattr\preconf\json.py", line 19, in loads     
  |     return self.structure(loads(data, **kwargs), cl)
  |   File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattrs\converters.py", line 281, in structure 
  |     return self._structure_func.dispatch(cl)(obj, cl)
  |   File "<cattrs generated structure __main__.Test>", line 14, in structure_Test
  |     if errors: raise __c_cve('While structuring Test', errors, __cl)
  | cattrs.errors.ClassValidationError: While structuring Test (2 sub-exceptions)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Test>", line 5, in structure_Test
    |     res['a'] = __c_structure_a(o['a'])
    | ValueError: invalid literal for int() with base 10: 'foo'
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Test>", line 10, in structure_Test
    |     res['b'] = __c_structure_b(o['b'])
    | ValueError: invalid literal for int() with base 10: 'bar'
    +------------------------------------

Case 2: Adding trio first

Now we add trio as the first import in our test.py script:

import trio
from attrs import define
from cattrs.preconf.json import make_converter as json_converter


@define
class Test:
    a: int
    b: int

converter = json_converter()

def main():
    s = '{"a": "foo", "b": "bar"}' 

    print(converter.loads(s, Test))

if __name__ == "__main__":
    main()

Executing it:

python test.py
Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 19, in <module>
    main()
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 16, in main
    print(converter.loads(s, Test))
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattr\preconf\json.py", line 19, in loads
    return self.structure(loads(data, **kwargs), cl)
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattrs\converters.py", line 281, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "<cattrs generated structure __main__.Test>", line 14, in structure_Test
    if errors: raise __c_cve('While structuring Test', errors, __cl)
cattrs.errors.ClassValidationError: While structuring Test (2 sub-exceptions)

The nice tree traceback is gone. We only get the top level error ClassValidationError. Unfortunately without having the sub exceptions, it is pretty much useless for pinpointing the exact error.

Case 3: adding trio last

Third test, putting trio import last.

from attrs import define
from cattrs.preconf.json import make_converter as json_converter
import trio


@define
class Test:
    a: int
    b: int

converter = json_converter()

def main():
    s = '{"a": "foo", "b": "bar"}'

    print(converter.loads(s, Test))

if __name__ == "__main__":
    main()

output:

python test.py
C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\trio\_core\_multierror.py:511: RuntimeWarning: You seem to already have a custom sys.excepthook 
handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
  warnings.warn(
Error in sys.excepthook:
Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 253, in exceptiongroup_excepthook
    sys.stderr.write("".join(traceback.format_exception(etype, value, tb)))
  File "C:\Users\REDACTED\conda\envs\python310\lib\traceback.py", line 135, in format_exception
    return list(te.format(chain=chain))
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\trio\_core\_multierror.py", line 436, in traceback_exception_format
    yield from traceback_exception_original_format(self, chain=chain)
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 234, in traceback_exception_format
    yield from exc.exceptions[i].format(chain=chain, _ctx=_ctx)
TypeError: traceback_exception_format() got an unexpected keyword argument '_ctx'

Original exception was:
Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 19, in <module>
    main()
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 16, in main
    print(converter.loads(s, Test))
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattr\preconf\json.py", line 19, in loads
    return self.structure(loads(data, **kwargs), cl)
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattrs\converters.py", line 281, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "<cattrs generated structure __main__.Test>", line 14, in structure_Test
cattrs.errors.ClassValidationError: While structuring Test (2 sub-exceptions)

Now this one is worse, because the default handling of ClassValidationError fails too
due to the unexpected keyword argument '_ctx', but at least the not very informative
ClassValidationError is still reported. Also trio warns about its MultiError
sys.excepthook monkeypatching not made because it was already patched. Despite trio
supposedly acting as a good citizen according to its warning, the exceptiongroup
monkeypatching is broken.

Current state of affairs

This kind of sys.excepthook monkeypatching business stepping on each other scores pretty high in terms of hard to find bugs because that's the typical errors at a distance. Obviously here it is not so bad because we know that both trio and exceptiongroup are in this monkeypatching business but when the bug happens in your transitive dependencies, that's very hard. As of now, I still don't know which library caused my original error when I reported the issue.

Mitigation proposed in PR #21

Now we try to see if the mitigation proposed in PR #21 can work:

Case 2 modified:

We protect the failing line in a try Except block, and we use exceptiongroup.format_exception function

import trio
from attrs import define
from cattrs.preconf.json import make_converter as json_converter
from cattrs.errors import ClassValidationError
from exceptiongroup import format_exception


@define
class Test:
    a: int
    b: int

converter = json_converter()

def main():
    s = '{"a": "foo", "b": "bar"}'

    try:
        print(converter.loads(s, Test))
    except ClassValidationError as e:
        format_exception(e)

if __name__ == "__main__":
    main()

output:

python test.py
Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 19, in main
    print(converter.loads(s, Test))
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattr\preconf\json.py", line 19, in loads
    return self.structure(loads(data, **kwargs), cl)
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattrs\converters.py", line 281, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "<cattrs generated structure __main__.Test>", line 14, in structure_Test
    if errors: raise __c_cve('While structuring Test', errors, __cl)
cattrs.errors.ClassValidationError: While structuring Test (2 sub-exceptions)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 24, in <module>
    main()
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 21, in main
    format_exception(e)
  File "C:\Users\REDACTED\conda\envs\python310\lib\functools.py", line 889, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 310, in format_exception
    return list(
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 234, in traceback_exception_format
    yield from exc.exceptions[i].format(chain=chain, _ctx=_ctx)
TypeError: traceback_exception_format() got an unexpected keyword argument '_ctx'

Unfortunately, it doesn't produce the expected results. It is even worse and we are like case 3, but inverted.

Case 3 modified:

As in case 3, trio is imported last, and the rest of the modifications are like Case 2 modified just above:

from attrs import define
from cattrs.preconf.json import make_converter as json_converter
from cattrs.errors import ClassValidationError
from exceptiongroup import format_exception
import trio


@define
class Test:
    a: int
    b: int

converter = json_converter()

def main():
    s = '{"a": "foo", "b": "bar"}'

    try:
        print(converter.loads(s, Test))
    except ClassValidationError as e:
        format_exception(e)

if __name__ == "__main__":
    main()

output:

python test.py
C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\trio\_core\_multierror.py:511: RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
  warnings.warn(
Error in sys.excepthook:
Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 253, in exceptiongroup_excepthook
    sys.stderr.write("".join(traceback.format_exception(etype, value, tb)))
  File "C:\Users\REDACTED\conda\envs\python310\lib\traceback.py", line 135, in format_exception
    return list(te.format(chain=chain))
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\trio\_core\_multierror.py", line 436, in traceback_exception_format
    yield from traceback_exception_original_format(self, chain=chain)
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 234, in traceback_exception_format
    yield from exc.exceptions[i].format(chain=chain, _ctx=_ctx)
TypeError: traceback_exception_format() got an unexpected keyword argument '_ctx'

Original exception was:
Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 19, in main
    print(converter.loads(s, Test))
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattr\preconf\json.py", line 19, in loads
    return self.structure(loads(data, **kwargs), cl)
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattrs\converters.py", line 281, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "<cattrs generated structure __main__.Test>", line 14, in structure_Test
cattrs.errors.ClassValidationError: While structuring Test (2 sub-exceptions)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 24, in <module>
    main()
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 21, in main
    format_exception(e)
  File "C:\Users\REDACTED\conda\envs\python310\lib\functools.py", line 889, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 310, in format_exception
    return list(
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\trio\_core\_multierror.py", line 436, in traceback_exception_format
    yield from traceback_exception_original_format(self, chain=chain)
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 234, in traceback_exception_format
    yield from exc.exceptions[i].format(chain=chain, _ctx=_ctx)
TypeError: traceback_exception_format() got an unexpected keyword argument '_ctx'

This one is the worst of all. We have two times the
TypeError: traceback_exception_format() got an unexpected keyword argument '_ctx' and in between the unformatted ClassValidationError. And also trio doing its warning like it did in Case 3.

Conclusion

I did not dive in the code really, but as an outsider user, this seems a little bit worrying that once you have two monkeypatching libraries for sys.excepthook, all chances are off. And good luck finding which ones are the source... At least I know that
for trio, we'll eventually be compatible: python-trio/trio#2213

The other worry, is that once the excepthook broken, there don't seem to be any way to "unbroken" it. At least it's only broken for ExceptionGroup subclasses, not the regular Exception ones.

[agronholm]

There are two observations to be made here:

1. You're calling format_exception() which only formats the exception and doesn't print anything to the console
2. The call to format_exception() itself crashes, possibly due to a bug

Finally, I'll ask: what exactly would you expect me to do in this situation (besides fixing the bug mentioned above)?

[matmel]

Thanks,

1. You're calling format_exception() which only formats the exception and doesn't print anything to the console

Oooops sorry, here is the amended script that would solve my problem in a meaningful way I think:

import trio
import sys
from attrs import define
from cattrs.preconf.json import make_converter as json_converter
from cattrs.errors import ClassValidationError
from exceptiongroup import format_exception


@define
class Test:
    a: int
    b: int

converter = json_converter()

def main():
    s = '{"a": "foo", "b": "bar"}'

    try:
        print(converter.loads(s, Test))
    except ClassValidationError as e:
        print("".join(format_exception(e)))
        sys.exit(1)


if __name__ == "__main__":
    main()

And the output would be ideally the same as in case 1:

python test.py
  + Exception Group Traceback (most recent call last):
  |   File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 18, in <module>
  |     main()
  |   File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 15, in main
  |     print(converter.loads(s, Test))
  |   File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattr\preconf\json.py", line 19, in loads     
  |     return self.structure(loads(data, **kwargs), cl)
  |   File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattrs\converters.py", line 281, in structure 
  |     return self._structure_func.dispatch(cl)(obj, cl)
  |   File "<cattrs generated structure __main__.Test>", line 14, in structure_Test
  |     if errors: raise __c_cve('While structuring Test', errors, __cl)
  | cattrs.errors.ClassValidationError: While structuring Test (2 sub-exceptions)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Test>", line 5, in structure_Test
    |     res['a'] = __c_structure_a(o['a'])
    | ValueError: invalid literal for int() with base 10: 'foo'
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Test>", line 10, in structure_Test
    |     res['b'] = __c_structure_b(o['b'])
    | ValueError: invalid literal for int() with base 10: 'bar'
    +------------------------------------

[agronholm]

I was able to reproduce the earlier error even without trio. I think it's fixable.

[agronholm]

Confirmed; I committed the fix and an accompanying test to the PR branch.

[matmel]

I launched my test cases again with you latest fix, it solved one of them, but not all

This one works (trio imported last):

import sys
from attrs import define
from cattrs.preconf.json import make_converter as json_converter
from cattrs.errors import ClassValidationError
from exceptiongroup import format_exception, catch, ExceptionGroup
import trio


@define
class Test:
    a: int
    b: int

converter = json_converter()


def err_handler(excgroup: ExceptionGroup) -> None:
    for exc in excgroup.exceptions:
        print(f"Caught exception: \n{''.join(format_exception(exc))}")

def main():
    s = '{"a": "foo", "b": "bar"}'

    try:
        print(converter.loads(s, Test))
    except ClassValidationError as e:
        print("".join(format_exception(e)))
        sys.exit(1)

if __name__ == "__main__":
    main()

Output:

python test.py
  + Exception Group Traceback (most recent call last):
  |   File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 25, in main
  |     print(converter.loads(s, Test))
  |   File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattr\preconf\json.py", line 19, in loads
  |     return self.structure(loads(data, **kwargs), cl)
  |   File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattrs\converters.py", line 281, in structure
  |     return self._structure_func.dispatch(cl)(obj, cl)
  |   File "<cattrs generated structure __main__.Test>", line 14, in structure_Test
  |     if errors: raise __c_cve('While structuring Test', errors, __cl)
  | cattrs.errors.ClassValidationError: While structuring Test (2 sub-exceptions)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Test>", line 5, in structure_Test
    |     res['a'] = __c_structure_a(o['a'])
    | ValueError: invalid literal for int() with base 10: 'foo'
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Test>", line 10, in structure_Test
    |     res['b'] = __c_structure_b(o['b'])
    | ValueError: invalid literal for int() with base 10: 'bar'
    +------------------------------------

This one does not work (trio imported first):

import trio
import sys
from attrs import define
from cattrs.preconf.json import make_converter as json_converter
from cattrs.errors import ClassValidationError
from exceptiongroup import format_exception, catch, ExceptionGroup


@define
class Test:
    a: int
    b: int

converter = json_converter()


def err_handler(excgroup: ExceptionGroup) -> None:
    for exc in excgroup.exceptions:
        print(f"Caught exception: \n{''.join(format_exception(exc))}")

def main():
    s = '{"a": "foo", "b": "bar"}'

    try:
        print(converter.loads(s, Test))
    except ClassValidationError as e:
        print("".join(format_exception(e)))
        sys.exit(1)

if __name__ == "__main__":
    main()

will produce this output:

python test.py
C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\trio\_core\_multierror.py:511: RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
  warnings.warn(
Error in sys.excepthook:
Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 254, in exceptiongroup_excepthook
    sys.stderr.write("".join(traceback.format_exception(etype, value, tb)))
  File "C:\Users\REDACTED\conda\envs\python310\lib\traceback.py", line 135, in format_exception
    return list(te.format(chain=chain))
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\trio\_core\_multierror.py", line 436, in traceback_exception_format
    yield from traceback_exception_original_format(self, chain=chain)
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 235, in traceback_exception_format
    yield from exc.exceptions[i].format(chain=chain, _ctx=_ctx)
TypeError: traceback_exception_format() got an unexpected keyword argument '_ctx'

Original exception was:
Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 25, in main
    print(converter.loads(s, Test))
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattr\preconf\json.py", line 19, in loads
    return self.structure(loads(data, **kwargs), cl)
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\cattrs\converters.py", line 281, in structure
    return self._structure_func.dispatch(cl)(obj, cl)
  File "<cattrs generated structure __main__.Test>", line 14, in structure_Test
cattrs.errors.ClassValidationError: While structuring Test (2 sub-exceptions)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 31, in <module>
    main()
  File "C:\Users\REDACTED\exceptiongroup\debug\test.py", line 27, in main
    print("".join(format_exception(e)))
  File "C:\Users\REDACTED\conda\envs\python310\lib\functools.py", line 889, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 311, in format_exception
    return list(
  File "C:\Users\REDACTED\exceptiongroup\.venv\lib\site-packages\trio\_core\_multierror.py", line 436, in traceback_exception_format
    yield from traceback_exception_original_format(self, chain=chain)
  File "C:\Users\REDACTED\exceptiongroup\src\exceptiongroup\_formatting.py", line 235, in traceback_exception_format
    yield from exc.exceptions[i].format(chain=chain, _ctx=_ctx)
TypeError: traceback_exception_format() got an unexpected keyword argument '_ctx'

[agronholm]

For me it was the other way around. The one where trio is imported last crashed with TypeError while the second one properly displayed the exception group.

[agronholm]

The problem stems from trio._core._multierror unconditionally monkey patching TracebackException, thus overwriting the patching done by exceptiongroup, which of course assumes that the patching it did is still there.

I managed to fix this on my end. Give it a try please.