Skip to content

Expected failures are tried to be listed erroneously as failures and reframe crashes #3526

@vkarak

Description

@vkarak

This happens only when there are also real failures so that a failure report is tried to be generated. Here is how to reproduced:

  1. Apply the following patch:
diff --git a/unittests/resources/checks/frontend_checks.py b/unittests/resources/checks/frontend_checks.py
index 4e0304ce0..3d2d5cae7 100644
--- a/unittests/resources/checks/frontend_checks.py
+++ b/unittests/resources/checks/frontend_checks.py
@@ -77,7 +77,7 @@ class SanityFailureCheck(BaseFrontendCheck):
 class PerformanceFailureCheck(BaseFrontendCheck):
     reference = {
         '*': {
-            'perf': (20, -0.1, 0.1, 'Gflop/s')
+            'perf': xfail('bug 123', (20, -0.1, 0.1, 'Gflop/s'))
         }
     }
 
  1. Run
./bin/reframe -c unittests/resources/checks/frontend_checks.py -r

And the output is the following:

FAILURE INFO for PerformanceFailureCheck (run: 1/1)
  * Description: 
  * System partition: generic:default
  * Environment: builtin
  * Test file: /Users/vkarakasis/Devel/reframe/unittests/resources/checks/frontend_checks.py
  * Stage directory: /Users/vkarakasis/Devel/reframe/stage/generic/default/builtin/PerformanceFailureCheck
  * Node list: vkarakasis-mlt
  * Job type: local (id=31514)
  * Dependencies (conceptual): []
  * Dependencies (actual): []
  * Maintainers: []
  * Failing phase: None
  * Rerun with '-n /c4348ca0 -p builtin --system generic -r'
ERROR: run session stopped: key error: 'fail_info'
ERROR: Traceback (most recent call last):
  File "/Users/vkarakasis/Devel/reframe/reframe/frontend/cli.py", line 1672, in main
    printer.failure_report(
    ~~~~~~~~~~~~~~~~~~~~~~^
        report,
        ^^^^^^^
        rerun_info=not options.distribute,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        global_stats=options.duration or options.reruns
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/vkarakasis/Devel/reframe/reframe/frontend/printer.py", line 185, in failure_report
    _print_failure_info(r, run_no, len(report['runs']))
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/vkarakasis/Devel/reframe/reframe/frontend/printer.py", line 148, in _print_failure_info
    if isinstance(rec['fail_info']['exc_value'], BuildError):
                  ~~~^^^^^^^^^^^^^
KeyError: 'fail_info'

The PerformanceFailureCheck shouldn't have been listed at all.

Metadata

Metadata

Assignees

Type

Projects

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions