How to run tests and test suites

Published

February 11, 2026

ValidMind provides many built-in tests and test suites, which help you produce documentation during stages of the model lifecycle, where you need to ensure that your work satisfies regulatory and risk management requirements.

Explore tests

Start by exploring the ValidMind Library's available tests and tests suites:

When do I use tests and test suites?

While you have the flexibility to decide when to use which ValidMind tests, we have identified a few typical scenarios with their own characteristics and needs:

To document and validate your dataset:

For generic tabular datasets: use the tabular_dataset test suite.
For time-series datasets: use the time_series_dataset test suite.

To document and validate your model:

For binary classification models: use the classifier test suite.
For time series models: use the timeseries test suite.

To document a binary classification model and the relevant dataset end-to-end:

Use the classifier_full_suite test suite.

Run tests

Next, learn how to use the ValidMind Library to run tests on datasets and models:

Run dataset based tests

Use the ValidMind Library's run_test function to run built-in or custom tests that take any dataset or model as input. These tests generate outputs in the form of text, tables, and images that get populated in model documentation.

Run comparison tests

Use the ValidMind Library's run_test function to run built-in or custom tests that take any combination of datasets or models as inputs. Comparison tests allow you to run existing test over different groups of inputs and produces a single consolidated list of outputs in the form of text, tables, and images that get populated in model documentation.

Customize test result descriptions

When you run ValidMind tests, test descriptions are automatically generated with LLM using the test results, the test name, and the static test definitions provided in the test's docstring. While this metadata offers valuable high-level overviews of tests, insights produced by the LLM-based descriptions may not always align with your specific use…

Enable PII detection in tests

Learn how to enable and configure Personally Identifiable Information (PII) detection when running tests with the ValidMind Library. Choose whether or not to include PII in test descriptions generated, or whether or not to include PII in test results logged to the ValidMind Platform.

Dataset Column Filters when Running Tests

To run a test on a dataset but only include certain columns from that dataset, you can use the new columns option. This is done by passing a dictionary for the dataset input for the test instead of the dataset object or dataset input ID directly. This dictionary should have the following keys:

Run tests with multiple datasets

To support running tests that require more than one dataset, ValidMind provides a mechanim that allows you to pass multiple datasets as inputs.

Understand and utilize `RawData` in ValidMind tests

Test functions in ValidMind can return a special object called RawData, which holds intermediate or unprocessed data produced somewhere in the test logic but not returned as part of the test's visible output, such as in tables or figures.

Document multiple results for the same test

Documentation templates facilitate the presentation of multiple unique test results for a single test.

Run individual documentation sections

For targeted testing, you can run tests on individual sections or specific groups of sections in your model documentation.

Run documentation tests with custom configurations

When running documentation tests, you can configure inputs and parameters for individual tests by passing a config as a parameter.

Can I use my own tests?

Absolutely! ValidMind supports custom tests that you develop yourself or that are provided by third-party test libraries, also referred to as test providers.

We provide instructions with code examples that you can adapt:

Implement custom tests

Custom tests extend the functionality of ValidMind, allowing you to document any model or use case with added flexibility.

Integrate external test providers

What's next

Learn more about using the other features of the ValidMind Library and browse our code samples by use case:

How to use ValidMind Library features

Browse our range of Jupyter Notebooks demonstrating how to use the core features of the ValidMind Library. Use these how-to notebooks to get familiar with the library's capabilities and apply them to your own use cases.

Code samples

Our Jupyter Notebook code samples showcase the capabilities and features of the ValidMind Library, while also providing you with useful examples that you can build on and adapt for your own use cases.

Explore tests

Explore tests

Explore test suites

Test sandbox beta

Test descriptions

When do I use tests and test suites?

Run tests

Run dataset based tests

Run comparison tests

Customize test result descriptions

Enable PII detection in tests

Dataset Column Filters when Running Tests

Run tests with multiple datasets

Understand and utilize RawData in ValidMind tests

Document multiple results for the same test

Run individual documentation sections

Run documentation tests with custom configurations

Can I use my own tests?

Implement custom tests

Integrate external test providers

What's next

How to use ValidMind Library features

Code samples

Test sandbox ^beta

Understand and utilize `RawData` in ValidMind tests