Add CLI

[bollwyvl]

Thanks for maintaining this!

This adds a CLI conda-recipe-v2-schema with two subcommands, and its dependencies under a [cli] extra, and some tests.

• generate the schema JSON (but without the PYTHONPATH-like side-effects of -m)
• validate (conda-forge) recipes while working on the schema

It uses the JSON schema and pydantic to try to get the most helpful results: not that the pydantic errors are all that much better, but the schema by itself doesn't help sometimes, and could point to some better schema conventions (e.g. if/then instead of oneOf can be good for more understandable errors).

My test case (as a prelude to another PR for #/tests/*/python/python_version/if):

pixi r validate --work-dir=build/.cache/crs -c py-rattler

Details

"https://raw.githubusercontent.com/conda-forge/py-rattler-feedstock/refs/heads/main/recipe/recipe.yaml":
- message: |-
    {'context': {'name': 'py-rattler', 'python_name': 'py_rattler', 'version': '0.20.0'}, 'recipe': {'name': 'py-rattler', 'version': '${{ version }}'}, 'source': {'url': 'https://pypi.org/packages/source/${{ name[0] }}/${{ name }}/${{ python_name }}-${{ version }}.tar.gz', 'sha256': 'bab20b8c96c77cc640ee7e88653e72f8faf86d0a3922e844d15b47e29bc1d969'}, 'build': {'number': 0}, 'outputs': [{'package': {'name': 'py-rattler'}, 'build': {'skip': 'is_abi3 and not is_python_min', 'script': 'build_base', 'python': {'version_independent': True}}, 'requirements': {'build': [{'if': 'build_platform != target_platform', 'then': ['python', 'cross-python_${{ target_platform }}', 'maturin >=1.2.2,<2']}, "${{ compiler('c') }}", '${{ stdlib("c") }}', "${{ compiler('rust') }}", 'cargo-bundle-licenses'], 'host': ['python', {'if': 'is_abi3', 'then': 'python-abi3'}, 'maturin >=1.2.2,<2', 'pip', {'if': 'unix', 'then': ['openssl']}], 'run': ['python']}, 'tests': [{'python': {'imports': ['rattler'], 'python_version': [{'if': 'is_abi3', 'then': "${{ python_min ~ '.*' }}"}, '*']}}, {'if': 'is_abi3', 'then': {'script': [{'if': 'win', 'then': 'abi3audit %PREFIX%/Lib/site-packages/rattler/rattler.pyd -s -v --assume-minimum-abi3 ${{ python_min }}'}, {'if': 'not win', 'then': ['export SP_DIR=$(python -c "import site; print(site.getsitepackages()[0])")', 'abi3audit $SP_DIR/rattler/rattler.abi3.so -s -v --assume-minimum-abi3 ${{ python_min }}']}], 'requirements': {'run': ['abi3audit']}}}]}, {'package': {'name': 'py_rattler'}, 'build': {'noarch': 'generic', 'skip': 'not (linux and x86_64 and is_python_min)'}, 'requirements': {'run': ['${{ pin_subpackage(\'py-rattler\', upper_bound="x.x.x") }}']}, 'tests': [{'python': {'imports': ['rattler']}}], 'about': {'license_file': []}}], 'about': {'homepage': 'https://github.com/conda/rattler', 'license': 'BSD-3-Clause', 'license_file': ['LICENSE', 'py-rattler/THIRDPARTY.yml'], 'summary': 'A blazing fast library to work with the conda ecosystem', 'description': 'Rattler is a library that provides common functionality used within the conda\necosystem. The goal of the library is to enable programs and other libraries to\neasily interact with the conda ecosystem without being dependent on Python. Its\nprimary use case is as a library that you can use to provide conda related\nworkflows in your own tools.\n', 'repository': 'https://github.com/conda/rattler'}, 'extra': {'feedstock-name': 'py-rattler', 'recipe-maintainers': ['baszalmstra', 'pavelzw']}} is not valid under any of the given schemas
  path: '#/'
  schema_path: '#/anyOf/'
- ComplexRecipe: |-
    23 validation errors for ComplexRecipe
    outputs.Output
      Input should be a valid dictionary or instance of Output [type=model_type, input_value=[{'package': {'name': 'py...: {'license_file': []}}], input_type=list]
        For further information visit https://errors.pydantic.dev/2.8/v/model_type
    outputs.IfStatement
      Input should be a valid dictionary or instance of IfStatement [type=model_type, input_value=[{'package': {'name': 'py...: {'license_file': []}}], input_type=list]
        For further information visit https://errors.pydantic.dev/2.8/v/model_type
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.ScriptTestElement.python
      Extra inputs are not permitted [type=extra_forbidden, input_value={'imports': ['rattler'], ...n_min ~ '.*' }}"}, '*']}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.PythonTestElement.python.python_version.str
      Input should be a valid string [type=string_type, input_value=[{'if': 'is_abi3', 'then'...on_min ~ '.*' }}"}, '*'], input_type=list]
        For further information visit https://errors.pydantic.dev/2.8/v/string_type
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.PythonTestElement.python.python_version.list[str].0
      Input should be a valid string [type=string_type, input_value={'if': 'is_abi3', 'then':...{ python_min ~ '.*' }}"}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/string_type
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.PerlTestElement.perl
      Field required [type=missing, input_value={'python': {'imports': ['..._min ~ '.*' }}"}, '*']}}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/missing
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.PerlTestElement.python
      Extra inputs are not permitted [type=extra_forbidden, input_value={'imports': ['rattler'], ...n_min ~ '.*' }}"}, '*']}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.RTestElement.r
      Field required [type=missing, input_value={'python': {'imports': ['..._min ~ '.*' }}"}, '*']}}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/missing
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.RTestElement.python
      Extra inputs are not permitted [type=extra_forbidden, input_value={'imports': ['rattler'], ...n_min ~ '.*' }}"}, '*']}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.DownstreamTestElement.downstream
      Field required [type=missing, input_value={'python': {'imports': ['..._min ~ '.*' }}"}, '*']}}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/missing
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.DownstreamTestElement.python
      Extra inputs are not permitted [type=extra_forbidden, input_value={'imports': ['rattler'], ...n_min ~ '.*' }}"}, '*']}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.PackageContentTest.package_contents
      Field required [type=missing, input_value={'python': {'imports': ['..._min ~ '.*' }}"}, '*']}}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/missing
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.PackageContentTest.python
      Extra inputs are not permitted [type=extra_forbidden, input_value={'imports': ['rattler'], ...n_min ~ '.*' }}"}, '*']}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.IfStatement[Union[ScriptTestElement, PythonTestElement, PerlTestElement, RTestElement, DownstreamTestElement, PackageContentTest]].if
      Field required [type=missing, input_value={'python': {'imports': ['..._min ~ '.*' }}"}, '*']}}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/missing
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.IfStatement[Union[ScriptTestElement, PythonTestElement, PerlTestElement, RTestElement, DownstreamTestElement, PackageContentTest]].then
      Field required [type=missing, input_value={'python': {'imports': ['..._min ~ '.*' }}"}, '*']}}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/missing
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.IfStatement[Union[ScriptTestElement, PythonTestElement, PerlTestElement, RTestElement, DownstreamTestElement, PackageContentTest]].python
      Extra inputs are not permitted [type=extra_forbidden, input_value={'imports': ['rattler'], ...n_min ~ '.*' }}"}, '*']}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.Output.tests.0.list[union[ScriptTestElement,PythonTestElement,PerlTestElement,RTestElement,DownstreamTestElement,PackageContentTest,IfStatement[Union[ScriptTestElement, PythonTestElement, PerlTestElement, RTestElement, DownstreamTestElement, PackageContentTest]]]]
      Input should be a valid list [type=list_type, input_value={'python': {'imports': ['..._min ~ '.*' }}"}, '*']}}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/list_type
    outputs.list[union[Output,IfStatement]].0.IfStatement.if
      Field required [type=missing, input_value={'package': {'name': 'py-...run': ['abi3audit']}}}]}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/missing
    outputs.list[union[Output,IfStatement]].0.IfStatement.then
      Field required [type=missing, input_value={'package': {'name': 'py-...run': ['abi3audit']}}}]}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/missing
    outputs.list[union[Output,IfStatement]].0.IfStatement.package
      Extra inputs are not permitted [type=extra_forbidden, input_value={'name': 'py-rattler'}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.IfStatement.build
      Extra inputs are not permitted [type=extra_forbidden, input_value={'skip': 'is_abi3 and not...ion_independent': True}}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.IfStatement.requirements
      Extra inputs are not permitted [type=extra_forbidden, input_value={'build': [{'if': 'build_...']}], 'run': ['python']}, input_type=dict]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden
    outputs.list[union[Output,IfStatement]].0.IfStatement.tests
      Extra inputs are not permitted [type=extra_forbidden, input_value=[{'python': {'imports': [...'run': ['abi3audit']}}}], input_type=list]
        For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden

[bollwyvl]

Seems fast enough.. against my local conda-forge feedstocks:

time pixi r conda-recipe-v2-schema --work-dir=build/.cache/crs $(ls ../feedstocks/*/recipe/recipe.yaml) 

# ... a lot of "is not valid under any of"

!!! 8 findings in 293 recipes

real	0m3.029s
user	0m2.896s
sys	0m0.095s

[Hofer-Julian]

Nice idea. I had a few comments

[Hofer-Julian]

I don't get this logic. Why would I not always use the class from the pydantic model?

[bollwyvl]

not always use the class

Because the pydantic model is an intermediate that isn't published anywhere, while the JSON schema is already in use by many tools, if only via rawgithubusercontent or whtatever URLs.

[Hofer-Julian]

Then I don't understand what you would like us to do with this script :)

I assumed the point of it is that you check out this repo and run it as a Pixi task

[Hofer-Julian]

Why not only catch Pydantic exceptions here?

[bollwyvl]

Welp, I don't care to know much about the pydantic exception API and what it might throw today, or change on a patch release tomorrow. In this context of a tiny dev tool that exists to help write schema, I'd rather not have to try/catch anything, and get back a list of errors (a la Validator.iter_errors), and do the "normal thing" of exploding on uncaught exceptions.

[pavelzw]

in general, i'm wondering whether a tool that just does this for all yaml/json files would be more helpful. you could check at https://www.schemastore.org/ whether the filename recipe.yaml has a json schema and compare it against that. this would be more general than the conda-forge use case and i can imagine me using it in more places than only recipe.yaml

[pavelzw]

best check before building, maybe such a tool already exists somewhere

[Hofer-Julian]

Yeah, I agree. If this is a general JSON schema validator, then this repo is probably not the right place for it

[bollwyvl]

yeah, sure schema everywhere, hooray!

again, as mentioned, the intent here is to make it easier for contributors to this repo to describe reproducible schema issues and their fixes.

schemastore

while useful, schemastore is... kinda bad on many levels (privacy, accuracy).

general

yeah, there are a lot of related tools (see also #29)... but YAML is weird enough across implementations (see: executable !!tags, lack of correct anchor support, etc.) that the finer points of being valid data sometimes is missed. Anyhow, all those tools don't know about pydantic (for good reasons) much less this repo's pydantic.

the conda-forge use case

i'll wager there are more github.com/conda-forge/.../recipe.yaml than anywhere else public, so doesn't seem like much of a stretch.