SBRA-370: Unit tests for the weather forcast use case by A669015 · Pull Request #16 · AI4SIM/model-collection

A669015 · 2022-03-18T16:31:25Z

This PR adds a unit test suite to cover the code of the weather forcast use case.

Should be rebased on master before merging.

github-actions · 2022-03-18T16:33:21Z

Coverage report for weather_forecast/gwd/ use-case

File	Coverage		Missing
All files	`94%`	✅
data.py	`98%`	✅	`163`
models.py	`90%`	✅	`40-43` `96` `106`
trainer.py	`88%`	✅	`74-75` `79`

Minimum allowed coverage is 80%

Generated by 🐒 cobertura-action against e816e65

github-actions · 2022-03-18T16:34:06Z

Coverage report for combustion/gnns/ use-case

File	Coverage		Missing
All files	`87%`	✅
data.py	`94%`	✅	`84` `149` `194` `206`
models.py	`68%`	❌	`85` `116-132` `159-163` `213-217` `238-242`
plotters.py	`98%`	✅	`50-51`
trainer.py	`77%`	❌	`69-74` `78-79`

Minimum allowed coverage is 80%

Generated by 🐒 cobertura-action against e816e65

alxyok · 2022-03-21T16:37:20Z

weather_forecast/gwd/config.py

-                    filemode='w', 
-                    format='%(name)s - %(levelname)s - %(message)s')
+
+logging.basicConfig(filename=os.path.join(logs_path, f'{_experiment_name}.log'),


not even sure this one is used anywhere...

Not sure too, we could embed this point in the whole config refactoring.

alxyok · 2022-03-21T16:37:43Z

weather_forecast/gwd/data.py

    x_feat = 191
    y_feat = 126
-    shard_len = 2355840
+    shard_len = 2355840  # FIXME: 'shard_len' is hardcoded, but should be dynamically set.""


alxyok · 2022-03-21T16:38:48Z

weather_forecast/gwd/config.py

 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
-# 


we should mv this out of each UC. this is just common management stuff..

Yes agree, but that should require a reffactoring at the repo level.
We could propose an US to refactor that, when all AC of the current US will be reach ?

alxyok · 2022-03-21T16:39:59Z

weather_forecast/gwd/data.py

        self.batch_size = batch_size
        self.num_workers = num_workers
        super().__init__()
+        self.train = None


is this required by pylint?

Not raised by flake8 (by pylint, possible). But more generally that is a good practice to define all the attributes in the constructor of the class.

alxyok · 2022-03-21T16:41:21Z

weather_forecast/gwd/models.py

-        self.x_mean = stats['x_mean'].to(self.device)
-        self.x_std = stats['x_std'].to(self.device)
-        self.y_std = stats['y_std'].max().to(self.device)
+        # Init attributes to fake data


is this a fix for the fact that lightning creates the model before the datamodule?

I am not very familiar with lightning, so I don't know if the model is created before the datamodule or not.
But we see with @cbovalo, the stats.pt file is not present on a fresh training, and while the attributes must be defined to continue, we have to init them. The values have been chosen arbitrarly...

alxyok · 2022-03-21T16:43:15Z

weather_forecast/gwd/models.py

            nn.MaxPool1d(pool_size),
            nn.Flatten(),
-            nn.Linear((init_feat * 4 - conv_size) // 2 * (init_feat * 2), out_channels),
+            nn.Linear(480, out_channels),


this should not be hard coded...

Yes, aggree, but with @cbovalo we did not find a way to parametrize this input value. Tthe previous formula was leading to a dimension issue, and the one found on the torch documentation was dependent on several "hidden" parameters (e.g. default values of some unset parameters of MaxPool1d).
So I do not see how to not hard-code it for now. Nevertheless, there is globally a refactoring work that should be done on the use case to parametrize the data dimensions, there is a lot of hard-coded dimension in the code. Maybe a dedicated US could tackle this point ?

weather_forecast/gwd/tests/.gitignore

weather_forecast/gwd/trainer.py

github-actions · 2022-03-28T09:20:56Z

Coverage report for combustion/unets/ use-case

File	Coverage		Missing
All files	`93%`	✅
data.py	`93%`	✅	`61` `72` `74` `113` `116`
models.py	`87%`	✅	`41-42` `45` `48`
trainer.py	`77%`	❌	`59-61` `65-66` `69`

Minimum allowed coverage is 80%

Generated by 🐒 cobertura-action against e816e65

* fix job name * Add branch and PR creation

* Adding pangu weather (#1) * re-add pangu-weather after branch cleanup * reduced spatial resolution for tests * fixed isort issue * fixed black issue * fixed accelerator issue * added .coveragerc and .gitignore * removed unnecessary requirements, and testing github repository name * changed pangu-weather to global-emulation, updated requirement with pip freeze, added global-emulation to images-ci-cd matrix * changed image ci cd to only push the Docker images in model-collection * fixed requirements, removed weatherlearn mentions from readme, renamed pangu into panguweather * added nox back in requirements * fixed panguweather import, removed unused imports * removed debug test from images-ci-cd, deleted config_pangu_micro, updated DropPath to use timm directly, removed unused models (MockPanguModel and CustomPanguModel), removed generate_3d_attention_mask_old and its associated test, aligned trainer.py with the others in weather-forecast * removed unused Dockerfiles * added git to dockerfile * made the init of transformer_module more pythonic * renamed global emulation to nwp emulation * renamed global emulation to nwp emulation in the cicd * [SBRA-837] Add mypy automatic checking to CI/CD (#4) * Add a common mypy.ini file * Add specific mypy config for pangu, Add mypy session to nox * add mypy_cache to gitignore * Add mypy session to ci/cd * Improve PanguWeather type hints * fix pangu lint * Add mypy to gravity-wave-drag * fix test_utils import * fix unit test on data * Add mypy checks to ecrad-3d-correction * Add mypy checks to combustion unet * Add mypy check to combustion gnn * fix format * fix unit test wrt previous code modification * [SBRA-870] Polish code for open access release (#5) * fix time_horizon error print * fix typo * fix casual comments * remove useless comments * improve comment * improve print info --------- Co-authored-by: mikael jacquemont <mikael.jacquemont@eviden.com> * fix format after black update (#6) * Bump mlflow from 2.17.2 to 2.20.3 in /weather-forecast/nwp-emulation/panguweather in the pip group across 1 directory (#3) * Bump mlflow Bumps the pip group with 1 update in the /weather-forecast/nwp-emulation/panguweather directory: [mlflow](https://github.com/mlflow/mlflow). Updates `mlflow` from 2.17.2 to 2.20.3 - [Release notes](https://github.com/mlflow/mlflow/releases) - [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.md) - [Commits](mlflow/mlflow@v2.17.2...v2.20.3) --- updated-dependencies: - dependency-name: mlflow dependency-version: 2.20.3 dependency-type: direct:production dependency-group: pip ... Signed-off-by: dependabot[bot] <support@github.com> * Apply black * Fix mlflow-skinny version * Revert "Fix mlflow-skinny version" This reverts commit 1bf04081ea5f324e91993f9ac24cf718dac5a6f7. * Revert "Apply black" This reverts commit 5c1ab8075cb29dfa13694168f3048dff1b8d62f5. * Fix mlflow-skinny version --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Mikaël Jacquemont <mikael.jacquemont@atos.net> * Refactor projects install (#8) * use uv instead of pip and python * Let uv manage python and dependencies * fix python version for all nox sessions * fix find wheel version * remove useless uv sync * update pangu * update cnns * update ecrad unets * update cnf unets * update gnns * update ci/cd workflow * update docker build * update doc * use large runner for docker image workflow * add dependabot.yml file * Bump the uv group across 4 directories with 1 update (#9) * Bump the uv group across 4 directories with 1 update Bumps the uv group with 1 update in the /weather-forecast/nwp-emulation/panguweather directory: [torch](https://github.com/pytorch/pytorch). Bumps the uv group with 1 update in the /weather-forecast/gravity-wave-drag/cnns directory: [torch](https://github.com/pytorch/pytorch). Bumps the uv group with 1 update in the /weather-forecast/ecrad-3d-correction/unets directory: [torch](https://github.com/pytorch/pytorch). Bumps the uv group with 1 update in the /reactive-flows/cnf-combustion/unets directory: [torch](https://github.com/pytorch/pytorch). Updates `torch` from 2.6.0 to 2.8.0 - [Release notes](https://github.com/pytorch/pytorch/releases) - [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md) - [Commits](pytorch/pytorch@v2.6.0...v2.8.0) Updates `torch` from 2.6.0 to 2.8.0 - [Release notes](https://github.com/pytorch/pytorch/releases) - [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md) - [Commits](pytorch/pytorch@v2.6.0...v2.8.0) Updates `torch` from 2.6.0 to 2.8.0 - [Release notes](https://github.com/pytorch/pytorch/releases) - [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md) - [Commits](pytorch/pytorch@v2.6.0...v2.8.0) Updates `torch` from 2.6.0 to 2.8.0 - [Release notes](https://github.com/pytorch/pytorch/releases) - [Changelog](https://github.com/pytorch/pytorch/blob/main/RELEASE.md) - [Commits](pytorch/pytorch@v2.6.0...v2.8.0) --- updated-dependencies: - dependency-name: torch dependency-version: 2.8.0 dependency-type: direct:production dependency-group: uv - dependency-name: torch dependency-version: 2.8.0 dependency-type: direct:production dependency-group: uv - dependency-name: torch dependency-version: 2.8.0 dependency-type: direct:production dependency-group: uv - dependency-name: torch dependency-version: 2.8.0 dependency-type: direct:production dependency-group: uv ... Signed-off-by: dependabot[bot] <support@github.com> * fix mypy issue * replacing float casting by item() --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: VINCENT, LIONEL <lionel.vincent.dm@atos.net> Co-authored-by: mikael10j <mikael.jacquemont@eviden.com> * improve install using explicit index and find-link for torch and pyg (#10) * Bump the uv group across 5 directories with 8 updates (#11) Bumps the uv group with 8 updates in the /reactive-flows/cnf-combustion/gnns directory: | Package | From | To | | --- | --- | --- | | [certifi](https://github.com/certifi/python-certifi) | `2022.12.7` | `2024.7.4` | | [filelock](https://github.com/tox-dev/py-filelock) | `3.20.0` | `3.20.3` | | [idna](https://github.com/kjd/idna) | `3.4` | `3.7` | | [pillow](https://github.com/python-pillow/Pillow) | `12.0.0` | `12.1.1` | | [requests](https://github.com/psf/requests) | `2.28.1` | `2.32.4` | | [setuptools](https://github.com/pypa/setuptools) | `70.2.0` | `78.1.1` | | [urllib3](https://github.com/urllib3/urllib3) | `1.26.13` | `2.6.3` | | [werkzeug](https://github.com/pallets/werkzeug) | `3.1.5` | `3.1.6` | Bumps the uv group with 1 update in the /reactive-flows/cnf-combustion/unets directory: [werkzeug](https://github.com/pallets/werkzeug). Bumps the uv group with 1 update in the /weather-forecast/ecrad-3d-correction/unets directory: [werkzeug](https://github.com/pallets/werkzeug). Bumps the uv group with 4 updates in the /weather-forecast/gravity-wave-drag/cnns directory: [filelock](https://github.com/tox-dev/py-filelock), [idna](https://github.com/kjd/idna), [setuptools](https://github.com/pypa/setuptools) and [werkzeug](https://github.com/pallets/werkzeug). Bumps the uv group with 5 updates in the /weather-forecast/nwp-emulation/panguweather directory: | Package | From | To | | --- | --- | --- | | [certifi](https://github.com/certifi/python-certifi) | `2022.12.7` | `2024.7.4` | | [filelock](https://github.com/tox-dev/py-filelock) | `3.20.0` | `3.20.3` | | [idna](https://github.com/kjd/idna) | `3.4` | `3.7` | | [pillow](https://github.com/python-pillow/Pillow) | `12.0.0` | `12.1.1` | | [setuptools](https://github.com/pypa/setuptools) | `70.2.0` | `78.1.1` | Updates `certifi` from 2022.12.7 to 2024.7.4 - [Commits](certifi/python-certifi@2022.12.07...2024.07.04) Updates `filelock` from 3.20.0 to 3.20.3 - [Release notes](https://github.com/tox-dev/py-filelock/releases) - [Changelog](https://github.com/tox-dev/filelock/blob/main/docs/changelog.rst) - [Commits](tox-dev/filelock@3.20.0...3.20.3) Updates `idna` from 3.4 to 3.7 - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](kjd/idna@v3.4...v3.7) Updates `pillow` from 12.0.0 to 12.1.1 - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@12.0.0...12.1.1) Updates `requests` from 2.28.1 to 2.32.4 - [Release notes](https://github.com/psf/requests/releases) - [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md) - [Commits](psf/requests@v2.28.1...v2.32.4) Updates `setuptools` from 70.2.0 to 78.1.1 - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](pypa/setuptools@v70.2.0...v78.1.1) Updates `urllib3` from 1.26.13 to 2.6.3 - [Release notes](https://github.com/urllib3/urllib3/releases) - [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst) - [Commits](urllib3/urllib3@1.26.13...2.6.3) Updates `werkzeug` from 3.1.5 to 3.1.6 - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@3.1.5...3.1.6) Updates `werkzeug` from 3.1.5 to 3.1.6 - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@3.1.5...3.1.6) Updates `werkzeug` from 3.1.5 to 3.1.6 - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@3.1.5...3.1.6) Updates `filelock` from 3.20.0 to 3.20.3 - [Release notes](https://github.com/tox-dev/py-filelock/releases) - [Changelog](https://github.com/tox-dev/filelock/blob/main/docs/changelog.rst) - [Commits](tox-dev/filelock@3.20.0...3.20.3) Updates `idna` from 3.4 to 3.7 - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](kjd/idna@v3.4...v3.7) Updates `setuptools` from 70.2.0 to 78.1.1 - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](pypa/setuptools@v70.2.0...v78.1.1) Updates `werkzeug` from 3.1.5 to 3.1.6 - [Release notes](https://github.com/pallets/werkzeug/releases) - [Changelog](https://github.com/pallets/werkzeug/blob/main/CHANGES.rst) - [Commits](pallets/werkzeug@3.1.5...3.1.6) Updates `certifi` from 2022.12.7 to 2024.7.4 - [Commits](certifi/python-certifi@2022.12.07...2024.07.04) Updates `filelock` from 3.20.0 to 3.20.3 - [Release notes](https://github.com/tox-dev/py-filelock/releases) - [Changelog](https://github.com/tox-dev/filelock/blob/main/docs/changelog.rst) - [Commits](tox-dev/filelock@3.20.0...3.20.3) Updates `idna` from 3.4 to 3.7 - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](kjd/idna@v3.4...v3.7) Updates `pillow` from 12.0.0 to 12.1.1 - [Release notes](https://github.com/python-pillow/Pillow/releases) - [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst) - [Commits](python-pillow/Pillow@12.0.0...12.1.1) Updates `setuptools` from 70.2.0 to 78.1.1 - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](pypa/setuptools@v70.2.0...v78.1.1) --- updated-dependencies: - dependency-name: certifi dependency-version: 2024.7.4 dependency-type: indirect dependency-group: uv - dependency-name: filelock dependency-version: 3.20.3 dependency-type: indirect dependency-group: uv - dependency-name: idna dependency-version: '3.7' dependency-type: indirect dependency-group: uv - dependency-name: pillow dependency-version: 12.1.1 dependency-type: indirect dependency-group: uv - dependency-name: requests dependency-version: 2.32.4 dependency-type: indirect dependency-group: uv - dependency-name: setuptools dependency-version: 78.1.1 dependency-type: indirect dependency-group: uv - dependency-name: urllib3 dependency-version: 2.6.3 dependency-type: indirect dependency-group: uv - dependency-name: werkzeug dependency-version: 3.1.6 dependency-type: indirect dependency-group: uv - dependency-name: werkzeug dependency-version: 3.1.6 dependency-type: indirect dependency-group: uv - dependency-name: werkzeug dependency-version: 3.1.6 dependency-type: indirect dependency-group: uv - dependency-name: filelock dependency-version: 3.20.3 dependency-type: indirect dependency-group: uv - dependency-name: idna dependency-version: '3.7' dependency-type: indirect dependency-group: uv - dependency-name: setuptools dependency-version: 78.1.1 dependency-type: indirect dependency-group: uv - dependency-name: werkzeug dependency-version: 3.1.6 dependency-type: indirect dependency-group: uv - dependency-name: certifi dependency-version: 2024.7.4 dependency-type: indirect dependency-group: uv - dependency-name: filelock dependency-version: 3.20.3 dependency-type: indirect dependency-group: uv - dependency-name: idna dependency-version: '3.7' dependency-type: indirect dependency-group: uv - dependency-name: pillow dependency-version: 12.1.1 dependency-type: indirect dependency-group: uv - dependency-name: setuptools dependency-version: 78.1.1 dependency-type: indirect dependency-group: uv ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix README (#12) * fix README * Add missing line Co-authored-by: Mikaël Jacquemont <mikael.jacquemont@atos.net> --------- Co-authored-by: Mikaël Jacquemont <mikael.jacquemont@atos.net> * add workflow to automate PR proposal in public when merging in private (#13) * fix branch name (#14) * fix dependabot.yaml (#15) * Automate PR proposal in public - fix credentials (#16) * fix job name * Add branch and PR creation --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: MarcMichouxEviden <106808701+MarcMichouxAtos@users.noreply.github.com> Co-authored-by: Mikaël Jacquemont <mikael.jacquemont@atos.net> Co-authored-by: mikael jacquemont <mikael.jacquemont@eviden.com> Co-authored-by: Lionel VINCENT <97742788+A669015@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: VINCENT, LIONEL <lionel.vincent.dm@atos.net>

* fix job name * Add branch and PR creation

A669015 requested review from alxyok, cbovalo, leonicoletti and rmslm March 18, 2022 16:31

alxyok suggested changes Mar 21, 2022

View reviewed changes

alxyok approved these changes Mar 28, 2022

View reviewed changes

A669015 force-pushed the test-wf branch from 8a55fd6 to 3819952 Compare March 28, 2022 09:18

leonicoletti approved these changes Mar 28, 2022

View reviewed changes

cbovalo approved these changes Mar 28, 2022

View reviewed changes

A669015 added 11 commits March 28, 2022 15:11

models.py: fix double variable placement

0c87a99

requirements.txt: fix missing requirement

e01079c

tests: add unit test for data.py module

d1fd4d2

tests: add unit tests for config.py module

7746fd2

models.py: add unit tests for the models

4b7e397

trainer.py: add unit tests for the trainer module

d09c780

tests: improve tests

8f63103

test_trainer.py: fix gpu tests

84f1703

trainer.py: fix CLI require data imported

184e240

tests: change after PR review

ba8c8db

nox_ref_file.py: improve test reporting

e816e65

A669015 force-pushed the test-wf branch from 3819952 to e816e65 Compare March 28, 2022 13:11

A669015 merged commit 955407d into main Mar 28, 2022

A669015 deleted the test-wf branch April 5, 2022 13:33

ai4sim-pr-app bot pushed a commit that referenced this pull request Mar 2, 2026

Automate PR proposal in public - fix credentials (#16)

8d30bdc

* fix job name * Add branch and PR creation

github-actions bot pushed a commit that referenced this pull request Mar 2, 2026

Automate PR proposal in public - fix credentials (#16)

4b4c69f

* fix job name * Add branch and PR creation

Conversation

A669015 commented Mar 18, 2022

Uh oh!

github-actions bot commented Mar 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Mar 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions bot commented Mar 18, 2022 •

edited

Loading

github-actions bot commented Mar 18, 2022 •

edited

Loading

github-actions bot commented Mar 28, 2022 •

edited

Loading