Regression Tests - input files are not copied

[pschuenemann]

Hello,

After compiling OpenFAST with Visual Studio, I wanted to test my compiled executable by running the regression tests. Thus, I followed the instructions given in https://openfast.readthedocs.io/en/dev/source/testing/regression_test_windows.html, i.e. I tried to run the test manually by using the python script manualRegressionTest.py. However, all of the test cases failed!

To solve the issue I had a deeper look and found that there is no reg_tests subdirectory in my build directory and thus no input files for the test cases, which I assume is the reason for the failed tests.

I have expected, that this directory and its content is created automatically, when running the script manualRegressionTest.py as it is also described in section 5.2.2. of the documentation at https://openfast.readthedocs.io/en/dev/source/testing/regression_test.html:

In both modes of execution a subdirectory is created in the build directory called reg_tests where all of the input files for the test cases are copied and all of the locally generated outputs are stored.

I also had a look into manualRegressionTest.py, but couldn't find any command that is creating the build/reg_tests directory and its content.

Is that a bug in manualRegressionTest.py or is the documentation just missing some further steps, which I'm not aware of, right now, that has to be done before running manualRegressionTest.py in order to create the build/reg_tests directory beforehand?

Best regards,
Paul

[rafmudaf]

Did you compile on Windows with a Visual Studio solution generated by CMake or with the included solution in the vs-build directory?

If you used cmake, using the flag -DBUILD_TESTING=ON will copy all of the files to the appropriate location in the build directory.

If you used the vs-build solution, there is indeed a missing step in the documentation. In this case, you'll need to copy the contents of openfast/reg_tests/r-test to openfast/build/reg_tests.

The action item here is to update the testing documentation. There have been a few updates to this documentation over the past year, but the methods for Mac and windows have become more consistent so that needs to be documented.

[pschuenemann]

I am using the included solution from the vs-build directory. Copying the contents of openfast/reg_tests/r-test to openfast/build/reg_tests solved my problem. To prevent others from running into the same problem, I created an update to the documentation and filed a pull request. (I hope that's fine and the request is correct, because it's my first one.)

Thanks @rafmudaf for this solution. With this I was able to run all regression tests and most of them have passed. The only ones that have not passed are those using linearization. However, I have manually compared my locally created results with the baseline results: The only differences are in outputting very small numbers near to zero, like for example my local results show 0.000000000E+000 where the baseline shows -1.772120205E-032. As those are the only differences I am not worried about that to much, or should I? What do you think?
Anyhow, I'm quite shure that those failed regression tests are not related with this current issue.

P.S.: For those interested, as an example for my failed tests, I am attaching my results for the Ideal_Beam_Free_Free_Linear case along with the baseline results and a comparison report:
localResults_pschuenemann_Ideal_Beam_Free_Free_Linear.zip
baselineResults_Ideal_Beam_Free_Free_Linear.zip
comparisonReport.zip

[rafmudaf]

fixed by #255