Simplify the process of creating baselines using Kaleido and improve image & other export test systems#5724
Simplify the process of creating baselines using Kaleido and improve image & other export test systems#5724
Conversation
OK. For that we need to install |
archmoj
force-pushed
the
switch-to-kaleido
branch
from
92c8306 to
0f9292c
Compare
| name: install kaleido v0.2.1 | ||
| command: python3 -m pip install kaleido==0.2.1 |
There was a problem hiding this comment.
| name: install kaleido v0.2.1 | |
| command: python3 -m pip install kaleido==0.2.1 | |
| name: install kaleido | |
| command: pip install kaleido==0.2.1 |
again, in this env we should only have one pip to worry about... also let's not repeat the version number. (applies several places below too)
| # install required fonts (if missing) on ubuntu | ||
| sudo cp -r .circleci/fonts/ /usr/share/ && sudo fc-cache -f | ||
| # upgrade pip (if needed) | ||
| python3 -m pip install --upgrade pip |
There was a problem hiding this comment.
To be clear re: my earlier comments - here the instructions are for local use, so using the longer python3 -m pip syntax is reasonable as a way to indicate to users that it's important to do this with Python 3 if nothing else :)
| # 'old' image server and we use orca | ||
| # | ||
| # having problem creating baselines for 2 mapbox mocks using kaleido | ||
| # we must use orca |
There was a problem hiding this comment.
But other mapbox mocks work? what happens with these two?
There was a problem hiding this comment.
I get the following error cc: @jonmmease
Transform failed with error code 525: Mapbox error.
when try to generate baselines for these two:
test/image/compare_pixels_test.js
Outdated
| } | ||
| // mapbox behave differently from run-to-run | ||
| // skip error on mapbox diff | ||
| if(mockName.substr(7) === 'mapbox_') continue; |
There was a problem hiding this comment.
wait, so we're including mapbox images in these tests, and generating diffs from them, but not allowing those diffs to fail? When do we actually look at these diffs then?
There was a problem hiding this comment.
We should have a look at diffs during the release cycle.
There was a problem hiding this comment.
I could add a note here after https://github.com/plotly/dev-docs/pull/188 and this PR merged.
There was a problem hiding this comment.
To filter mapbox mocks this should be substr(0, 7) not substr(7).
There was a problem hiding this comment.
Fixes are in b1de0bb.
Thanks to @antoinerg comment, now we test everything with zero tolerance except mapbox_* and gl3d_bunny-hull which are tested with certain tolerance.
|
Next we should possibly drop |
OK. I found The only remaining |
2 participants
Closes #5625
Supersedes #3237 and #2615 🎉 🎉
Bye bye
testbedimage server and possiblynpm run docker -- run(for ubuntu users)Hello
Kaleido!@jonmmease
@plotly/plotly_js
Please review commit by commit.
TODOs:
testbed