Ci quality workflows (#1423)

* Add inference tests

* Clean up

* Rename test graph file

* Add readme for tests

* Separate server fixture

* test file name change

* Assert images are generated

* Clean up comments

* Add __init__.py so tests can run with command line `pytest`

* Fix command line args for pytest

* Loop all samplers/schedulers in test_inference.py

* Ci quality workflows compare (#1)

* Add image comparison tests

* Comparison tests do not pass with empty metadata

* Ensure tests are run in correct order

* Save image files  with test name

* Update tests readme

* Reduce step counts in tests to ~halve runtime

* Ci quality workflows build (#2)

* Add build test github workflow
This commit is contained in:
enzymezoo-code
2023-09-18 22:18:06 -05:00
committed by GitHub
parent b92bf8196e
commit 26cd8405dd
10 changed files with 728 additions and 0 deletions

29
tests/README.md Normal file
View File

@@ -0,0 +1,29 @@
# Automated Testing
## Running tests locally
Additional requirements for running tests:
```
pip install pytest
pip install websocket-client==1.6.1
opencv-python==4.6.0.66
scikit-image==0.21.0
```
Run inference tests:
```
pytest tests/inference
```
## Quality regression test
Compares images in 2 directories to ensure they are the same
1) Run an inference test to save a directory of "ground truth" images
```
pytest tests/inference --output_dir tests/inference/baseline
```
2) Make code edits
3) Run inference and quality comparison tests
```
pytest
```