ONNX CI Pipelines#
CI pipelines matrix:
When it runs
Config
Test
Linux-CI
Every PR
- Ubuntu-18.04
- DEBUG=1 or 0
- ONNX_USE_LITE_PROTO=OFF
- ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
- ONNX_BUILD_TESTS=1
- ONNX_WERROR=ON
- ONNX_ML=1 or 0
- ONNX C++ tests
- Style check (flake8, mypy, and clang-format)
- Test doc generation
- Test proto generation
- Verify node test generation
Windows-CI
Every PR
- windows-2019
- ONNX_USE_LITE_PROTO=ON
- ONNX_USE_PROTOBUF_SHARED_LIBS=ON
- ONNX_BUILD_TESTS=1
- ONNX_WERROR=ON
- ONNX_ML=1 or 0
- Test building ONNX in conda environment
- Test doc generation
- Test proto generation
- Verify node test generation
Mac-CI
Every PR
- macOS-10.15
- DEBUG=1
- ONNX_USE_LITE_PROTO=ON or OFF
- ONNX_ML=1 or 0
- ONNX_BUILD_TESTS=1
- ONNX_WERROR=ON
- ONNX C++ tests
- Test doc generation
- Test proto generation
- Verify node test generation
Windows_No_Exception CI
Every PR
- vs2019-winlatest
- ONNX_DISABLE_EXCEPTIONS=ON
- ONNX_USE_LITE_PROTO=ON
- ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
- ONNX_ML=1
- ONNX_USE_MSVC_STATIC_RUNTIME=ON
- ONNX_DISABLE_STATIC_REGISTRATION=ON or OFF
- Only ONNX C++ tests
- Test selective schema loading
Lint / Lint Python
Every PR
- ubuntu-latest
- Not required – it shows lint warnings for suggestions in PR
- flake8 (but required by
style.sh
in another CI) - pyflakes
- misspell
- shellcheck
- pylint
- mypy (but required by
style.sh
in another CI)
Lint / Enforce style
Every PR
- ubuntu-latest
- flake8
- isort
- black
- mypy
- clang-format
- unix line endings
- c++ namespace rules
- Auto-generated files are up to date
WindowsRelease
- Main branch
- Release branch
- Weekly(1)
- Latest Windows
- x86 and x64
- ONNX_USE_LITE_PROTO=ON
- ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
- ONNX_ML=1
- ONNX_USE_MSVC_STATIC_RUNTIME=OFF
- Release Windows wheel
- Release onnx-weekly package
- Verify with different dependency versions - latest and min supported numpy version, latest and min supported protobuf version(2)
- Verify ONNX with the latest ONNX Runtime PyPI package(3).
LinuxRelease_aarch64
- Main branch
- Release branch
- Weekly
- Latest manylinux2014_aarch64
- ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
- ONNX_ML=1
- ONNX_USE_LITE_PROTO=ON
- Release Linux aarch64 wheel
- Release onnx-weekly package
- Verify with different dependency versions - latest numpy version, latest and min supported protobuf version
- Verify ONNX with the latest ONNX Runtime PyPI package
LinuxRelease_x86_64
- Main branch
- Release branch
- Weekly
- Latest LinuxRelease_x86_64
- ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
- ONNX_ML=1
- ONNX_USE_LITE_PROTO=ON
- Release Linux x86_64 wheel
- Release onnx-weekly package
- Test TEST_HUB=1(4)
- Verify with different dependency versions - latest numpy version, latest and min supported protobuf version
- Verify ONNX with the latest ONNX Runtime PyPI package.
MacRelease
- Main branch
- Release branch
- Weekly
- macos-11
- MACOSX_DEPLOYMENT_TARGET=10.12(5)
- ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
- ONNX_ML=1
- ONNX_USE_LITE_PROTO=ON
- Release Mac wheel
- Release onnx-weekly package
- Verify with different dependency versions - latest numpy version, latest and min supported protobuf version
- Verify ONNX with the latest ONNX Runtime PyPI package.
- Test source distribution generation
- Test build with source distribution
- Release onnx-weekly source distribution
Weekly CI with latest onnx.checker
weekly(6)
- macos-latest
- MACOSX_DEPLOYMENT_TARGET=10.12
- ONNX_USE_PROTOBUF_SHARED_LIBS=OFF
- ONNX_ML=1
- Test latest ONNX checker
- Test latest ONNX shape inference
- With all models from onnx/models(7)
(1) When the release CIs will run:
After a PR has been merged into main/rel-* branch
Run weekly (Sunday midnight) and release Python wheel to onnx-weekly package on TestPyPI.
Any PR targeting rel-* branch
To manually run them, add a PR label “run release CIs” (only maintainers have permission).
(2) Minimum supported versions are listed here.
(3) Test ONNX Python wheel with
onnxruntime.InferenceSession
from latest ONNXRuntime. Please note that ONNX Runtime does not support Windows-x86 thus its verification is skipped.(4) TEST_HUB=1 will test onnx.hub by using this API to download an ONNX model from onnx/models. This test is restricted to only 1 pipeline for saving quota usage.
(5) Although the build environment is macos-11, use MACOSX_DEPLOYMENT_TARGET=10.12 and -p macosx_10_12_x86_64 to force the wheel to support 10.12+.
(6):
The ONNX Model Zoo test will run weekly (Sunday midnight)
To manually trigger it, add a PR label “test ONNX Model Zoo” (only maintainers have permission). Please note that it will need a lot of download bandwidth from onnx/models so use it with caution.
(7) Some old deprecated models (opset-1) are skipped.