<== blog page - 2/3 ==> Blog benchmark (4) onnx (8)
blog page - 2/3#
Numpy API for ONNX and scikit-learn (part I)#
2021-05-05
sklearn-onnx converts most of the pipelines including numerical preprocessing or predictors but it fails whenever custom code is involved. That covers the use of FunctionTransformer or a new model inheriting from BaseEstimator. To be successful, the conversion needs a way to convert the custom code into ONNX. The proposed solution here is bypass that complex steps (rewrite a python function with ONNX operators) by directly writing the custom code with ONNX operators. However, even though most of the operator are close to numpy functions, they are not the same. To avoid spending time looking at them, many numpy functions were implementing with ONNX operators. The custom function or predictor can then just be implemented with this API to build a unique ONNX graph executed with a runtime.
…
Parallelization of Random Forest predictions#
2020-11-27
I’ve been struggling to understand why the first implementation of TreeEnsemble could not get as fast as scikit-learn implementation for a RandomForest when the number of observations was 100.000 or above, 100 trees and a depth >= 10. The only difference was that the computation was parallelized by trees and not by observations. These observations are benchmarked in Benchmark Random Forests, Tree Ensemble, (AoS and SoA) (Benchmark Random Forests, Tree Ensemble, Multi-Classification for the multiclass version).
…
x / y != x * (1 / y)#
2020-06-09
I was recently investigating issue onnxruntime/4130 in notebook Discrepencies with ONNX. While looking into a way to solve it, I finally discovered that this is not an easy problem.
…
Custom C++ TopK#
2019-12-16
It started with the fact the python runtime for the AdaBoostRegressor was quite slow. I noticed three operators were quite slow even though their implementation was based on numpy: TopK, ArrayFeatureExtractor and GatherElement. I made a custom implementation of the first two.
…
RandomForestClassifier - prediction for one observation#
2019-12-04
I was meeting with Olivier Grisel this morning and we were wondering why scikit-learn was slow to compute the prediction of a random forest for one observation compare to what onnxruntime does, and more specically some optimized C++ code inspired from onnxruntime. We used py-spy and wrote the following script:
…
The bug which makes you waste time#
2019-10-04
It is not a bug but it is something which makes
you waste some significant time just to understand
what’s going on. asv would refuse to detect
the benchmark I was trying to set up just because
filenames did contain dots. So, for asv
don’t add a file name.option.py but use
name_option.py.
A couple of benchmark for tries:
bench1, bench2.
Operator CDist#
2019-09-16
Notebooks Pairwise distances with ONNX (pdist) shows how much slower
an ONNX implementation of function
cdist, from 3 to 10 times slower.
One way to optimize the converted model is to
create dedicated operator such as one for function
cdist. Tutorial Converters with options
explains how to tell function to_onnx to use
the custom operator CDist.
Float, double with ONNX#
2019-08-23
Replicating what a library does, scikit-learn for example, is different from implementing a function defined in a paper. Every trick needs to be replicated. scikit-learn trees implement a prediction function which takes float features and compares them to double thresholds. Knowning the ONNX assumes that comparison only happens numbers of the same type, you end up with discrepencies.
…
ONNX updates#
2019-08-02
The python runtime is now almost complete for all the supported numerical operator implemented in sklearn-onnx. A couple of notebooks introduces a couple of way to investigates issues, to benchmark ONNX models with onnxruntime or python runtime, to check the differences between the same model. It also extend ONNX with operators not in the specification to experiment some assumptions and check it is more efficient. Notebook Precision loss due to float32 conversion with ONNX introduces a way to guess the margins introduced by the conversion from double to single. There also exists a function to convert numpy function into ONNX (see Create custom ONNX graphs with AST). Its coverage is probably low but it will improve.
ONNX, runtime#
2019-06-25
Somebody asked me one day if it would be difficult to write a runtime for ONNX in Rust. I just replied that it should not take that long but it would require to implement a way to goes through the nodes of the ONNX graph and to have an implementation for every ONNX Operators…
…
<== blog page - 2/3 ==> 2021-08 (3) 2022-02 (1) 2022-05 (1) 2022-06 (1) 2022-11 (1)