:orphan: |rss_image| :ref:`<== ` **blog page - 2/3** :ref:`==> ` :ref:`Blog ` :ref:`benchmark (4) ` :ref:`onnx (8) ` .. |rss_image| image:: feed-icon-16x16.png :target: ../_downloads/rss.xml :alt: RSS ---- .. _ap-main-1: blog page - 2/3 +++++++++++++++ .. blogpostagg:: :title: Numpy API for ONNX and scikit-learn (part I) :date: 2021-05-05 :keywords: ONNX,API,numpy,scikit-learn :categories: API :rawfile: 2021/2021-05-05_numpyapionnx1.rst :epkg:`sklearn-onnx` converts most of the pipelines including numerical preprocessing or predictors but it fails whenever custom code is involved. That covers the use of `FunctionTransformer `_ or a new model inheriting from `BaseEstimator `_. To be successful, the conversion needs a way to convert the custom code into ONNX. The proposed solution here is bypass that complex steps (rewrite a python function with ONNX operators) by directly writing the custom code with ONNX operators. However, even though most of the operator are close to :epkg:`numpy` functions, they are not the same. To avoid spending time looking at them, many :epkg:`numpy` functions were implementing with ONNX operators. The custom function or predictor can then just be implemented with this API to build a unique ONNX graph executed with a runtime. ... .. blogpostagg:: :title: Parallelization of Random Forest predictions :date: 2020-11-27 :keywords: scikit-learn,parallelization,Random Forest :categories: runtime :rawfile: 2020/2020-11-27_parallelisation.rst I've been struggling to understand why the first implementation of TreeEnsemble could not get as fast as *scikit-learn* implementation for a RandomForest when the number of observations was 100.000 or above, 100 trees and a depth >= 10. The only difference was that the computation was parallelized by trees and not by observations. These observations are benchmarked in :ref:`l-example-tree-ensemble-reg-bench` (:ref:`l-example-tree-ensemble-cls-bench-multi` for the multiclass version). ... .. blogpostagg:: :title: x / y != x * (1 / y) :date: 2020-06-09 :keywords: scikit-learn,float inverse,compilation,StandardScaler :categories: runtime :rawfile: 2020/2020-06-09_float_inverse.rst I was recently investigating issue `onnxruntime/4130 `_ in notebook :ref:`onnxdiscrepenciesrst`. While looking into a way to solve it, I finally discovered that this is not an easy problem. ... .. blogpostagg:: :title: Custom C++ TopK :date: 2019-12-16 :keywords: scikit-learn,topk,argpartition :categories: benchmark :rawfile: 2019/2019-12-16_topk.rst It started with the fact the python runtime for the AdaBoostRegressor was quite slow. I noticed three operators were quite slow even though their implementation was based on :epkg:`numpy`: *TopK*, *ArrayFeatureExtractor* and *GatherElement*. I made a custom implementation of the first two. ... .. blogpostagg:: :title: RandomForestClassifier - prediction for one observation :date: 2019-12-04 :keywords: scikit-learn,py-spy,benchmark,one-off prediction :categories: benchmark :rawfile: 2019/2019-12-04_py-spy.rst I was meeting with Olivier Grisel this morning and we were wondering why :epkg:`scikit-learn` was slow to compute the prediction of a random forest for one observation compare to what :epkg:`onnxruntime` does, and more specically some optimized C++ code inspired from :epkg:`onnxruntime`. We used :epkg:`py-spy` and wrote the following script: ... .. blogpostagg:: :title: The bug which makes you waste time :date: 2019-10-04 :keywords: asv,bug :categories: benchmark :rawfile: 2019/2019-10-04_asv.rst It is not a bug but it is something which makes you waste some significant time just to understand what's going on. :epkg:`asv` would refuse to detect the benchmark I was trying to set up just because filenames did contain dots. So, for :epkg:`asv` don't add a file ``name.option.py`` but use ``name_option.py``. A couple of benchmark for tries: :epkg:`bench1`, :epkg:`bench2`. .. blogpostagg:: :title: Operator CDist :date: 2019-09-16 :keywords: onnxruntime,cdist :categories: onnx :rawfile: 2019/2019-09-16_cdist.rst Notebooks :ref:`onnxpdistrst` shows how much slower an :epkg:`ONNX` implementation of function :epkg:`cdist`, from 3 to 10 times slower. One way to optimize the converted model is to create dedicated operator such as one for function :epkg:`cdist`. Tutorial :ref:`l-onnx-tutorial-optim` explains how to tell function :func:`to_onnx ` to use the custom operator `CDist`. .. blogpostagg:: :title: Float, double with ONNX :date: 2019-08-23 :keywords: onnx,float,double :categories: onnx :rawfile: 2019/2019-08-23_onnx_float_double.rst Replicating what a library does, :epkg:`scikit-learn` for example, is different from implementing a function defined in a paper. Every trick needs to be replicated. :epkg:`scikit-learn` trees implement a prediction function which takes float features and compares them to double thresholds. Knowning the :epkg:`ONNX` assumes that comparison only happens numbers of the same type, you end up with discrepencies. ... .. blogpostagg:: :title: ONNX updates :date: 2019-08-02 :keywords: onnx,onnxrt,update :categories: onnx :rawfile: 2019/2019-08-02_onnx_changes.rst The python runtime is now almost complete for all the supported numerical operator implemented in :epkg:`sklearn-onnx`. A couple of notebooks introduces a couple of way to investigates issues, to benchmark ONNX models with :epkg:`onnxruntime` or python runtime, to check the differences between the same model. It also extend ONNX with operators not in the specification to experiment some assumptions and check it is more efficient. Notebook :ref:`onnxshakerrst` introduces a way to guess the margins introduced by the conversion from double to single. There also exists a function to convert numpy function into ONNX (see :ref:`l-numpy2onnx-tutorial`). Its coverage is probably low but it will improve. .. blogpostagg:: :title: ONNX, runtime :date: 2019-06-25 :keywords: onnx,onnxrt :categories: onnx :rawfile: 2019/2019-06-25_runtime.rst Somebody asked me one day if it would be difficult to write a runtime for :epkg:`ONNX` in :epkg:`Rust`. I just replied that it should not take that long but it would require to implement a way to goes through the nodes of the :epkg:`ONNX` graph and to have an implementation for every :epkg:`ONNX Operators`... ... ---- |rss_image| :ref:`<== ` **blog page - 2/3** :ref:`==> ` :ref:`2021-08 (3) ` :ref:`2022-02 (1) ` :ref:`2022-05 (1) ` :ref:`2022-06 (1) ` :ref:`2022-11 (1) `