When to parallelize?#

That is the question. Parallize computation takes some time to set up, it is not the right solution in every case. The following example studies the parallelism introduced into the runtime of TreeEnsembleRegressor to see when it is best to do it.

from pprint import pprint
import numpy
from pandas import DataFrame
import matplotlib.pyplot as plt
from tqdm import tqdm
from sklearn import config_context
from sklearn.datasets import make_regression
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.model_selection import train_test_split
from cpyquickhelper.numbers import measure_time
from pyquickhelper.pycode.profiling import profile
from mlprodict.onnx_conv import to_onnx, register_rewritten_operators
from mlprodict.onnxrt import OnnxInference
from mlprodict.tools.model_info import analyze_model

Available optimisations on this machine.

from mlprodict.testing.experimental_c_impl.experimental_c import code_optimisation
print(code_optimisation())
AVX-omp=8

Training and converting a model#

data = make_regression(50000, 20)
X, y = data
X_train, X_test, y_train, y_test = train_test_split(X, y)

hgb = HistGradientBoostingRegressor(max_iter=100, max_depth=6)
hgb.fit(X_train, y_train)
print(hgb)
HistGradientBoostingRegressor(max_depth=6)

Let’s get more statistics about the model itself.

pprint(analyze_model(hgb))
{'_predictors.max|tree_.max_depth': 6,
 '_predictors.size': 100,
 '_predictors.sum|tree_.leave_count': 3100,
 '_predictors.sum|tree_.node_count': 6100,
 'train_score_.shape': 101,
 'validation_score_.shape': 101}

And let’s convert it.

register_rewritten_operators()
onx = to_onnx(hgb, X_train[:1].astype(numpy.float32))
oinf = OnnxInference(onx, runtime='python_compiled')
print(oinf)
OnnxInference(...)
    def compiled_run(dict_inputs, yield_ops=None, context=None, attributes=None):
        if yield_ops is not None:
            raise NotImplementedError('yields_ops should be None.')
        # inputs
        X = dict_inputs['X']
        (variable, ) = n0_treeensembleregressor_3(X)
        return {
            'variable': variable,
        }

The runtime of the forest is in the following object.

print(oinf.sequence_[0].ops_)
print(oinf.sequence_[0].ops_.rt_)
TreeEnsembleRegressor_3(
    op_type=TreeEnsembleRegressor
    aggregate_function=b'SUM',
    base_values=[1.2382386],
    base_values_as_tensor=[],
    domain=ai.onnx.ml,
    inplaces={},
    ir_version=8,
    n_targets=1,
    nodes_falsenodeids=[32 17 10 ... 60  0  0],
    nodes_featureids=[ 3 12 17 ... 14  0  0],
    nodes_hitrates=[1. 1. 1. ... 1. 1. 1.],
    nodes_hitrates_as_tensor=[],
    nodes_missing_value_tracks_true=[1 1 0 ... 1 0 0],
    nodes_modes=[b'BRANCH_LEQ' b'BRANCH_LEQ' b'BRANCH_LEQ' ... b'BRANCH_LEQ' b'LEAF'
 b'LEAF'],
    nodes_nodeids=[ 0  1  2 ... 58 59 60],
    nodes_treeids=[ 0  0  0 ... 99 99 99],
    nodes_truenodeids=[ 1  2  3 ... 59  0  0],
    nodes_values=[ 0.08383127  0.21855506 -0.14405032 ...  0.54634726  0.
  0.        ],
    nodes_values_as_tensor=[],
    parallel=(60, 128, 20),
    post_transform=b'NONE',
    runtime=None,
    target_ids=[0 0 0 ... 0 0 0],
    target_nodeids=[ 5  6  8 ... 57 59 60],
    target_opset=3,
    target_treeids=[ 0  0  0 ... 99 99 99],
    target_weights=[-30.602661  -19.76563   -19.78937   ...   0.6851268   1.6637601
   3.518802 ],
    target_weights_as_tensor=[],
)
<mlprodict.onnxrt.ops_cpu.op_tree_ensemble_regressor_p_.RuntimeTreeEnsembleRegressorPFloat object at 0x7f18847f29b0>

And the threshold used to start parallelizing based on the number of observations.

print(oinf.sequence_[0].ops_.rt_.omp_N_)
20

Profiling#

This step involves pyinstrument to measure where the time is spent. Both scikit-learn and mlprodict runtime are called so that the prediction times can be compared.

X32 = X_test.astype(numpy.float32)


def runlocal():
    with config_context(assume_finite=True):
        for i in range(0, 100):
            oinf.run({'X': X32[:1000]})
            hgb.predict(X_test[:1000])


print("profiling...")
txt = profile(runlocal, pyinst_format='text')
print(txt[1])
profiling...

  _     ._   __/__   _ _  _  _ _/_   Recorded: 09:50:19 AM Samples:  1214
 /_//_/// /_\ / //_// / //_'/ //     Duration: 1.608     CPU time: 12.567
/   _/                      v4.4.0

Program: somewhere/workspace/mlprodict/mlprodict_UT_39_std/_doc/examples/plot_parallelism.py

1.599 profile  ../pycode/profiling.py:455
`- 1.598 runlocal  plot_parallelism.py:91
      [48 frames hidden]  plot_parallelism, sklearn, <built-in>...

Now let’s measure the performance the average computation time per observations for 2 to 100 observations. The runtime implemented in mlprodict parallizes the computation after a given number of observations.

obs = []
for N in tqdm(list(range(2, 21))):
    m = measure_time("oinf.run({'X': x})",
                     {'oinf': oinf, 'x': X32[:N]},
                     div_by_number=True,
                     number=20)
    m['N'] = N
    m['RT'] = 'ONNX'
    obs.append(m)

    with config_context(assume_finite=True):
        m = measure_time("hgb.predict(x)",
                         {'hgb': hgb, 'x': X32[:N]},
                         div_by_number=True,
                         number=15)
    m['N'] = N
    m['RT'] = 'SKL'
    obs.append(m)

df = DataFrame(obs)
num = ['min_exec', 'average', 'max_exec']
for c in num:
    df[c] /= df['N']
df.head()
  0%|          | 0/19 [00:00<?, ?it/s]
  5%|5         | 1/19 [00:00<00:16,  1.12it/s]
 11%|#         | 2/19 [00:01<00:15,  1.11it/s]
 16%|#5        | 3/19 [00:02<00:14,  1.11it/s]
 21%|##1       | 4/19 [00:03<00:13,  1.11it/s]
 26%|##6       | 5/19 [00:04<00:12,  1.11it/s]
 32%|###1      | 6/19 [00:05<00:11,  1.11it/s]
 37%|###6      | 7/19 [00:06<00:10,  1.11it/s]
 42%|####2     | 8/19 [00:07<00:09,  1.11it/s]
 47%|####7     | 9/19 [00:08<00:09,  1.11it/s]
 53%|#####2    | 10/19 [00:09<00:08,  1.10it/s]
 58%|#####7    | 11/19 [00:09<00:07,  1.10it/s]
 63%|######3   | 12/19 [00:10<00:06,  1.09it/s]
 68%|######8   | 13/19 [00:11<00:05,  1.08it/s]
 74%|#######3  | 14/19 [00:12<00:04,  1.08it/s]
 79%|#######8  | 15/19 [00:13<00:03,  1.07it/s]
 84%|########4 | 16/19 [00:14<00:02,  1.06it/s]
 89%|########9 | 17/19 [00:15<00:01,  1.07it/s]
 95%|#########4| 18/19 [00:16<00:00,  1.07it/s]
100%|##########| 19/19 [00:17<00:00,  1.07it/s]
100%|##########| 19/19 [00:17<00:00,  1.09it/s]
average deviation min_exec max_exec repeat number ttime context_size N RT
0 0.000024 0.000012 0.000021 0.000041 10 20 0.000480 232 2 ONNX
1 0.002933 0.000077 0.002902 0.003033 10 15 0.058665 232 2 SKL
2 0.000016 0.000002 0.000015 0.000017 10 20 0.000471 232 3 ONNX
3 0.001996 0.000234 0.001948 0.002223 10 15 0.059888 232 3 SKL
4 0.000012 0.000003 0.000012 0.000014 10 20 0.000494 232 4 ONNX


Graph.

fig, ax = plt.subplots(1, 2, figsize=(10, 4))
df[df.RT == 'ONNX'].set_index('N')[num].plot(ax=ax[0])
ax[0].set_title("Average ONNX prediction time per observation in a batch.")
df[df.RT == 'SKL'].set_index('N')[num].plot(ax=ax[1])
ax[1].set_title(
    "Average scikit-learn prediction time\nper observation in a batch.")
Average ONNX prediction time per observation in a batch., Average scikit-learn prediction time per observation in a batch.
Text(0.5, 1.0, 'Average scikit-learn prediction time\nper observation in a batch.')

Gain from parallelization#

There is a clear gap between after and before 10 observations when it is parallelized. Does this threshold depends on the number of trees in the model? For that we compute for each model the average prediction time up to 10 and from 10 to 20.

def parallized_gain(df):
    df = df[df.RT == 'ONNX']
    df10 = df[df.N <= 10]
    t10 = sum(df10['average']) / df10.shape[0]
    df10p = df[df.N > 10]
    t10p = sum(df10p['average']) / df10p.shape[0]
    return t10 / t10p


print('gain', parallized_gain(df))
gain 1.5061734208260684

Measures based on the number of trees#

We trained many models with different number of trees to see how the parallelization gain is moving. One models is trained for every distinct number of trees and then the prediction time is measured for different number of observations.

tries_set = [2, 5, 8] + list(range(10, 50, 5)) + list(range(50, 101, 10))
tries = [(nb, N) for N in range(2, 21, 2) for nb in tries_set]

training

models = {100: (hgb, oinf)}
for nb in tqdm(set(_[0] for _ in tries)):
    if nb not in models:
        hgb = HistGradientBoostingRegressor(max_iter=nb, max_depth=6)
        hgb.fit(X_train, y_train)
        onx = to_onnx(hgb, X_train[:1].astype(numpy.float32))
        oinf = OnnxInference(onx, runtime='python_compiled')
        models[nb] = (hgb, oinf)
  0%|          | 0/17 [00:00<?, ?it/s]
  6%|5         | 1/17 [00:00<00:04,  3.90it/s]
 12%|#1        | 2/17 [00:01<00:10,  1.47it/s]
 24%|##3       | 4/17 [00:01<00:04,  2.82it/s]
 29%|##9       | 5/17 [00:03<00:09,  1.29it/s]
 35%|###5      | 6/17 [00:03<00:07,  1.52it/s]
 41%|####1     | 7/17 [00:04<00:07,  1.27it/s]
 47%|####7     | 8/17 [00:05<00:06,  1.48it/s]
 53%|#####2    | 9/17 [00:06<00:06,  1.20it/s]
 59%|#####8    | 10/17 [00:06<00:05,  1.34it/s]
 65%|######4   | 11/17 [00:08<00:06,  1.12s/it]
 71%|#######   | 12/17 [00:10<00:05,  1.17s/it]
 76%|#######6  | 13/17 [00:10<00:04,  1.02s/it]
 82%|########2 | 14/17 [00:11<00:02,  1.06it/s]
 88%|########8 | 15/17 [00:13<00:02,  1.31s/it]
 94%|#########4| 16/17 [00:15<00:01,  1.37s/it]
100%|##########| 17/17 [00:16<00:00,  1.22s/it]
100%|##########| 17/17 [00:16<00:00,  1.05it/s]

prediction time

obs = []

for nb, N in tqdm(tries):
    hgb, oinf = models[nb]
    m = measure_time("oinf.run({'X': x})",
                     {'oinf': oinf, 'x': X32[:N]},
                     div_by_number=True,
                     number=50)
    m['N'] = N
    m['nb'] = nb
    m['RT'] = 'ONNX'
    obs.append(m)

df = DataFrame(obs)
num = ['min_exec', 'average', 'max_exec']
for c in num:
    df[c] /= df['N']
df.head()
  0%|          | 0/170 [00:00<?, ?it/s]
  5%|4         | 8/170 [00:00<00:02, 71.59it/s]
  9%|9         | 16/170 [00:00<00:02, 58.21it/s]
 14%|#3        | 23/170 [00:00<00:02, 60.52it/s]
 18%|#7        | 30/170 [00:00<00:02, 56.00it/s]
 21%|##1       | 36/170 [00:00<00:02, 53.33it/s]
 25%|##4       | 42/170 [00:00<00:02, 54.64it/s]
 28%|##8       | 48/170 [00:00<00:02, 49.07it/s]
 32%|###1      | 54/170 [00:01<00:02, 49.40it/s]
 35%|###5      | 60/170 [00:01<00:02, 49.40it/s]
 38%|###8      | 65/170 [00:01<00:02, 44.12it/s]
 41%|####1     | 70/170 [00:01<00:02, 44.67it/s]
 45%|####4     | 76/170 [00:01<00:02, 46.15it/s]
 48%|####7     | 81/170 [00:01<00:02, 40.15it/s]
 51%|#####     | 86/170 [00:01<00:02, 39.99it/s]
 54%|#####4    | 92/170 [00:01<00:01, 43.28it/s]
 57%|#####7    | 97/170 [00:02<00:01, 38.31it/s]
 59%|#####9    | 101/170 [00:02<00:01, 36.27it/s]
 63%|######2   | 107/170 [00:02<00:01, 40.55it/s]
 66%|######5   | 112/170 [00:02<00:01, 38.05it/s]
 68%|######8   | 116/170 [00:02<00:01, 33.51it/s]
 71%|#######   | 120/170 [00:02<00:01, 34.47it/s]
 74%|#######4  | 126/170 [00:02<00:01, 38.18it/s]
 76%|#######6  | 130/170 [00:03<00:01, 34.02it/s]
 79%|#######8  | 134/170 [00:03<00:01, 30.99it/s]
 82%|########1 | 139/170 [00:03<00:00, 34.36it/s]
 84%|########4 | 143/170 [00:03<00:00, 35.70it/s]
 86%|########6 | 147/170 [00:03<00:00, 31.33it/s]
 89%|########8 | 151/170 [00:03<00:00, 28.42it/s]
 92%|#########1| 156/170 [00:03<00:00, 32.18it/s]
 94%|#########4| 160/170 [00:03<00:00, 33.67it/s]
 96%|#########6| 164/170 [00:04<00:00, 29.39it/s]
 99%|#########8| 168/170 [00:04<00:00, 26.73it/s]
100%|##########| 170/170 [00:04<00:00, 38.63it/s]
average deviation min_exec max_exec repeat number ttime context_size N nb RT
0 0.000012 3.402441e-07 0.000012 0.000012 10 50 0.000241 232 2 2 ONNX
1 0.000012 3.591181e-07 0.000012 0.000013 10 50 0.000246 232 2 5 ONNX
2 0.000013 2.975464e-07 0.000012 0.000013 10 50 0.000251 232 2 8 ONNX
3 0.000013 2.356329e-07 0.000013 0.000013 10 50 0.000257 232 2 10 ONNX
4 0.000013 3.697757e-07 0.000013 0.000014 10 50 0.000264 232 2 15 ONNX


Let’s compute the gains.

gains = []
for nb in set(df['nb']):
    gain = parallized_gain(df[df.nb == nb])
    gains.append(dict(nb=nb, gain=gain))

dfg = DataFrame(gains)
dfg = dfg.sort_values('nb').reset_index(drop=True).copy()
dfg
nb gain
0 2 3.346527
1 5 3.095013
2 8 2.896627
3 10 2.770305
4 15 2.511757
5 20 2.295138
6 25 2.123142
7 30 1.991839
8 35 1.899327
9 40 1.800172
10 45 1.755309
11 50 1.705960
12 60 1.586818
13 70 3.060065
14 80 2.860915
15 90 2.729481
16 100 2.705024


Graph.

ax = dfg.set_index('nb').plot()
ax.set_title(
    "Parallelization gain depending\non the number of trees\n(max_depth=6).")
Parallelization gain depending on the number of trees (max_depth=6).
Text(0.5, 1.0, 'Parallelization gain depending\non the number of trees\n(max_depth=6).')

That does not answer the question we are looking for as we would like to know the best threshold th which defines the number of observations for which we should parallelized. This number depends on the number of trees. A gain > 1 means the parallization should happen Here, even two observations is ok. Let’s check with lighter trees (max_depth=2), maybe in that case, the conclusion is different.

models = {100: (hgb, oinf)}
for nb in tqdm(set(_[0] for _ in tries)):
    if nb not in models:
        hgb = HistGradientBoostingRegressor(max_iter=nb, max_depth=2)
        hgb.fit(X_train, y_train)
        onx = to_onnx(hgb, X_train[:1].astype(numpy.float32))
        oinf = OnnxInference(onx, runtime='python_compiled')
        models[nb] = (hgb, oinf)

obs = []
for nb, N in tqdm(tries):
    hgb, oinf = models[nb]
    m = measure_time("oinf.run({'X': x})",
                     {'oinf': oinf, 'x': X32[:N]},
                     div_by_number=True,
                     number=50)
    m['N'] = N
    m['nb'] = nb
    m['RT'] = 'ONNX'
    obs.append(m)

df = DataFrame(obs)
num = ['min_exec', 'average', 'max_exec']
for c in num:
    df[c] /= df['N']
df.head()
  0%|          | 0/17 [00:00<?, ?it/s]
  6%|5         | 1/17 [00:00<00:03,  4.61it/s]
 12%|#1        | 2/17 [00:00<00:04,  3.22it/s]
 24%|##3       | 4/17 [00:00<00:02,  5.30it/s]
 29%|##9       | 5/17 [00:01<00:03,  3.35it/s]
 35%|###5      | 6/17 [00:01<00:03,  3.54it/s]
 41%|####1     | 7/17 [00:02<00:03,  3.14it/s]
 47%|####7     | 8/17 [00:02<00:02,  3.34it/s]
 53%|#####2    | 9/17 [00:02<00:02,  2.95it/s]
 59%|#####8    | 10/17 [00:02<00:02,  3.11it/s]
 65%|######4   | 11/17 [00:03<00:02,  2.48it/s]
 71%|#######   | 12/17 [00:04<00:02,  2.39it/s]
 76%|#######6  | 13/17 [00:04<00:01,  2.59it/s]
 82%|########2 | 14/17 [00:04<00:01,  2.71it/s]
 88%|########8 | 15/17 [00:05<00:00,  2.22it/s]
 94%|#########4| 16/17 [00:05<00:00,  2.16it/s]
100%|##########| 17/17 [00:06<00:00,  2.33it/s]
100%|##########| 17/17 [00:06<00:00,  2.76it/s]

  0%|          | 0/170 [00:00<?, ?it/s]
  5%|4         | 8/170 [00:00<00:02, 76.13it/s]
  9%|9         | 16/170 [00:00<00:02, 65.01it/s]
 14%|#3        | 23/170 [00:00<00:02, 66.14it/s]
 18%|#7        | 30/170 [00:00<00:02, 65.51it/s]
 22%|##1       | 37/170 [00:00<00:02, 60.33it/s]
 26%|##5       | 44/170 [00:00<00:02, 62.08it/s]
 30%|###       | 51/170 [00:00<00:02, 56.00it/s]
 34%|###4      | 58/170 [00:00<00:01, 59.75it/s]
 38%|###8      | 65/170 [00:01<00:01, 57.07it/s]
 42%|####1     | 71/170 [00:01<00:01, 55.28it/s]
 46%|####5     | 78/170 [00:01<00:01, 56.63it/s]
 49%|####9     | 84/170 [00:01<00:01, 52.00it/s]
 54%|#####3    | 91/170 [00:01<00:01, 54.37it/s]
 57%|#####7    | 97/170 [00:01<00:01, 53.22it/s]
 61%|######    | 103/170 [00:01<00:01, 49.21it/s]
 65%|######4   | 110/170 [00:01<00:01, 52.65it/s]
 68%|######8   | 116/170 [00:02<00:01, 49.32it/s]
 72%|#######1  | 122/170 [00:02<00:00, 48.52it/s]
 75%|#######5  | 128/170 [00:02<00:00, 50.20it/s]
 79%|#######8  | 134/170 [00:02<00:00, 46.03it/s]
 82%|########1 | 139/170 [00:02<00:00, 46.63it/s]
 85%|########5 | 145/170 [00:02<00:00, 48.37it/s]
 88%|########8 | 150/170 [00:02<00:00, 44.71it/s]
 91%|#########1| 155/170 [00:02<00:00, 44.10it/s]
 95%|#########4| 161/170 [00:03<00:00, 47.13it/s]
 98%|#########7| 166/170 [00:03<00:00, 43.56it/s]
100%|##########| 170/170 [00:03<00:00, 51.25it/s]
average deviation min_exec max_exec repeat number ttime context_size N nb RT
0 0.000012 3.167176e-07 0.000012 0.000012 10 50 0.000240 232 2 2 ONNX
1 0.000012 3.575039e-07 0.000012 0.000013 10 50 0.000241 232 2 5 ONNX
2 0.000012 2.188225e-07 0.000012 0.000012 10 50 0.000244 232 2 8 ONNX
3 0.000012 3.150617e-07 0.000012 0.000013 10 50 0.000244 232 2 10 ONNX
4 0.000012 3.038314e-07 0.000012 0.000013 10 50 0.000248 232 2 15 ONNX


Measures.

gains = []
for nb in set(df['nb']):
    gain = parallized_gain(df[df.nb == nb])
    gains.append(dict(nb=nb, gain=gain))

dfg = DataFrame(gains)
dfg = dfg.sort_values('nb').reset_index(drop=True).copy()
dfg
nb gain
0 2 3.429179
1 5 3.312187
2 8 3.204141
3 10 3.119820
4 15 2.955329
5 20 2.834290
6 25 2.728680
7 30 2.631016
8 35 2.534944
9 40 2.448582
10 45 2.344460
11 50 2.261824
12 60 2.130212
13 70 2.962409
14 80 2.925913
15 90 2.881071
16 100 2.707353


Graph.

ax = dfg.set_index('nb').plot()
ax.set_title(
    "Parallelization gain depending\non the number of trees\n(max_depth=3).")
Parallelization gain depending on the number of trees (max_depth=3).
Text(0.5, 1.0, 'Parallelization gain depending\non the number of trees\n(max_depth=3).')

The conclusion is somewhat the same but it shows that the bigger the number of trees is the bigger the gain is and under the number of cores of the processor.

Moving the theshold#

The last experiment consists in comparing the prediction time with or without parallelization for different number of observation.

hgb = HistGradientBoostingRegressor(max_iter=40, max_depth=6)
hgb.fit(X_train, y_train)
onx = to_onnx(hgb, X_train[:1].astype(numpy.float32))
oinf = OnnxInference(onx, runtime='python_compiled')


obs = []
for N in tqdm(list(range(2, 51))):
    oinf.sequence_[0].ops_.rt_.omp_N_ = 100
    m = measure_time("oinf.run({'X': x})",
                     {'oinf': oinf, 'x': X32[:N]},
                     div_by_number=True,
                     number=20)
    m['N'] = N
    m['RT'] = 'ONNX'
    m['PARALLEL'] = False
    obs.append(m)

    oinf.sequence_[0].ops_.rt_.omp_N_ = 1
    m = measure_time("oinf.run({'X': x})",
                     {'oinf': oinf, 'x': X32[:N]},
                     div_by_number=True,
                     number=50)
    m['N'] = N
    m['RT'] = 'ONNX'
    m['PARALLEL'] = True
    obs.append(m)

df = DataFrame(obs)
num = ['min_exec', 'average', 'max_exec']
for c in num:
    df[c] /= df['N']
df.head()
  0%|          | 0/49 [00:00<?, ?it/s]
  8%|8         | 4/49 [00:00<00:01, 34.46it/s]
 16%|#6        | 8/49 [00:00<00:01, 29.91it/s]
 24%|##4       | 12/49 [00:00<00:01, 28.52it/s]
 31%|###       | 15/49 [00:00<00:01, 27.16it/s]
 37%|###6      | 18/49 [00:00<00:01, 25.78it/s]
 43%|####2     | 21/49 [00:00<00:01, 24.41it/s]
 49%|####8     | 24/49 [00:00<00:01, 23.01it/s]
 55%|#####5    | 27/49 [00:01<00:01, 20.86it/s]
 61%|######1   | 30/49 [00:01<00:00, 20.13it/s]
 67%|######7   | 33/49 [00:01<00:00, 19.34it/s]
 71%|#######1  | 35/49 [00:01<00:00, 18.76it/s]
 76%|#######5  | 37/49 [00:01<00:00, 18.19it/s]
 80%|#######9  | 39/49 [00:01<00:00, 17.61it/s]
 84%|########3 | 41/49 [00:01<00:00, 17.04it/s]
 88%|########7 | 43/49 [00:02<00:00, 16.51it/s]
 92%|#########1| 45/49 [00:02<00:00, 16.02it/s]
 96%|#########5| 47/49 [00:02<00:00, 15.55it/s]
100%|##########| 49/49 [00:02<00:00, 15.09it/s]
100%|##########| 49/49 [00:02<00:00, 19.72it/s]
average deviation min_exec max_exec repeat number ttime context_size N RT PARALLEL
0 0.000016 1.077364e-06 0.000015 0.000017 10 20 0.000311 232 2 ONNX False
1 0.000019 1.931788e-06 0.000019 0.000022 10 50 0.000386 232 2 ONNX True
2 0.000011 1.335087e-06 0.000011 0.000013 10 20 0.000345 232 3 ONNX False
3 0.000013 9.616338e-07 0.000013 0.000014 10 50 0.000394 232 3 ONNX True
4 0.000009 1.079845e-06 0.000009 0.000010 10 20 0.000375 232 4 ONNX False


Graph.

piv = df[['N', 'PARALLEL', 'average']].pivot('N', 'PARALLEL', 'average')
ax = piv.plot(logy=True)
ax.set_title("Prediction time with and without parallelization.")
Prediction time with and without parallelization.
somewhere/workspace/mlprodict/mlprodict_UT_39_std/_doc/examples/plot_parallelism.py:338: FutureWarning: In a future version of pandas all arguments of DataFrame.pivot will be keyword-only.
  piv = df[['N', 'PARALLEL', 'average']].pivot('N', 'PARALLEL', 'average')

Text(0.5, 1.0, 'Prediction time with and without parallelization.')

Parallelization is working.

plt.show()

Total running time of the script: ( 0 minutes 58.735 seconds)

Gallery generated by Sphinx-Gallery