module onnxrt.validate.validate_scenarios#

Short summary#

module mlprodict.onnxrt.validate.validate_scenarios

Scenarios for validation.

source on GitHub

Functions#

function

truncated documentation

build_custom_scenarios

Defines parameters values for some operators.

interpret_options_from_string

Converts a string into a dictionary.

Documentation#

Scenarios for validation.

source on GitHub

mlprodict.onnxrt.validate.validate_scenarios.build_custom_scenarios()#

Defines parameters values for some operators.

<<<

from mlprodict.onnxrt.validate.validate_scenarios import build_custom_scenarios
import pprint
pprint.pprint(build_custom_scenarios())

>>>

    {<class 'sklearn.neighbors._classification.KNeighborsClassifier'>: [('default_k3',
                                                                         {'algorithm': 'brute',
                                                                          'n_neighbors': 3},
                                                                         {'conv_options': [{<class 'sklearn.neighbors._classification.KNeighborsClassifier'>: {'optim': 'cdist',
                                                                                                                                                               'zipmap': False}}]}),
                                                                        ('weights_k3',
                                                                         {'algorithm': 'brute',
                                                                          'n_neighbors': 3,
                                                                          'weights': 'distance'},
                                                                         {'conv_options': [{<class 'sklearn.neighbors._classification.KNeighborsClassifier'>: {'optim': 'cdist',
                                                                                                                                                               'zipmap': False}}]})],
     <class 'sklearn.neighbors._classification.RadiusNeighborsClassifier'>: [('default_k3',
                                                                              {'algorithm': 'brute'},
                                                                              {'conv_options': [{<class 'sklearn.neighbors._classification.RadiusNeighborsClassifier'>: {'optim': 'cdist',
                                                                                                                                                                         'zipmap': False}}]}),
                                                                             ('weights_k3',
                                                                              {'algorithm': 'brute',
                                                                               'weights': 'distance'},
                                                                              {'conv_options': [{<class 'sklearn.neighbors._classification.RadiusNeighborsClassifier'>: {'optim': 'cdist',
                                                                                                                                                                         'zipmap': False}}]})],
     <class 'sklearn.neighbors._regression.KNeighborsRegressor'>: [('default_k3',
                                                                    {'algorithm': 'brute',
                                                                     'n_neighbors': 3},
                                                                    {'conv_options': [{<class 'sklearn.neighbors._regression.KNeighborsRegressor'>: {'optim': 'cdist'}}]}),
                                                                   ('weights_k3',
                                                                    {'algorithm': 'brute',
                                                                     'n_neighbors': 3,
                                                                     'weights': 'distance'},
                                                                    {'conv_options': [{<class 'sklearn.neighbors._regression.KNeighborsRegressor'>: {'optim': 'cdist'}}]})],
     <class 'sklearn.neighbors._regression.RadiusNeighborsRegressor'>: [('default_k3',
                                                                         {'algorithm': 'brute'},
                                                                         {'conv_options': [{},
                                                                                           {<class 'sklearn.neighbors._regression.RadiusNeighborsRegressor'>: {'optim': 'cdist'}}]}),
                                                                        ('weights_k3',
                                                                         {'algorithm': 'brute',
                                                                          'weights': 'distance'},
                                                                         {'conv_options': [{<class 'sklearn.neighbors._regression.RadiusNeighborsRegressor'>: {'optim': 'cdist'}}]})],
     <class 'sklearn.neighbors._lof.LocalOutlierFactor'>: [('novelty',
                                                            {'novelty': True})],
     <class 'sklearn.model_selection._search.GridSearchCV'>: [('cl',
                                                               {'estimator': LogisticRegression(solver='liblinear'),
                                                                'n_jobs': 1,
                                                                'param_grid': {'fit_intercept': [False,
                                                                                                 True]}},
                                                               {'conv_options': [{<class 'sklearn.model_selection._search.GridSearchCV'>: {'zipmap': False}}],
                                                                'subset_problems': ['b-cl',
                                                                                    'm-cl',
                                                                                    '~b-cl-64']}),
                                                              ('reg',
                                                               {'estimator': LinearRegression(),
                                                                'n_jobs': 1,
                                                                'param_grid': {'fit_intercept': [False,
                                                                                                 True]}},
                                                               ['b-reg',
                                                                'm-reg',
                                                                '~b-reg-64']),
                                                              ('reg',
                                                               {'estimator': KMeans(),
                                                                'n_jobs': 1,
                                                                'param_grid': {'n_clusters': [2,
                                                                                              3]}},
                                                               ['cluster'])],
     <class 'sklearn.model_selection._search.RandomizedSearchCV'>: [('cl',
                                                                     {'estimator': LogisticRegression(solver='liblinear'),
                                                                      'param_distributions': {'fit_intercept': [False,
                                                                                                                True]}}),
                                                                    ('reg',
                                                                     {'estimator': LinearRegression(),
                                                                      'param_distributions': {'fit_intercept': [False,
                                                                                                                True]}})],
     <class 'sklearn.linear_model._stochastic_gradient.SGDClassifier'>: [('log',
                                                                          {'loss': 'log'},
                                                                          {'conv_options': [{<class 'sklearn.linear_model._stochastic_gradient.SGDClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.linear_model._ridge.RidgeClassifier'>: [('default',
                                                              {},
                                                              {'conv_options': [{<class 'sklearn.linear_model._ridge.RidgeClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.linear_model._ridge.RidgeClassifierCV'>: [('default',
                                                                {},
                                                                {'conv_options': [{<class 'sklearn.linear_model._ridge.RidgeClassifierCV'>: {'zipmap': False}}]})],
     <class 'sklearn.svm._classes.SVC'>: [('linear',
                                           {'kernel': 'linear',
                                            'probability': True},
                                           {'conv_options': [{<class 'sklearn.svm._classes.SVC'>: {'zipmap': False}}]}),
                                          ('poly',
                                           {'kernel': 'poly', 'probability': True},
                                           {'conv_options': [{<class 'sklearn.svm._classes.SVC'>: {'zipmap': False}}]}),
                                          ('rbf',
                                           {'kernel': 'rbf', 'probability': True},
                                           {'conv_options': [{<class 'sklearn.svm._classes.SVC'>: {'zipmap': False}}]}),
                                          ('sigmoid',
                                           {'kernel': 'sigmoid',
                                            'probability': True},
                                           {'conv_options': [{<class 'sklearn.svm._classes.SVC'>: {'zipmap': False}}]})],
     <class 'sklearn.svm._classes.NuSVC'>: [('prob', {'probability': True})],
     <class 'sklearn.svm._classes.SVR'>: [('linear', {'kernel': 'linear'}),
                                          ('poly', {'kernel': 'poly'}),
                                          ('rbf', {'kernel': 'rbf'}),
                                          ('sigmoid', {'kernel': 'sigmoid'})],
     <class 'sklearn.linear_model._passive_aggressive.PassiveAggressiveClassifier'>: [('logreg',
                                                                                       {},
                                                                                       {'conv_options': [{<class 'sklearn.linear_model._passive_aggressive.PassiveAggressiveClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.linear_model._perceptron.Perceptron'>: [('logreg',
                                                              {},
                                                              {'conv_options': [{<class 'sklearn.linear_model._perceptron.Perceptron'>: {'zipmap': False}}]})],
     <class 'sklearn.tree._classes.DecisionTreeClassifier'>: [('default',
                                                               {},
                                                               {'conv_options': [{<class 'sklearn.tree._classes.DecisionTreeClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.tree._classes.ExtraTreeClassifier'>: [('default',
                                                            {},
                                                            {'conv_options': [{<class 'sklearn.tree._classes.ExtraTreeClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.ensemble._forest.RandomForestClassifier'>: [('default',
                                                                  {'n_estimators': 10},
                                                                  {'conv_options': [{<class 'sklearn.ensemble._forest.RandomForestClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.ensemble._forest.RandomForestRegressor'>: [('default',
                                                                 {'n_estimators': 10})],
     <class 'sklearn.ensemble._forest.ExtraTreesClassifier'>: [('default',
                                                                {'n_estimators': 10},
                                                                {'conv_options': [{<class 'sklearn.ensemble._forest.ExtraTreesClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.ensemble._forest.ExtraTreesRegressor'>: [('default',
                                                               {'n_estimators': 10})],
     <class 'sklearn.ensemble._iforest.IsolationForest'>: [('default',
                                                            {'n_estimators': 10})],
     <class 'sklearn.ensemble._weight_boosting.AdaBoostClassifier'>: [('default',
                                                                       {'n_estimators': 10},
                                                                       {'conv_options': [{<class 'sklearn.ensemble._weight_boosting.AdaBoostClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.ensemble._weight_boosting.AdaBoostRegressor'>: [('default',
                                                                      {'n_estimators': 10})],
     <class 'sklearn.ensemble._gb.GradientBoostingClassifier'>: [('default',
                                                                  {'n_estimators': 200},
                                                                  {'conv_options': [{<class 'sklearn.ensemble._gb.GradientBoostingClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.ensemble._gb.GradientBoostingRegressor'>: [('default',
                                                                 {'n_estimators': 200})],
     <class 'sklearn.ensemble._voting.VotingClassifier'>: [('logreg-noflatten',
                                                            {'estimators': [('lr1',
                                                                             LogisticRegression(solver='liblinear')),
                                                                            ('lr2',
                                                                             LogisticRegression(fit_intercept=False, solver='liblinear'))],
                                                             'flatten_transform': False,
                                                             'voting': 'soft'},
                                                            {'conv_options': [{<class 'sklearn.ensemble._voting.VotingClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.ensemble._voting.VotingRegressor'>: [('linreg',
                                                           {'estimators': [('lr1',
                                                                            LinearRegression()),
                                                                           ('lr2',
                                                                            LinearRegression(fit_intercept=False))]})],
     <class 'sklearn.ensemble._stacking.StackingRegressor'>: [('linreg',
                                                               {'estimators': [('lr1',
                                                                                LinearRegression()),
                                                                               ('lr2',
                                                                                LinearRegression(fit_intercept=False))]})],
     <class 'sklearn.ensemble._stacking.StackingClassifier'>: [('logreg',
                                                                {'estimators': [('lr1',
                                                                                 LogisticRegression(solver='liblinear')),
                                                                                ('lr2',
                                                                                 LogisticRegression(fit_intercept=False, solver='liblinear'))]},
                                                                {'conv_options': [{<class 'sklearn.ensemble._stacking.StackingClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingRegressor'>: [('default',
                                                                                                           {'max_iter': 100})],
     <class 'sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingClassifier'>: [('default',
                                                                                                            {'max_iter': 100},
                                                                                                            {'conv_options': [{<class 'sklearn.ensemble._hist_gradient_boosting.gradient_boosting.HistGradientBoostingClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.multioutput.MultiOutputRegressor'>: [('linreg',
                                                           {'estimator': LinearRegression()})],
     <class 'sklearn.multioutput.MultiOutputClassifier'>: [('logreg',
                                                            {'estimator': LogisticRegression(solver='liblinear')},
                                                            {'conv_options': [{<class 'sklearn.multioutput.MultiOutputClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.multioutput.ClassifierChain'>: [('logreg',
                                                      {'base_estimator': LogisticRegression(solver='liblinear')})],
     <class 'sklearn.multioutput.RegressorChain'>: [('linreg',
                                                     {'base_estimator': LinearRegression()})],
     <class 'sklearn.neural_network._multilayer_perceptron.MLPClassifier'>: [('default',
                                                                              {},
                                                                              {'conv_options': [{<class 'sklearn.neural_network._multilayer_perceptron.MLPClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.feature_selection._univariate_selection.SelectPercentile'>: [('p50',
                                                                                   {'percentile': 50})],
     <class 'sklearn.feature_selection._univariate_selection.SelectKBest'>: [('k2',
                                                                              {'k': 2})],
     <class 'sklearn.feature_selection._univariate_selection.SelectFwe'>: [('alpha100',
                                                                            {'alpha': 0.5})],
     <class 'sklearn.feature_selection._rfe.RFE'>: [('reg',
                                                     {'estimator': LinearRegression()})],
     <class 'sklearn.feature_selection._rfe.RFECV'>: [('reg',
                                                       {'estimator': LinearRegression()})],
     <class 'sklearn.feature_selection._from_model.SelectFromModel'>: [('rf',
                                                                        {'estimator': DecisionTreeRegressor()})],
     <class 'sklearn.random_projection.GaussianRandomProjection'>: [('eps95',
                                                                     {'eps': 0.95})],
     <class 'sklearn.random_projection.SparseRandomProjection'>: [('eps95',
                                                                   {'eps': 0.95})],
     <class 'sklearn.preprocessing._data.Normalizer'>: [('l2', {'norm': 'l2'}),
                                                        ('l1', {'norm': 'l1'}),
                                                        ('max', {'norm': 'max'})],
     <class 'sklearn.preprocessing._data.PowerTransformer'>: [('yeo-johnson',
                                                               {'method': 'yeo-johnson'}),
                                                              ('box-cox',
                                                               {'method': 'box-cox'})],
     <class 'sklearn.linear_model._logistic.LogisticRegression'>: [('liblinear',
                                                                    {'solver': 'liblinear'},
                                                                    {'conv_options': [{},
                                                                                      {<class 'sklearn.linear_model._logistic.LogisticRegression'>: {'zipmap': False}}],
                                                                     'optim': [None,
                                                                               'onnx'],
                                                                     'subset_problems': ['b-cl',
                                                                                         '~b-cl-64',
                                                                                         'm-cl']}),
                                                                   ('liblinear-dec',
                                                                    {'solver': 'liblinear'},
                                                                    {'conv_options': [{<class 'sklearn.linear_model._logistic.LogisticRegression'>: {'raw_scores': True,
                                                                                                                                                     'zipmap': False}}],
                                                                     'subset_problems': ['~b-cl-dec',
                                                                                         '~m-cl-dec']})],
     <class 'sklearn.linear_model._logistic.LogisticRegressionCV'>: [('default',
                                                                      {},
                                                                      {'conv_options': [{<class 'sklearn.linear_model._logistic.LogisticRegressionCV'>: {'zipmap': False}}]})],
     <class 'sklearn.decomposition._dict_learning.SparseCoder'>: None,
     <class 'sklearn.decomposition._lda.LatentDirichletAllocation'>: [('default',
                                                                       {'n_components': 2})],
     <class 'sklearn.gaussian_process._gpr.GaussianProcessRegressor'>: [('expsine',
                                                                         {'alpha': 20.0,
                                                                          'kernel': ExpSineSquared(length_scale=1, periodicity=1)},
                                                                         {'conv_options': [{<class 'sklearn.gaussian_process._gpr.GaussianProcessRegressor'>: {'optim': 'cdist'}}]}),
                                                                        ('dotproduct',
                                                                         {'alpha': 100.0,
                                                                          'kernel': DotProduct(sigma_0=1)},
                                                                         {'conv_options': [{},
                                                                                           {<class 'sklearn.gaussian_process._gpr.GaussianProcessRegressor'>: {'optim': 'cdist'}}]}),
                                                                        ('rational',
                                                                         {'alpha': 100.0,
                                                                          'kernel': RationalQuadratic(alpha=1, length_scale=1)},
                                                                         {'conv_options': [{<class 'sklearn.gaussian_process._gpr.GaussianProcessRegressor'>: {'optim': 'cdist'}}]}),
                                                                        ('rbf',
                                                                         {'alpha': 100.0,
                                                                          'kernel': RBF(length_scale=1)},
                                                                         {'conv_options': [{<class 'sklearn.gaussian_process._gpr.GaussianProcessRegressor'>: {'optim': 'cdist'}}]})],
     <class 'sklearn.multiclass.OneVsRestClassifier'>: [('logreg',
                                                         {'estimator': LogisticRegression(solver='liblinear')},
                                                         {'conv_options': [{<class 'sklearn.multiclass.OneVsOneClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.multiclass.OneVsOneClassifier'>: [('logreg',
                                                        {'estimator': LogisticRegression(solver='liblinear')},
                                                        {'conv_options': [{<class 'sklearn.multiclass.OneVsOneClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.multiclass.OutputCodeClassifier'>: [('logreg',
                                                          {'estimator': LogisticRegression(solver='liblinear')},
                                                          {'conv_options': [{<class 'sklearn.multiclass.OneVsOneClassifier'>: {'zipmap': False}}]})],
     <class 'sklearn.gaussian_process._gpc.GaussianProcessClassifier'>: [('expsine',
                                                                          {'kernel': ExpSineSquared(length_scale=1, periodicity=1)},
                                                                          {'conv_options': [{},
                                                                                            {<class 'sklearn.gaussian_process._gpc.GaussianProcessClassifier'>: {'optim': 'cdist'}}]}),
                                                                         ('dotproduct',
                                                                          {'kernel': DotProduct(sigma_0=1)},
                                                                          {'conv_options': [{<class 'sklearn.gaussian_process._gpc.GaussianProcessClassifier'>: {'optim': 'cdist'}}]}),
                                                                         ('rational',
                                                                          {'kernel': RationalQuadratic(alpha=1, length_scale=1)},
                                                                          {'conv_options': [{<class 'sklearn.gaussian_process._gpc.GaussianProcessClassifier'>: {'optim': 'cdist'}}]}),
                                                                         ('rbf',
                                                                          {'kernel': RBF(length_scale=1)},
                                                                          {'conv_options': [{<class 'sklearn.gaussian_process._gpc.GaussianProcessClassifier'>: {'optim': 'cdist'}}]})],
     <class 'sklearn.calibration.CalibratedClassifierCV'>: [('sgd',
                                                             {'base_estimator': SGDClassifier()}),
                                                            ('default', {})],
     <class 'sklearn.feature_extraction._dict_vectorizer.DictVectorizer'>: [('default',
                                                                             {})],
     <class 'sklearn.feature_extraction._hash.FeatureHasher'>: [('default', {})]}

source on GitHub

mlprodict.onnxrt.validate.validate_scenarios.interpret_options_from_string(st)#

Converts a string into a dictionary.

Parameters:

st – string

Returns:

evaluated object

source on GitHub