module ml.neural_tree#

Inheritance diagram of mlstatpy.ml.neural_tree

Short summary#

module mlstatpy.ml.neural_tree

Conversion from tree to neural network.

source on GitHub

Classes#

class

truncated documentation

BaseNeuralTreeNet

Classifier or regressor following scikit-learn API.

NeuralTreeNet

Node ensemble.

NeuralTreeNetClassifier

Classifier following scikit-learn API.

NeuralTreeNetRegressor

Regressor following scikit-learn API.

Functions#

function

truncated documentation

label_class_to_softmax_output

Converts a binary class label into a matrix with two columns of probabilities.

Properties#

property

truncated documentation

_repr_html_

HTML representation of estimator. This is redundant with the logic of _repr_mimebundle_. The latter should …

_repr_html_

HTML representation of estimator. This is redundant with the logic of _repr_mimebundle_. The latter should …

_repr_html_

HTML representation of estimator. This is redundant with the logic of _repr_mimebundle_. The latter should …

shape

Returns the shape of the coefficients.

training_weights

Returns the weights.

Static Methods#

staticmethod

truncated documentation

_create_from_tree_compact

Implements strategy “compact”. See @see meth create_from_tree.

_create_from_tree_one

Implements strategy “one”. See @see meth create_from_tree.

create_from_tree

Creates a NeuralTreeNet instance from a DecisionTreeClassifier

onnx_converter

Converts this model into ONNX.

onnx_converter

Converts this model into ONNX.

onnx_converter

Converts this model into ONNX.

onnx_shape_calculator

Shape calculator when converting this model into ONNX. See :epkg:`skearn-onnx`.

onnx_shape_calculator

Shape calculator when converting this model into ONNX. See :epkg:`skearn-onnx`.

onnx_shape_calculator

Shape calculator when converting this model into ONNX. See :epkg:`skearn-onnx`.

Methods#

method

truncated documentation

__getitem__

Retrieves node and attributes for node i.

__init__

__init__

__init__

__init__

__len__

Returns the number of nodes

__repr__

usual

_common_loss_dloss

Common beginning to methods loss, dlossds, dlossdw.

_get_output_node_attr

Retrieves the output nodes. nb_last is the number of expected outputs.

_predict_one

_update_members

Updates internal members.

append

Appends a node into the graph.

clear

Clear all nodes

copy

decision_function

Returns the classification probabilities.

decision_function

Returns the classification probabilities.

decision_function

Returns the classification probabilities.

dlossds

Computes the loss derivative against the inputs.

fill_cache

Creates a cache with intermediate results.

fit

Trains the estimator.

fit

Trains the estimator.

fit

Trains the estimator.

gradient_backward

Computes the gradient in X.

loss

Computes the loss due to prediction error. Returns a float.

predict

predict

Returns the predicted classes.

predict

Returns the predicted classes.

predict_proba

Returns the classification probabilities.

to_dot

Exports the neural network into dot.

update_training_weights

Updates weights.

Documentation#

Conversion from tree to neural network.

source on GitHub

class mlstatpy.ml.neural_tree.BaseNeuralTreeNet(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)#

Bases : BaseEstimator

Classifier or regressor following scikit-learn API.

Paramètres:
  • estimator – instance of NeuralTreeNet.

  • X – training set

  • y – training labels

  • optimizer – optimizer, by default, it is SGDOptimizer.

  • max_iter – number maximum of iterations

  • early_th – early stopping threshold

  • verbose – more verbose

  • lr – to overwrite learning_rate_init if optimizer is None (unused otherwise)

  • lr_schedule – to overwrite lr_schedule if optimizer is None (unused otherwise)

  • l1 – L1 regularization if optimizer is None (unused otherwise)

  • l2 – L2 regularization if optimizer is None (unused otherwise)

  • momentum – used if optimizer is None

source on GitHub

__init__(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)#
decision_function(X)#

Returns the classification probabilities.

Paramètres:

X – inputs

Renvoie:

probabilities

source on GitHub

fit(X, y, sample_weights=None)#

Trains the estimator.

Paramètres:
  • X – input features

  • y – expected classes (binary)

  • sample_weights – sample weights

Renvoie:

self

source on GitHub

static onnx_converter()#

Converts this model into ONNX.

source on GitHub

static onnx_shape_calculator()#

Shape calculator when converting this model into ONNX. See :epkg:`skearn-onnx`.

source on GitHub

class mlstatpy.ml.neural_tree.NeuralTreeNet(dim, empty=True)#

Bases : _TrainingAPI

Node ensemble.

Paramètres:
  • dim – space dimension

  • empty – empty network, other adds an identity node

<<<

import numpy
from mlstatpy.ml.neural_tree import NeuralTreeNode, NeuralTreeNet

w1 = numpy.array([-0.5, 0.8, -0.6])

neu = NeuralTreeNode(w1[1:], bias=w1[0], activation='sigmoid')
net = NeuralTreeNet(2, empty=True)
net.append(neu, numpy.arange(2))

ide = NeuralTreeNode(numpy.array([1.]),
                     bias=numpy.array([0.]),
                     activation='identity')

net.append(ide, numpy.arange(2, 3))

X = numpy.abs(numpy.random.randn(10, 2))
pred = net.predict(X)
print(pred)

>>>

    [[1.4   0.912 0.518 0.518]
     [0.426 3.023 0.122 0.122]
     [1.128 0.237 0.565 0.565]
     [0.744 0.102 0.509 0.509]
     [1.383 0.228 0.615 0.615]
     [0.581 0.204 0.461 0.461]
     [1.323 0.628 0.545 0.545]
     [1.037 1.161 0.409 0.409]
     [2.418 0.511 0.755 0.755]
     [0.943 0.167 0.538 0.538]]

source on GitHub

__getitem__(i)#

Retrieves node and attributes for node i.

__init__(dim, empty=True)#
__len__()#

Returns the number of nodes

__repr__()#

usual

_common_loss_dloss(X, y, cache=None)#

Common beginning to methods loss, dlossds, dlossdw.

source on GitHub

static _create_from_tree_compact(tree, k=1.0)#

Implements strategy “compact”. See @see meth create_from_tree.

static _create_from_tree_one(tree, k=1.0)#

Implements strategy “one”. See @see meth create_from_tree.

_get_output_node_attr(nb_last)#

Retrieves the output nodes. nb_last is the number of expected outputs.

source on GitHub

_predict_one(X)#
_update_members(node=None, attr=None)#

Updates internal members.

append(node, inputs)#

Appends a node into the graph.

Paramètres:
  • node – node to add

  • inputs – index of input nodes

source on GitHub

clear()#

Clear all nodes

static create_from_tree(tree, k=1.0, arch='one')#

Creates a NeuralTreeNet instance from a DecisionTreeClassifier

Paramètres:
Renvoie:

NeuralTreeNet

The function only works for binary problems. Available architecture: * “one”: the method adds nodes with one output, there

is no soecific definition of layers,

  • “compact”: the adds two nodes, the first computes the threshold, the second one computes the leaves output, a final node merges all outputs into one

See notebook Un arbre de décision en réseaux de neurones for examples.

source on GitHub

dlossds(X, y, cache=None)#

Computes the loss derivative against the inputs.

source on GitHub

fill_cache(X)#

Creates a cache with intermediate results.

source on GitHub

gradient_backward(graddx, X, inputs=False, cache=None)#

Computes the gradient in X.

Paramètres:
  • graddx – existing gradient against the inputs

  • X – computes the gradient in X

  • inputs – if False, derivative against the coefficients, otherwise against the inputs.

  • cache – cache intermediate results to avoid more computation

Renvoie:

gradient

source on GitHub

loss(X, y, cache=None)#

Computes the loss due to prediction error. Returns a float.

source on GitHub

property shape#

Returns the shape of the coefficients.

to_dot(X=None)#

Exports the neural network into dot.

Paramètres:

X – input as an example

source on GitHub

property training_weights#

Returns the weights.

update_training_weights(X, add=True)#

Updates weights.

Paramètres:
  • grad – vector to add to the weights such as gradient

  • add – addition or replace

source on GitHub

class mlstatpy.ml.neural_tree.NeuralTreeNetClassifier(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)#

Bases : ClassifierMixin, BaseNeuralTreeNet

Classifier following scikit-learn API.

Paramètres:
  • estimator – instance of NeuralTreeNet.

  • X – training set

  • y – training labels

  • optimizer – optimizer, by default, it is SGDOptimizer.

  • max_iter – number maximum of iterations

  • early_th – early stopping threshold

  • verbose – more verbose

  • lr – to overwrite learning_rate_init if optimizer is None (unused otherwise)

  • lr_schedule – to overwrite lr_schedule if optimizer is None (unused otherwise)

  • l1 – L1 regularization if optimizer is None (unused otherwise)

  • l2 – L2 regularization if optimizer is None (unused otherwise)

  • momentum – used if optimizer is None

source on GitHub

__init__(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)#
predict(X)#

Returns the predicted classes.

Paramètres:

X – inputs

Renvoie:

classes

source on GitHub

predict_proba(X)#

Returns the classification probabilities.

Paramètres:

X – inputs

Renvoie:

probabilities

source on GitHub

class mlstatpy.ml.neural_tree.NeuralTreeNetRegressor(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)#

Bases : RegressorMixin, BaseNeuralTreeNet

Regressor following scikit-learn API.

Paramètres:
  • estimator – instance of NeuralTreeNet.

  • X – training set

  • y – training labels

  • optimizer – optimizer, by default, it is SGDOptimizer.

  • max_iter – number maximum of iterations

  • early_th – early stopping threshold

  • verbose – more verbose

  • lr – to overwrite learning_rate_init if optimizer is None (unused otherwise)

  • lr_schedule – to overwrite lr_schedule if optimizer is None (unused otherwise)

  • l1 – L1 regularization if optimizer is None (unused otherwise)

  • l2 – L2 regularization if optimizer is None (unused otherwise)

  • momentum – used if optimizer is None

source on GitHub

__init__(estimator, optimizer=None, max_iter=100, early_th=None, verbose=False, lr=None, lr_schedule=None, l1=0.0, l2=0.0, momentum=0.9)#
predict(X)#

Returns the predicted classes.

Paramètres:

X – inputs

Renvoie:

classes

source on GitHub

mlstatpy.ml.neural_tree.label_class_to_softmax_output(y_label)#

Converts a binary class label into a matrix with two columns of probabilities.

<<<

import numpy
from mlstatpy.ml.neural_tree import label_class_to_softmax_output

y_label = numpy.array([0, 1, 0, 0])
soft_y = label_class_to_softmax_output(y_label)
print(soft_y)

>>>

    [[1. 0.]
     [0. 1.]
     [1. 0.]
     [1. 0.]]

source on GitHub