module ml._neural_tree_node

Inheritance diagram of mlstatpy.ml._neural_tree_node

Short summary

module mlstatpy.ml._neural_tree_node

Conversion from tree to neural network.

source on GitHub

Classes

class

truncated documentation

NeuralTreeNode

One node in a neural network.

Properties

property

truncated documentation

bias

Returns the weights.

input_weights

Returns the weights.

ndim

Returns the input dimension.

training_weights

Returns the weights stored in the neuron.

Static Methods

staticmethod

truncated documentation

_dleakyrelu

Derivative of the Leaky Relu function.

_drelu

Derivative of the Relu function.

_dsigmoid

Derivativ of the sigmoid function.

_dsoftmax

Derivative of the softmax function.

_leakyrelu

Leaky Relu function.

_relu

Relu function.

_softmax

Derivative of the softmax function.

get_activation_dloss_function

Returns the derivative of the default loss function based on the activation function. It returns a function …

get_activation_function

Returns the activation function. It returns a function y=f(x).

get_activation_gradient_function

Returns the activation function. It returns a function y=f”(x). About the sigmoid:

get_activation_loss_function

Returns a default loss function based on the activation function. It returns two functions g=loss(x,y).

Methods

method

truncated documentation

__eq__

__getstate__

usual

__init__

__repr__

usual

__setstate__

usual

_common_loss_dloss

Common beginning to methods loss, dlossds, dlossdw.

_predict

Computes inputs of the activation function.

_set_fcts

dlossds

Computes the loss derivative due to prediction error.

fill_cache

Creates a cache with intermediate results. lX is the results before the activation function, aX

gradient_backward

Computes the gradients at point X.

loss

Computes the loss. Returns a float.

predict

Computes neuron outputs.

update_training_weights

Updates weights.

Documentation

Conversion from tree to neural network.

source on GitHub

class mlstatpy.ml._neural_tree_node.NeuralTreeNode(weights, bias=None, activation='sigmoid', nodeid=- 1, tag=None)[source]

Bases : mlstatpy.ml._neural_tree_api._TrainingAPI

One node in a neural network.

source on GitHub

Paramètres
  • weights – weights

  • bias – bias, if None, draws a random number

  • activation – activation function

  • nodeid – node id

  • tag – unused but to add information on how this node was created

source on GitHub

__eq__(obj)[source]

Return self==value.

__getstate__()[source]

usual

__hash__ = None
__init__(weights, bias=None, activation='sigmoid', nodeid=- 1, tag=None)[source]
Paramètres
  • weights – weights

  • bias – bias, if None, draws a random number

  • activation – activation function

  • nodeid – node id

  • tag – unused but to add information on how this node was created

source on GitHub

__repr__()[source]

usual

__setstate__(state)[source]

usual

_common_loss_dloss(X, y, cache=None)[source]

Common beginning to methods loss, dlossds, dlossdw.

source on GitHub

static _dleakyrelu(x)[source]

Derivative of the Leaky Relu function.

static _drelu(x)[source]

Derivative of the Relu function.

static _dsigmoid(x)[source]

Derivativ of the sigmoid function.

static _dsoftmax(x)[source]

Derivative of the softmax function.

static _leakyrelu(x)[source]

Leaky Relu function.

_predict(X)[source]

Computes inputs of the activation function.

static _relu(x)[source]

Relu function.

_set_fcts()[source]
static _softmax(x)[source]

Derivative of the softmax function.

property bias

Returns the weights.

dlossds(X, y, cache=None)[source]

Computes the loss derivative due to prediction error.

source on GitHub

fill_cache(X)[source]

Creates a cache with intermediate results. lX is the results before the activation function, aX is the results after the activation function, the prediction.

source on GitHub

static get_activation_dloss_function(activation)[source]

Returns the derivative of the default loss function based on the activation function. It returns a function df(x,y)/dw, df(w)/dw where w are the weights.

source on GitHub

static get_activation_function(activation)[source]

Returns the activation function. It returns a function y=f(x).

source on GitHub

static get_activation_gradient_function(activation)[source]

Returns the activation function. It returns a function y=f”(x). About the sigmoid:

System Message: WARNING/2 (\begin{array}{l} f(x) &=& )

latex exited with error [stdout] This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020/Debian) (preloaded format=latex) restricted \write18 enabled. entering extended mode (./math.tex LaTeX2e <2020-02-02> patch level 5 L3 programming layer <2020-07-17> (/usr/share/texlive/texmf-dist/tex/latex/base/article.cls Document Class: article 2019/12/20 v1.4l Standard LaTeX document class (/usr/share/texlive/texmf-dist/tex/latex/base/size12.clo)) (/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty For additional information on amsmath, use the `?' option. (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty) (/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty)) (/usr/share/texlive/texmf-dist/tex/latex/amscls/amsthm.sty) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty)) (/usr/share/texlive/texmf-dist/tex/latex/anyfontsize/anyfontsize.sty) (/usr/share/texlive/texmf-dist/tex/latex/tools/bm.sty) (/usr/share/texlive/texmf-dist/tex/latex/etex-pkg/etex.sty) (/usr/share/texlive/texmf-dist/tex/latex/base/fixltx2e.sty Package fixltx2e Warning: fixltx2e is not required with releases after 2015 (fixltx2e) All fixes are now in the LaTeX kernel. (fixltx2e) See the latexrelease package for details. ) (/usr/share/texlive/texmf-dist/tex/latex/cmap/cmap.sty Package cmap Warning: pdftex in DVI mode - exiting. ) (/usr/share/texlive/texmf-dist/tex/latex/fast-diagram/fast-diagram.sty (/usr/share/texlive/texmf-dist/tex/latex/pgf/frontendlayer/tikz.sty (/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgf.sty (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfrcs.sty (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-common-lists.t ex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfutil-latex.def (/usr/share/texlive/texmf-dist/tex/latex/ms/everyshi.sty)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfrcs.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/pgf.revision.tex))) (/usr/share/texlive/texmf-dist/tex/latex/pgf/basiclayer/pgfcore.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty) (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/graphics.cfg) (/usr/share/texlive/texmf-dist/tex/latex/graphics-def/dvips.def))) (/usr/share/texlive/texmf-dist/tex/latex/pgf/systemlayer/pgfsys.sty (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeysfiltered.code.t ex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgf.cfg) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-dvips.def (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsys-common-postsc ript.def))) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsyssoftpath.code. tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/systemlayer/pgfsysprotocol.code. tex)) (/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty (/usr/share/texlive/texmf-dist/tex/latex/graphics-cfg/color.cfg)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcore.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathcalc.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathutil.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathparser.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.basic.code .tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.trigonomet ric.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.random.cod e.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.comparison .code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.base.code. tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.round.code .tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.misc.code. tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfunctions.integerari thmetics.code.tex))) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmathfloat.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfint.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepoints.code.te x) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathconstruct. code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathusage.code .tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorescopes.code.te x) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoregraphicstate.c ode.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransformation s.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorequick.code.tex ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreobjects.code.t ex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepathprocessing .code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorearrows.code.te x) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreshade.code.tex ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreimage.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoreexternal.code. tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorelayers.code.te x) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcoretransparency.c ode.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorepatterns.code. tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/basiclayer/pgfcorerdf.code.tex)) ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleshapes.code.tex ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmoduleplot.code.tex ) (/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-0-65 .sty) (/usr/share/texlive/texmf-dist/tex/latex/pgf/compatibility/pgfcomp-version-1-18 .sty)) (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgffor.sty (/usr/share/texlive/texmf-dist/tex/latex/pgf/utilities/pgfkeys.sty (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgfkeys.code.tex)) (/usr/share/texlive/texmf-dist/tex/latex/pgf/math/pgfmath.sty (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/utilities/pgffor.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/math/pgfmath.code.tex))) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/tikz.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/pgflibraryplothandlers .code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/modules/pgfmodulematrix.code.tex ) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarytopaths.code.tex))) (/usr/share/texlive/texmf-dist/tex/latex/base/ifthen.sty) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibrarycalc.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryfit.code.tex) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshapes.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshapes.geometric.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibraryshape s.geometric.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshapes.misc.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibraryshape s.misc.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshapes.symbols.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibraryshape s.symbols.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshapes.arrows.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibraryshape s.arrows.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshapes.callouts.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibraryshape s.callouts.code.tex)) (/usr/share/texlive/texmf-dist/tex/generic/pgf/frontendlayer/tikz/libraries/tik zlibraryshapes.multipart.code.tex (/usr/share/texlive/texmf-dist/tex/generic/pgf/libraries/shapes/pgflibraryshape s.multipart.code.tex))) (/usr/share/texlive/texmf-dist/tex/latex/relsize/relsize.sty) (/usr/share/texlive/texmf-dist/tex/latex/xargs/xargs.sty (/usr/share/texlive/texmf-dist/tex/latex/xkeyval/xkeyval.sty (/usr/share/texlive/texmf-dist/tex/generic/xkeyval/xkeyval.tex (/usr/share/texlive/texmf-dist/tex/generic/xkeyval/xkvutils.tex))))) (/usr/share/texlive/texmf-dist/tex/latex/titlesec/titlesec.sty) (/usr/share/texlive/texmf-dist/tex/latex/eepic/epic.sty Enhancements to Picture Environment. Version 1.2 - Released June 1, 1986 ) (/usr/share/texlive/texmf-dist/tex/latex/eepic/eepic.sty Extension to Epic and LaTeX. Version 1.1e - Released Dec 21, 1999 ) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xy.sty (/usr/share/texlive/texmf-dist/tex/generic/xypic/xy.tex Bootstrap'ing: catcodes, docmode, (/usr/share/texlive/texmf-dist/tex/generic/xypic/xyrecat.tex ) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xyidioms.tex) Xy-pic version 3.8.9 <2013/10/06> Copyright (c) 1991-2013 by Kristoffer H. Rose <krisrose@tug.org> and others Xy-pic is free software: see the User's Guide for details. Loading kernel: messages; fonts; allocations: state, direction, utility macros; pictures: \xy, positions, objects, decorations; kernel objects: directionals, circles, text; options; algorithms: directions, edges, connections; Xy-pic loaded) (/usr/share/texlive/texmf-dist/tex/generic/iftex/ifpdf.sty (/usr/share/texlive/texmf-dist/tex/generic/iftex/iftex.sty)) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xyall.tex Xy-pic option: All features v.3.8 (/usr/share/texlive/texmf-dist/tex/generic/xypic/xycurve.tex Xy-pic option: Curve and Spline extension v.3.12 curve, circles, loaded) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xyframe.tex Xy-pic option: Frame and Bracket extension v.3.14 loaded) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xycmtip.tex Xy-pic option: Computer Modern tip extension v.3.7 (/usr/share/texlive/texmf-dist/tex/generic/xypic/xytips.tex Xy-pic option: More Tips extension v.3.11 loaded) loaded) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xyline.tex Xy-pic option: Line styles extension v.3.10 loaded) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xyrotate.tex Xy-pic option: Rotate and Scale extension v.3.8 loaded) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xycolor.tex Xy-pic option: Colour extension v.3.11 loaded) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xymatrix.tex Xy-pic option: Matrix feature v.3.14 loaded) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xyarrow.tex Xy-pic option: Arrow and Path feature v.3.9 path, \ar, loaded) (/usr/share/texlive/texmf-dist/tex/generic/xypic/xygraph.tex Xy-pic option: Graph feature v.3.11 loaded) loaded)) (/usr/share/texlive/texmf-dist/tex/latex/l3backend/l3backend-dvips.def) (./math.aux) ABD: EveryShipout initializing macros (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsa.fd) (/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsb.fd) ! LaTeX Error: \begin{split} on input line 69 ended by \end{equation*}. See the LaTeX manual or LaTeX Companion for explanation. Type H <return> for immediate help. ... l.71 \end{equation*} ! Extra alignment tab has been changed to \cr. <recently read> \endtemplate l.71 \end{equation*} ! Extra alignment tab has been changed to \cr. <recently read> \endtemplate l.71 \end{equation*} ! LaTeX Error: \begin{array} on input line 71 ended by \end{split}. See the LaTeX manual or LaTeX Companion for explanation. Type H <return> for immediate help. ... l.71 \end{equation*} ! Extra alignment tab has been changed to \cr. <recently read> \endtemplate l.71 \end{equation*} ! Extra alignment tab has been changed to \cr. <recently read> \endtemplate l.71 \end{equation*} ! LaTeX Error: \begin{array} on input line 71 ended by \end{split}. See the LaTeX manual or LaTeX Companion for explanation. Type H <return> for immediate help. ... l.71 \end{equation*} ! Missing $ inserted. <inserted text> $ l.72 \end{document} ! Missing } inserted. <inserted text> } l.72 \end{document} ! Missing } inserted. <inserted text> } l.72 \end{document} ! Missing } inserted. <inserted text> } l.72 \end{document} ! Missing \cr inserted. <inserted text> \cr l.72 \end{document} ! Missing { inserted. <inserted text> { l.72 \end{document} ! Missing { inserted. <inserted text> { l.72 \end{document} ! Extra }, or forgotten $. <recently read> } l.72 \end{document} ! Missing $ inserted. <inserted text> $ l.72 \end{document} ! Missing } inserted. <inserted text> } l.72 \end{document} ! LaTeX Error: \begin{equation*} on input line 68 ended by \end{document}. See the LaTeX manual or LaTeX Companion for explanation. Type H <return> for immediate help. ... l.72 \end{document} (./math.aux) ! You can't use `\end' in internal vertical mode. \enddocument ... \endgroup \deadcycles \z@ \@@end l.72 \end{document} ! LaTeX Error: \begin{equation*} on input line 68 ended by \end{document}. See the LaTeX manual or LaTeX Companion for explanation. Type H <return> for immediate help. ... l.72 \end{document} ! Missing } inserted. <inserted text> } l.72 \end{document} ! Missing $ inserted. <inserted text> $ l.72 \end{document} ! Display math should end with $$. <to be read again> \endgroup l.72 \end{document} ) ! Emergency stop. <*> math.tex No pages of output. Transcript written on math.log.

rac{1}{1 + e^{-x}} \

f”(x) &=&

rac{e^{-x}}{(1 + e^{-x})^2} = f(x)(1-f(x))

end{array}}

source on GitHub

static get_activation_loss_function(activation)[source]

Returns a default loss function based on the activation function. It returns two functions g=loss(x,y).

source on GitHub

gradient_backward(graddx, X, inputs=False, cache=None)[source]

Computes the gradients at point X.

Paramètres
  • graddx – existing gradient against the inputs

  • X – computes the gradient in X

  • inputs – if False, derivative against the coefficients, otherwise against the inputs.

  • cache – cache intermediate results

Renvoie

gradient

source on GitHub

property input_weights

Returns the weights.

loss(X, y, cache=None)[source]

Computes the loss. Returns a float.

source on GitHub

property ndim

Returns the input dimension.

predict(X)[source]

Computes neuron outputs.

property training_weights

Returns the weights stored in the neuron.

update_training_weights(X, add=True)[source]

Updates weights.

Paramètres
  • grad – vector to add to the weights such as gradient

  • add – addition or replace

source on GitHub