module benchhelper.benchmark

Inheritance diagram of pyquickhelper.benchhelper.benchmark

Short summary

module pyquickhelper.benchhelper.benchmark

Helpers to benchmark something

source on GitHub

Classes

class

truncated documentation

BenchMark

Class to help benchmarking. You should overwrite method init, bench, end, graphs.

Properties

property

truncated documentation

Appendix

Returns the metrics.

Graphs

Returns images of graphs.

Metadata

Returns the metrics.

Metrics

Returns the metrics.

Name

Returns the name of the benchmark.

Methods

method

truncated documentation

__init__

_convert

Converts a value knowing its column, its type into something readable.

bench

Runs the benchmark. Overwrite this method.

end

Cleans. Overwrites this method.

fLOG

Logs something.

graphs

Builds graphs after the benchmark was run.

init

Initialisation. Overwrite this method.

meta_to_df

Converts meta data into a dataframe

report

Produces a report.

run

Runs the benchmark.

to_df

Converts the metrics into a dataframe.

uncache

overwrite this method to uncache some previous run

Documentation

Helpers to benchmark something

source on GitHub

class pyquickhelper.benchhelper.benchmark.BenchMark(name, clog=None, fLOG=<function noLOG>, path_to_images='.', cache_file=None, pickle_module=None, progressbar=None, **params)[source][source]

Bases: object

Class to help benchmarking. You should overwrite method init, bench, end, graphs.

source on GitHub

Parameters
  • name – name of the test

  • clogCustomLog or string

  • fLOG – logging function

  • params – extra parameters

  • path_to_images – path to images

  • cache_file – cache file

  • pickle_module – pickle or dill if you need to serialize functions

  • progressbar – relies on tqdm, example tnrange

If cache_file is specified, the class will store the results of the method bench. On a second run, the function load the cache and run modified or new run (in params_list).

source on GitHub

property Appendix[source]

Returns the metrics.

source on GitHub

property Graphs[source]

Returns images of graphs.

source on GitHub

class LocalGraph(func_gen, filename=None, title=None, root=None)[source][source]

Bases: object

Information about graphs.

source on GitHub

Parameters
  • func_gen – function generating the graph

  • filename – filename

  • title – title

  • root – path should be relative to this one

source on GitHub

__init__(func_gen, filename=None, title=None, root=None)[source][source]
Parameters
  • func_gen – function generating the graph

  • filename – filename

  • title – title

  • root – path should be relative to this one

source on GitHub

add(name, value)[source][source]

Adds an attribute.

Parameters
  • name – name of the attribute

  • value – value

source on GitHub

plot(ax=None, text=True, **kwargs)[source][source]

Draws the graph again.

Parameters
  • ax – axis

  • text – add text on the graph

  • kwargs – additional parameters

Returns

axis

source on GitHub

to_html()[source][source]

Renders as HTML.

source on GitHub

to_rst()[source][source]

Renders as :ekg:`rst`.

source on GitHub

property Metadata[source]

Returns the metrics.

source on GitHub

property Metrics[source]

Returns the metrics.

source on GitHub

property Name[source]

Returns the name of the benchmark.

source on GitHub

__init__(name, clog=None, fLOG=<function noLOG>, path_to_images='.', cache_file=None, pickle_module=None, progressbar=None, **params)[source][source]
Parameters
  • name – name of the test

  • clogCustomLog or string

  • fLOG – logging function

  • params – extra parameters

  • path_to_images – path to images

  • cache_file – cache file

  • pickle_module – pickle or dill if you need to serialize functions

  • progressbar – relies on tqdm, example tnrange

If cache_file is specified, the class will store the results of the method bench. On a second run, the function load the cache and run modified or new run (in params_list).

source on GitHub

_convert(df, i, col, ty, value)[source][source]

Converts a value knowing its column, its type into something readable.

Parameters
  • df – dataframe

  • i – line index

  • col – column name

  • ty – type

  • value – value to convert

Returns

value

source on GitHub

bench(**params)[source][source]

Runs the benchmark. Overwrite this method.

Parameters

params – parameters

Returns

metrics as a dictionary, appendix as a dictionary

The results of this method will be cached if a cache_file was specified in the constructor.

source on GitHub

end()[source][source]

Cleans. Overwrites this method.

source on GitHub

fLOG(*l, **p)[source][source]

Logs something.

source on GitHub

graphs(path_to_images)[source][source]

Builds graphs after the benchmark was run.

Parameters

path_to_images – path to images

Returns

a list of LocalGraph

Every returned graph must contain a function which creates the graph. The function must accepts two parameters ax and text. Example:

def local_graph(ax=None, text=True, figsize=(5,5)):
    vx = ...
    vy = ...
    btrys = set(df["_btry"])
    ymin = df[vy].min()
    ymax = df[vy].max()
    decy = (ymax - ymin) / 50
    colors = cm.rainbow(numpy.linspace(0, 1, len(btrys)))
    if len(btrys) == 0:
        raise ValueError("The benchmark is empty.")
    if ax is None:
        fig, ax = plt.subplots(1, 1, figsize=figsize)
        ax.grid(True)
    for i, btry in enumerate(sorted(btrys)):
        subset = df[df["_btry"]==btry]
        if subset.shape[0] > 0:
            subset.plot(x=vx, y=vy, kind="scatter", label=btry, ax=ax, color=colors[i])
        if text:
            tx = subset[vx].mean()
            ty = subset[vy].mean()
            ax.text(tx, ty + decy, btry, size='small',
                    color=colors[i], ha='center', va='bottom')
    ax.set_xlabel(vx)
    ax.set_ylabel(vy)
    return ax

source on GitHub

init()[source][source]

Initialisation. Overwrite this method.

source on GitHub

meta_to_df(convert=False, add_link=False, format='html')[source][source]

Converts meta data into a dataframe

Parameters
  • convert – if True, calls method _convert on each cell

  • add_link – add hyperlink

  • format – format for hyperlinks (html or rst)

Returns

dataframe

source on GitHub

report(css=None, template_html=None, template_rst=None, engine='mako', filecsv=None, filehtml=None, filerst=None, params_html=None, title=None, description=None)[source][source]

Produces a report.

Parameters
  • css – css (will take the default one if empty)

  • template_html – template HTML (mako or jinja2)

  • template_rst – template RST (mako or jinja2)

  • engine'mako’ or ‘jinja2'

  • filehtml – report will written in this file if not None

  • filecsv – metrics will be written as a flat table

  • filerst – metrics will be written as a RST table

  • params_html – parameter to send to function pandas.DataFrame.to_html

  • title – title (Name if any)

  • description – add a description

Returns

dictionary {format: content}

You can define your own template by looking into the default ones defines in this class (see the bottom of this file). By default, HTML and RST report are generated.

source on GitHub

run(params_list)[source][source]

Runs the benchmark.

Parameters

params_list – list of dictionaries

source on GitHub

to_df(convert=False, add_link=False, format='html')[source][source]

Converts the metrics into a dataframe.

Parameters
  • convert – if True, calls method _convert on each cell

  • add_link – add hyperlink

  • format – format for hyperlinks (html or rst)

Returns

dataframe

source on GitHub

uncache(cache)[source][source]

overwrite this method to uncache some previous run

source on GitHub