module utils.nvprof2json

Short summary

module onnxcustom.utils.nvprof2json

Converts traces from nvprof. The source comes from nvprof2json.

source on GitHub

Classes

class	truncated documentation
Cbids	List of events.

Functions

function	truncated documentation
_demangle	Demangle a C++ identifier using c++filt
_munge_time	Take a time from nvprof and convert it into a chrome://tracing time.
_sizeof_fmt	Format size with metric units (like nvvp)
convert_trace_to_json	Converts traces produced by nvprof and saved with format sqlite3 (extension .sql). The output format …
json_to_dataframe	Converts a json dump obtained with function convert_trace_to_json() to a dataframe.
json_to_dataframe_streaming	Converts a big json dump (from convert_trace_to_json()) to a dataframe. The function processes the data by streaming …

Documentation

Converts traces from nvprof. The source comes from nvprof2json.

source on GitHub

class onnxcustom.utils.nvprof2json.Cbids(value)

Bases: IntEnum

List of events.

onnxcustom.utils.nvprof2json._demangle(name)

Demangle a C++ identifier using c++filt

source on GitHub

onnxcustom.utils.nvprof2json._munge_time(t)

Take a time from nvprof and convert it into a chrome://tracing time.

source on GitHub

onnxcustom.utils.nvprof2json._sizeof_fmt(num, suffix='B')

Format size with metric units (like nvvp)

source on GitHub

onnxcustom.utils.nvprof2json.convert_trace_to_json(filename, output=None, temporary_file=None, verbose=0, fLOG=None)

Converts traces produced by nvprof and saved with format sqlite3 (extension .sql). The output format follows Trace Event Format.

Parameters:

• filename – filename

• output – output file or None

• temporary_file – if the file needs to be unzipped, this file will be created to be the unzipped file, it is not cleaned after the unzipping.

• verbose – verbosity

• fLOG – logging function

Returns:

json (if output is None, the list of events otherwise)

This file, if not too big, can be viewed with chrome-tracing. The traces are usually generated by using a command line similar to:

nvprof -o gpu_profile.sql python plot_gpu_training.py

source on GitHub

onnxcustom.utils.nvprof2json.json_to_dataframe(js)

Converts a json dump obtained with function convert_trace_to_json to a dataframe.

Parameters:

js – a filename, a json string, a stream containing json

Returns:

a dataframe

source on GitHub

onnxcustom.utils.nvprof2json.json_to_dataframe_streaming(js, chunksize=100000, flatten=False, **kwargs)

Converts a big json dump (from convert_trace_to_json) to a dataframe. The function processes the data by streaming to avoid loading huge data in memory. Returns an iterator on dataframes. The function relies on pandas_streaming.

Parameters:

• js – a filename, a json string, a stream containing json

• chunksize – see pandas_streaming.df.StreamingDataFrame.read_json()

• flatten – see pandas_streaming.df.StreamingDataFrame.read_json()

• kwargs – see pandas_streaming.df.StreamingDataFrame.read_json()

Returns:

a dataframe

source on GitHub