.. _profilingrst:

====================
Hierchical profiling
====================


.. only:: html

    **Links:** :download:`notebook <profiling.ipynb>`, :downloadlink:`html <profiling2html.html>`, :download:`PDF <profiling.pdf>`, :download:`python <profiling.py>`, :downloadlink:`slides <profiling.slides.html>`, :githublink:`GitHub|_doc/notebooks/profiling.ipynb|*`


`pyinstrument <https://github.com/joerick/pyinstrument>`__ has a nice
rendering when it comes to display the profiling of a function in a
notebook. Let’s see how to get the same with this library.

.. code:: ipython3

    from jyquickhelper import add_notebook_menu
    add_notebook_menu()






.. contents::
    :local:





The code to profile
-------------------

.. code:: ipython3

    import time
    
    def f0(t):
        time.sleep(t)
    
    def f1(t):
        time.sleep(t)
    
    def f2():
        f1(0.1)
        f1(0.01)
    
    def f3():
        f0(0.2)
        f1(0.5)
    
    def f4():
        f2()
        f3()

.. code:: ipython3

    from pyquickhelper.pycode.profiling import profile
    stat, text = profile(f4)

This is the default rendering for module
`cProfile <https://docs.python.org/3/library/profile.html>`__.

.. code:: ipython3

    print(text)


.. parsed-literal::
             12 function calls in 0.829 seconds
       Ordered by: cumulative time
       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
            1    0.000    0.000    0.829    0.829 <ipython-input-2-61d7d6b6a99d>:17(f4)
            4    0.829    0.207    0.829    0.207 {built-in method time.sleep}
            1    0.000    0.000    0.711    0.711 <ipython-input-2-61d7d6b6a99d>:13(f3)
            3    0.000    0.000    0.622    0.207 <ipython-input-2-61d7d6b6a99d>:6(f1)
            1    0.000    0.000    0.207    0.207 <ipython-input-2-61d7d6b6a99d>:3(f0)
            1    0.000    0.000    0.118    0.118 <ipython-input-2-61d7d6b6a99d>:9(f2)
            1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
    
    
    


Renders a profile as a dataframe
--------------------------------

.. code:: ipython3

    from pyquickhelper.pycode.profiling import profile2df
    
    df = profile2df(stat)
    df






.. raw:: html

    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>fct</th>
          <th>file</th>
          <th>ncalls1</th>
          <th>ncalls2</th>
          <th>tin</th>
          <th>cum_tin</th>
          <th>tall</th>
          <th>cum_tall</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>f4</td>
          <td>&lt;ipython-input-2-61d7d6b6a99d&gt;:17</td>
          <td>1</td>
          <td>1</td>
          <td>0.000026</td>
          <td>0.000026</td>
          <td>0.828829</td>
          <td>0.828829</td>
        </tr>
        <tr>
          <th>1</th>
          <td>&lt;built-in method time.sleep&gt;</td>
          <td>~:0</td>
          <td>4</td>
          <td>4</td>
          <td>0.828624</td>
          <td>0.828624</td>
          <td>0.828624</td>
          <td>0.828624</td>
        </tr>
        <tr>
          <th>2</th>
          <td>f3</td>
          <td>&lt;ipython-input-2-61d7d6b6a99d&gt;:13</td>
          <td>1</td>
          <td>1</td>
          <td>0.000030</td>
          <td>0.000030</td>
          <td>0.711066</td>
          <td>0.711066</td>
        </tr>
        <tr>
          <th>3</th>
          <td>f1</td>
          <td>&lt;ipython-input-2-61d7d6b6a99d&gt;:6</td>
          <td>3</td>
          <td>3</td>
          <td>0.000113</td>
          <td>0.000113</td>
          <td>0.621887</td>
          <td>0.621887</td>
        </tr>
        <tr>
          <th>4</th>
          <td>f0</td>
          <td>&lt;ipython-input-2-61d7d6b6a99d&gt;:3</td>
          <td>1</td>
          <td>1</td>
          <td>0.000014</td>
          <td>0.000014</td>
          <td>0.206864</td>
          <td>0.206864</td>
        </tr>
        <tr>
          <th>5</th>
          <td>f2</td>
          <td>&lt;ipython-input-2-61d7d6b6a99d&gt;:9</td>
          <td>1</td>
          <td>1</td>
          <td>0.000022</td>
          <td>0.000022</td>
          <td>0.117736</td>
          <td>0.117736</td>
        </tr>
        <tr>
          <th>6</th>
          <td>&lt;method 'disable' of '_lsprof.Profiler' objects&gt;</td>
          <td>~:0</td>
          <td>1</td>
          <td>1</td>
          <td>0.000004</td>
          <td>0.000004</td>
          <td>0.000004</td>
          <td>0.000004</td>
        </tr>
      </tbody>
    </table>
    </div>



Renders a profile as indented text
----------------------------------

.. code:: ipython3

    from pyquickhelper.pycode.profiling import profile2graph
    
    gr = profile2graph(stat)
    print(gr[0].to_text(fct_width=20))


.. parsed-literal::
    <built-in...e.sleep> --  4  4 -- 0.82862 0.82862 -- ~:0:<built-in method time.sleep> (<built-in method time.sleep>)
    f1                   --  3  3 -- 0.00011 0.62189 -- <ipython-input-2-61d7d6b6a99d>:6 (f1)
        <built-...sleep> --  3  3 -- 0.62177 0.62177 -- ~:0:<built-in method time.sleep> (<built-in method time.sleep>) +++
    f4                   --  1  1 -- 0.00003 0.82883 -- <ipython-input-2-61d7d6b6a99d>:17 (f4)
        f2               --  1  1 -- 0.00002 0.11774 -- <ipython-input-2-61d7d6b6a99d>:9 (f2)
            f1           --  2  2 -- 0.00007 0.11771 -- <ipython-input-2-61d7d6b6a99d>:6 (f1) +++
        f3               --  1  1 -- 0.00003 0.71107 -- <ipython-input-2-61d7d6b6a99d>:13 (f3)
            f0           --  1  1 -- 0.00001 0.20686 -- <ipython-input-2-61d7d6b6a99d>:3 (f0)
                <bu...p> --  1  1 -- 0.20685 0.20685 -- ~:0:<built-in method time.sleep> (<built-in method time.sleep>) +++
            f1           --  1  1 -- 0.00004 0.50417 -- <ipython-input-2-61d7d6b6a99d>:6 (f1) +++


Renders a profile as json
-------------------------

.. code:: ipython3

    from pyquickhelper.pycode.profiling import profile2graph
    
    gr = profile2graph(stat)
    js = gr[0].to_json(indent=4)
    print(js)


.. parsed-literal::
    {
        "profile": {
            "4-0.82862:::<built-in method time.sleep>": {
                "details": {
                    "fct": "<built-in method time.sleep>",
                    "where": "~:0:<built-in method time.sleep>",
                    "nc1": 4,
                    "nc2": 4,
                    "tin": 0.8286236,
                    "tall": 0.8286236,
                    "indent": 0,
                    "ncalls": 0
                }
            },
            "3-0.62189:::f1": {
                "details": {
                    "fct": "f1",
                    "where": "<ipython-input-2-61d7d6b6a99d>:6",
                    "nc1": 3,
                    "nc2": 3,
                    "tin": 0.00011260000000000001,
                    "tall": 0.6218869,
                    "indent": 0,
                    "ncalls": 1
                },
                "calls": {
                    "3-0.62177:<built-in method time.sleep>": {
                        "details": {
                            "fct": "<built-in method time.sleep>",
                            "where": "~:0:<built-in method time.sleep>",
                            "nc1": 3,
                            "nc2": 3,
                            "tin": 0.6217743,
                            "tall": 0.6217743,
                            "indent": 0,
                            "ncalls": 1
                        }
                    }
                }
            },
            "1-0.82883:::f4": {
                "details": {
                    "fct": "f4",
                    "where": "<ipython-input-2-61d7d6b6a99d>:17",
                    "nc1": 1,
                    "nc2": 1,
                    "tin": 2.6300000000000002e-05,
                    "tall": 0.8288287000000001,
                    "indent": 0,
                    "ncalls": 2
                },
                "calls": {
                    "1-0.11774:f2": {
                        "details": {
                            "fct": "f2",
                            "where": "<ipython-input-2-61d7d6b6a99d>:9",
                            "nc1": 1,
                            "nc2": 1,
                            "tin": 2.19e-05,
                            "tall": 0.1177363,
                            "indent": 1,
                            "ncalls": 1
                        },
                        "calls": {
                            "2-0.11771:f1": {
                                "details": {
                                    "fct": "f1",
                                    "where": "<ipython-input-2-61d7d6b6a99d>:6",
                                    "nc1": 2,
                                    "nc2": 2,
                                    "tin": 7.25e-05,
                                    "tall": 0.11771440000000001,
                                    "indent": 1,
                                    "ncalls": 1
                                }
                            }
                        }
                    },
                    "1-0.71107:f3": {
                        "details": {
                            "fct": "f3",
                            "where": "<ipython-input-2-61d7d6b6a99d>:13",
                            "nc1": 1,
                            "nc2": 1,
                            "tin": 3.0100000000000003e-05,
                            "tall": 0.7110661,
                            "indent": 1,
                            "ncalls": 2
                        },
                        "calls": {
                            "1-0.20686:f0": {
                                "details": {
                                    "fct": "f0",
                                    "where": "<ipython-input-2-61d7d6b6a99d>:3",
                                    "nc1": 1,
                                    "nc2": 1,
                                    "tin": 1.4200000000000001e-05,
                                    "tall": 0.2068635,
                                    "indent": 2,
                                    "ncalls": 1
                                },
                                "calls": {
                                    "1-0.20685:<built-in method time.sleep>": {
                                        "details": {
                                            "fct": "<built-in method time.sleep>",
                                            "where": "~:0:<built-in method time.sleep>",
                                            "nc1": 1,
                                            "nc2": 1,
                                            "tin": 0.2068493,
                                            "tall": 0.2068493,
                                            "indent": 2,
                                            "ncalls": 1
                                        }
                                    }
                                }
                            },
                            "1-0.50417:f1": {
                                "details": {
                                    "fct": "f1",
                                    "where": "<ipython-input-2-61d7d6b6a99d>:6",
                                    "nc1": 1,
                                    "nc2": 1,
                                    "tin": 4.0100000000000006e-05,
                                    "tall": 0.5041725,
                                    "indent": 1,
                                    "ncalls": 2
                                }
                            }
                        }
                    }
                }
            }
        }
    }


Or in a better way:

.. code:: ipython3

    import json
    from pstats import SortKey
    from jyquickhelper import JSONJS
    
    JSONJS(gr[0].to_json(as_str=False, sort_key=SortKey.CUMULATIVE))






.. raw:: html

    <div id="0d90ae3c-78c8-46c3-bfdd-567e793ba61e" style="height: 100%; width:100%;"></div>
    <script>

            require(["https://rawgit.com/caldwell/renderjson/master/renderjson.js"], function() {
            document.getElementById('0d90ae3c-78c8-46c3-bfdd-567e793ba61e').appendChild(renderjson({"profile": {"1-0.82883:::f4": {"details": {"fct": "f4", "where": "<ipython-input-2-61d7d6b6a99d>:17", "nc1": 1, "nc2": 1, "tin": 2.6300000000000002e-05, "tall": 0.8288287000000001, "indent": 0, "ncalls": 2}, "calls": {"1-0.71107:f3": {"details": {"fct": "f3", "where": "<ipython-input-2-61d7d6b6a99d>:13", "nc1": 1, "nc2": 1, "tin": 3.0100000000000003e-05, "tall": 0.7110661, "indent": 1, "ncalls": 2}, "calls": {"1-0.50417:f1": {"details": {"fct": "f1", "where": "<ipython-input-2-61d7d6b6a99d>:6", "nc1": 1, "nc2": 1, "tin": 4.0100000000000006e-05, "tall": 0.5041725, "indent": 1, "ncalls": 2}}, "1-0.20686:f0": {"details": {"fct": "f0", "where": "<ipython-input-2-61d7d6b6a99d>:3", "nc1": 1, "nc2": 1, "tin": 1.4200000000000001e-05, "tall": 0.2068635, "indent": 2, "ncalls": 1}, "calls": {"1-0.20685:<built-in method time.sleep>": {"details": {"fct": "<built-in method time.sleep>", "where": "~:0:<built-in method time.sleep>", "nc1": 1, "nc2": 1, "tin": 0.2068493, "tall": 0.2068493, "indent": 2, "ncalls": 1}}}}}}, "1-0.11774:f2": {"details": {"fct": "f2", "where": "<ipython-input-2-61d7d6b6a99d>:9", "nc1": 1, "nc2": 1, "tin": 2.19e-05, "tall": 0.1177363, "indent": 1, "ncalls": 1}, "calls": {"2-0.11771:f1": {"details": {"fct": "f1", "where": "<ipython-input-2-61d7d6b6a99d>:6", "nc1": 2, "nc2": 2, "tin": 7.25e-05, "tall": 0.11771440000000001, "indent": 1, "ncalls": 1}}}}}}, "4-0.82862:::<built-in method time.sleep>": {"details": {"fct": "<built-in method time.sleep>", "where": "~:0:<built-in method time.sleep>", "nc1": 4, "nc2": 4, "tin": 0.8286236, "tall": 0.8286236, "indent": 0, "ncalls": 0}}, "3-0.62189:::f1": {"details": {"fct": "f1", "where": "<ipython-input-2-61d7d6b6a99d>:6", "nc1": 3, "nc2": 3, "tin": 0.00011260000000000001, "tall": 0.6218869, "indent": 0, "ncalls": 1}, "calls": {"3-0.62177:<built-in method time.sleep>": {"details": {"fct": "<built-in method time.sleep>", "where": "~:0:<built-in method time.sleep>", "nc1": 3, "nc2": 3, "tin": 0.6217743, "tall": 0.6217743, "indent": 0, "ncalls": 1}}}}}}))
            }); 
    </script>