module helpgen.process_notebooks

Short summary

module pyquickhelper.helpgen.process_notebooks

Contains the main function to generate the documentation for a module designed the same way as this one, generate_help_sphinx().

source on GitHub

Functions

function

truncated documentation

_preprocess_notebook

Preprocesses the content of a notebook.

_process_notebooks_in

The notebook conversion does not handle images from url for pdf and docx. They could be downloaded first …

_process_notebooks_in_private

This function fails in nbconvert 6.0 when the conversion is called more than once. The conversion probably changes the …

_process_notebooks_in_private_cmd

add_link_to_notebook

Adds a link to the notebook in HTML format and does a little bit of cleaning for various format.

add_tag_for_slideshow

Modifies a notebook to add tag for a slideshow.

build_all_notebooks_coverage

Creates a rst page (gallery) with links to all notebooks and information about coverage. It relies on function …

build_notebooks_gallery

Creates a rst page (gallery) with links to all notebooks. For each notebook, it creates a snippet.

build_thumbail_in_gallery

Returns rst code for a notebook.

find_pdflatex

Returns the executable for latex.

process_notebooks

Converts notebooks into html, rst, latex, pdf, python, docx using …

Documentation

Contains the main function to generate the documentation for a module designed the same way as this one, generate_help_sphinx.

source on GitHub

pyquickhelper.helpgen.process_notebooks._preprocess_notebook(notebook_content)[source]

Preprocesses the content of a notebook.

Parameters

notebook_content – notebook content

Returns

modified content

source on GitHub

pyquickhelper.helpgen.process_notebooks._process_notebooks_in(notebooks, outfold, build, latex_path=None, pandoc_path=None, formats=('ipynb', 'html', 'python', 'rst', 'slides', 'pdf', 'github'), fLOG=<function fLOG>, exc=True, nblinks=None, remove_unicode_latex=False, notebook_replacements=None)[source]

The notebook conversion does not handle images from url for pdf and docx. They could be downloaded first and replaced by local files.

Note

nbconvert introduced a commit which breaks the conversion of notebooks in latex if they have a cell outputting svg (see PR 910).

Use xelatex if possible.

source on GitHub

pyquickhelper.helpgen.process_notebooks._process_notebooks_in_private(fnbcexe, list_args, options_args)[source]

This function fails in nbconvert 6.0 when the conversion is called more than once. The conversion probably changes the initial state.

source on GitHub

pyquickhelper.helpgen.process_notebooks._process_notebooks_in_private_cmd(fnbcexe, list_args, options_args, fLOG)[source]

Adds a link to the notebook in HTML format and does a little bit of cleaning for various format.

Parameters
  • file – notebook.html

  • nb – notebook (.ipynb)

  • pdf – if True, add a link to the PDF, assuming it will exists at the same location

  • html – if True, add a link to the HTML conversion

  • python – if True, add a link to the Python conversion

  • slides – if True, add a link to the HTML slides

  • exc – raises an exception (True) or a warning (False)

  • github – add a link to the notebook on github

  • notebook – location of the notebook (file might be a copy)

  • nblinks – dictionary {ref: url}

  • notebook_replacements – stirng replacement in notebooks

  • fLOG – logging function

Returns

list of generated files

The function does some cleaning too in the files.

source on GitHub

pyquickhelper.helpgen.process_notebooks.add_tag_for_slideshow(ipy, folder, encoding='utf8')[source]

Modifies a notebook to add tag for a slideshow.

Parameters
  • ipy – notebook file

  • folder – where to write the new notebook

  • encoding – encoding

Returns

written file

source on GitHub

pyquickhelper.helpgen.process_notebooks.build_all_notebooks_coverage(nbs, fileout, module_name, dump=None, badge=True, too_old=30, fLOG=<function noLOG>)[source]

Creates a rst page (gallery) with links to all notebooks and information about coverage. It relies on function notebook_coverage.

Parameters
  • nbs – list of notebooks to consider or tuple(full path, rst),

  • fileout – file to create

  • module_name – module name

  • dump – dump containing information about notebook execution (or None for the default one)

  • badge – builds an image with the notebook coverage

  • too_old – drop executions older than too_old days from now

  • fLOG – logging function

Returns

dataframe which contains the data

source on GitHub

Creates a rst page (gallery) with links to all notebooks. For each notebook, it creates a snippet.

Parameters
  • nbs – list of notebooks to consider or tuple(full path, rst),

  • fileout – file to create

  • layout'classic' or 'table'

  • neg_pattern – do not consider notebooks matching this regular expression

  • snippet_folder – folder where to find custom snippet for notebooks, the snippet should have the same name as the notebook itself, snippet must have extension .png

  • fLOG – logging function

Returns

created file name

Example for parameter nbs:

('challenges\city_tour\city_tour_1.ipynb',
    'ensae_projects\_doc\notebooks\challenges\city_tour\city_tour_1.ipynb')
('challenges\city_tour\city_tour_1_solution.ipynb',
    'ensae_projects\_doc\notebooks\challenges\city_tour\city_tour_1_solution.ipynb')
('challenges\city_tour\city_tour_data_preparation.ipynb',
    'ensae_projects\_doc\notebooks\challenges\city_tour\city_tour_data_preparation.ipynb')
('challenges\city_tour\city_tour_long.ipynb',
    'ensae_projects\_doc\notebooks\challenges\city_tour\city_tour_long.ipynb')
('cheat_sheets\chsh_files.ipynb',
    'ensae_projects\_doc\notebooks\cheat_sheets\chsh_files.ipynb')
('cheat_sheets\chsh_geo.ipynb',
    'ensae_projects\_doc\notebooks\cheat_sheets\chsh_geo.ipynb')

nbs can be a folder, in that case, the function will build the list of all notebooks in that folder. nbs can be a list of tuple. the function adds a thumbnail, organizes the list of notebook as a galley, it adds a link on notebook coverage. Parameters layout, neg_pattern were added.

Changed in version 1.7: Modifies the function to bypass the generation of a snippet if a custom one was found. Parameter snippet_folder was added.

source on GitHub

Returns rst code for a notebook.

Parameters
  • nbfile – notebook file

  • folder_snippet – where to store the snippet

  • relative – the path to the snippet will be relative to this folder

  • rst_link – rst link

  • layout'classic' or 'table'

  • snippet_folder – folder where to find custom snippet for notebooks, the snippet should have the same name as the notebook itself, snippet must have extension .png

Returns

RST

Modifies the function to bypass the generation of a snippet if a custom one was found. Parameter snippet_folder was added.

source on GitHub

pyquickhelper.helpgen.process_notebooks.find_pdflatex(latex_path)[source]

Returns the executable for latex.

Parameters

latex_path – path to look (only on Windows)

Returns

executable

New in version 1.7.

source on GitHub

pyquickhelper.helpgen.process_notebooks.process_notebooks(notebooks, outfold, build, latex_path=None, pandoc_path=None, formats='ipynb, html, python, rst, slides, pdf, github', fLOG=<function fLOG>, exc=True, remove_unicode_latex=False, nblinks=None, notebook_replacements=None)[source]

Converts notebooks into html, rst, latex, pdf, python, docx using nbconvert.

Parameters
  • notebooks – list of notebooks or comma separated values

  • outfold – folder which will contains the outputs

  • build – temporary folder which contains all produced files

  • pandoc_path – path to pandoc

  • formats – list of formats to convert into (pdf format means latex then compilation), or comma separated values

  • latex_path – path to the latex compiler

  • fLOG – logging function

  • exc – raises an exception (True) or a warning (False) if an error happens

  • nblinks – dictionary {ref: url} or a string in json format

  • remove_unicode_latex – remove unicode characters for latex (to avoid failing)

  • notebook_replacements – string replacement in a notebook before conversion or a string in json format

Returns

list of tuple [(file, created or skipped)]

This function relies on pandoc. It also needs modules pywin32, pygments.

pywin32 might have some issues to find its DLL, look import_pywin32.

The latex compilation uses MiKTeX. The conversion into Word document directly uses pandoc. It still has an issue with table.

Some latex templates (for nbconvert) uses [commandchars=\\\{\}]{\|} which allows commands \\ and it does not compile. The one used here is report. Some others bugs can be found at: schlichtanders/latex_test.html. For example, you must not let spaces between symbol $ and the formulas it indicates.

If pandoc_path is None, uses find_pandoc_path to guess it. If latex_path is None, uses find_latex_path to guess it.

Convert a notebook into multiple formats

from pyquickhelper.ipythonhelper import process_notebooks
process_notebooks("td1a_correction_session7.ipynb",
                  "dest_folder", "dest_folder",
                  formats=("ipynb", "html", "python", "rst", "slides", "pdf",
                           "docx", "github")])

For latex and pdf, a custom processor was added to handle raw data and add \begin{verbatim} and \end{verbatim}. Format github adds a link to file on github.

Todo(bug): check differences between _process_notebooks_in_private and _process_notebooks_in_private_cmd

For latex and pdf, the custom preprocessor is not taken into account. by function _process_notebooks_in_private.

source on GitHub