module helpgen.process_notebooks
¶
Short summary¶
module pyquickhelper.helpgen.process_notebooks
Contains the main function to generate the documentation
for a module designed the same way as this one, generate_help_sphinx()
.
Functions¶
function |
truncated documentation |
---|---|
Preprocesses the content of a notebook. |
|
The notebook conversion does not handle images from url for pdf and docx. They could be downloaded first … |
|
This function fails in nbconvert 6.0 when the conversion is called more than once. The conversion probably changes the … |
|
Adds a link to the notebook in HTML format and does a little bit of cleaning for various format. |
|
Modifies a notebook to add tag for a slideshow. |
|
Creates a rst page (gallery) with links to all notebooks and information about coverage. It relies on function … |
|
Creates a rst page (gallery) with links to all notebooks. For each notebook, it creates a snippet. |
|
Returns rst code for a notebook. |
|
Returns the executable for latex. |
|
Converts notebooks into html, rst, latex, pdf, python, docx using … |
Documentation¶
Contains the main function to generate the documentation
for a module designed the same way as this one, generate_help_sphinx
.
- pyquickhelper.helpgen.process_notebooks._preprocess_notebook(notebook_content)[source]¶
Preprocesses the content of a notebook.
- Parameters:
notebook_content – notebook content
- Returns:
modified content
- pyquickhelper.helpgen.process_notebooks._process_notebooks_in(notebooks, outfold, build, latex_path=None, pandoc_path=None, formats=('ipynb', 'html', 'python', 'rst', 'slides', 'pdf', 'github'), fLOG=<function fLOG>, exc=True, nblinks=None, remove_unicode_latex=False, notebook_replacements=None)[source]¶
The notebook conversion does not handle images from url for pdf and docx. They could be downloaded first and replaced by local files.
Note
nbconvert introduced a commit which breaks the conversion of notebooks in latex if they have a cell outputting svg (see PR 910).
Use xelatex if possible.
- pyquickhelper.helpgen.process_notebooks._process_notebooks_in_private(fnbcexe, list_args, options_args)[source]¶
This function fails in nbconvert 6.0 when the conversion is called more than once. The conversion probably changes the initial state.
- pyquickhelper.helpgen.process_notebooks._process_notebooks_in_private_cmd(fnbcexe, list_args, options_args, fLOG)[source]¶
- pyquickhelper.helpgen.process_notebooks.add_link_to_notebook(file, nb, pdf, html, python, slides, exc=True, github=False, notebook=None, nblinks=None, fLOG=None, notebook_replacements=None)[source]¶
Adds a link to the notebook in HTML format and does a little bit of cleaning for various format.
- Parameters:
file – notebook.html
nb – notebook (.ipynb)
pdf – if True, add a link to the PDF, assuming it will exists at the same location
html – if True, add a link to the HTML conversion
python – if True, add a link to the Python conversion
slides – if True, add a link to the HTML slides
exc – raises an exception (True) or a warning (False)
github – add a link to the notebook on github
notebook – location of the notebook (file might be a copy)
nblinks – dictionary
{ref: url}
notebook_replacements – stirng replacement in notebooks
fLOG – logging function
- Returns:
list of generated files
The function does some cleaning too in the files.
- pyquickhelper.helpgen.process_notebooks.add_tag_for_slideshow(ipy, folder, encoding='utf8')[source]¶
Modifies a notebook to add tag for a slideshow.
- Parameters:
ipy – notebook file
folder – where to write the new notebook
encoding – encoding
- Returns:
written file
- pyquickhelper.helpgen.process_notebooks.build_all_notebooks_coverage(nbs, fileout, module_name, dump=None, badge=True, too_old=30, fLOG=<function noLOG>)[source]¶
Creates a rst page (gallery) with links to all notebooks and information about coverage. It relies on function
notebook_coverage
.- Parameters:
nbs – list of notebooks to consider or tuple(full path, rst),
fileout – file to create
module_name – module name
dump – dump containing information about notebook execution (or None for the default one)
badge – builds an image with the notebook coverage
too_old – drop executions older than too_old days from now
fLOG – logging function
- Returns:
dataframe which contains the data
- pyquickhelper.helpgen.process_notebooks.build_notebooks_gallery(nbs, fileout, layout='classic', neg_pattern=None, snippet_folder=None, fLOG=<function noLOG>)[source]¶
Creates a rst page (gallery) with links to all notebooks. For each notebook, it creates a snippet.
- Parameters:
nbs – list of notebooks to consider or tuple(full path, rst),
fileout – file to create
layout –
'classic'
or'table'
neg_pattern – do not consider notebooks matching this regular expression
snippet_folder – folder where to find custom snippet for notebooks, the snippet should have the same name as the notebook itself, snippet must have extension
.png
fLOG – logging function
- Returns:
created file name
Example for parameter nbs:
('challenges\city_tour\city_tour_1.ipynb', 'ensae_projects\_doc\notebooks\challenges\city_tour\city_tour_1.ipynb') ('challenges\city_tour\city_tour_1_solution.ipynb', 'ensae_projects\_doc\notebooks\challenges\city_tour\city_tour_1_solution.ipynb') ('challenges\city_tour\city_tour_data_preparation.ipynb', 'ensae_projects\_doc\notebooks\challenges\city_tour\city_tour_data_preparation.ipynb') ('challenges\city_tour\city_tour_long.ipynb', 'ensae_projects\_doc\notebooks\challenges\city_tour\city_tour_long.ipynb') ('cheat_sheets\chsh_files.ipynb', 'ensae_projects\_doc\notebooks\cheat_sheets\chsh_files.ipynb') ('cheat_sheets\chsh_geo.ipynb', 'ensae_projects\_doc\notebooks\cheat_sheets\chsh_geo.ipynb')
nbs can be a folder, in that case, the function will build the list of all notebooks in that folder. nbs can be a list of tuple. the function adds a thumbnail, organizes the list of notebook as a galley, it adds a link on notebook coverage. The function bypasses the generation of a snippet if a custom one was found.
- pyquickhelper.helpgen.process_notebooks.build_thumbail_in_gallery(nbfile, folder_snippet, relative, rst_link, layout, snippet_folder=None, fLOG=None)[source]¶
Returns rst code for a notebook.
- Parameters:
nbfile – notebook file
folder_snippet – where to store the snippet
relative – the path to the snippet will be relative to this folder
rst_link – rst link
layout –
'classic'
or'table'
snippet_folder – folder where to find custom snippet for notebooks, the snippet should have the same name as the notebook itself, snippet must have extension
.png
- Returns:
RST
Modifies the function to bypass the generation of a snippet if a custom one was found. Parameter snippet_folder was added.
- pyquickhelper.helpgen.process_notebooks.find_pdflatex(latex_path)[source]¶
Returns the executable for latex.
- Parameters:
latex_path – path to look (only on Windows)
- Returns:
executable
- pyquickhelper.helpgen.process_notebooks.process_notebooks(notebooks, outfold, build, latex_path=None, pandoc_path=None, formats='ipynb, html, python, rst, slides, pdf, github', fLOG=<function fLOG>, exc=True, remove_unicode_latex=False, nblinks=None, notebook_replacements=None)[source]¶
Converts notebooks into html, rst, latex, pdf, python, docx using nbconvert.
- Parameters:
notebooks – list of notebooks or comma separated values
outfold – folder which will contains the outputs
build – temporary folder which contains all produced files
pandoc_path – path to pandoc
formats – list of formats to convert into (pdf format means latex then compilation), or comma separated values
latex_path – path to the latex compiler
fLOG – logging function
exc – raises an exception (True) or a warning (False) if an error happens
nblinks – dictionary
{ref: url}
or a string in json formatremove_unicode_latex – remove unicode characters for latex (to avoid failing)
notebook_replacements – string replacement in a notebook before conversion or a string in json format
- Returns:
list of tuple [(file, created or skipped)]
This function relies on pandoc. It also needs modules pywin32, pygments.
pywin32 might have some issues to find its DLL, look
import_pywin32
.The latex compilation uses MiKTeX. The conversion into Word document directly uses pandoc. It still has an issue with table.
Some latex templates (for nbconvert) uses
[commandchars=\\\{\}]{\|}
which allows commands\\
and it does not compile. The one used here isreport
. Some others bugs can be found at: schlichtanders/latex_test.html. For example, you must not let spaces between symbol$
and the formulas it indicates.If pandoc_path is None, uses
find_pandoc_path
to guess it. If latex_path is None, usesfind_latex_path
to guess it.Convert a notebook into multiple formats
from pyquickhelper.ipythonhelper import process_notebooks process_notebooks("td1a_correction_session7.ipynb", "dest_folder", "dest_folder", formats=("ipynb", "html", "python", "rst", "slides", "pdf", "docx", "github")])
For latex and pdf, a custom processor was added to handle raw data and add
\begin{verbatim}
and\end{verbatim}
. Format github adds a link to file on github.Todo(bug): check differences between _process_notebooks_in_private and _process_notebooks_in_private_cmd
For latex and pdf, the custom preprocessor is not taken into account. by function
_process_notebooks_in_private
.