module `helpgen.utils_sphinx_doc`¶

Short summary¶

module pyquickhelper.helpgen.utils_sphinx_doc

various helpers to produce a Sphinx documentation

Classes¶

class	truncated documentation
`useless_class_UnicodeStringIOThreadSafe`	avoid conversion problem between str and char, class protected again Thread issue

Functions¶

function	truncated documentation
`_private_migrating_doxygen_doc`
`_private_process_one_file`	Copies one file from the source to the documentation folder. It processes some comments in doxygen format (@ param, …
`add_file_rst`	Creates a rst file for every source file.
`apply_modification_template`	See `add_file_rst()`.
`copy_source_files`	Copies all sources files (input) into a folder (output), apply on each of them a modification.
`doc_checking`	Example of a doc string.
`filecontent_to_rst`	Produces a .rst file which contains the file. It adds a title and a label based on the filename (no folder included). …
`fix_incomplete_references`	Looks into every file .rst or .py for incomplete reference. Example
`migrating_doxygen_doc`	Migrates the doxygen documentation to rst format.
`prepare_file_for_sphinx_help_generation`	Prepares all files for Sphinx generation.
`process_copy_images`	Looks into every file .rst or .py for images (
`produces_indexes`	Produces a file for each category of object found in the module.
`remove_undesired_part_for_documentation`	Some files contains blocs inserted between the two lines:
`replace_relative_import_fct`	Takes a python file and replaces all relative imports it was able to find by an import which can be processed …
`validate_file_for_help`	Accepts or rejects a file to be copied in the help folder.

Methods¶

method	truncated documentation
`__init__`	creates a lock

Documentation¶

various helpers to produce a Sphinx documentation

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc._private_migrating_doxygen_doc(rows, index_first_line, filename, debug=False, silent=False)[source]¶

pyquickhelper.helpgen.utils_sphinx_doc._private_process_one_file(fullname, to, silent, fmod, replace_relative_import, use_sys)[source]¶

Copies one file from the source to the documentation folder. It processes some comments in doxygen format (@ param, @ return). It replaces relatives imports by a regular import.

Parameters:

fullname – name of the file
to – location (folder)
silent – no logs if True
fmod – modification functions
replace_relative_import – replace relative import
use_sys – remove_undesired_part_for_documentation

Returns:

extension, number of lines, number of lines in documentation

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.add_file_rst(rootm, store_obj, actions, template='\n__FULLNAME_UNDERLINED__\n\n\n\n\n.. inheritance-diagram:: __FULLNAMENOEXT__\n\n\nShort summary\n+++++++++++++\n\n__DOCUMENTATION__\n\n\n__CLASSES__\n\n__FUNCTIONS__\n\n__PROPERTIES__\n\n__STATICMETHODS__\n\n__METHODS__\n\nDocumentation\n+++++++++++++\n\n.. automodule:: __FULLNAMENOEXT__\n :members:\n :special-members: __init__\n :show-inheritance:\n\n__ADDEDMEMBERS__\n\n', rootrep=('_doc.sphinxdoc.source.pyquickhelper.', ''), fmod=<function <lambda>>, softfile=<function <lambda>>, mapped_function=None, indexes=None, additional_sys_path=None, fLOG=<function noLOG>)[source]¶

Creates a rst file for every source file.

Parameters:

rootm – root of the module (for relative import)
store_obj – to keep table of all objects
actions – output from copy_source_files
template – rst template to produce
rootrep – file name in the documentation contains some folders which are not desired in the documentation
fmod – applies modification to the instanciated template
softfile – softfile is a function (f : filename –> True or False), when it is True, the documentation is lighter (no special members)
mapped_function – list of 2-tuple (pattern, function). Every file matching the pattern will be copied to the documentation folder, its content will be sent to the function and will produce a file <filename>.rst. Example: [ (".*[.]sql$", filecontent_to_rst) ] The function takes two parameters: full_filename, content_filename. It returns a string (the rst file) or a tuple (rst file, short description). By default (if function is None), the function filecontent_to_rst will be called except for .rst file for which nothing is done.
indexes – to index some information { dictionary label:IndexInformation (…) }, the function populates it
additional_sys_path – additional path to include to sys.path before importing a module (will be removed afterwards)
fLOG – logging function

Returns:

list of written files stored in RstFileHelp

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.apply_modification_template(rootm, store_obj, template, fullname, rootrep, softfile, indexes, additional_sys_path, fLOG=<function noLOG>)[source]¶

See add_file_rst.

Parameters:

rootm – root of the module
store_obj – keep track of all objects extracted from the module
template – rst template to produce
fullname – full name of the file
rootrep – file name in the documentation contains some folders which are not desired in the documentation
softfile – a function (f : filename –> True or False), when it is True, the documentation is lighter (no special members)
indexes – dictionary with the label and some information (IndexInformation)
additional_sys_path – additional path to include to sys.path before importing a module (will be removed afterwards)
fLOG – logging function

Returns:

content of a .rst file

Why doesn’t the documentation show compiled submodules?

The instruction .. automodule:: <name> only shows objects obj which verify obj.__module__ == name. This is always the case for modules written in Python but not necessarily for module compiled from C language. When the module is declared, the following structure contains the module name in second position. This name must not be the submodule shortname but the name the module has is the package. The C file pyquickhelper/helpgen/compiled.c implements submodule pyquickhelper.helpgen.compiled, this value must replace <fullname> in the structure below, not simply compiled.

static struct PyModuleDef moduledef = {
        PyModuleDef_HEAD_INIT,
        "<fullname>",
        "Helper for parallelization with threads with C++.",
        sizeof(struct module_state),
        fonctions,
        NULL,
        threader_module_traverse,
        threader_module_clear,
        NULL
};

Warning

This function still needs some improvments for C++ modules on MacOSX.

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.copy_source_files(input, output, fmod=<function <lambda>>, silent=False, filter=None, remove=True, softfile=<function <lambda>>, fexclude=<function <lambda>>, addfilter=None, replace_relative_import=False, copy_add_ext=None, use_sys=None, fLOG=<function fLOG>)[source]¶

Copies all sources files (input) into a folder (output), apply on each of them a modification.

Parameters:

input – input folder
output – output folder (it will be cleaned each time)
fmod – modifies the content of each file, this function takes a string and returns a string
silent – if True, do not stop when facing an issue with doxygen documentation
filter – if None, process only file related to python code, otherwise, use this filter to select file (regular expression). If this parameter is None or is empty, the default value is something like: "(.+[.]py$)|(.+[.]pyd$)|(.+[.]cpp$)|(.+[.]h$)|(.+[.]dll$))".
remove – if True, remove every files in the output folder first
softfile – softfile is a function (f : filename –> True or False), when it is True, the documentation is lighter (no special members)
fexclude – function to exclude some files from the help
addfilter – additional filter, it should look like: "(.+[.]pyx$)|(.+[.]pyh$)"
replace_relative_import – replace relative import
copy_add_ext – additional extension file to copy
use_sys – see remove_undesired_part_for_documentation
fLOG – logging function

Returns:

list of copied files

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.doc_checking()[source]¶

Example of a doc string.

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.filecontent_to_rst(filename, content)[source]¶

Produces a .rst file which contains the file. It adds a title and a label based on the filename (no folder included).

Parameters:

filename – filename
content – content

Returns:

new content

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.fix_incomplete_references(folder_source, store_obj, issues=None, fLOG=<function fLOG>)[source]¶

Looks into every file .rst or .py for incomplete reference. Example:

:class:`name`  --> :class:`name <...>`.

Parameters:

folder_source – folder where to look for sources
store_obj – container for indexed objects
issues – if not None (a list), it will add issues (function, message)
fLOG – logging function

Returns:

list of fixed references

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.migrating_doxygen_doc(content, filename, silent=False, log=False, debug=False)[source]¶

Migrates the doxygen documentation to rst format.

Parameters:

content – file content
filename – filename (to display useful error messages)
silent – if silent, do not raise an exception
log – if True, write some information in the logs (not only exceptions)
debug – display more information on the output if True

Returns:

statistics, new content file

Function private_migrating_doxygen_doc enumerates the list of conversion which will be done.

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.prepare_file_for_sphinx_help_generation(store_obj, input, output, subfolders, fmod_copy=<function <lambda>>, template='\n__FULLNAME_UNDERLINED__\n\n\n\n\n.. inheritance-diagram:: __FULLNAMENOEXT__\n\n\nShort summary\n+++++++++++++\n\n__DOCUMENTATION__\n\n\n__CLASSES__\n\n__FUNCTIONS__\n\n__PROPERTIES__\n\n__STATICMETHODS__\n\n__METHODS__\n\nDocumentation\n+++++++++++++\n\n.. automodule:: __FULLNAMENOEXT__\n :members:\n :special-members: __init__\n :show-inheritance:\n\n__ADDEDMEMBERS__\n\n', rootrep=('_doc.sphinxdoc.source.project_name.', ''), fmod_res=<function <lambda>>, silent=False, optional_dirs=None, softfile=<function <lambda>>, fexclude=<function <lambda>>, mapped_function=None, fexclude_index=<function <lambda>>, issues=None, additional_sys_path=None, replace_relative_import=False, module_name=None, copy_add_ext=None, use_sys=None, auto_rst_generation=True, fLOG=<function fLOG>)[source]¶

Prepares all files for Sphinx generation.

Parameters:

store_obj – to keep track of all objects, it should be a dictionary
input – input folder
output – output folder (it will be cleaned each time)
subfolders – list of subfolders to copy from input to output, two cases: * a string input/<sub> –> output/<sub> * a tuple input/<sub[0]> –> output/<sub[1]>
fmod_copy – modifies the content of each file, this function takes a string and the filename and returns a string f(content, filename) --> string
template – rst template to produce
rootrep – file name in the documentation contains some folders which are not desired in the documentation
fmod_res – applies modification to the instanciated template
silent – if True, do not stop when facing an issue with doxygen migration
optional_dirs – list of tuple with a list of folders (source, copy, filter) to copy for the documentation, example: ( <folder_help>, "coverage", ".*" )
softfile – softfile is a function (f : filename –> True or False), when it is True, the documentation is lighter (no special members)
fexclude – function to exclude some files from the help
fexclude_index – function to exclude some files from the indices
mapped_function – list of 2-tuple (pattern, function). Every file matching the pattern will be copied to the documentation folder, its content will be sent to the function and will produce a file <filename>.rst. Example: [ (".*[.]sql$", filecontent_to_rst) ] The function takes two parameters: full_filename, content_filename. It returns a string (the rst file) or a tuple (rst file, short description). By default (if function is None), the function filecontent_to_rst will be called.
issues – if not None (a list), the function will store some issues here.
additional_sys_path – additional paths to includes to sys.path when import a module (will be removed afterwards)
replace_relative_import – replace relative import
module_name – module name (cannot be None)
copy_add_ext – additional file extension to copy
use_sys – remove_undesired_part_for_documentation
auto_rst_generation – add a file .rst for each source file
fLOG – logging function

Returns:

list of written files stored in RstFileHelp

Example:

prepare_file_for_sphinx_help_generation (
            {},
            ".",
            "_doc/sphinxdoc/source/",
            subfolders      = [
                                ("src/" + project_var_name,
                                 project_var_name),
                                 ],
            silent          = True,
            rootrep         = ("_doc.sphinxdoc.source.%s." %
                               (project_var_name,), ""),
            optional_dirs   = optional_dirs,
            mapped_function = [ (".*[.]tohelp$", None) ] )

It produces a file with the number of lines and files per extension.

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.process_copy_images(folder_source, folder_images)[source]¶

Looks into every file .rst or .py for images (.. image:: imagename), if this image was found in directory folder_images, then the image is copied closes to the file.

Parameters:

folder_source – folder where to look for sources
folder_images – folder where to look for images

Returns:

list of copied images

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.produces_indexes(store_obj, indexes, fexclude_index, titles=None, correspondances=None, fLOG=<function fLOG>)[source]¶

Produces a file for each category of object found in the module.

Parameters:

store_obj – list of collected object, it is a dictionary key : ModuleMemberDoc or key : [ list of ModuleMemberDoc ]
indexes – list of things to index, dictionary { label : IndexInformation }
fexclude_index – to exclude files from the indices
titles – each type is mapped to a title to add to the rst file
correspondances – each type is mapped to a label to add to the rst file
fLOG – logging function

Returns:

dictionary: { type : rst content of the index }

Default values if titles of correspondances is None:

title = {"method": "Methods",
         "staticmethod": "Static Methods",
         "property": "Properties",
         "function": "Functions",
         "class": "Classes",
         "module": "Modules"}

correspondances = {"method": "l-methods",
                   "function": "l-functions",
                   "staticmethod": "l-staticmethods",
                   "property": "l-properties",
                   "class": "l-classes",
                   "module": "l-modules"}

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.remove_undesired_part_for_documentation(content, filename, use_sys)[source]¶

Some files contains blocs inserted between the two lines:

# -- HELP BEGIN EXCLUDE --
# -- HELP END EXCLUDE --

Those lines will be commented out.

Parameters:

content – file content
filename – for error message
use_sys – string or None, enables, disables a section based on variables added to sys module

Returns:

modified file content

If the parameter use_sys is false, the section of code will be commented out. If true, the section can be enabled. It relies on the following code:

import sys
if hasattr(sys, "<use_sys>") and sys.<use_sys>:

    # section to enable or disables

The string <use_sys> will be replaced by the value of parameter use_sys.

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.replace_relative_import_fct(fullname, content=None)[source]¶

Takes a python file and replaces all relative imports it was able to find by an import which can be processed by Python if the file were the main file.

Parameters:

fullname – name of the file
content – a preprocessed content of the file of the content if it is None

Returns:

content of the file without relative imports

Does not change imports in comments.

source on GitHub

class pyquickhelper.helpgen.utils_sphinx_doc.useless_class_UnicodeStringIOThreadSafe[source]¶

Bases: str

avoid conversion problem between str and char, class protected again Thread issue

source on GitHub

creates a lock

source on GitHub

__init__()[source]¶

creates a lock

source on GitHub

pyquickhelper.helpgen.utils_sphinx_doc.validate_file_for_help(filename, fexclude=<function <lambda>>)[source]¶

Accepts or rejects a file to be copied in the help folder.

Parameters:

filename – filename
fexclude – function to exclude some files

Returns:

boolean

source on GitHub

module `helpgen.utils_sphinx_doc`¶

Short summary¶

Classes¶

Functions¶

Methods¶

Documentation¶

Links

Contents

Information

Related Topics

This Page

module helpgen.utils_sphinx_doc¶

Short summary¶

Classes¶

Functions¶

Methods¶

Documentation¶

module `helpgen.utils_sphinx_doc`¶