Examples

  1. A python script which generates documentation

  2. Clone a git repository

  3. Convert a notebook into multiple formats

  4. Convert a notebook into slides

  5. Encrypted and compressed backup

  6. File autocompletion in IPython

  7. How to display a formula

  8. How to test a Sphinx directive?

  9. List files from FTP site

  10. Open a add a form in a notebook to ask parameters to a user

  11. Produce HTML documentation for a function or class

  12. Run a notebook end to end

  13. Run a program using the command line

  14. Run help generation

  15. Simple configuration file for Sphinx

  16. Transfer files to webste through FTP

  17. Transfer updated files to a website

  18. Visualize the difference between two text files or strings

  19. synchronize two folders

A python script which generates documentation

The following code prints the version of Python on the standard output. It is added to the documentation:

.. runpython::
    :showcode:

    import sys
    print("sys.version_info=", str(sys.version_info))

If give the following results:

    sys.version_info= sys.version_info(major=3, minor=7, micro=2, releaselevel='final', serial=0)

Options showcode can be used to display the code. The option rst will assume the output is in RST format and must be interpreted. showout will complement the RST output with the raw format.

(original entry : sphinx_runpython_extension.py:docstring of pyquickhelper.sphinxext.sphinx_runpython_extension.RunPythonDirective, line 4)

Clone a git repository

clone("local_folder", "github.com", "sdpython", "pyquickhelper")

(original entry : pygit_helper.py:docstring of pyquickhelper.loghelper.repositories.pygit_helper.clone, line 15)

Convert a notebook into multiple formats

from pyquickhelper.ipythonhelper import process_notebooks
process_notebooks("td1a_correction_session7.ipynb",
                "dest_folder", "dest_folder",
                formats=("ipynb", "html", "python", "rst", "slides", "pdf",
                         "docx", "present", "github")])

(original entry : process_notebooks.py:docstring of pyquickhelper.helpgen.process_notebooks.process_notebooks, line 44)

Convert a notebook into slides

By default, the function automatically adds sections if there is none and it copies the javascript from reveal.js at the right place.

from pyquickhelper.helpgen import nb2slides
nb2slides("nb.ipynb", "convert.slides.html")

(original entry : process_notebook_api.py:docstring of pyquickhelper.helpgen.process_notebook_api.nb2slides, line 12)

Encrypted and compressed backup

Here is an example which stores everything on hard drive. A second run only modifies files updated between the two processes. A modified file does not remove the previous version, it creates a new file. Example:

from pyquickhelper.loghelper import fLOG
from pyquickhelper.filehelper import FileTreeNode, EncryptedBackup
from pyensae.remote import TransferAPIFile

key_crypt = "crypt"

local = os.path.normpath(os.path.join(os.path.dirname(__file__), ".."))
this = os.path.normpath(os.path.dirname(__file__))
file_status=os.path.join(this, "backup_status.txt")
file_map=os.path.join(this, "backup_mapping.txt")

backup = True
if backup:
    # code to backup
    root = os.path.normpath(os.path.join(os.path.dirname(__file__)))
    api = TransferAPIFile("f:\\mycryptedbackup")
    ft = FileTreeNode(root, repository=True)
    enc = EncryptedBackup(
        key=key_crypt,
        file_tree_node=ft,
        transfer_api=api,
        root_local=local,
        file_status=file_status,
        file_map=file_map,
        fLOG=print)

    enc.start_transfering()

restore = not backup
if restore:
    # code to restore
    root = os.path.normpath(os.path.join(os.path.dirname(__file__)))
    api = TransferAPIFile("f:\\mycryptedbackup")
    enc = EncryptedBackup(
        key=key_crypt,
        file_tree_node=None,
        transfer_api=api,
        root_local=local,
        file_status=file_status,
        file_map=file_map,
        fLOG=print)

    dest=os.path.join(this, "_temp")
    enc.retrieve_all(dest)

(original entry : encrypted_backup.py:docstring of pyquickhelper.filehelper.encrypted_backup.EncryptedBackup, line 6)

File autocompletion in IPython

The following code:

from pyquickhelper.ipythonhelper import AutoCompletionFile
d = AutoCompletionFile(".")

Will produce the following auto completion picture:

_images/completion.png

(original entry : kindofcompletion.py:docstring of pyquickhelper.ipythonhelper.kindofcompletion.AutoCompletionFile, line 4)

How to display a formula

We want to check this formula to successfully converted.

\left \{ \begin{array}{l} \min_{x,y} \left \{ x^2 + y^2 - xy + y \right \}
\\ \text{sous contrainte} \; x + 2y = 1 \end{array}\right .

Brackets and backslashes might be an issue.

(original entry : utils_sphinx_doc_helpers.py:docstring of pyquickhelper.helpgen.utils_sphinx_doc_helpers.example_function_latex, line 4)

How to test a Sphinx directive?

The following code defines a simple directive definedbased on an existing one. It also defined what to do if a new node is inserted in the documentation.

from docutils import nodes
from pyquickhelper.helpgen import rst2html

class runpythonthis_node(nodes.Structural, nodes.Element):
    pass

class RunPythonThisDirective (RunPythonDirective):
    runpython_class = runpythonthis_node

def visit_node(self, node):
    self.body.append("<p><b>visit_node</b></p>")
def depart_node(self, node):
    self.body.append("<p><b>depart_node</b></p>")

content = '''
            test a directive
            ================

            .. runpythonthis::

                print("this code shoud appear" + "___")
            '''.replace("                    ", "")
            # to remove spaces at the beginning of the line

tives = [ ("runpythonthis", RunPythonThisDirective,
           runpythonthis_node, visit_node, depart_node) ]

html = rst2html(content, writer="html", keep_warnings=True,
                directives=tives)

Unfortunately, this functionality is only tested on Python 3. It might not work on Python 2.7. The function produces files if the document contains latex converted into image.

(original entry : rst_converters.py:docstring of pyquickhelper.helpgen.rst_converters.rst2html, line 61)

List files from FTP site

from pyquickhelper.filehelper import TransferFTP
ftp = TransferFTP("ftp....", "login", "password")
res = ftp.ls("path")
for v in res:
    print(v["name"])
ftp.close()

(original entry : ftp_transfer.py:docstring of pyquickhelper.filehelper.ftp_transfer.TransferFTP.ls, line 8)

Open a add a form in a notebook to ask parameters to a user

_images/form.png

Cell 1:

from pyquickhelper.ipythonhelper import open_html_form
params = { "module":, "version":"v..." }
open_html_form (params, title="try the password *", key_save="form1")

Cell 2:

print(form1)

We can execute a simple action after the button Ok is pressed. This second trick comes from this notebook. The code displays whatever comes from function custom_action in this case. You should return "" to display nothing.

def custom_action(x):
    x["combined"] = x["first_name"] + " " + x["last_name"]
    return x

params = { "first_name":"", "last_name":"" }
open_html_form(params, title="enter your name", key_save="my_address",
               hook="custom_action(my_address)")

(original entry : html_forms.py:docstring of pyquickhelper.ipythonhelper.html_forms.open_html_form, line 19)

Produce HTML documentation for a function or class

The following code can display the dosstring in HTML format to display it in a notebook.

from pyquickhelper.helpgen import docstring2html
import sklearn.linear_model
docstring2html(sklearn.linear_model.LogisticRegression)

(original entry : rst_converters.py:docstring of pyquickhelper.helpgen.rst_converters.docstring2html, line 25)

Run a notebook end to end

from pyquickhelper.ipythonhelper import run_notebook
run_notebook("source.ipynb", working_dir="temp",
            outfilename="modified.ipynb",
            additional_path=["custom_path"] )

(original entry : run_notebook.py:docstring of pyquickhelper.ipythonhelper.run_notebook.run_notebook, line 39)

Run a program using the command line

from pyquickhelper.loghelper import run_cmd
out, err = run_cmd("python setup.py install", wait=True)

(original entry : run_cmd.py:docstring of pyquickhelper.loghelper.run_cmd.run_cmd, line 34)

Run help generation

# from the main folder which contains folder src or the sources
generate_help_sphinx("pyquickhelper")

(original entry : sphinx_main.py:docstring of pyquickhelper.helpgen.sphinx_main.generate_help_sphinx, line 59)

Simple configuration file for Sphinx

We assume a module is configurated using the same structure as pyquickhelper. The file conf.py could just contain:

# -*- coding: utf-8 -*-
import sys, os, datetime, re
import solar_theme
from pyquickhelper.helpgen.default_conf import set_sphinx_variables

sys.path.insert(0, os.path.abspath(os.path.join(os.path.split(__file__)[0])))
set_sphinx_variables(__file__, "pyquickhelper", "Xavier Dupré", 2014,
                     "solar_theme", solar_theme.theme_path, locals())

# custom settings
...

setup.py must contain a string such as __version__ = 3.4. Close to the setup, there must be a file version.txt. You overwrite a value by giving a variable another value after the fucntion is called.

Some parts of the code can be disabled before generating the documentation. Those parts are surrounded by:

# -- HELP BEGIN EXCLUDE --
import module
# -- HELP END EXCLUDE --

If enable_disabled_parts is set to a string, these sections will become:

# -- HELP BEGIN EXCLUDE --
if hasattr(sys, <enable_disabled_parts>) and sys.<enable_disabled_parts>:
    import module
# -- HELP END EXCLUDE --

(original entry : default_conf.py:docstring of pyquickhelper.helpgen.default_conf.set_sphinx_variables, line 46)

Transfer files to webste through FTP

Simple sketch to transfer a list of files to a website through FTP

ftp = TransferFTP('ftp.<website>', alias, password, fLOG=print)

issues = [ ]
done = [ ]
notdone = [ ]
for file in files :

    try :
        r = ftp.transfer (file, path)
        if r : done.append( (file, path) )
        else : notdone.append ( (file, path) )
    except Exception as e :
        issues.append( (file, e) )

try :
    ftp.close()
except Exception as e :
    print ("unable to close FTP connection using ftp.close")

(original entry : ftp_transfer.py:docstring of pyquickhelper.filehelper.ftp_transfer.TransferFTP, line 4)

Transfer updated files to a website

The following code shows how to transfer the content of a folder to website through FTP protocol.

ftn  = FileTreeNode("c:/somefolder")
ftp  = TransferFTP("ftp.website.fr", "login", "password", fLOG=print)
fftp = FolderTransferFTP (ftn, ftp, "status_file.txt",
        root_web = "/www/htdocs/app/pyquickhelper/helpsphinx")

fftp.start_transfering()
ftp.close()

(original entry : ftp_transfer_files.py:docstring of pyquickhelper.filehelper.ftp_transfer_files.FolderTransferFTP, line 5)

Visualize the difference between two text files or strings

with open("file1.txt","r",encoding="utf8") as f:
    text1 = f.read()
with open("file2.txt","r",encoding="utf8") as f:
    text2 = f.read()
pg = create_visual_diff_through_html(text1,text2)
with open("page.html","w",encoding="utf8") as f:
    f.write(pg)
import webbrowser
webbrowser.open("page.html")

(original entry : visual_sync.py:docstring of pyquickhelper.filehelper.visual_sync.create_visual_diff_through_html, line 20)

synchronize two folders

The following function synchronizes a folder with another one on a USB drive or a network drive. To minimize the number of access to the other location, it stores the status of the previous synchronization in a file (status_copy.txt in the below example). Next time, the function goes through the directory and sub-directories to synchronize and only propagates the modifications which happened since the last modification. The function filter_copy defines what file to synchronize or not.

def filter_copy(file):
    return "_don_t_synchronize_" not in file

synchronize_folder( "c:/mydata",
                    "g:/mybackup",
                    hash_size = 0,
                    filter_copy = filter_copy,
                    file_date = "c:/status_copy.txt")

The function is able to go through 90.000 files and 90 Gb in 12 minutes (for an update).

(original entry : synchelper.py:docstring of pyquickhelper.filehelper.synchelper.synchronize_folder, line 49)