Command lines

  1. Analyses profiling results produced by module cProfile

  2. Clean files

  3. Compares two files

  4. Convert RST document into HTML

  5. Convert a notebook into a different format

  6. Decrypt a file

  7. Decrypt a string

  8. Encrypt a file

  9. Encrypt a string

  10. Explore the content of a directory

  11. Merge images into PDF

  12. Repeat script execution every n seconds

  13. Returns statistics about the documentation of a module

  14. Run a notebook

  15. Runs a uvicorn application

  16. Synchronize two folders

  17. Upload one or several files to a FTP server

Analyses profiling results produced by module cProfile

The command line produces a flat output like method print_stats or a hierchical output showing function calls.

<<<

python -m pyquickhelper profile_stat --help

>>>

usage: profile_stat [-h] [-f FILE_STAT] [-o OUTPUT] [-c CALLS] [-v VERBOSE]
                    [-cl CLEAN_PREFIXES] [-s SORT_KEY] [-fc FCT_WIDTH]

Analyses the output of a profiling measured by module :mod:`cProfile`.

optional arguments:
  -h, --help            show this help message and exit
  -f FILE_STAT, --file_stat FILE_STAT
                        filename, profiling statistics (default: None)
  -o OUTPUT, --output OUTPUT
                        output file, the extension determines the format,
                        `.txt` for a text output, `.csv` for a comma separated
                        value, `.xlsx` for excel output (default: )
  -c CALLS, --calls CALLS
                        flat output (False) or hierchical output (True), the
                        hierarchical output shows the call stack (default:
                        True)
  -v VERBOSE, --verbose VERBOSE
                        more verbosity (default: False)
  -cl CLEAN_PREFIXES, --clean_prefixes CLEAN_PREFIXES
                        prefixes to clean from the output, separated by `;`
                        (default: )
  -s SORT_KEY, --sort_key SORT_KEY
                        `line` or `cumulative` or `time` (if calls is True)
                        (default: line)
  -fc FCT_WIDTH, --fct_width FCT_WIDTH
                        number of character dedicatedd to the function name
                        (if calls is True) (default: 50)

(original entry : profile_cli.py:docstring of pyquickhelper.cli.profile_cli.profile_stat, line 19)

Clean files

The command line cleans files in a folder.

<<<

python -m pyquickhelper clean_files --help

>>>

usage: clean_files [-h] [-f FOLDER] [-p POSREG] [-n NEGREG] [--op OP]

Cleans ``\r`` in files a folder and subfolders with a given extensions.
Backslashes are replaces by ``/``. The regular expressions applies on the
relative path starting from *folder*.

optional arguments:
  -h, --help            show this help message and exit
  -f FOLDER, --folder FOLDER
                        folder to clean (default: .)
  -p POSREG, --posreg POSREG
                        regular expression to select files to process
                        (default: .*[.]((py)|(rst))$)
  -n NEGREG, --negreg NEGREG
                        regular expression to skip files to process (default:
                        .*[.]git/.*)
  --op OP               kind of cleaning to do, options are CR, CRB, pep8, see
                        below for more details (default: CR)

(original entry : cli_helper.py:docstring of pyquickhelper.cli.cli_helper.cli_main_helper, line 50)

Compares two files

The command calls function create_visual_diff_through_html_files and produces a HTML page with shows the differences between two files. Example:

python -m pyquickhelper visual_diff -f <file1> -fi <file2> --browser=1 --page=diff.html

It works better with chrome.

<<<

python -m pyquickhelper visual_diff --help

>>>

usage: visual_diff [-h] [-f FILE1] [-fi FILE2] [-e ENCODING] [-p PAGE]
                   [-b BROWSER] [-n NOTEBOOK] [-c CONTEXT_SIZE]
                   [-i INLINE_VIEW]

Calls function :func:`create_visual_diff_through_html
<pyquickhelper.filehelper.visual_sync.create_visual_diff_through_html>` with
the content of two files.

optional arguments:
  -h, --help            show this help message and exit
  -f FILE1, --file1 FILE1
                        first file (anything such as an url, a file, a string,
                        a stream) (default: None)
  -fi FILE2, --file2 FILE2
                        second file (anything such as an url, a file, a
                        string, a stream) (default: None)
  -e ENCODING, --encoding ENCODING
                        encoding (default: utf8)
  -p PAGE, --page PAGE  if not None, saves the results in file (default: )
  -b BROWSER, --browser BROWSER
                        open browser ? (default: False)
  -n NOTEBOOK, --notebook NOTEBOOK
                        if True, the function assumes the outcome will be
                        displayed from a notebook and does things accordingly
                        (default: False)
  -c CONTEXT_SIZE, --context_size CONTEXT_SIZE
                        to display everything (None) or just the changes > 0
                        (default: )
  -i INLINE_VIEW, --inline_view INLINE_VIEW
                        only for notebook, True: one column, False: two
                        columns (default: False)

(original entry : visual_sync.py:docstring of pyquickhelper.filehelper.visual_sync.create_visual_diff_through_html_files, line 24)

Convert RST document into HTML

Converts RST documents into HTML or even RST.

<<<

python -m pyquickhelper sphinx_rst --help

>>>

usage: sphinx_rst [-h] [-i INPUT] [-w WRITER] [-k KEEP_WARNINGS]
                  [-d DIRECTIVES] [-l LANGUAGE] [-la LAYOUT] [-o OUTPUT]

Converts a string from *RST* to *HTML* or *RST* format.

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        text of filename (default: )
  -w WRITER, --writer WRITER
                        ``'html'`` for `HTML` format, ``'rst'`` for `RST`
                        format, ``'md'`` for `MD` format, ``'elatex'`` for
                        `latex` format, ``'doctree'`` to get the doctree,
                        *writer* can also be a tuple for custom formats and
                        must be like ``('buider_name', builder_class)``.
                        (default: html)
  -k KEEP_WARNINGS, --keep_warnings KEEP_WARNINGS
                        keep_warnings in the final HTML (default: False)
  -d DIRECTIVES, --directives DIRECTIVES
                        new directives to add, comma separated values
                        (default: )
  -l LANGUAGE, --language LANGUAGE
                        language (default: en)
  -la LAYOUT, --layout LAYOUT
                        ``'docutils'``, ``'sphinx'``, ``'sphinx_body'``, see
                        below. (default: sphinx)
  -o OUTPUT, --output OUTPUT
                        document name, the function adds the extension
                        (default: output)

(original entry : simplified_fct.py:docstring of pyquickhelper.cli.simplified_fct.sphinx_rst, line 18)

Convert a notebook into a different format

The command line converts notebook into HTML, RST, PDF, slides… It calls nbconvert but adds some preprocessing before calling it.

<<<

python -m pyquickhelper convert_notebook --help

>>>

usage: convert_notebook [-h] [-f FILENAME] [-o OUTFOLD] [-b BUILD]
                        [-l LATEX_PATH] [-p PANDOC_PATH] [-fo FORMATS]
                        [--exc EXC] [-n NBLINKS] [-r REMOVE_UNICODE_LATEX]

Converts a notebook into a specific format.

optional arguments:
  -h, --help            show this help message and exit
  -f FILENAME, --filename FILENAME
                        notebook name (default: None)
  -o OUTFOLD, --outfold OUTFOLD
                        notebook is first copied into this directory to make
                        some preprocessing. This directory must exist,
                        directory ``_convertnb`` will be created otherwise.
                        (default: )
  -b BUILD, --build BUILD
                        can be the current one (default: )
  -l LATEX_PATH, --latex_path LATEX_PATH
                        if format includes latex (default: )
  -p PANDOC_PATH, --pandoc_path PANDOC_PATH
                        for word format (default: )
  -fo FORMATS, --formats FORMATS
                        list of formats to use (comma separated), full list is
                        ``ipynb,html,python,rst,slides,pdf,github`` (default:
                        html,python)
  --exc EXC             raises an exception of be silent (default: True)
  -n NBLINKS, --nblinks NBLINKS
                        to add some link (default: )
  -r REMOVE_UNICODE_LATEX, --remove_unicode_latex REMOVE_UNICODE_LATEX
                        should not be necessary (default: False)

(original entry : notebook.py:docstring of pyquickhelper.cli.notebook.convert_notebook, line 16)

Decrypt a file

Decrypt a file from the command line.

decrypt_file –help

usage: decrypt_file [-h] source dest password

decrypt_file a file does not work well in Python 2.7 with pycryptodome

positional arguments:
  source      file to decrypt_file
  dest        location of the decrypt_fileed file
  password    password, usually an ascii string with 16x characters

optional arguments:
  -h, --help  show this help message and exit

(original entry : encryption_file_cli.py:docstring of pyquickhelper.cli.encryption_file_cli.decrypt_file, line 6)

Decrypt a string

Decrypts an encrypted string from the command line.

decrypt –help

usage: decrypt [-h] [-r REGEX] source dest password

decrypt a folder. For a second run, the program looks into file status to
avoid crypting same file gain, it does only modified files it does not work
well in Python 2.7 with pycryptodome.

positional arguments:
  source                folder to decrypt
  dest                  location of the decrypted files
  password              password, usually an ascii string with 16x characters

optional arguments:
  -h, --help            show this help message and exit
  -r REGEX, --regex REGEX
                        the script can retrieve only a subpart of the data
                        defined by a regular expression

(original entry : encryption_cli.py:docstring of pyquickhelper.cli.encryption_cli.decrypt, line 9)

Encrypt a file

Encrypt a file from the command line.

encrypt_file –help

usage: encrypt_file [-h] source dest password

encrypt_file a file does not work well in Python 2.7 with pycryptodome

positional arguments:
  source      file to encrypt_file
  dest        location of the encrypt_fileed file
  password    password, usually an ascii string with 16x characters

optional arguments:
  -h, --help  show this help message and exit

(original entry : encryption_file_cli.py:docstring of pyquickhelper.cli.encryption_file_cli.encrypt_file, line 6)

Encrypt a string

Encrypts a string from the command line.

encrypt –help

usage: encrypt [-h] [-s STATUS] [-m MAP] source dest password

encrypt a folder. For a second run, the program looks into file status to
avoid crypting same file gain, it does only modified files it does not work
well in Python 2.7 with pycryptodome.

positional arguments:
  source                folder to encrypt
  dest                  location of the encrypted files
  password              password, usually an ascii string with 16x characters

optional arguments:
  -h, --help            show this help message and exit
  -s STATUS, --status STATUS
                        to keep track of what was done
  -m MAP, --map MAP     mapping between raw files and crypted files

(original entry : encryption_cli.py:docstring of pyquickhelper.cli.encryption_cli.encrypt, line 9)

Explore the content of a directory

The command calls function explore_folder and makes the list of all files in a directory or all folders. Example:

python -m pyquickhelper ls -f _mynotebooks -r f -p .*[.]ipynb -n checkpoints -fu 1

It works better with chrome. An example to change file names:

python -m pyquickhelper ls -f myfolder -p .*[.]py -r f -n pycache -fu 1 -s test_(.*) -su unit_\1

Or another to automatically create git commands to rename files:

python -m pyquickhelper ls -f _mynotebooks -r f -p .*[.]ipynb -s "(.*)[.]ipynb" -su "git mv \1.ipynb \1~.ipynb"

<<<

python -m pyquickhelper ls --help

>>>

usage: ls [-h] [-f FOLDER] [-p PATTERN] [-n NEG_PATTERN] [-fu FULLNAME]
          [-r RETURN_ONLY] [-re RECURSIVE] [-s SUB_PATTERN] [-su SUB_REPLACE]

Returns the list of files included in a folder and its subfolders. Returned
names can be modified if *sub_pattern* is specified.

optional arguments:
  -h, --help            show this help message and exit
  -f FOLDER, --folder FOLDER
                        (str) folder (default: None)
  -p PATTERN, --pattern PATTERN
                        (str) if None, get all files, otherwise, it is a
                        regular expression, the filename must verify (with the
                        folder if fullname is True) (default: )
  -n NEG_PATTERN, --neg_pattern NEG_PATTERN
                        (str) negative pattern (default: )
  -fu FULLNAME, --fullname FULLNAME
                        (bool) if True, include the subfolder while checking
                        the regex (pattern) (default: False)
  -r RETURN_ONLY, --return_only RETURN_ONLY
                        (str) to return folders and files (*=None*), only the
                        files (*='f'*) or only the folders (*='d') (default: )
  -re RECURSIVE, --recursive RECURSIVE
                        (bool) look into subfolders (default: True)
  -s SUB_PATTERN, --sub_pattern SUB_PATTERN
                        (str) replacements pattern, the output is then
                        modified accordingly to this regular expression
                        (default: )
  -su SUB_REPLACE, --sub_replace SUB_REPLACE
                        (str) if sub_pattern is specified, this second pattern
                        specifies how to replace (default: )

(original entry : synchelper.py:docstring of pyquickhelper.filehelper.synchelper.explore_folder, line 20)

Merge images into PDF

Merges one or several images into a single PDF document.

<<<

python -m pyquickhelper images2pdf --help

>>>

usage: images2pdf [-h] [-i IMAGES] [-o OUTPUT]

Merges multiples images into one single pdf. Relies on `img2pdf`. If an image
name contains ``'*'``, the function assumes it is a pattern and uses
`*py.glob`.

optional arguments:
  -h, --help            show this help message and exit
  -i IMAGES, --images IMAGES
                        images to merge, it can be a comma separated values
                        (default: None)
  -o OUTPUT, --output OUTPUT
                        output filename or stream (default: None)

(original entry : img_export.py:docstring of pyquickhelper.imghelper.img_export.images2pdf, line 10)

Repeat script execution every n seconds

The command line runs the execution a script on a regular basis.

<<<

python -m pyquickhelper repeat_script --help

>>>

usage: repeat_script [-h] [-s SCRIPT] [-e EVERY_SECOND]
                     [-st STOP_AFTER_SECOND] [-o OUTFILE] [-er ERRFILE]
                     [--exc EXC] [-v VERBOSE]

Runs a python script on a regular basis. The function is not multithreaded, it
returns when all execution are done.

optional arguments:
  -h, --help            show this help message and exit
  -s SCRIPT, --script SCRIPT
                        script to run (default: None)
  -e EVERY_SECOND, --every_second EVERY_SECOND
                        every second (default: 20)
  -st STOP_AFTER_SECOND, --stop_after_second STOP_AFTER_SECOND
                        stop after a given time or never if -1 (default: -1)
  -o OUTFILE, --outfile OUTFILE
                        file which receives the standard output (default:
                        out.log)
  -er ERRFILE, --errfile ERRFILE
                        file which receives the standard error (default:
                        err.log)
  --exc EXC             True to stop if an exception is raised, False to
                        continue (default: True)
  -v VERBOSE, --verbose VERBOSE
                        prints out every execution (default: 1)

(original entry : script_exec.py:docstring of pyquickhelper.cli.script_exec.repeat_script, line 14)

Returns statistics about the documentation of a module

The command line returns a table with the number of lines of code and documentatation.

<<<

python -m pyquickhelper code_stat --help

>>>

usage: code_stat [-h] [-n NAMES] [-o OUTPUT]

Returns statistics about the documentation of a module.

optional arguments:
  -h, --help            show this help message and exit
  -n NAMES, --names NAMES
                        module name comma separated value (default: None)
  -o OUTPUT, --output OUTPUT
                        output file name (default: )

(original entry : code_cli.py:docstring of pyquickhelper.cli.code_cli.code_stat, line 8)

Run a notebook

The command line runs a notebook and stores the modified notebook.

<<<

python -m pyquickhelper run_notebook --help

>>>

usage: run_notebook [-h] [-f FILENAME] [-p PROFILE_DIR] [-w WORKING_DIR]
                    [-s SKIP_EXCEPTIONS] [-o OUTFILENAME] [-a ADDITIONAL_PATH]
                    [-k KERNEL_NAME] [-l LOG_LEVEL] [-st STARTUP_TIMEOUT]
                    [-v VERBOSE] [-r RAISE_EXCEPTION]

Runs a notebook end to end, it is inspired from module runipy.

optional arguments:
  -h, --help            show this help message and exit
  -f FILENAME, --filename FILENAME
                        notebook filename (default: None)
  -p PROFILE_DIR, --profile_dir PROFILE_DIR
                        profile directory (default: )
  -w WORKING_DIR, --working_dir WORKING_DIR
                        working directory (default: )
  -s SKIP_EXCEPTIONS, --skip_exceptions SKIP_EXCEPTIONS
                        skip exceptions (default: False)
  -o OUTFILENAME, --outfilename OUTFILENAME
                        if not None, saves the output in this notebook
                        (default: )
  -a ADDITIONAL_PATH, --additional_path ADDITIONAL_PATH
                        additional paths for import (comma separated)
                        (default: )
  -k KERNEL_NAME, --kernel_name KERNEL_NAME
                        kernel name, it can be None (default: python)
  -l LOG_LEVEL, --log_level LOG_LEVEL
                        Choices: (0, 10, 20, 30=default, 40, 50, 'DEBUG',
                        'INFO', 'WARN', 'ERROR', 'CRITICAL') (default: 30)
  -st STARTUP_TIMEOUT, --startup_timeout STARTUP_TIMEOUT
                        wait for this long for the kernel to be ready, see
                        `wait_for_ready <https://github.com/jupyter/jupyter_cl
                        ient/blob/master/jupyter_client/blocking/client.py#L84
                        >`_ (default: 300)
  -v VERBOSE, --verbose VERBOSE
                        0 for standard logging, 1 for more (default: 0)
  -r RAISE_EXCEPTION, --raise_exception RAISE_EXCEPTION
                        raise an exception if a cell raises one (default:
                        True)

(original entry : notebook.py:docstring of pyquickhelper.cli.notebook.run_notebook, line 22)

Runs a uvicorn application

Runs a uvicorn application.

<<<

python -m pyquickhelper uvicorn_app --help

>>>

usage: uvicorn_app [-h] [-p PATH] [--pwd PWD] [-po PORT] [-ho HOST]

Runs a uvicorn application. It should be used for testing not for production.
Use ``host:post/redoc`` or ``host:post/docs`` to get a web page in order to
submit files.

optional arguments:
  -h, --help            show this help message and exit
  -p PATH, --path PATH  filename for the databse (default: dummy_db.db3)
  --pwd PWD             password (default: dummy)
  -po PORT, --port PORT
                        port (default: 8798)
  -ho HOST, --host HOST
                        host (default: 127.0.0.1)

(original entry : uvicorn_cli.py:docstring of pyquickhelper.cli.uvicorn_cli.uvicorn_app, line 11)

Synchronize two folders

Synchronizes two folders from the command line.

pyq_sync –help

usage: synchronize_folder [-h] [--p1 P1] [--p2 P2] [-ha HASH_SIZE] [-r REPO1]
                          [-re REPO2] [-s SIZE_DIFFERENT] [-n NO_DELETION]
                          [-f FILTER] [-fi FILTER_COPY] [-a AVOID_COPY]
                          [-fil FILE_DATE] [-c COPY_1TO2] [-cr CREATE_DEST]

Synchronizes two folders (or copy if the second is empty), it only copies more
recent files. It can walk through a `git` repository or SVN.

optional arguments:
  -h, --help            show this help message and exit
  --p1 P1               (str) first path (default: None)
  --p2 P2               (str) second path (default: None)
  -ha HASH_SIZE, --hash_size HASH_SIZE
                        (bool) to check whether or not two files are different
                        (default: 1048576)
  -r REPO1, --repo1 REPO1
                        (bool) assuming the first folder is under SVN or GIT,
                        it uses pysvn to get the list of files (avoiding any
                        extra files) (default: False)
  -re REPO2, --repo2 REPO2
                        (bool) assuming the second folder is under SVN or GIT,
                        it uses pysvn to get the list of files (avoiding any
                        extra files) (default: False)
  -s SIZE_DIFFERENT, --size_different SIZE_DIFFERENT
                        (bool) if True, a file will be copied only if size are
                        different, otherwise, it will be copied if the first
                        file is more recent (default: True)
  -n NO_DELETION, --no_deletion NO_DELETION
                        (bool) if a file is found in the second folder and not
                        in the first one, if will be removed unless
                        no_deletion is True (default: False)
  -f FILTER, --filter FILTER
                        (str) None to accept every file, a string if it is a
                        regular expression, a function for something more
                        complex: function ``(fullname) --> True`` (every file
                        is considered in lower case), (use `*py.re.search`)
                        (default: None)
  -fi FILTER_COPY, --filter_copy FILTER_COPY
                        (str) None to accept every file, a string if it is a
                        regular expression, a function for something more
                        complex: function (fullname) --> True (default: None)
  -a AVOID_COPY, --avoid_copy AVOID_COPY
                        (bool) if True, just return the list of files which
                        should be copied but does not do the copy (default:
                        False)
  -fil FILE_DATE, --file_date FILE_DATE
                        (str) filename which contains information about when
                        the last sync was done (default: None)
  -c COPY_1TO2, --copy_1to2 COPY_1TO2
                        (bool) only copy files from *p1* to *p2* (default:
                        False)
  -cr CREATE_DEST, --create_dest CREATE_DEST
                        (bool) create destination directory if not exist
                        (default: False)

(original entry : pyq_sync_cli.py:docstring of pyquickhelper.cli.pyq_sync_cli.pyq_sync, line 6)

Upload one or several files to a FTP server

Uploads a file, a list of files, files defined by a pattern to a FTP server using FTP or SFTP protocol.

<<<

python -m pyquickhelper ftp_upload --help

>>>

usage: ftp_upload [-h] [-f FILES] [-d DEST] [-ho HOST] [-u USER] [--pwd PWD]
                  [-ft FTPS]

Pushes a file to a server using `FTP` or `SFTP`.

optional arguments:
  -h, --help            show this help message and exit
  -f FILES, --files FILES
                        local files to move, comma separated or defined with a
                        pattern if character ``*`` is used (default: None)
  -d DEST, --dest DEST  destination folder (default: None)
  -ho HOST, --host HOST
                        server name or ip address (default: None)
  -u USER, --user USER  user to log in (default: None)
  --pwd PWD             password for the user (default: None)
  -ft FTPS, --ftps FTPS
                        use `SFTP` or `FTP` (default: False)

(original entry : ftp_cli.py:docstring of pyquickhelper.cli.ftp_cli.ftp_upload, line 13)