module pycode.setup_helper

Short summary

module pyquickhelper.pycode.setup_helper

Helper for the setup

source on GitHub

Functions

function

truncated documentation

_get_dump_default_path

Proposes a default location to dump results about unit tests execution.

available_commands_list

Checks that on command handled by pyquickhelper is part of the arguments.

build_history_from_setup

Builds the history from github and pypi.

clean_notebooks_for_numbers

Upgrades notebooks to the latest format and cleans notebooks execution numbers and rearranges the JSON file.

clean_space_for_setup

copy27_for_setup

Prepares a copy of the source for Python 2.7, assuming the module follows the same design as pyquickhelper. …

get_available_build_commands

Returns commands which build the package.

get_available_setup_commands

Returns the list of commands pyquickhelper implements or allows.

get_folder

Returns the folder which contains setup.py.

get_script_extension

Returns the scripts extension based on the system it is running on.

hash_list

Proposes a hash for the list of arguments.

process_standard_options_for_setup

Processes the standard options the module pyquickhelper is able to process assuming the module which calls this function …

process_standard_options_for_setup_help

Prints the added options available through this module.

run_pylint_for_setup

Applies pylint on subfolder folder.

run_unittests_for_setup

Runs the unit tests and computes the coverage, stores the results in _doc/sphinxdoc/source/coverage assuming …

standard_help_for_setup

Standard function which generates help assuming they follow the same design as pyquickhelper.

write_module_scripts

Writes a couple of scripts which allow a user to be faster on some tasks or to easily get information about the module. …

write_pyproj

Creates a project pyproj

write_version_for_setup

Extracts the version number, the function writes the files version.txt in this folder.

Documentation

Helper for the setup

source on GitHub

pyquickhelper.pycode.setup_helper._get_dump_default_path(location, module_name, argv)[source][source]

Proposes a default location to dump results about unit tests execution.

Parameters
  • location – location of the module

  • module_name – module name

  • argv – argument on the command line

Returns

location of the dump

The result is None for remote continuous integration.

source on GitHub

pyquickhelper.pycode.setup_helper.available_commands_list(argv)[source][source]

Checks that on command handled by pyquickhelper is part of the arguments.

Parameters

argvsys.args

Returns

bool

source on GitHub

pyquickhelper.pycode.setup_helper.build_history_from_setup(dest, owner, module, existing_history=None, fLOG=<function noLOG>)[source][source]

Builds the history from github and pypi.

Parameters
  • dest – history will be written in this file

  • owner – owner of the package on github

  • module – module name

  • existing_history – existing history, retrieves existing issues stored in that file

  • fLOG – logging function

Returns

history

source on GitHub

pyquickhelper.pycode.setup_helper.clean_notebooks_for_numbers(file_or_folder)[source][source]

Upgrades notebooks to the latest format and cleans notebooks execution numbers and rearranges the JSON file.

Parameters

file_or_folder – file setup.py or folder which contains it

Returns

impacted files

source on GitHub

pyquickhelper.pycode.setup_helper.clean_space_for_setup(file_or_folder, file_filter=None)[source][source]

Does some cleaning within the module, apply pep8 rules.

Parameters
Returns

impacted files

source on GitHub

pyquickhelper.pycode.setup_helper.copy27_for_setup(file_or_folder)[source][source]

Prepares a copy of the source for Python 2.7, assuming the module follows the same design as pyquickhelper.

Parameters

file_or_folder – file setup.py or folder which contains it

source on GitHub

pyquickhelper.pycode.setup_helper.get_available_build_commands()[source][source]

Returns commands which build the package.

source on GitHub

pyquickhelper.pycode.setup_helper.get_available_setup_commands()[source][source]

Returns the list of commands pyquickhelper implements or allows.

source on GitHub

pyquickhelper.pycode.setup_helper.get_folder(file_or_folder)[source][source]

Returns the folder which contains setup.py.

Parameters

file_or_folder – file setup.py or folder which contains it

Returns

folder

source on GitHub

pyquickhelper.pycode.setup_helper.get_script_extension()[source][source]

Returns the scripts extension based on the system it is running on.

Returns

bat or sh

source on GitHub

pyquickhelper.pycode.setup_helper.hash_list(argv, size=8)[source][source]

Proposes a hash for the list of arguments.

Parameters
  • argv – list of arguments on the command line.

  • size – size of the hash

Returns

string

source on GitHub

pyquickhelper.pycode.setup_helper.process_standard_options_for_setup(argv, file_or_folder, project_var_name, module_name=None, unittest_modules=None, pattern_copy='.*[.]((ico)|(7z)|(dll)|(so)|(yml)|(rst)|(ipynb)|(gif)|(jpg)|(jpeg)|(png)|(txt)|(zip)|(gz)|(html)|(exe)|(js)|(css)|(tex)|(data)|(csv)|(tpl))$', requirements=None, port=8067, blog_list=None, default_engine_paths=None, extra_ext=None, add_htmlhelp=False, setup_params=None, coverage_options=None, coverage_exclude_lines=None, func_sphinx_begin=None, func_sphinx_end=None, additional_notebook_path=None, additional_local_path=None, copy_add_ext=None, nbformats=('ipynb', 'html', 'python', 'rst', 'slides', 'pdf', 'present', 'github'), layout=None, direct_call=False, additional_ut_path=None, skip_function=<function default_skip_function>, covtoken=None, hook_print=True, stdout=None, stderr=None, use_run_cmd=False, filter_warning=None, file_filter_pep8=None, github_owner=None, existing_history=None, coverage_root='src', fexclude=None, fLOG=<function noLOG>)[source][source]

Processes the standard options the module pyquickhelper is able to process assuming the module which calls this function follows the same design as pyquickhelper, it will process the following options:

    Commands processed by pyquickhelper:
      build27           build the wheel for Python 2.7 (if available), it requires to run copy27 first
      build_script      produce various scripts to build the module
      build_sphinx      build the documentation
      build_wheel       build the wheel
      clean_pyd         clean file ``*.pyd``
      clean_space       clean unnecessary spaces in the code, applies :epkg:`pycodestyle` on all files
      copy27            create a modified copy of the module to run on Python 2.7 (if available), it requires to run copy27 first
      copy_dist         copy documentation to folder dist
      copy_sphinx       modify and copy sources to _doc/sphinxdoc/source/<module>
      history           builds the history of the package in RST
      local_jenkins     sets up a local jenkins server
      run27             run the unit tests for the Python 2.7
      run_pylint        run pylint on the sources, allowed parameters <pattern> <neg_pattern>
      setup_hook        call function setup_hook which initializes the module before running unit tests
      unittests         run the unit tests which do not contain test_LONG, test_SKIP or test_GUI in their file name
      unittests_GUI     run the unit tests which contain test_GUI their file name
      unittests_LONG    run the unit tests which contain test_LONG their file name
      unittests_SKIP    run the unit tests which contain test_SKIP their file name
      write_version     write a file ``version.txt`` with the version number (assuming sources are host with git)
Parameters
  • argv – = sys.argv

  • file_or_folder – file setup.py or folder which contains it

  • project_var_name – display name of the module

  • module_name – module name, None if equal to project_var_name (import <module_name>)

  • unittest_modules – modules added for the unit tests, see py3to2_convert_tree

  • pattern_copy – see py3to2_convert_tree

  • requirements – dependencies, fetched with a local pipy server from http://localhost:port/

  • port – port for the local pipy server

  • blog_list – list of blog to listen for this module (usually stored in module.__blog__)

  • default_engine_paths – define the default location for python engine, should be dictionary { engine: path }, see below.

  • extra_ext – extra file extension to process (add a page for each of them, ex ["doc"])

  • add_htmlhelp – run HTML Help too (only on Windows)

  • setup_params – parameters send to call_setup_hook

  • coverage_options – see main_wrapper_tests

  • coverage_exclude_lines – see main_wrapper_tests

  • func_sphinx_begin – function called before the documentation generation, it gets the same parameters as this function (all named), use **args**

  • func_sphinx_end – function called after the documentation generation, it gets the same parameters as this function (all named), use **args**

  • additional_notebook_path – additional paths to add when launching the notebook

  • additional_local_path – additional paths to add when running a local command

  • copy_add_ext – additional file extensions to copy

  • nbformats – requested formats for the notebooks conversion

  • layout – list of formats sphinx should generate such as html, latex, pdf, docx, it is a list of tuple (layout, build directory, parameters to override), if None –> ["html", "pdf"]

  • additional_ut_path – additional paths to add when running unit tests

  • skip_function – function to skip unit tests, see @ee fn main_wrapper_tests

  • covtoken – token used when publishing coverage report to codecov, more in main_wrapper_tests

  • fLOG – logging function

  • hook_print – enable, disable print when calling _setup_hook

  • stdout – redirect stdout for unit test if not None

  • stderr – redirect stderr for unit test if not None

  • use_run_cmd – to run the sphinx documentation with run_cmd and not os.system

  • filter_warning – see main_wrapper_tests

  • file_filter_pep8 – function to filter out files for which checking pep8 (see remove_extra_spaces_folder)

  • github_ownergithub owner of the package

  • existing_history – existing history, retrieves existing issues stored in that file

  • coverage_root – see main_wrapper_tests

  • direct_callgenerate_help_sphinx

  • fexclude – function which tells which file not to copy in the folder used to build the documentation

Returns

True (an option was processed) or False, the file setup.py should call function setup

The command build_script is used, the flag --private can be used to avoid producing scripts to publish the module on Pypi.

An example for default_engine_paths:

default_engine_paths = {
    "windows": {
        "__PY35__": None,
        "__PY37_X64__": "c:\Python37_x64",
        "__PY36_X64__": "c:\Python36_x64",
        "__PY35_X64__": "c:\Python35_x64",
        "__PY27_X64__": "c:\Anaconda2",
    },
}

Parameters coverage_options, coverage_exclude_lines, copy_add_ext were added for function main_wrapper_tests. Parameter unittest_modules accepts a list of string and 2-uple. If it is a 2-uple, the first string is used to convert Python 3 code into Python 2 (in case the local folder is different from the module name), the second string is used to add local path to the variable PYTHON_PATH. If it is a single string, it means both name strings are equal. Parameters func_sphinx_begin and func_sphinx_end were added to pre-process or post-process the documentation. Parameter additional_notebook_path was added to specify some additional paths when preparing the script auto_cmd_notebook.bat. Parameters layout, nbformats were added for function generate_help_sphinx. The coverage computation can be disable by specifying coverage_options["disable_coverage"] = True. Parameter covtoken was added to post the coverage report to codecov. Option -e and -g were added to filter file by regular expressions (in with e, out with g).

Changed in version 1.8: Parameters coverage_root, direct_call were added.

source on GitHub

pyquickhelper.pycode.setup_helper.process_standard_options_for_setup_help(argv)[source][source]

Prints the added options available through this module.

source on GitHub

pyquickhelper.pycode.setup_helper.run_pylint_for_setup(folder, pattern='.*[.]py$', neg_pattern=None, verbose=False, pylint_ignore=None, fLOG=<function noLOG>)[source][source]

Applies pylint on subfolder folder.

Parameters
  • folder – folder where to look

  • pattern – file to checks

  • neg_pattern – negative pattern

@parm pylint_ignore ignore these pylint warnings or errors

Parameters
  • verbose – verbose

  • fLOG – logging function

source on GitHub

pyquickhelper.pycode.setup_helper.run_unittests_for_setup(file_or_folder, skip_function=<function default_skip_function>, setup_params=None, only_setup_hook=False, coverage_options=None, coverage_exclude_lines=None, additional_ut_path=None, covtoken=None, hook_print=True, stdout=None, stderr=None, filter_warning=None, dump_coverage=None, add_coverage_folder=None, coverage_root='src', fLOG=<function noLOG>)[source][source]

Runs the unit tests and computes the coverage, stores the results in _doc/sphinxdoc/source/coverage assuming the module follows the same design as pyquickhelper.

Parameters

See function main_wrapper_tests. The coverage computation can be disabled by specifying coverage_options["disable_coverage"] = True. covtoken was added to post the coverage report to codecov. Parameter dump_coverage dumps the unit test coverage in another location.

Changed in version 1.8: Parameter coverage_root was added.

source on GitHub

pyquickhelper.pycode.setup_helper.standard_help_for_setup(argv, file_or_folder, project_var_name, module_name=None, extra_ext=None, add_htmlhelp=False, copy_add_ext=None, nbformats=('ipynb', 'html', 'python', 'rst', 'slides', 'pdf', 'present'), layout=None, use_run_cmd=False, direct_call=False, fexclude=None, fLOG=<function noLOG>)[source][source]

Standard function which generates help assuming they follow the same design as pyquickhelper.

Parameters
  • argv – it should be sys.argv

  • file_or_folder – file setup.py or folder which contains it

  • project_var_name – display name of the module

  • module_name – module name, None if equal to project_var_name (import <module_name>)

  • extra_ext – extra file extension to process (ex ["doc"])

  • add_htmlhelp – run HTML Help too (only on Windows)

  • copy_add_ext – additional extension of files to copy

  • nbformats – notebooks format to generate

  • layout – layout for the documentation, if None –> ["html", "pdf"]

  • use_run_cmd – use function run_cmd instead of os.system to build the documentation

  • direct_call – see generate_help_sphinx

  • fexclude – function which tells which file not to copy in the folder used to build the documentation

  • fLOG – logging function

The function outputs some information through function fLOG.

A page will be added for each extra file extension mentioned in extra_ext if some of these were found.

source on GitHub

pyquickhelper.pycode.setup_helper.write_module_scripts(folder, platform='linux', blog_list=None, default_engine_paths=None, command=None)[source][source]

Writes a couple of scripts which allow a user to be faster on some tasks or to easily get information about the module.

Parameters
  • folder – where to write the script

  • platform – platform

  • blog_list – blog list to follow, should be attribute __blog__ of the module

  • command – None to generate scripts for all commands or a value in [blog, doc].

  • default_engine_paths – default engines (or python distributions)

Returns

list of written scripts

The function produces the following files:

  • auto_rss_list.xml: list of rss stream to follow

  • auto_rss_database.db3: stores blog posts

  • auto_rss_server.py: runs a server which updates the scripts and runs a server. It also open the default browser.

  • auto_rss_server.(bat|sh): run auto_run_server.py, the file on Linux might be missing if there is an equivalent python script

How to generate auto_rss_server.py?

The following code generates the script auto_rss_local.py which runs a local server to read blog posts included in the documentation (it uses module pyrsslocal):

from pyquickhelper.pycode import write_module_scripts, __blog__
write_module_scripts(".", blog_list=__blog__, command="blog")

source on GitHub

pyquickhelper.pycode.setup_helper.write_pyproj(file_or_folder, location=None)[source][source]

Creates a project pyproj to work with PTVS (Python Tools for Visual Studio)

Parameters
  • file_or_folder – file setup.py or folder which contains it

  • location – if not None, stores the project into this folder

This functionality fails with Python 2.7 (encoding).

source on GitHub

pyquickhelper.pycode.setup_helper.write_version_for_setup(file_or_folder, exc=False)[source][source]

Extracts the version number, the function writes the files version.txt in this folder.

Parameters
  • file_or_folder – file setup.py or folder which contains it

  • exc – raises an exception if cannot look into git folder

Returns

version number

Changed in version 1.8: Parameter exc was added.

source on GitHub