module pycode.setup_helper

Short summary

module pyquickhelper.pycode.setup_helper

Helper for the setup

source on GitHub

Functions

function

truncated documentation

_get_dump_default_path

Proposes a default location to dump results about unit tests execution.

available_commands_list

Checks that on command handled by pyquickhelper is part of the arguments.

build_history_from_setup

Builds the history from github and pypi.

clean_notebooks_for_numbers

Upgrades notebooks to the latest format and cleans notebooks execution numbers and rearranges the JSON file.

clean_space_for_setup

copy27_for_setup

Prepares a copy of the source for Python 2.7, assuming the module follows the same design as pyquickhelper. …

get_available_build_commands

Returns commands which build the package.

get_available_setup_commands

Returns the list of commands pyquickhelper implements or allows.

get_folder

Returns the folder which contains setup.py.

get_script_extension

Returns the scripts extension based on the system it is running on.

hash_list

Proposes a hash for the list of arguments.

process_standard_options_for_setup

Processes the standard options the module pyquickhelper is able to process assuming the module which calls this function …

process_standard_options_for_setup_help

Prints the added options available through this module.

run_pylint_for_setup

Applies pylint on subfolder folder.

run_unittests_for_setup

Runs the unit tests and computes the coverage, stores the results in _doc/sphinxdoc/source/coverage assuming …

standard_help_for_setup

Standard function which generates help assuming they follow the same design as pyquickhelper.

write_module_scripts

Writes a couple of scripts which allow a user to be faster on some tasks or to easily get information about the module. …

write_pyproj

Creates a project pyproj

write_version_for_setup

Extracts the version number, the function writes the files version.txt in this folder.

Documentation

Helper for the setup

source on GitHub

pyquickhelper.pycode.setup_helper._get_dump_default_path(location, module_name, argv)[source][source]

Proposes a default location to dump results about unit tests execution.

Parameters
  • location – location of the module

  • module_name – module name

  • argv – argument on the command line

Returns

location of the dump

The result is None for remote continuous integration.

source on GitHub

pyquickhelper.pycode.setup_helper.available_commands_list(argv)[source][source]

Checks that on command handled by pyquickhelper is part of the arguments.

Parameters

argvsys.args

Returns

bool

source on GitHub

pyquickhelper.pycode.setup_helper.build_history_from_setup(dest, owner, module, existing_history=None, skip_issues=None, fLOG=<function noLOG>)[source][source]

Builds the history from github and pypi.

Parameters
  • dest – history will be written in this file

  • owner – owner of the package on github

  • module – module name

  • existing_history – existing history, retrieves existing issues stored in that file

  • skip_issues – skip a given list of issues when building the history

  • fLOG – logging function

Returns

history

source on GitHub

pyquickhelper.pycode.setup_helper.clean_notebooks_for_numbers(file_or_folder)[source][source]

Upgrades notebooks to the latest format and cleans notebooks execution numbers and rearranges the JSON file.

Parameters

file_or_folder – file setup.py or folder which contains it

Returns

impacted files

source on GitHub

pyquickhelper.pycode.setup_helper.clean_space_for_setup(file_or_folder, file_filter=None)[source][source]

Does some cleaning within the module, apply pep8 rules.

Parameters
Returns

impacted files

source on GitHub

pyquickhelper.pycode.setup_helper.copy27_for_setup(file_or_folder)[source][source]

Prepares a copy of the source for Python 2.7, assuming the module follows the same design as pyquickhelper.

Parameters

file_or_folder – file setup.py or folder which contains it

source on GitHub

pyquickhelper.pycode.setup_helper.get_available_build_commands()[source][source]

Returns commands which build the package.

source on GitHub

pyquickhelper.pycode.setup_helper.get_available_setup_commands()[source][source]

Returns the list of commands pyquickhelper implements or allows.

source on GitHub

pyquickhelper.pycode.setup_helper.get_folder(file_or_folder)[source][source]

Returns the folder which contains setup.py.

Parameters

file_or_folder – file setup.py or folder which contains it

Returns

folder

source on GitHub

pyquickhelper.pycode.setup_helper.get_script_extension()[source][source]

Returns the scripts extension based on the system it is running on.

Returns

bat or sh

source on GitHub

pyquickhelper.pycode.setup_helper.hash_list(argv, size=8)[source][source]

Proposes a hash for the list of arguments.

Parameters
  • argv – list of arguments on the command line.

  • size – size of the hash

Returns

string

source on GitHub

pyquickhelper.pycode.setup_helper.process_standard_options_for_setup(argv, file_or_folder, project_var_name, module_name=None, unittest_modules=None, pattern_copy='.*[.]((ico)|(7z)|(dll)|(so)|(yml)|(rst)|(ipynb)|(gif)|(jpg)|(jpeg)|(png)|(txt)|(zip)|(gz)|(html)|(exe)|(js)|(css)|(tex)|(data)|(csv)|(tpl))$', requirements=None, port=8067, blog_list=None, default_engine_paths=None, extra_ext=None, add_htmlhelp=False, setup_params=None, coverage_options=None, coverage_exclude_lines=None, func_sphinx_begin=None, func_sphinx_end=None, additional_notebook_path=None, additional_local_path=None, copy_add_ext=None, nbformats=('ipynb', 'html', 'python', 'rst', 'slides', 'pdf', 'github'), layout=None, direct_call=False, additional_ut_path=None, skip_function=<function default_skip_function>, covtoken=None, hook_print=True, stdout=None, stderr=None, use_run_cmd=False, filter_warning=None, file_filter_pep8=None, github_owner=None, existing_history=None, coverage_root='src', fexclude=None, skip_issues=None, fLOG=<function noLOG>)[source][source]

Processes the standard options the module pyquickhelper is able to process assuming the module which calls this function follows the same design as pyquickhelper, it will process the following options:

    Commands processed by pyquickhelper:
      build27           build the wheel for Python 2.7 (if available), it requires to run copy27 first
      build_script      produce various scripts to build the module
      build_sphinx      build the documentation
      build_wheel       build the wheel
      clean_pyd         clean file ``*.pyd``
      clean_space       clean unnecessary spaces in the code, applies :epkg:`pycodestyle` on all files
      copy27            create a modified copy of the module to run on Python 2.7 (if available), it requires to run copy27 first
      copy_dist         copy documentation to folder dist
      copy_sphinx       modify and copy sources to _doc/sphinxdoc/source/<module>
      history           builds the history of the package in RST
      local_jenkins     sets up jobs on a local jenkins server
      run27             run the unit tests for the Python 2.7
      run_pylint        run pylint on the sources, allowed parameters <pattern> <neg_pattern>
      setup_hook        call function setup_hook which initializes the module before running unit tests
      unittests         run the unit tests which do not contain test_LONG, test_SKIP or test_GUI in their file name
      unittests_GUI     run the unit tests which contain test_GUI their file name
      unittests_LONG    run the unit tests which contain test_LONG their file name
      unittests_SKIP    run the unit tests which contain test_SKIP their file name
      write_version     write a file ``version.txt`` with the version number (assuming sources are host with git)
Parameters
  • argv – = sys.argv

  • file_or_folder – file setup.py or folder which contains it

  • project_var_name – display name of the module

  • module_name – module name, None if equal to project_var_name (import <module_name>)

  • unittest_modules – modules added for the unit tests, see py3to2_convert_tree

  • pattern_copy – see py3to2_convert_tree

  • requirements – dependencies, fetched with a local pipy server from http://localhost:port/

  • port – port for the local pipy server

  • blog_list – list of blog to listen for this module (usually stored in module.__blog__)

  • default_engine_paths – define the default location for python engine, should be dictionary { engine: path }, see below.

  • extra_ext – extra file extension to process (add a page for each of them, ex ["doc"])

  • add_htmlhelp – run HTML Help too (only on Windows)

  • setup_params – parameters send to call_setup_hook

  • coverage_options – see main_wrapper_tests

  • coverage_exclude_lines – see main_wrapper_tests

  • func_sphinx_begin – function called before the documentation generation, it gets the same parameters as this function (all named), use **args**

  • func_sphinx_end – function called after the documentation generation, it gets the same parameters as this function (all named), use **args**

  • additional_notebook_path – additional paths to add when launching the notebook

  • additional_local_path – additional paths to add when running a local command

  • copy_add_ext – additional file extensions to copy

  • nbformats – requested formats for the notebooks conversion

  • layout – list of formats sphinx should generate such as html, latex, pdf, docx, it is a list of tuple (layout, build directory, parameters to override), if None –> ["html", "pdf"]

  • additional_ut_path – additional paths to add when running unit tests

  • skip_function – function to skip unit tests, see @ee fn main_wrapper_tests

  • covtoken – token used when publishing coverage report to codecov, more in main_wrapper_tests

  • fLOG – logging function

  • hook_print – enable, disable print when calling _setup_hook

  • stdout – redirect stdout for unit test if not None

  • stderr – redirect stderr for unit test if not None

  • use_run_cmd – to run the sphinx documentation with run_cmd and not os.system

  • filter_warning – see main_wrapper_tests

  • file_filter_pep8 – function to filter out files for which checking pep8 (see remove_extra_spaces_folder)

  • github_ownergithub owner of the package

  • existing_history – existing history, retrieves existing issues stored in that file

  • coverage_root – see main_wrapper_tests

  • direct_callgenerate_help_sphinx

  • fexclude – function which tells which file not to copy in the folder used to build the documentation

  • skip_issues – skip a given list of issues when building the history

Returns

True (an option was processed) or False, the file setup.py should call function setup

The command build_script is used, the flag --private can be used to avoid producing scripts to publish the module on Pypi.

An example for default_engine_paths:

default_engine_paths = {
    "windows": {
        "__PY35__": None,
        "__PY37_X64__": "c:\Python37_x64",
        "__PY36_X64__": "c:\Python36_x64",
        "__PY35_X64__": "c:\Python35_x64",
        "__PY27_X64__": "c:\Anaconda2",
    },
}

Parameters coverage_options, coverage_exclude_lines, copy_add_ext were added for function main_wrapper_tests. Parameter unittest_modules accepts a list of string and 2-uple. If it is a 2-uple, the first string is used to convert Python 3 code into Python 2 (in case the local folder is different from the module name), the second string is used to add local path to the variable PYTHON_PATH. If it is a single string, it means both name strings are equal. Parameters func_sphinx_begin and func_sphinx_end were added to pre-process or post-process the documentation. Parameter additional_notebook_path was added to specify some additional paths when preparing the script auto_cmd_notebook.bat. Parameters layout, nbformats were added for function generate_help_sphinx. The coverage computation can be disable by specifying coverage_options["disable_coverage"] = True. Parameter covtoken was added to post the coverage report to codecov. Option -e and -g were added to filter file by regular expressions (in with e, out with g).

Changed in version 1.8: Parameters coverage_root, direct_call were added.

source on GitHub

pyquickhelper.pycode.setup_helper.process_standard_options_for_setup_help(argv)[source][source]

Prints the added options available through this module.

source on GitHub

pyquickhelper.pycode.setup_helper.run_pylint_for_setup(folder, pattern='.*[.]py$', neg_pattern=None, verbose=False, pylint_ignore=None, fLOG=<function noLOG>)[source][source]

Applies pylint on subfolder folder.

Parameters
  • folder – folder where to look

  • pattern – file to checks

  • neg_pattern – negative pattern

@parm pylint_ignore ignore these pylint warnings or errors

Parameters
  • verbose – verbose

  • fLOG – logging function

source on GitHub

pyquickhelper.pycode.setup_helper.run_unittests_for_setup(file_or_folder, skip_function=<function default_skip_function>, setup_params=None, only_setup_hook=False, coverage_options=None, coverage_exclude_lines=None, additional_ut_path=None, covtoken=None, hook_print=True, stdout=None, stderr=None, filter_warning=None, dump_coverage=None, add_coverage_folder=None, coverage_root='src', fLOG=<function noLOG>)[source][source]

Runs the unit tests and computes the coverage, stores the results in _doc/sphinxdoc/source/coverage assuming the module follows the same design as pyquickhelper.

Parameters

See function main_wrapper_tests. The coverage computation can be disabled by specifying coverage_options["disable_coverage"] = True. covtoken was added to post the coverage report to codecov. Parameter dump_coverage dumps the unit test coverage in another location.

Changed in version 1.8: Parameter coverage_root was added.

source on GitHub

pyquickhelper.pycode.setup_helper.standard_help_for_setup(argv, file_or_folder, project_var_name, module_name=None, extra_ext=None, add_htmlhelp=False, copy_add_ext=None, nbformats=('ipynb', 'html', 'python', 'rst', 'slides', 'pdf'), layout=None, use_run_cmd=False, direct_call=False, fexclude=None, fLOG=<function noLOG>)[source][source]

Standard function which generates help assuming they follow the same design as pyquickhelper.

Parameters
  • argv – it should be sys.argv

  • file_or_folder – file setup.py or folder which contains it

  • project_var_name – display name of the module

  • module_name – module name, None if equal to project_var_name (import <module_name>)

  • extra_ext – extra file extension to process (ex ["doc"])

  • add_htmlhelp – run HTML Help too (only on Windows)

  • copy_add_ext – additional extension of files to copy

  • nbformats – notebooks format to generate

  • layout – layout for the documentation, if None –> ["html", "pdf"]

  • use_run_cmd – use function run_cmd instead of os.system to build the documentation

  • direct_call – see generate_help_sphinx

  • fexclude – function which tells which file not to copy in the folder used to build the documentation

  • fLOG – logging function

The function outputs some information through function fLOG.

A page will be added for each extra file extension mentioned in extra_ext if some of these were found.

source on GitHub

pyquickhelper.pycode.setup_helper.write_module_scripts(folder, platform='linux', blog_list=None, default_engine_paths=None, command=None)[source][source]

Writes a couple of scripts which allow a user to be faster on some tasks or to easily get information about the module.

Parameters
  • folder – where to write the script

  • platform – platform

  • blog_list – blog list to follow, should be attribute __blog__ of the module

  • command – None to generate scripts for all commands or a value in [blog, doc].

  • default_engine_paths – default engines (or python distributions)

Returns

list of written scripts

The function produces the following files:

  • auto_rss_list.xml: list of rss stream to follow

  • auto_rss_database.db3: stores blog posts

  • auto_rss_server.py: runs a server which updates the scripts and runs a server. It also open the default browser.

  • auto_rss_server.(bat|sh): run auto_run_server.py, the file on Linux might be missing if there is an equivalent python script

How to generate auto_rss_server.py?

The following code generates the script auto_rss_local.py which runs a local server to read blog posts included in the documentation (it uses module pyrsslocal):

from pyquickhelper.pycode import write_module_scripts, __blog__
write_module_scripts(".", blog_list=__blog__, command="blog")

source on GitHub

pyquickhelper.pycode.setup_helper.write_pyproj(file_or_folder, location=None)[source][source]

Creates a project pyproj to work with PTVS (Python Tools for Visual Studio)

Parameters
  • file_or_folder – file setup.py or folder which contains it

  • location – if not None, stores the project into this folder

This functionality fails with Python 2.7 (encoding).

source on GitHub

pyquickhelper.pycode.setup_helper.write_version_for_setup(file_or_folder, exc=False)[source][source]

Extracts the version number, the function writes the files version.txt in this folder.

Parameters
  • file_or_folder – file setup.py or folder which contains it

  • exc – raises an exception if cannot look into git folder

Returns

version number

Changed in version 1.8: Parameter exc was added.

source on GitHub