Command lines

  1. Clean files

  2. Compares two files

  3. Decrypt a file

  4. Decrypt a string

  5. Encrypt a file

  6. Encrypt a string

  7. Explore the content of a directory

  8. Synchronize two folders

Clean files

The command line cleans files in a folder.

<<<

python -m pyquickhelper clean_files --help

>>>

--SCRIPT---m pyquickhelper clean_files --help
--OUT--
usage: clean_files [-h] [-f FOLDER] [-p POSREG] [-n NEGREG] [--op OP]

Cleans `\r` in files a folder and subfolders with a given extensions.
Backslashes are replaces by `/`. The regular expressions applies on the
relative path starting from *folder*. The following cleaning are available: *
`'CR'`: replaces `'\r\n'` by `'\n'` * `'pep8'`: applies *pep8* convention

optional arguments:
  -h, --help            show this help message and exit
  -f FOLDER, --folder FOLDER
                        folder to clean (default: .)
  -p POSREG, --posreg POSREG
                        regular expression to select files to process
                        (default: .*[.]((py)|(rst))$)
  -n NEGREG, --negreg NEGREG
                        regular expression to skip files to process (default:
                        .*[.]git/.*)
  --op OP               kind of cleaning to do, see below for the available
                        option (default: CR)


--ERR--
somewhere/workspace/pyquickhelper/pyquickhelper_UT_37_std/_doc/sphinxdoc/source/pyquickhelper/helpgen/sphinxm_mock_app.py:393: RemovedInSphinx30Warning: sphinx.application.CONFIG_FILENAME is now deprecated. Please refer CHANGES to grasp the changes of Sphinx API.
  new_extensions=new_extensions)
somewhere/workspace/pyquickhelper/pyquickhelper_UT_37_std/_doc/sphinxdoc/source/pyquickhelper/helpgen/sphinxm_mock_app.py:370: RemovedInSphinx30Warning: sphinx.application.CONFIG_FILENAME is now deprecated. Please refer CHANGES to grasp the changes of Sphinx API.
  new_extensions=new_extensions)
--PATH--
None

(original entry : cli_helper.py:docstring of pyquickhelper.cli.cli_helper.cli_main_helper, line 50)

Compares two files

The command calls function create_visual_diff_through_html_files and produces a HTML page with shows the differences between two files. Example:

python -m pyquickhelper visual_diff -f <file1> -fi <file2> --browser=1 --page=diff.html

It works better with chrome.

<<<

python -m pyquickhelper visual_diff --help

>>>

--SCRIPT---m pyquickhelper visual_diff --help
--OUT--
usage: visual_diff [-h] [-f FILE1] [-fi FILE2] [-e ENCODING] [-p PAGE]
                   [-b BROWSER] [-n NOTEBOOK] [-c CONTEXT_SIZE]
                   [-i INLINE_VIEW]

Calls function :py:func:`create_visual_diff_through_html
<pyquickhelper.filehelper.visual_sync.create_visual_diff_through_html>` with
the content of two files. An example of the results is shown in blog post
:py:ref:`b-diffview`. The function now uses :py:func:`read_content_ufs
<pyquickhelper.filehelper.anyfhelper.read_content_ufs>` to retrieve the
content.

optional arguments:
  -h, --help            show this help message and exit
  -f FILE1, --file1 FILE1
                        first file (anything such as an url, a file, a string,
                        a stream) (default: None)
  -fi FILE2, --file2 FILE2
                        second file (anything such as an url, a file, a
                        string, a stream) (default: None)
  -e ENCODING, --encoding ENCODING
                        encoding (default: utf8)
  -p PAGE, --page PAGE  if not None, saves the results in file (default: )
  -b BROWSER, --browser BROWSER
                        open browser ? (default: False)
  -n NOTEBOOK, --notebook NOTEBOOK
                        if True, the function assumes the outcome will be
                        displayed from a notebook and does things accordingly
                        (default: False)
  -c CONTEXT_SIZE, --context_size CONTEXT_SIZE
                        to display everything (None) or just the changes > 0
                        (default: )
  -i INLINE_VIEW, --inline_view INLINE_VIEW
                        only for notebook, True: one column, False: two
                        columns (default: False)


--ERR--
somewhere/workspace/pyquickhelper/pyquickhelper_UT_37_std/_doc/sphinxdoc/source/pyquickhelper/helpgen/sphinxm_mock_app.py:393: RemovedInSphinx30Warning: sphinx.application.CONFIG_FILENAME is now deprecated. Please refer CHANGES to grasp the changes of Sphinx API.
  new_extensions=new_extensions)
somewhere/workspace/pyquickhelper/pyquickhelper_UT_37_std/_doc/sphinxdoc/source/pyquickhelper/helpgen/sphinxm_mock_app.py:370: RemovedInSphinx30Warning: sphinx.application.CONFIG_FILENAME is now deprecated. Please refer CHANGES to grasp the changes of Sphinx API.
  new_extensions=new_extensions)
--PATH--
None

(original entry : visual_sync.py:docstring of pyquickhelper.filehelper.visual_sync.create_visual_diff_through_html_files, line 24)

Decrypt a file

Decrypt a file from the command line.

decrypt_file –help

usage: decrypt_file [-h] source dest password

decrypt_file a file does not work well in Python 2.7 with pycryptodome

positional arguments:
  source      file to decrypt_file
  dest        location of the decrypt_fileed file
  password    password, usually an ascii string with 16x characters

optional arguments:
  -h, --help  show this help message and exit

(original entry : encryption_file_cli.py:docstring of pyquickhelper.cli.encryption_file_cli.decrypt_file, line 6)

Decrypt a string

Decrypts an encrypted string from the command line.

decrypt –help

usage: decrypt [-h] [-r REGEX] source dest password

decrypt a folder. For a second run, the program looks into file status to
avoid crypting same file gain, it does only modified files it does not work
well in Python 2.7 with pycryptodome.

positional arguments:
  source                folder to decrypt
  dest                  location of the decrypted files
  password              password, usually an ascii string with 16x characters

optional arguments:
  -h, --help            show this help message and exit
  -r REGEX, --regex REGEX
                        the script can retrieve only a subpart of the data
                        defined by a regular expression

(original entry : encryption_cli.py:docstring of pyquickhelper.cli.encryption_cli.decrypt, line 9)

Encrypt a file

Encrypt a file from the command line.

encrypt_file –help

usage: encrypt_file [-h] source dest password

encrypt_file a file does not work well in Python 2.7 with pycryptodome

positional arguments:
  source      file to encrypt_file
  dest        location of the encrypt_fileed file
  password    password, usually an ascii string with 16x characters

optional arguments:
  -h, --help  show this help message and exit

(original entry : encryption_file_cli.py:docstring of pyquickhelper.cli.encryption_file_cli.encrypt_file, line 6)

Encrypt a string

Encrypts a string from the command line.

encrypt –help

usage: encrypt [-h] [-s STATUS] [-m MAP] source dest password

encrypt a folder. For a second run, the program looks into file status to
avoid crypting same file gain, it does only modified files it does not work
well in Python 2.7 with pycryptodome.

positional arguments:
  source                folder to encrypt
  dest                  location of the encrypted files
  password              password, usually an ascii string with 16x characters

optional arguments:
  -h, --help            show this help message and exit
  -s STATUS, --status STATUS
                        to keep track of what was done
  -m MAP, --map MAP     mapping between raw files and crypted files

(original entry : encryption_cli.py:docstring of pyquickhelper.cli.encryption_cli.encrypt, line 9)

Explore the content of a directory

The command calls function explore_folder and makes the list of all files in a directory or all folders. Example:

python -m pyquickhelper ls -f _mynotebooks -r f -p .*[.]ipynb -n checkpoints -fu 1

It works better with chrome. An example to change file names:

python -m pyquickhelper ls -f myfolder -p .*[.]py -r f -n pycache -fu 1 -s test_(.*) -su unit_\1

Or another to automatically create git commands to rename files:

python -m pyquickhelper ls -f _mynotebooks -r f -p .*[.]ipynb -s "(.*)[.]ipynb" -su "git mv \1.ipynb \1~.ipynb"

<<<

python -m pyquickhelper ls --help

>>>

--SCRIPT---m pyquickhelper ls --help
--OUT--
usage: ls [-h] [-f FOLDER] [-p PATTERN] [-n NEG_PATTERN] [-fu FULLNAME]
          [-r RETURN_ONLY] [-re RECURSIVE] [-s SUB_PATTERN] [-su SUB_REPLACE]

Returns the list of files included in a folder and its subfolders. Returned
names can be modified if *sub_pattern* is specified.

optional arguments:
  -h, --help            show this help message and exit
  -f FOLDER, --folder FOLDER
                        (str) folder (default: None)
  -p PATTERN, --pattern PATTERN
                        (str) if None, get all files, otherwise, it is a
                        regular expression, the filename must verify (with the
                        folder if fullname is True) (default: )
  -n NEG_PATTERN, --neg_pattern NEG_PATTERN
                        (str) negative pattern (default: )
  -fu FULLNAME, --fullname FULLNAME
                        (bool) if True, include the subfolder while checking
                        the regex (pattern) (default: False)
  -r RETURN_ONLY, --return_only RETURN_ONLY
                        (str) to return folders and files (=None), only the
                        files (='f') or only the folders (*='d') <string>:10:
                        (WARNING/2) Inline emphasis start-string without end-
                        string. (default: )
  -re RECURSIVE, --recursive RECURSIVE
                        (bool) look into subfolders (default: True)
  -s SUB_PATTERN, --sub_pattern SUB_PATTERN
                        (str) replacements pattern, the output is then
                        modified accordingly to this regular expression
                        (default: )
  -su SUB_REPLACE, --sub_replace SUB_REPLACE
                        (str) if sub_pattern is specified, this second pattern
                        specifies how to replace (default: )


--ERR--
/var/lib/jenkins/workspace/pyquickhelper/pyquickhelper_UT_37_std/_doc/sphinxdoc/source/pyquickhelper/helpgen/sphinxm_mock_app.py:393: RemovedInSphinx30Warning: sphinx.application.CONFIG_FILENAME is now deprecated. Please refer CHANGES to grasp the changes of Sphinx API.
  new_extensions=new_extensions)
/var/lib/jenkins/workspace/pyquickhelper/pyquickhelper_UT_37_std/_doc/sphinxdoc/source/pyquickhelper/helpgen/sphinxm_mock_app.py:370: RemovedInSphinx30Warning: sphinx.application.CONFIG_FILENAME is now deprecated. Please refer CHANGES to grasp the changes of Sphinx API.
  new_extensions=new_extensions)
--PATH--
None

(original entry : synchelper.py:docstring of pyquickhelper.filehelper.synchelper.explore_folder, line 24)

Synchronize two folders

Synchronizes two folders from the command line.

pyq_sync –help

usage: synchronize_folder [-h] [--p1 P1] [--p2 P2] [-ha HASH_SIZE] [-r REPO1]
                          [-re REPO2] [-s SIZE_DIFFERENT] [-n NO_DELETION]
                          [-f FILTER] [-fi FILTER_COPY] [-a AVOID_COPY]
                          [-fil FILE_DATE] [-c COPY_1TO2] [-cr CREATE_DEST]

Synchronizes two folders (or copy if the second is empty), it only copies more
recent files. It can walk through a *git* repository or SVN. if `file_date` is
mentioned, the second folder is not explored. Only the modified files will be
taken into account (except for the first sync). .. _indexexref-ex0:
synchronize two folders: The following function synchronizes a folder with
another one on a USB drive or a network drive. To minimize the number of
access to the other location, it stores the status of the previous
synchronization in a file (`status_copy.txt` in the below example). Next time,
the function goes through the directory and sub-directories to synchronize and
only propagates the modifications which happened since the last modification.
The function `filter_copy` defines what file to synchronize or not.:: def
filter_copy(file): return "_don_t_synchronize_" not in file
synchronize_folder( "c:/mydata", "g:/mybackup", hash_size = 0, filter_copy =
filter_copy, file_date = "c:/status_copy.txt") The function is able to go
through 90.000 files and 90 Gb in 12 minutes (for an update).

optional arguments:
  -h, --help            show this help message and exit
  --p1 P1               (str) first path (default: None)
  --p2 P2               (str) second path (default: None)
  -ha HASH_SIZE, --hash_size HASH_SIZE
                        (bool) to check whether or not two files are different
                        (default: 1048576)
  -r REPO1, --repo1 REPO1
                        (bool) assuming the first folder is under SVN or GIT,
                        it uses pysvn to get the list of files (avoiding any
                        extra files) (default: False)
  -re REPO2, --repo2 REPO2
                        (bool) assuming the second folder is under SVN or GIT,
                        it uses pysvn to get the list of files (avoiding any
                        extra files) (default: False)
  -s SIZE_DIFFERENT, --size_different SIZE_DIFFERENT
                        (bool) if True, a file will be copied only if size are
                        different, otherwise, it will be copied if the first
                        file is more recent (default: True)
  -n NO_DELETION, --no_deletion NO_DELETION
                        (bool) if a file is found in the second folder and not
                        in the first one, if will be removed unless
                        no_deletion is True (default: False)
  -f FILTER, --filter FILTER
                        (str) None to accept every file, a string if it is a
                        regular expression, a function for something more
                        complex: function (fullname) --> True (every file is
                        considered in lower case), (use *py.re.search)
                        (default: None)
  -fi FILTER_COPY, --filter_copy FILTER_COPY
                        (str) None to accept every file, a string if it is a
                        regular expression, a function for something more
                        complex: function (fullname) --> True (default: None)
  -a AVOID_COPY, --avoid_copy AVOID_COPY
                        (bool) if True, just return the list of files which
                        should be copied but does not do the copy (default:
                        False)
  -fil FILE_DATE, --file_date FILE_DATE
                        (str) filename which contains information about when
                        the last sync was done (default: None)
  -c COPY_1TO2, --copy_1to2 COPY_1TO2
                        (bool) only copy files from p1 to p2 (default: False)
  -cr CREATE_DEST, --create_dest CREATE_DEST
                        (bool) create destination directory if not exist
                        (default: False)

(original entry : pyq_sync_cli.py:docstring of pyquickhelper.cli.pyq_sync_cli.pyq_sync, line 6)