module loghelper.flog

Short summary

module pyquickhelper.loghelper.flog

logging functionalities

The function fLOG (or fLOG) is used to logged everything into a log file.

from pyquickhelper.loghelper.flog import fLOG
fLOG(OutputPrint = True)                  # the logs are also displayed in the output stream
fLOG(LogPath     = "...")                 # chanages the path returned by GetPath
fLOG("un", "deux", 4, ["gt"])             # log everything in a log file

from pyquickhelper.loghelper.flog import GetPath ()
print GetPath()                           # return the log path (file temp_log.txt)

fLOG(LogPath = "c:/temp/log_path")        # change the log path, creates it if it does not exist

Functions

function	truncated documentation
_check_source	Check the existence of a file, downloads it if not existing.
_check_url_file	If url is an url, download the file and return the downloaded if it has already been downloaded, it is not downloaded …
_check_zip_file	This function tests if a file is a zip file (extension zip), if it is the case, it unzips it into another file and return …
_first_more_recent	Checks if the first file (opened url) is more recent of the second file (path).
_get_file_txt	build a filename knowing an url, same name but in default_path
_get_file_url	build a filename knowing an url
_this_fLOG	Other name private to this module.
download	Download a file to the folder path_unzip if not present, if the downloading is interrupted, the next time, it will start …
fLOG	Builds a message on a single line with the date, it deals with encoding issues.
fLOGFormat	Formats a message.
get_default_value_type
get_prefix	Returns a prefix for a file based on time.
get_relative_path	private function, return the relative path or absolute between a folder and a file, use relpath …
GetLogFile	Returns a file name containing the log
GetPath	returns a path where the log file is stored.
GetSepLine	Always returns \n
guess_machine_parameter	Determines many parameters on this machine:
guess_type_list	guess the type of a list
guess_type_value	Guessees the type of a value.
guess_type_value_type
init	initialisation
IsEmptyString	Empty string or not?
load_content_file_with_encoding	Tries different encoding to load a file, tries utf8, latin1 and None.
noLOG	does nothing
Print	if True, redirect everything which is displayed to the standard output
removedirs	Removes all files and folders in folder.
unzip	Unzips a file into the temporary folder, the function expects to have only one zipped file.

Documentation

logging functionalities

The function fLOG (or fLOG) is used to logged everything into a log file.

from pyquickhelper.loghelper.flog import fLOG
fLOG(OutputPrint = True)                  # the logs are also displayed in the output stream
fLOG(LogPath     = "...")                 # chanages the path returned by GetPath
fLOG("un", "deux", 4, ["gt"])             # log everything in a log file

from pyquickhelper.loghelper.flog import GetPath ()
print GetPath()                           # return the log path (file temp_log.txt)

fLOG(LogPath = "c:/temp/log_path")        # change the log path, creates it if it does not exist

Warning

This module inserts static variable in module sys. I was done to deal with several instances of the same module in earlier versions of python.

source on GitHub

pyquickhelper.loghelper.flog.GetLogFile(physical=False, filename=None)

Returns a file name containing the log

Parameters:

• physical – use a physical file or not

• filename – file name (if physical is True, default value is temp_log.txt)

Returns:

a pointer to a log file

Return type:

str

Raises:

OSError – if this file cannot be created

source on GitHub

pyquickhelper.loghelper.flog.GetPath()

returns a path where the log file is stored.

Returns:

path to the logs

source on GitHub

pyquickhelper.loghelper.flog.GetSepLine()

Always returns \n

source on GitHub

pyquickhelper.loghelper.flog.IsEmptyString(s)

Empty string or not?

Parameters:

s – any string (str, None)

Returns:

is it empty or not?

Return type:

bool

Raises:

PQHException – when a type is unexpected

source on GitHub

pyquickhelper.loghelper.flog.Print(redirect=True)

if True, redirect everything which is displayed to the standard output

source on GitHub

pyquickhelper.loghelper.flog._check_source(fileurl, path_unzip, outfile, flatten=True, fLOG=<function noLOG>)

Check the existence of a file, downloads it if not existing.

Parameters:

• fileurl – can be an url, a zip file, a text file

• path_unzip – if None, unzip the file where it stands, otherwise, put it in path

• outfile – if None, the function will assign a filename unless this parameter is specified

• flatten – extract all files into the same directory

• fLOG – logging function

Returns:

a text file name

if it is:

• an url: download it and copy it into default_path

• a zipfile: beside the true file

• a text file: do nothing

If the file has already been downloaded and unzipped, it is not done twice.

source on GitHub

pyquickhelper.loghelper.flog._check_url_file(url, path_download, outfile, fLOG=<function noLOG>)

If url is an url, download the file and return the downloaded if it has already been downloaded, it is not downloaded again.

Parameters:

• url – url

• path_download – download the file here

• outfile – if None, the function will assign a filename unless this parameter is specified

• fLOG – logging function

Returns:

the filename

source on GitHub

pyquickhelper.loghelper.flog._check_zip_file(filename, path_unzip, outfile, flatten=True, fLOG=<function noLOG>)

This function tests if a file is a zip file (extension zip), if it is the case, it unzips it into another file and return the new name, if the unzipped file already exists, the file is not unzipped a second time.

Parameters:

• filename – any filename (.zip or not), if txt, it has no effect

• path_unzip – if None, unzip it where it stands, otherwise, place it into path

• outfile – if None, the function will assign a filename unless this parameter is specified

• flatten – unzip all files into the same directory

• fLOG – logging function

Returns:

the unzipped file or filename if the format was not zip

source on GitHub

pyquickhelper.loghelper.flog._first_more_recent(f1, path)

Checks if the first file (opened url) is more recent of the second file (path).

Parameters:

• f1 – opened url

• path – path name

Returns:

boolean

source on GitHub

pyquickhelper.loghelper.flog._get_file_txt(zipname)

build a filename knowing an url, same name but in default_path

Parameters:

zipname – filename of the zip

Returns:

filename

source on GitHub

pyquickhelper.loghelper.flog._get_file_url(url, path)

build a filename knowing an url

Parameters:

• url – url

• path – where to download the file

Returns:

filename

source on GitHub

pyquickhelper.loghelper.flog._this_fLOG(*args, **kwargs)

Other name private to this module.

source on GitHub

pyquickhelper.loghelper.flog.download(httpfile, path_unzip=None, outfile=None, flatten=True, fLOG=None)

Download a file to the folder path_unzip if not present, if the downloading is interrupted, the next time, it will start from where it stopped. Before downloading, the function creates a temporary file, which means the downloading has began. If the connection is lost, an exception is raised and the program stopped. Next time, the program will detect the existence of the temporary file and will start downloading from where it previously stopped. After it ends, the temporary file is removed.

Parameters:

• httpfile – (str) url

• path_unzip – (str) path where to unzip the file, if None, choose GetPath ()

• outfile – (str) if None, the function will assign a filename unless this parameter is specified

• flatten – (bool) put all files in the same folder (forget subfolders)

• fLOG – (str) logging function

Returns:

local file name

source on GitHub

pyquickhelper.loghelper.flog.fLOG(*args, **kwargs)

Builds a message on a single line with the date, it deals with encoding issues.

Parameters:

• args – list of fields

• kwargs – dictionary of fields (see below)

Raises:

OSError – When the log file cannot be created.

About parameter p:

• if p contains OutputPrint, call Print(OutputPrint)

• if p contains LogPath, it calls init(v)

• if p contains LogFile, it changes the log file name (it creates a new one, the previous is closed).

• if p contains LogPathAdd, it adds this path to the temporary file

• if p contains Lock, it locks option OutputPrint

• if p contains UnLock, it unlocks option OutputPrint

• if p contains _pp, it uses pprint

Example:

fLOG (LogPath = "###", LogPathAdd = __file__, OutputPrint = True)

How to activate the logs?

The following instruction will do:

fLOG(OutputPrint=True)

To log everything into a file:

fLOG(OutputPrint=True, LogFile="log_file.txt")

Parameter OutputStream allows to print the message on a different stream.

source on GitHub

pyquickhelper.loghelper.flog.fLOGFormat(sep, *args, **kwargs)

Formats a message.

Parameters:

• sep – line separator

• args – list of anything

• kwargs – dictioanry of anything

Returns:

string

if _pp is True, the function uses pprint.

source on GitHub

pyquickhelper.loghelper.flog.get_default_value_type(ty, none=True)

Parameters:

• ty – type in guess_type_value_type

• none – if True and all values are empty, return None

Returns:

a default value for this type

source on GitHub

pyquickhelper.loghelper.flog.get_prefix()

Returns a prefix for a file based on time.

source on GitHub

pyquickhelper.loghelper.flog.get_relative_path(folder, file, exists=True, absolute=True)

private function, return the relative path or absolute between a folder and a file, use relpath

Parameters:

• folder – folder

• file – file

• exists – check existence

• absolute – if True return a path which starts from the root

Returns:

relative path

Return type:

str

source on GitHub

pyquickhelper.loghelper.flog.guess_machine_parameter()

Determines many parameters on this machine: - machine name - user name - domain…

Returns:

dictionary { name : value }

source on GitHub

pyquickhelper.loghelper.flog.guess_type_list(args, tolerance=0.01, none=True)

guess the type of a list

Parameters:

• args – list

• tolerance – let’s denote m as the frequency of the most representative type, and m2 the second one, if m2 > m * tolerance –> str

• none – if True and all values are empty, return None

Returns:

type, length (order of preference (int, float, str)) the parameter length has a meaning only for str result

source on GitHub

pyquickhelper.loghelper.flog.guess_type_value(x, none=None)

Guessees the type of a value.

Parameters:

• x – type

• none – if True and all values are empty, return None

Returns:

type

Warning

if an integer starts with a zero, then it is a string

source on GitHub

pyquickhelper.loghelper.flog.guess_type_value_type(none=True)

Parameters:

none – if True and all values are empty, return None

Returns:

the list of types recognized by guess_type_value

source on GitHub

pyquickhelper.loghelper.flog.init(path=None, filename=None, create=True, path_add=None)

initialisation

Parameters:

• path – new path, if path==*”###”*, then uses d:\temp\log_pyquickhelper if it exists or c:\temp\log_pyquickhelper if not

• filename – new filename

• create – force the creation

• path_add – subfolder to append to the current folder

This function is also called when LogPath is specified while calling function fLOG.

source on GitHub

pyquickhelper.loghelper.flog.load_content_file_with_encoding(filename)

Tries different encoding to load a file, tries utf8, latin1 and None.

Parameters:

filename – filename

Returns:

couple (content, encoding)

source on GitHub

pyquickhelper.loghelper.flog.noLOG(*args, **kwargs)

does nothing

source on GitHub

pyquickhelper.loghelper.flog.removedirs(folder, silent=False, use_command_line=False)

Removes all files and folders in folder.

Parameters:

• folder – folder

• silent – silent mode or not

• use_command_line – see below

Returns:

list of not remove files or folders

Sometimes it fails due to PermissionError exception, in that case, you can try to remove the folder through the command line rmdir /q /s + <folder>. In that case, the function does not return the list of removed files but the output of the command line

source on GitHub

pyquickhelper.loghelper.flog.unzip(file, path_unzip=None, outfile=None, flatten=True, fLOG=<function noLOG>)

Unzips a file into the temporary folder, the function expects to have only one zipped file.

Parameters:

• file – (str) zip files

• path_unzip – (str) where to unzip the file, if None, choose GetPath ()

• outfile – (str) if None, the function will assign a filename unless this parameter is specified

• flatten – (bool) put all files in the same folder (forget subfolders)

Returns:

expanded file name

source on GitHub