module loghelper.flog

Short summary

module pyquickhelper.loghelper.flog

logging functionalities

The function fLOG (or fLOG) is used to logged everything into a log file.

from pyquickhelper.loghelper.flog import fLOG
fLOG(OutputPrint = True)                  # the logs are also displayed in the output stream
fLOG(LogPath     = "...")                 # chanages the path returned by GetPath
fLOG("un", "deux", 4, ["gt"])             # log everything in a log file

from pyquickhelper.loghelper.flog import GetPath ()
print GetPath()                           # return the log path (file temp_log.txt)

fLOG(LogPath = "c:/temp/log_path")        # change the log path, creates it if it does not exist

Functions

function

truncated documentation

_check_source

Check the existence of a file, downloads it if not existing.

_check_url_file

If url is an url, download the file and return the downloaded if it has already been downloaded, it is not downloaded …

_check_zip_file

This function tests if a file is a zip file (extension zip), if it is the case, it unzips it into another file and return …

_first_more_recent

Checks if the first file (opened url) is more recent of the second file (path).

_get_file_txt

build a filename knowing an url, same name but in default_path

_get_file_url

build a filename knowing an url

_this_fLOG

other name private to this module

download

Download a file to the folder path_unzip if not present, if the downloading is interrupted, the next time, it will start …

fLOG

Builds a message on a single line with the date, it deals with encoding issues.

fLOGFormat

Formats a message.

get_default_value_type

get_prefix

Returns a prefix for a file based on time.

get_relative_path

private function, return the relative path or absolute between a folder and a file, use relpath

GetLogFile

Returns a file name containing the log

GetPath

returns a path where the log file is stored.

GetSepLine

return always \n

guess_machine_parameter

determine many parameters on this machine

guess_type_list

guess the type of a list

guess_type_value

guess the type of a value

guess_type_value_type

init

initialisation

IsEmptyString

empty string or not?

load_content_file_with_encoding

try different encoding to load a file, tries utf8, latin1 and None

noLOG

does nothing

Print

if True, redirect everything which is displayed to the standard output

removedirs

remove all files and folder in folder

unzip

Unzips a file into the temporary folder, the function expects to have only one zipped file.

Documentation

logging functionalities

The function fLOG (or fLOG) is used to logged everything into a log file.

from pyquickhelper.loghelper.flog import fLOG
fLOG(OutputPrint = True)                  # the logs are also displayed in the output stream
fLOG(LogPath     = "...")                 # chanages the path returned by GetPath
fLOG("un", "deux", 4, ["gt"])             # log everything in a log file

from pyquickhelper.loghelper.flog import GetPath ()
print GetPath()                           # return the log path (file temp_log.txt)

fLOG(LogPath = "c:/temp/log_path")        # change the log path, creates it if it does not exist

Warning

This module inserts static variable in module sys. I was done to deal with several instances of the same module in earlier versions of python.

source on GitHub

pyquickhelper.loghelper.flog.GetLogFile(physical=False, filename=None)[source][source]

Returns a file name containing the log

Parameters
  • physical – use a physical file or not

  • filename – file name (if physical is True, default value is temp_log.txt)

Returns

a pointer to a log file

Return type

str

Raises

OSError – if this file cannot be created

source on GitHub

pyquickhelper.loghelper.flog.GetPath()[source][source]

returns a path where the log file is stored.

Returns

path to the logs

source on GitHub

pyquickhelper.loghelper.flog.GetSepLine()[source][source]

return always \n

source on GitHub

pyquickhelper.loghelper.flog.IsEmptyString(s)[source][source]

empty string or not?

Parameters

s – any string (str, None)

Returns

is it empty or not?

Return type

bool

Raises

PQHException – when a type is unexpected

source on GitHub

pyquickhelper.loghelper.flog.Print(redirect=True)[source][source]

if True, redirect everything which is displayed to the standard output

source on GitHub

pyquickhelper.loghelper.flog._check_source(fileurl, path_unzip, outfile, flatten=True, fLOG=<function noLOG>)[source][source]

Check the existence of a file, downloads it if not existing.

Parameters
  • fileurl – can be an url, a zip file, a text file

  • path_unzip – if None, unzip the file where it stands, otherwise, put it in path

  • outfile – if None, the function will assign a filename unless this parameter is specified

  • flatten – extract all files into the same directory

  • fLOG – logging function

Returns

a text file name

if it is:
  • an url: download it and copy it into default_path

  • a zipfile: beside the true file

  • a text file: do nothing

If the file has already been downloaded and unzipped, it is not done twice.

source on GitHub

pyquickhelper.loghelper.flog._check_url_file(url, path_download, outfile, fLOG=<function noLOG>)[source][source]

If url is an url, download the file and return the downloaded if it has already been downloaded, it is not downloaded again.

Parameters
  • url – url

  • path_download – download the file here

  • outfile – if None, the function will assign a filename unless this parameter is specified

  • fLOG – logging function

Returns

the filename

source on GitHub

pyquickhelper.loghelper.flog._check_zip_file(filename, path_unzip, outfile, flatten=True, fLOG=<function noLOG>)[source][source]

This function tests if a file is a zip file (extension zip), if it is the case, it unzips it into another file and return the new name, if the unzipped file already exists, the file is not unzipped a second time.

Parameters
  • filename – any filename (.zip or not), if txt, it has no effect

  • path_unzip – if None, unzip it where it stands, otherwise, place it into path

  • outfile – if None, the function will assign a filename unless this parameter is specified

  • flatten – unzip all files into the same directory

  • fLOG – logging function

Returns

the unzipped file or filename if the format was not zip

source on GitHub

pyquickhelper.loghelper.flog._first_more_recent(f1, path)[source][source]

Checks if the first file (opened url) is more recent of the second file (path).

Parameters
  • f1 – opened url

  • path – path name

Returns

boolean

source on GitHub

pyquickhelper.loghelper.flog._get_file_txt(zipname)[source][source]

build a filename knowing an url, same name but in default_path

Parameters

zipname – filename of the zip

Returns

filename

source on GitHub

pyquickhelper.loghelper.flog._get_file_url(url, path)[source][source]

build a filename knowing an url

Parameters
  • url – url

  • path – where to download the file

Returns

filename

source on GitHub

pyquickhelper.loghelper.flog._this_fLOG(*l, **p)[source][source]

other name private to this module

source on GitHub

pyquickhelper.loghelper.flog.download(httpfile, path_unzip=None, outfile=None, flatten=True, fLOG=None)[source][source]

Download a file to the folder path_unzip if not present, if the downloading is interrupted, the next time, it will start from where it stopped. Before downloading, the function creates a temporary file, which means the downloading has began. If the connection is lost, an exception is raised and the program stopped. Next time, the program will detect the existence of the temporary file and will start downloading from where it previously stopped. After it ends, the temporary file is removed.

Parameters
  • httpfile – (str) url

  • path_unzip – (str) path where to unzip the file, if None, choose GetPath ()

  • outfile – (str) if None, the function will assign a filename unless this parameter is specified

  • flatten – (bool) put all files in the same folder (forget subfolders)

  • fLOG – (str) logging function

Returns

local file name

source on GitHub

pyquickhelper.loghelper.flog.fLOG(*l, **p)[source][source]

Builds a message on a single line with the date, it deals with encoding issues.

Parameters
  • l – list of fields

  • p – dictionary of fields (see below)

Raises

OSError – When the log file cannot be created.

About parameter p:

  • if p contains OutputPrint, call Print(OutputPrint)

  • if p contains LogPath, it calls init(v)

  • if p contains LogFile, it changes the log file name (it creates a new one, the previous is closed).

  • if p contains LogPathAdd, it adds this path to the temporary file

  • if p contains Lock, it locks option OutputPrint

  • if p contains UnLock, it unlocks option OutputPrint

  • if p contains _pp, it uses pprint

Example:

fLOG (LogPath = "###", LogPathAdd = __file__, OutputPrint = True)

How to activate the logs?

The following instruction will do:

fLOG(OutputPrint=True)

To log everything into a file:

fLOG(OutputPrint=True, LogFile="log_file.txt")

Changed in version 1.8: Parameter OutputStream allows to print the message on a different stream.

source on GitHub

pyquickhelper.loghelper.flog.fLOGFormat(sep, *l, **p)[source][source]

Formats a message.

Parameters
  • sep – line separator

  • l – list of anything

  • p – dictioanry of anything

Returns

string

if _pp is True, the function uses pprint.

source on GitHub

pyquickhelper.loghelper.flog.get_default_value_type(ty, none=True)[source][source]
Parameters
  • ty – type in guess_type_value_type

  • none – if True and all values are empty, return None

Returns

a default value for this type

source on GitHub

pyquickhelper.loghelper.flog.get_prefix()[source][source]

Returns a prefix for a file based on time.

source on GitHub

pyquickhelper.loghelper.flog.get_relative_path(folder, file, exists=True, absolute=True)[source][source]

private function, return the relative path or absolute between a folder and a file, use relpath

Parameters
  • folder – folder

  • file – file

  • exists – check existence

  • absolute – if True return a path which starts from the root

Returns

relative path

Return type

str

source on GitHub

pyquickhelper.loghelper.flog.guess_machine_parameter()[source][source]
determine many parameters on this machine
  • machine name

  • user name

  • domain…

Returns

dictionary { name : value }

source on GitHub

pyquickhelper.loghelper.flog.guess_type_list(l, tolerance=0.01, none=True)[source][source]

guess the type of a list

Parameters
  • l – list

  • tolerance – let’s denote m as the frequency of the most representative type, and m2 the second one, if m2 > m * tolerance –> str

  • none – if True and all values are empty, return None

Returns

type, length (order of preference (int, float, str)) the parameter length has a meaning only for str result

source on GitHub

pyquickhelper.loghelper.flog.guess_type_value(x, none=None)[source][source]

guess the type of a value

Parameters
  • x – type

  • none – if True and all values are empty, return None

Returns

type

Warning

if an integer starts with a zero, then it is a string

source on GitHub

pyquickhelper.loghelper.flog.guess_type_value_type(none=True)[source][source]
Parameters

none – if True and all values are empty, return None

Returns

the list of types recognized by guess_type_value

source on GitHub

pyquickhelper.loghelper.flog.init(path=None, filename=None, create=True, path_add=None)[source][source]

initialisation

Parameters
  • path – new path, if path==*”###”*, then uses d:\temp\log_pyquickhelper if it exists or c:\temp\log_pyquickhelper if not

  • filename – new filename

  • create – force the creation

  • path_add – subfolder to append to the current folder

This function is also called when LogPath is specified while calling function fLOG.

source on GitHub

pyquickhelper.loghelper.flog.load_content_file_with_encoding(filename)[source][source]

try different encoding to load a file, tries utf8, latin1 and None

Parameters

filename – filename

Returns

couple (content, encoding)

source on GitHub

pyquickhelper.loghelper.flog.noLOG(*l, **p)[source][source]

does nothing

source on GitHub

pyquickhelper.loghelper.flog.removedirs(folder, silent=False, use_command_line=False)[source][source]

remove all files and folder in folder

Parameters
  • folder – folder

  • silent – silent mode or not

  • use_command_line – see below

Returns

list of not remove files or folders

Sometimes it fails due to PermissionError exception, in that case, you can try to remove the folder through the command line rmdir /q /s + <folder>. In that case, the function does not return the list of removed files but the output of the command line

source on GitHub

pyquickhelper.loghelper.flog.unzip(file, path_unzip=None, outfile=None, flatten=True, fLOG=<function noLOG>)[source][source]

Unzips a file into the temporary folder, the function expects to have only one zipped file.

Parameters
  • file – (str) zip files

  • path_unzip – (str) where to unzip the file, if None, choose GetPath ()

  • outfile – (str) if None, the function will assign a filename unless this parameter is specified

  • flatten – (bool) put all files in the same folder (forget subfolders)

Returns

expanded file name

source on GitHub