module filehelper.download_helper

Short summary

module pyquickhelper.filehelper.download_helper

A function to download the content of a url.

source on GitHub

Classes

class	truncated documentation
InternetException	Exception for the function get_url_content_timeout()

Functions

function	truncated documentation
_hash_url
get_url_content_timeout	Downloads a file from internet (by default, it assumes it is text information, otherwise, encoding should be None).
get_urls_content_timeout	Downloads data from urls (by default, it assumes it is text information, otherwise, encoding should be None).
local_url	Replaces the url by a local file in a folder or an environment variable if folder is None.

Documentation

A function to download the content of a url.

source on GitHub

exception pyquickhelper.filehelper.download_helper.InternetException

Bases: Exception

Exception for the function get_url_content_timeout

source on GitHub

pyquickhelper.filehelper.download_helper._hash_url(url)

pyquickhelper.filehelper.download_helper.get_url_content_timeout(url, timeout=10, output=None, encoding='utf8', raise_exception=True, chunk=None, fLOG=None)

Downloads a file from internet (by default, it assumes it is text information, otherwise, encoding should be None).

Parameters:

• url – (str) url

• timeout – (int) in seconds, after this time, the function drops an returns None, -1 for forever

• output – (str) if None, the content is stored in that file

• encoding – (str) utf8 by default, but if it is None, the returned information is binary

• raise_exception – (bool) True to raise an exception, False to send a warnings

• chunk – (int|None) save data every chunk (only if output is not None)

• fLOG – logging function (only applies when chunk is not None)

Returns:

content of the url

If the function automatically detects that the downloaded data is in gzip format, it will decompress it.

The function raises the exception InternetException.

source on GitHub

pyquickhelper.filehelper.download_helper.get_urls_content_timeout(urls, timeout=10, folder=None, encoding=None, raise_exception=True, chunk=None, fLOG=None)

Downloads data from urls (by default, it assumes it is text information, otherwise, encoding should be None).

Parameters:

• urls – urls

• timeout – in seconds, after this time, the function drops an returns None, -1 for forever

• folder – if None, the content is stored in that file

• encoding – None by default, but if it is None, the returned information is binary

• raise_exception – True to raise an exception, False to send a warnings

• chunk – save data every chunk (only if output is not None)

• fLOG – logging function (only applies when chunk is not None)

Returns:

list of downloaded content

If the function automatically detects that the downloaded data is in gzip format, it will decompress it.

The function raises the exception InternetException.

source on GitHub

pyquickhelper.filehelper.download_helper.local_url(url, folder=None, envvar='REPO_LOCAL_URLS')

Replaces the url by a local file in a folder or an environment variable if folder is None.

Parameters:

• url – url to replace

• folder – local folder

• envvar – environment variable

Returns:

local file or url

source on GitHub