module filehelper.download_helper

Inheritance diagram of pyquickhelper.filehelper.download_helper

Short summary

module pyquickhelper.filehelper.download_helper

A function to download the content of a url.

source on GitHub

Classes

class

truncated documentation

InternetException

Exception for the function get_url_content_timeout()

Functions

function

truncated documentation

_hash_url

get_url_content_timeout

Downloads a file from internet (by default, it assumes it is text information, otherwise, encoding should be None).

get_urls_content_timeout

Downloads data from urls (by default, it assumes it is text information, otherwise, encoding should be None).

local_url

Replaces the url by a local file in a folder or an environment variable if folder is None.

Documentation

A function to download the content of a url.

source on GitHub

exception pyquickhelper.filehelper.download_helper.InternetException[source]

Bases: Exception

Exception for the function get_url_content_timeout

source on GitHub

pyquickhelper.filehelper.download_helper._hash_url(url)[source]
pyquickhelper.filehelper.download_helper.get_url_content_timeout(url, timeout=10, output=None, encoding='utf8', raise_exception=True, chunk=None, fLOG=None)[source]

Downloads a file from internet (by default, it assumes it is text information, otherwise, encoding should be None).

Parameters
  • url – (str) url

  • timeout – (int) in seconds, after this time, the function drops an returns None, -1 for forever

  • output – (str) if None, the content is stored in that file

  • encoding – (str) utf8 by default, but if it is None, the returned information is binary

  • raise_exception – (bool) True to raise an exception, False to send a warnings

  • chunk – (int|None) save data every chunk (only if output is not None)

  • fLOG – logging function (only applies when chunk is not None)

Returns

content of the url

If the function automatically detects that the downloaded data is in gzip format, it will decompress it.

The function raises the exception InternetException.

source on GitHub

pyquickhelper.filehelper.download_helper.get_urls_content_timeout(urls, timeout=10, folder=None, encoding=None, raise_exception=True, chunk=None, fLOG=None)[source]

Downloads data from urls (by default, it assumes it is text information, otherwise, encoding should be None).

Parameters
  • urls – urls

  • timeout – in seconds, after this time, the function drops an returns None, -1 for forever

  • folder – if None, the content is stored in that file

  • encoding – None by default, but if it is None, the returned information is binary

  • raise_exception – True to raise an exception, False to send a warnings

  • chunk – save data every chunk (only if output is not None)

  • fLOG – logging function (only applies when chunk is not None)

Returns

list of downloaded content

If the function automatically detects that the downloaded data is in gzip format, it will decompress it.

The function raises the exception InternetException.

source on GitHub

pyquickhelper.filehelper.download_helper.local_url(url, folder=None, envvar='REPO_LOCAL_URLS')[source]

Replaces the url by a local file in a folder or an environment variable if folder is None.

Parameters
  • url – url to replace

  • folder – local folder

  • envvar – environment variable

Returns

local file or url

source on GitHub