module filehelper.ftp_transfer_files

Inheritance diagram of pyquickhelper.filehelper.ftp_transfer_files

Short summary

module pyquickhelper.filehelper.ftp_transfer_files

Class to transfer files to a website using FTP, it only transfers updated files

source on GitHub

Classes

class

truncated documentation

FolderTransferFTP

This class aims at transfering a folder to a FTP website, it checks that a file was updated before transfering, …

FolderTransferFTPException

custom exception for FolderTransferFTP

Functions

function

truncated documentation

content_as_binary

determines if filename is binary or None before transfering it

Methods

method

truncated documentation

__init__

__str__

usual

close_stream

Closes a stream opened by preprocess_before_transfering().

iter_eligible_files

Iterates on eligible file for transfering (if they have been modified).

preprocess_before_transfering

Applies some preprocessing to the file to transfer. It adds the footer for example. It returns a stream …

start_transfering

Starts transfering files to a remote FTP website.

update_status

Updates the status of a file.

Documentation

Class to transfer files to a website using FTP, it only transfers updated files

source on GitHub

class pyquickhelper.filehelper.ftp_transfer_files.FolderTransferFTP(file_tree_node, ftp_transfer, file_status, root_local=None, root_web=None, footer_html=None, content_filter=None, is_binary=<function content_as_binary>, text_transform=None, filter_out=None, exc=False, force_allow=None, fLOG=<function noLOG>)[source]

Bases: object

This class aims at transfering a folder to a FTP website, it checks that a file was updated before transfering, TransferFTP .

Transfer updated files to a website

The following code shows how to transfer the content of a folder to website through FTP protocol.

ftn  = FileTreeNode("c:/somefolder")
ftp  = TransferFTP("ftp.website.fr", "login", "password", fLOG=print)
fftp = FolderTransferFTP (ftn, ftp, "status_file.txt",
        root_web = "/www/htdocs/app/pyquickhelper/helpsphinx")

fftp.start_transfering()
ftp.close()

The following example is more complete:

import sys, os
from pyquickhelper.filehelper import TransferFTP, FileTreeNode, FolderTransferFTP
import keyring

user = keyring.get_password("webtransfer", "user")
pwd = keyring.get_password("webtransfer", "pwd")

ftp = TransferFTP("ftp.website.fr", user, pwd, fLOG=print)

location = r"local_location/GitHub/%s/dist/html"
this = os.path.abspath(os.path.dirname(__file__))
rootw = "/root/subfolder/%s/helpsphinx"

for module in ["pyquickhelper", "pyensae"] :
    root = location % module

    # documentation
    sfile = os.path.join(this, "status_%s.txt" % module)
    ftn  = FileTreeNode(root)
    fftp = FolderTransferFTP (ftn, ftp, sfile,
                        root_web = rootw % module,
                        fLOG=print)

    fftp.start_transfering()

    # setup, wheels
    ftn  = FileTreeNode(os.path.join(root,".."), filter = lambda root, path, f, dir: not dir)
    fftp = FolderTransferFTP (ftn, ftp, sfile,
                        root_web = (rootw % module).replace("helpsphinx",""),
                        fLOG=print)

    fftp.start_transfering()

ftp.close()

source on GitHub

Parameters:
  • file_tree_nodeFileTreeNode

  • ftp_transferTransferFTP

  • file_status – file keeping the status for each file (date, hash of the content for the last upload)

  • root_local – local root

  • root_web – remote root on the website

  • footer_html – append this HTML code to any uploaded page (such a javascript code to count the audience) at the end of the file (before tag </body>)

  • content_filter – function which transform the content if a specific string is found in the file, if the result is None, it raises an exception indicating the file cannot be transfered (applies only on text files)

  • is_binary – function which determines if content of a files is binary or not

  • text_transform – function to transform the content of a text file before uploading it

  • filter_out – regular expression to exclude some files, it can also be a function.

  • exc – raise exception if not able to transfer

  • force_allow – the class does not transfer a file containing a set of specific strings except if they are in the list

  • fLOG – logging function

Function text_transform(self, filename, content) returns the modified content.

If filter_out is a function, the signature is:

def filter_out(full_file_name, filename):
    # ...
    return True # if the file is filtered out, False otherwise

Function filter_out receives another parameter (filename) to give more information when raising an exception.

source on GitHub

__init__(file_tree_node, ftp_transfer, file_status, root_local=None, root_web=None, footer_html=None, content_filter=None, is_binary=<function content_as_binary>, text_transform=None, filter_out=None, exc=False, force_allow=None, fLOG=<function noLOG>)[source]
Parameters:
  • file_tree_nodeFileTreeNode

  • ftp_transferTransferFTP

  • file_status – file keeping the status for each file (date, hash of the content for the last upload)

  • root_local – local root

  • root_web – remote root on the website

  • footer_html – append this HTML code to any uploaded page (such a javascript code to count the audience) at the end of the file (before tag </body>)

  • content_filter – function which transform the content if a specific string is found in the file, if the result is None, it raises an exception indicating the file cannot be transfered (applies only on text files)

  • is_binary – function which determines if content of a files is binary or not

  • text_transform – function to transform the content of a text file before uploading it

  • filter_out – regular expression to exclude some files, it can also be a function.

  • exc – raise exception if not able to transfer

  • force_allow – the class does not transfer a file containing a set of specific strings except if they are in the list

  • fLOG – logging function

Function text_transform(self, filename, content) returns the modified content.

If filter_out is a function, the signature is:

def filter_out(full_file_name, filename):
    # ...
    return True # if the file is filtered out, False otherwise

Function filter_out receives another parameter (filename) to give more information when raising an exception.

source on GitHub

__str__()[source]

usual

source on GitHub

close_stream(stream)[source]

Closes a stream opened by preprocess_before_transfering.

Parameters:

stream – stream to close

source on GitHub

iter_eligible_files()[source]

Iterates on eligible file for transfering (if they have been modified).

Returns:

iterator on file name

source on GitHub

preprocess_before_transfering(path, force_binary=False, force_allow=None)[source]

Applies some preprocessing to the file to transfer. It adds the footer for example. It returns a stream which should be closed by using method close_stream.

Parameters:
  • path – file name

  • force_binary – impose a binary transfer

  • force_allow – allow these strings even if they seem to be credentials

Returns:

binary stream, size

Bypass utf-8 encoding checking when the extension is .rst.txt.

source on GitHub

start_transfering(max_errors=20, delay=None)[source]

Starts transfering files to a remote FTP website.

Parameters:
  • max_errors – stops after this number of errors

  • delay – delay between two files

Returns:

list of transferred FileInfo

Raises:

FolderTransferFTPException – the class raises an exception (FolderTransferFTPException) more than max_errors issues happened

source on GitHub

update_status(file)[source]

Updates the status of a file.

Parameters:

file – filename

Returns:

FileInfo

source on GitHub

exception pyquickhelper.filehelper.ftp_transfer_files.FolderTransferFTPException[source]

Bases: Exception

custom exception for FolderTransferFTP

source on GitHub

pyquickhelper.filehelper.ftp_transfer_files.content_as_binary(filename)[source]

determines if filename is binary or None before transfering it

Parameters:

filename – filename

Returns:

boolean

source on GitHub

pyquickhelper.filehelper.ftp_transfer_files.random() x in the interval [0, 1).[source]