ftpsync package¶
ftpsync.resources module¶
(c) 2012-2024 Martin Wendt; see https://github.com/mar10/pyftpsync Licensed under the MIT license: https://www.opensource.org/licenses/mit-license.php
- class ftpsync.resources.DirectoryEntry(target, rel_path, name, size, mtime, unique)[source]¶
Bases:
_Resource
- as_string(other_resource=None)¶
- classification¶
(set by synchronizer._classify_entry()).
- Type:
str
- classify(peer_dir_meta)¶
Classify this entry as ‘new’, ‘unmodified’, or ‘modified’.
- get_rel_path()¶
- get_sync_info(key=None)¶
- is_file()¶
- is_local()¶
- mtime¶
Current file modification time stamp (for FTP targets adjusted using metadata information).
- Type:
float
- mtime_org¶
Modification time stamp (as reported by source FTP server).
- Type:
float
- name¶
File name.
- Type:
str
- ps_mtime¶
File modification time stamp at the time of last sync operation
- Type:
float
- ps_size¶
File size at the time of last sync operation
- Type:
int
- ps_utime¶
Time stamp of last sync operation
- Type:
float
- set_sync_info(local_file)¶
- size¶
Current file size
- Type:
int
- target¶
Parent target object.
- Type:
_Target
- unique¶
Unique id of file/directory.
- Type:
str
- was_deleted¶
May be set to true by synchronizer
- Type:
bool
- class ftpsync.resources.EntryPair(local, remote)[source]¶
Bases:
object
- property any_entry¶
Return the local entry (or the remote entry if it is None).
- is_dir¶
bool:
- local_classification¶
str:
- name¶
str:
- operation¶
str:
- re_class_reason¶
str:
- rel_path¶
str:
- remote_classification¶
str:
- class ftpsync.resources.FileEntry(target, rel_path, name, size, mtime, unique)[source]¶
Bases:
_Resource
- EPS_TIME = 2.01¶
- as_string(other_resource=None)¶
- classification¶
(set by synchronizer._classify_entry()).
- Type:
str
- classify(peer_dir_meta)¶
Classify this entry as ‘new’, ‘unmodified’, or ‘modified’.
- get_rel_path()¶
- get_sync_info(key=None)[source]¶
Get mtime/size when this resource was last synchronized with remote.
- is_dir()¶
- is_local()¶
- mtime¶
Current file modification time stamp (for FTP targets adjusted using metadata information).
- Type:
float
- mtime_org¶
Modification time stamp (as reported by source FTP server).
- Type:
float
- name¶
File name.
- Type:
str
- ps_mtime¶
File modification time stamp at the time of last sync operation
- Type:
float
- ps_size¶
File size at the time of last sync operation
- Type:
int
- ps_utime¶
Time stamp of last sync operation
- Type:
float
- set_sync_info(local_file)¶
- size¶
Current file size
- Type:
int
- target¶
Parent target object.
- Type:
_Target
- unique¶
Unique id of file/directory.
- Type:
str
- was_deleted¶
May be set to true by synchronizer
- Type:
bool
- class ftpsync.resources._Resource(target, rel_path, name, size, mtime, unique)[source]¶
Bases:
object
Common base class for files and directories.
- classification¶
(set by synchronizer._classify_entry()).
- Type:
str
- mtime¶
Current file modification time stamp (for FTP targets adjusted using metadata information).
- Type:
float
- mtime_org¶
Modification time stamp (as reported by source FTP server).
- Type:
float
- name¶
File name.
- Type:
str
- ps_mtime¶
File modification time stamp at the time of last sync operation
- Type:
float
- ps_size¶
File size at the time of last sync operation
- Type:
int
- ps_utime¶
Time stamp of last sync operation
- Type:
float
- size¶
Current file size
- Type:
int
- target¶
Parent target object.
- Type:
_Target
- unique¶
Unique id of file/directory.
- Type:
str
- was_deleted¶
May be set to true by synchronizer
- Type:
bool
ftpsync.synchronizers module¶
(c) 2012-2024 Martin Wendt; see https://github.com/mar10/pyftpsync Licensed under the MIT license: https://www.opensource.org/licenses/mit-license.php
- class ftpsync.synchronizers.BaseSynchronizer(local, remote, options)[source]¶
Bases:
object
Synchronizes two target instances in dry_run mode (also base class for other synchronizers).
- _before_sync(entry)[source]¶
Called by the synchronizer for each entry.
Return False to prevent the synchronizer’s default action.
- _dry_run_action(action)[source]¶
Called in dry-run mode after call to _log_action() and before exiting function.
- _resolve_shortcuts = {'l': 'local', 'r': 'remote', 's': 'skip'}¶
- _sync_dir()[source]¶
Traverse the local folder structure and remote peers.
This is the core algorithm that generates calls to self.sync_XXX() handler methods. _sync_dir() is called by self.run().
- _test_match_or_print(entry)[source]¶
Return True if entry matches filter. Otherwise print ‘skip’ and return False.
- is_script¶
True if this synchronizer is used by a command line script (e.g. pyftpsync.exe)
- Type:
bool
- on_conflict(pair)[source]¶
Called when resources have been modified on local and remote.
- Returns:
False to prevent visiting of children (if pair is a directory)
- on_mismatch(pair)[source]¶
Called for pairs that don’t match match and exclude filters.
A synchronizer may decide to implement –delete-unmatched and set pair.entry.was_deleted accordingly.
- re_classify_pair(pair)[source]¶
Allow derrived classes to override default classification and operation.
- Returns:
False to prevent default operation.
- resolve_all¶
Conflict resolution strategy
- Type:
str
- class ftpsync.synchronizers.BiDirSynchronizer(local, remote, options)[source]¶
Bases:
BaseSynchronizer
Synchronizer that performs up- and download operations as required.
Newer files override unmodified older files
When both files are newer than last sync -> conflict! Conflicts may be resolved by these options:
--resolve=old: use the older file --resolve=new: use the newer file --resolve=local: use the local file --resolve=remote: use the remote file --resolve=ask: prompt user for decision
When a file is missing: check if it existed in the past. If so, delete it. Otherwise copy it.
In order to know if a file was modified, deleted, or created since last sync, we store a snapshot of the directory in the local directory.
- _before_sync(entry)¶
Called by the synchronizer for each entry.
Return False to prevent the synchronizer’s default action.
- _compare_file(local, remote)¶
Byte compare two files (early out on first difference).
- _copy_file(src, dest, file_entry)¶
- _copy_recursive(src, dest, dir_entry)¶
- _dry_run_action(action)¶
Called in dry-run mode after call to _log_action() and before exiting function.
- _inc_stat(name, ofs=1)¶
- _interactive_resolve(pair)[source]¶
Return ‘local’, ‘remote’, or ‘skip’ to use local, remote resource or skip.
- _log_action(action, status, symbol, entry, min_level=3)¶
- _match(entry)¶
- _remove_dir(dir_entry)¶
- _remove_file(file_entry)¶
- _resolve_shortcuts = {'l': 'local', 'r': 'remote', 's': 'skip'}¶
- _sync_dir()¶
Traverse the local folder structure and remote peers.
This is the core algorithm that generates calls to self.sync_XXX() handler methods. _sync_dir() is called by self.run().
- _test_match_or_print(entry)¶
Return True if entry matches filter. Otherwise print ‘skip’ and return False.
- _tick()¶
Write progress info and move cursor to beginning of line.
- close()¶
- error_count() int ¶
- get_stats()¶
- is_script¶
True if this synchronizer is used by a command line script (e.g. pyftpsync.exe)
- Type:
bool
- on_error(exc, pair)¶
Called for pairs that don’t match match and exclude filters.
- problem_count() int ¶
- re_classify_pair(pair)¶
Allow derrived classes to override default classification and operation.
- Returns:
False to prevent default operation.
- resolve_all¶
Conflict resolution strategy
- Type:
str
- ftpsync.synchronizers.DEFAULT_OMIT = ['.DS_Store', '.git', '.hg', '.svn', '#recycle']¶
Default for –exclude CLI option Note: DirMetadata.META_FILE_NAME and LOCK_FILE_NAME are always ignored
- class ftpsync.synchronizers.DownloadSynchronizer(local, remote, options)[source]¶
Bases:
BiDirSynchronizer
- _before_sync(entry)¶
Called by the synchronizer for each entry.
Return False to prevent the synchronizer’s default action.
- _compare_file(local, remote)¶
Byte compare two files (early out on first difference).
- _copy_file(src, dest, file_entry)¶
- _copy_recursive(src, dest, dir_entry)¶
- _dry_run_action(action)¶
Called in dry-run mode after call to _log_action() and before exiting function.
- _inc_stat(name, ofs=1)¶
- _interactive_resolve(pair)[source]¶
Return ‘local’, ‘remote’, or ‘skip’ to use local, remote resource or skip.
- _log_action(action, status, symbol, entry, min_level=3)¶
- _match(entry)¶
- _print_pair_diff(pair)¶
- _remove_dir(dir_entry)¶
- _remove_file(file_entry)¶
- _resolve_shortcuts = {'l': 'local', 'r': 'remote', 's': 'skip'}¶
- _sync_dir()¶
Traverse the local folder structure and remote peers.
This is the core algorithm that generates calls to self.sync_XXX() handler methods. _sync_dir() is called by self.run().
- _test_match_or_print(entry)¶
Return True if entry matches filter. Otherwise print ‘skip’ and return False.
- _tick()¶
Write progress info and move cursor to beginning of line.
- close()¶
- error_count() int ¶
- get_stats()¶
- is_script¶
True if this synchronizer is used by a command line script (e.g. pyftpsync.exe)
- Type:
bool
- on_conflict(pair)¶
Return False to prevent visiting of children.
- on_copy_remote(pair)¶
Called when the remote resource should be copied to local.
- on_equal(pair)¶
Called for (unmodified, unmodified) pairs.
- on_error(exc, pair)¶
Called for pairs that don’t match match and exclude filters.
- on_mismatch(pair)[source]¶
Called for pairs that don’t match match and exclude filters.
If –delete-unmatched is on, remove the local resource.
- on_need_compare(pair)¶
Re-classify pair based on file attributes and options.
- problem_count() int ¶
- re_classify_pair(pair)[source]¶
Allow derrived classes to override default classification and operation.
- Returns:
False to prevent default operation.
- resolve_all¶
Conflict resolution strategy
- Type:
str
- class ftpsync.synchronizers.UploadSynchronizer(local, remote, options)[source]¶
Bases:
BiDirSynchronizer
- _before_sync(entry)¶
Called by the synchronizer for each entry.
Return False to prevent the synchronizer’s default action.
- _compare_file(local, remote)¶
Byte compare two files (early out on first difference).
- _copy_file(src, dest, file_entry)¶
- _copy_recursive(src, dest, dir_entry)¶
- _dry_run_action(action)¶
Called in dry-run mode after call to _log_action() and before exiting function.
- _inc_stat(name, ofs=1)¶
- _interactive_resolve(pair)[source]¶
Return ‘local’, ‘remote’, or ‘skip’ to use local, remote resource or skip.
- _log_action(action, status, symbol, entry, min_level=3)¶
- _match(entry)¶
- _print_pair_diff(pair)¶
- _remove_dir(dir_entry)¶
- _remove_file(file_entry)¶
- _resolve_shortcuts = {'l': 'local', 'r': 'remote', 's': 'skip'}¶
- _sync_dir()¶
Traverse the local folder structure and remote peers.
This is the core algorithm that generates calls to self.sync_XXX() handler methods. _sync_dir() is called by self.run().
- _test_match_or_print(entry)¶
Return True if entry matches filter. Otherwise print ‘skip’ and return False.
- _tick()¶
Write progress info and move cursor to beginning of line.
- close()¶
- error_count() int ¶
- get_stats()¶
- is_script¶
True if this synchronizer is used by a command line script (e.g. pyftpsync.exe)
- Type:
bool
- on_conflict(pair)¶
Return False to prevent visiting of children.
- on_copy_local(pair)¶
Called when the local resource should be copied to remote.
- on_equal(pair)¶
Called for (unmodified, unmodified) pairs.
- on_error(exc, pair)¶
Called for pairs that don’t match match and exclude filters.
- on_mismatch(pair)[source]¶
Called for pairs that don’t match match and exclude filters.
If –delete-unmatched is on, remove the remote resource.
- on_need_compare(pair)¶
Re-classify pair based on file attributes and options.
- problem_count() int ¶
- re_classify_pair(pair)[source]¶
Allow derrived classes to override default classification and operation.
- Returns:
False to prevent default operation.
- resolve_all¶
Conflict resolution strategy
- Type:
str
ftpsync.targets module¶
(c) 2012-2024 Martin Wendt; see https://github.com/mar10/pyftpsync Licensed under the MIT license: https://www.opensource.org/licenses/mit-license.php
- class ftpsync.targets.FsTarget(root_dir, extra_opts=None)[source]¶
Bases:
_Target
- DEFAULT_BLOCKSIZE = 16384¶
- check_write(name)¶
Raise exception if writing cur_dir/name is not allowed.
- copy_to_file(name, fp_dest, callback=None)¶
Write cur_dir/name to file-like fp_dest.
- Parameters:
name (str) – file name, located in self.curdir
fp_dest (file-like) – must support write() method
callback (function, optional) – Called like func(buf) for every written chunk
- encoding¶
Assumed encoding for this target. Used to decode binary paths.
- enter_subdir(name)¶
Temporarily changes the working directory to name.
Examples
- with target.enter_subdir(folder):
…
- get_base_name()¶
- get_id()¶
- get_option(key, default=None)¶
Return option from synchronizer (possibly overridden by target extra_opts).
- get_options_dict()¶
Return options from synchronizer (possibly overridden by own extra_opts).
- get_sync_info(name, key=None)¶
Get mtime/size when this target’s current dir was last synchronized with remote.
- is_local()¶
- is_remote(or_unbound=False)¶
- is_unbound()¶
- mtime_compare_eps¶
Maximum allowed difference between a reported mtime and the last known update time, before we classify the entry as ‘modified externally’
- pop_meta()¶
- push_meta()¶
- read_text(name)¶
Read text string from cur_dir/name using open_readable().
- remove_sync_info(name)¶
- root_dir¶
The target’s top-level folder
- server_time_ofs¶
Time difference between <local upload time> and the mtime that the server reports afterwards. The value is added to the ‘u’ time stored in meta data. (This is only a rough estimation, derived from the lock-file.)
- set_sync_info(name, mtime, size)¶
Store mtime/size when this resource was last synchronized with remote.
- synchronizer¶
Set by BaseSynchronizer.__init__(). May be None for tree command, etc.
- walk(pred=None, recursive=True)¶
Iterate over all target entries recursively.
- Parameters:
pred (function, optional) – Callback(
ftpsync.resources._Resource
) should return False to ignore entry. Default: None.recursive (bool, optional) – Pass False to generate top level entries only. Default: True.
- Yields:
- walk_tree(sort=True, files=False, pred=None, _prefixes=None)¶
Iterate over target hierarchy, depth-first, adding a connector prefix.
This iterator walks the tree nodes, but slightly delays the output, in order to add information if a node is the last sibling. This information is then used to create pretty tree connector prefixes.
- Parameters:
sort (bool)
files (bool)
pred (function, optional) – Callback(
ftpsync.resources._Resource
) should return False to ignore entry. Default: None.
- Yields:
- 3-tuple (
ftpsync.resources._Resource
, is_last_sibling, prefix,
)
- write_file(name, fp_src, blocksize=16384, callback=None)[source]¶
Write binary data from file-like to cur_dir/name.
- write_text(name, s)¶
Write string data to cur_dir/name using write_file().
- class ftpsync.targets._Target(root_dir, extra_opts)[source]¶
Bases:
object
Base class for
FsTarget
,FTPTarget
, etc.- DEFAULT_BLOCKSIZE = 16384¶
- copy_to_file(name, fp_dest, callback=None)[source]¶
Write cur_dir/name to file-like fp_dest.
- Parameters:
name (str) – file name, located in self.curdir
fp_dest (file-like) – must support write() method
callback (function, optional) – Called like func(buf) for every written chunk
- encoding¶
Assumed encoding for this target. Used to decode binary paths.
- enter_subdir(name)[source]¶
Temporarily changes the working directory to name.
Examples
- with target.enter_subdir(folder):
…
- get_option(key, default=None)[source]¶
Return option from synchronizer (possibly overridden by target extra_opts).
- get_options_dict()[source]¶
Return options from synchronizer (possibly overridden by own extra_opts).
- get_sync_info(name, key=None)[source]¶
Get mtime/size when this target’s current dir was last synchronized with remote.
- mtime_compare_eps¶
Maximum allowed difference between a reported mtime and the last known update time, before we classify the entry as ‘modified externally’
- root_dir¶
The target’s top-level folder
- server_time_ofs¶
Time difference between <local upload time> and the mtime that the server reports afterwards. The value is added to the ‘u’ time stored in meta data. (This is only a rough estimation, derived from the lock-file.)
- set_sync_info(name, mtime, size)[source]¶
Store mtime/size when this resource was last synchronized with remote.
- synchronizer¶
Set by BaseSynchronizer.__init__(). May be None for tree command, etc.
- walk(pred=None, recursive=True)[source]¶
Iterate over all target entries recursively.
- Parameters:
pred (function, optional) – Callback(
ftpsync.resources._Resource
) should return False to ignore entry. Default: None.recursive (bool, optional) – Pass False to generate top level entries only. Default: True.
- Yields:
- walk_tree(sort=True, files=False, pred=None, _prefixes=None)[source]¶
Iterate over target hierarchy, depth-first, adding a connector prefix.
This iterator walks the tree nodes, but slightly delays the output, in order to add information if a node is the last sibling. This information is then used to create pretty tree connector prefixes.
- Parameters:
sort (bool)
files (bool)
pred (function, optional) – Callback(
ftpsync.resources._Resource
) should return False to ignore entry. Default: None.
- Yields:
- 3-tuple (
ftpsync.resources._Resource
, is_last_sibling, prefix,
)
- ftpsync.targets._get_encoding_opt(synchronizer, extra_opts, default)[source]¶
Helper to figure out encoding setting inside constructors.
ftpsync.ftp_target module¶
(c) 2012-2024 Martin Wendt; see https://github.com/mar10/pyftpsync Licensed under the MIT license: https://www.opensource.org/licenses/mit-license.php
- class ftpsync.ftp_target.FTPTarget(path, host, port=0, username=None, password=None, tls=False, timeout=None, extra_opts=None)[source]¶
Bases:
_Target
Represents a synchronization target on an FTP server.
- path¶
Current working directory on FTP server.
- Type:
str
- ftp¶
Instance of ftplib.FTP.
- Type:
FTP
- host¶
hostname of FTP server
- Type:
str
- port¶
FTP port (defaults to 21)
- Type:
int
- username¶
- Type:
str
- password¶
- Type:
str
- DEFAULT_BLOCKSIZE = 8192¶
- MAX_SPOOL_MEM = 102400¶
- _ftp_pwd()[source]¶
Variant of self.ftp.pwd() that supports encoding-fallback.
- Returns:
Current working directory as native string.
- _ftp_retrlines_native(command, callback, encoding)[source]¶
A re-implementation of ftp.retrlines that returns lines as native str.
This is needed on Python 3, where ftp.retrlines() returns unicode str by decoding the incoming command response using ftp.encoding. This would fail for the whole request if a single line of the MLSD listing cannot be decoded. FTPTarget wants to fall back to Cp1252 if UTF-8 fails for a single line, so we need to process the raw original binary input lines.
On Python 2, the response is already bytes, but we try to decode in order to check validity and optionally re-encode from Cp1252.
- Parameters:
command (str) – A valid FTP command like ‘NLST’, ‘MLSD’, …
callback (function) –
- Called for every line with these args:
status (int): 0:ok 1:fallback used, 2:decode failed line (str): result line decoded using encoding.
If encoding is ‘utf-8’, a fallback to cp1252 is accepted.
encoding (str) – Coding that is used to convert the FTP response to str.
- Returns:
None
- check_write(name)¶
Raise exception if writing cur_dir/name is not allowed.
- copy_to_file(name, fp_dest, callback=None)[source]¶
Write cur_dir/name to file-like fp_dest.
- Parameters:
name (str) – file name, located in self.curdir
fp_dest (file-like) – must support write() method
callback (function, optional) – Called like func(buf) for every written chunk
- encoding¶
Assumed encoding for this target. Used to decode binary paths.
- enter_subdir(name)¶
Temporarily changes the working directory to name.
Examples
- with target.enter_subdir(folder):
…
- flush_meta()¶
Write additional meta information for current directory.
- get_option(key, default=None)¶
Return option from synchronizer (possibly overridden by target extra_opts).
- get_options_dict()¶
Return options from synchronizer (possibly overridden by own extra_opts).
- get_sync_info(name, key=None)¶
Get mtime/size when this target’s current dir was last synchronized with remote.
- is_local()¶
- is_remote(or_unbound=False)¶
- is_unbound()¶
- lock_data¶
written to ftp target root folder before synchronization starts. set to False, if write failed. Default: None
- Type:
dict
- mtime_compare_eps¶
Maximum allowed difference between a reported mtime and the last known update time, before we classify the entry as ‘modified externally’
- open_readable(name)[source]¶
Open cur_dir/name for reading.
Note: we read everything into a buffer that supports .read().
- Parameters:
name (str) – file name, located in self.curdir
- Returns:
file-like (must support read() method)
- open_writable(name)¶
Return file-like object opened in binary mode for cur_dir/name.
- pop_meta()¶
- push_meta()¶
- read_text(name)¶
Read text string from cur_dir/name using open_readable().
- remove_sync_info(name)¶
- root_dir¶
The target’s top-level folder
- server_time_ofs¶
Time difference between <local upload time> and the mtime that the server reports afterwards. The value is added to the ‘u’ time stored in meta data. (This is only a rough estimation, derived from the lock-file.)
- set_sync_info(name, mtime, size)¶
Store mtime/size when this resource was last synchronized with remote.
- support_utf8¶
True if server reports FEAT UTF8
- synchronizer¶
Set by BaseSynchronizer.__init__(). May be None for tree command, etc.
- walk(pred=None, recursive=True)¶
Iterate over all target entries recursively.
- Parameters:
pred (function, optional) – Callback(
ftpsync.resources._Resource
) should return False to ignore entry. Default: None.recursive (bool, optional) – Pass False to generate top level entries only. Default: True.
- Yields:
- walk_tree(sort=True, files=False, pred=None, _prefixes=None)¶
Iterate over target hierarchy, depth-first, adding a connector prefix.
This iterator walks the tree nodes, but slightly delays the output, in order to add information if a node is the last sibling. This information is then used to create pretty tree connector prefixes.
- Parameters:
sort (bool)
files (bool)
pred (function, optional) – Callback(
ftpsync.resources._Resource
) should return False to ignore entry. Default: None.
- Yields:
- 3-tuple (
ftpsync.resources._Resource
, is_last_sibling, prefix,
)
- write_file(name, fp_src, blocksize=8192, callback=None)[source]¶
Write file-like fp_src to cur_dir/name.
- Parameters:
name (str) – file name, located in self.curdir
fp_src (file-like) – must support read() method
blocksize (int, optional)
callback (function, optional) – Called like func(buf) for every written chunk
- write_text(name, s)¶
Write string data to cur_dir/name using write_file().