Command Line Interface¶
Use the --help
or -h
argument to get help:
$ pyftpsync --help
usage: pyftpsync [-h] [-v | -q] [--debug {classify}] [--case {strict,local,remote}] [-V]
{upload,download,sync,run,scan,tree} ...
Synchronize folders over FTP.
positional arguments:
{upload,download,sync,run,scan,tree}
sub-command help
upload copy new and modified files to remote folder
download copy new and modified files from remote folder to local target
sync synchronize new and modified files between remote folder and local target
run run pyftpsync with configuration from `pyftpsync.yaml` in current or parent folder
scan repair, purge, or check targets
tree list target folder structure
optional arguments:
-h, --help show this help message and exit
-v, --verbose increment verbosity by one (default: 3, range: 0..5)
-q, --quiet decrement verbosity by one
--debug {classify} enable additional specific logging (requires -v)
--case {strict,local,remote}
how to handle file names that only differ in casing (default: stop if ambigous files are encountered)
-V, --version show program's version number and exit
See also https://github.com/mar10/pyftpsync
$
run command¶
In addition to the direct invocation of upload, download, or sync
commands, version 3.x allows to define a pyftpsync_yaml
file
in your project’s root folder which then can be executed like so:
$ pyftpsync run
See also
See Run from pyftpsync.yaml for details.
scan command¶
TODO: The scan command can be used for different tasks.
tree command¶
Example: display a hierarchical directory listing:
$ pyftpsync tree sftp://demo:password@test.rebex.net
Connecting demo:*** to sftp://test.rebex.net
Could not write lock file: [Errno 13] Access denied.
[/]
+- [aspnet_client]
| `- [system_web]
| `- [4_0_30319]
`- [pub]
+- [192.168.1.114]
| +- [2020-10-07]
| +- [2020-10-08]
| `- [2020-10-09]
`- [example]
Scanning 0 files in 10 directories took 2.93 seconds.
Options --files
, --sort
, etc. are also available.
Target URLs¶
The local
and remote
target arguments can be file paths or URLs
(currently the ftp:
, ftps
, and sftp:
protocols are supported):
$ pyftpsync upload ~/temp ftp://example.com/target/folder
FTP URLs may contain credentials (not recommended):
$ pyftpsync upload ~/temp ftp://joe:secret@example.com/target/folder
Note
Replace ftp://
with ftps://
to enable TLS encryption.
Replace ftp://
with sftp://
to use the SFTP protocol.
Note
The content of the local target folder is synchronized with the content of
the remote target folder, so both folders must exist, with one exception:
If the remote target does not exist, but its parent folder does, the
--create-folder
option may be passed.
Note that pyftpsync also supports prompting for passwords and storing
passwords in the system keyring and .netrc
files (see below).
Authentication¶
FTP targets often require authentication. There are multiple ways to handle this:
Pass credentials with the target URL:
ftp://user:password@example.com/target/folder
Pass only a user name with the target URL:
ftp://user@example.com/target/folder
The CLI will prompt for a password (the library would raise an error).Don’t pass any credentials with the URL:
ftp://example.com/target/folder
pyftpsync will now
Try to lookup credentials for host (‘example.com’) in the system keyring storage.
Try to lookup credentials for host (‘example.com’) in the
.netrc
file in the user’s home directory.CLI will prompt for username and password.
Assume anonymous access.
If authentication fails, the CLI will prompt for a password again.
Credential discovery can be controlled by --no-keyring
, --no-netrc
,
and --no-prompt
options.
--prompt
will force prompting, even if lookup is possible.
--store-password
will save credentials to the system keyring storage upon
successful login.
Note
In order to use .netrc
on Windows, the %HOME% environment variable
should be set. If not, try this:
> set HOME=%USERPROFILE%
(see here).
Note
The SFTP protocol checks if the public key of the remote server is
known, by looking for an entry in the ~/.ssh/known_hosts
file.
This can be disabled by passing --no-verify-host-keys
, but a safer
and recommended solution is to add the real key using a tool like
ssh-keyscan HOST
.
Matching and Filtering¶
The --match
option filters processed files using on or more patterns
(using the fnmatch syntax).
Note: These patterns are only applied to files, not directories.
The --exclude
option is applied after --match
and removes entries from
processing. Unlike --match
, these patterns are also applied to directories.
Example:
$ pyftpsync scan /my/folder --list --match=*.js,*.css --exclude=.git,build,node_modules
Upload Files Syntax¶
Command specific help is available like so:
$ pyftpsync upload -h
usage: pyftpsync upload [-h] [-v | -q] [--debug {classify}] [--case {strict,local,remote}] [-n] [--progress] [--no-color]
[--ftp-active] [--migrate] [--no-verify-host-keys] [-m MATCH] [-x EXCLUDE] [--prompt | --no-prompt]
[--no-keyring] [--no-netrc] [--store-password] [--force] [--resolve {local,skip,ask}] [--delete]
[--delete-unmatched] [--create-folder] [--report-problems]
LOCAL REMOTE
positional arguments:
LOCAL path to local folder (default: .)
REMOTE path to remote folder
optional arguments:
-h, --help show this help message and exit
-v, --verbose increment verbosity by one (default: 3, range: 0..5)
-q, --quiet decrement verbosity by one
--debug {classify} enable additional specific logging (requires -v)
--case {strict,local,remote}
how to handle file names that only differ in casing (default: stop if ambigous files are encountered)
-n, --dry-run just simulate and log results, but don't change anything
--progress show progress info, even if redirected or verbose < 3
--no-color prevent use of ansi terminal color codes
--ftp-active use Active FTP mode instead of passive
--migrate replace meta data files from different pyftpsync versions with current format. Existing data will be
discarded.
--no-verify-host-keys
do not check SFTP connection against `~/.ssh/known_hosts`
-m MATCH, --match MATCH
wildcard for file names (but not directories) using fnmatch syntax (default: match all, separate
multiple values with ',')
-x EXCLUDE, --exclude EXCLUDE
wildcard of files and directories to exclude (applied after --match, default:
'.DS_Store,.git,.hg,.svn,#recycle')
--prompt always prompt for password
--no-prompt prevent prompting for invalid credentials
--no-keyring prevent use of the system keyring service for credential lookup
--no-netrc prevent use of .netrc file for credential lookup
--store-password save password to keyring if login succeeds
--force overwrite remote files, even if the target is newer (but no conflict was detected)
--resolve {local,skip,ask}
conflict resolving strategy (default: 'ask')
--delete remove remote files if they don't exist locally
--delete-unmatched remove remote files if they don't exist locally or don't match the current filter (implies '--delete'
option)
--create-folder Create remote folder if missing
--report-problems return exit code 10 if any conflict was skipped, a copy error occurred, etc.
$
Example: Upload Files¶
Upload all new and modified files from user’s temp folder to an FTP server. No files are changed on the local directory:
$ pyftpsync upload ~/temp sftp://example.com/target/folder
Add the --delete
option to remove all files from the remote target that
don’t exist locally:
$ pyftpsync upload ~/temp sftp://example.com/target/folder --delete
Add the --dry-run
option to switch to DRY-RUN mode, i.e. run in test mode
without modifying files:
$ pyftpsync upload ~/temp sftp://example.com/target/folder --delete --dry-run
Add one or more -v
options to increase output verbosity:
$ pyftpsync upload ~/temp sftp://example.com/target/folder --delete -vv
Mirror current directory to remote folder:
$ pyftpsync upload . sftp://example.com/target/folder --force --delete --resolve=local
Download Files Syntax¶
This is generally the same as upload with swapped targets.
Synchronize Files Syntax¶
$ pyftpsync sync -h
usage: pyftpsync sync [-h] [-v | -q] [--debug {classify}] [--case {strict,local,remote}] [-n] [--progress] [--no-color]
[--ftp-active] [--migrate] [--no-verify-host-keys] [-m MATCH] [-x EXCLUDE] [--prompt | --no-prompt]
[--no-keyring] [--no-netrc] [--store-password] [--resolve {old,new,local,remote,skip,ask}]
[--create-folder] [--report-problems]
LOCAL REMOTE
positional arguments:
LOCAL path to local folder (default: .)
REMOTE path to remote folder
optional arguments:
-h, --help show this help message and exit
-v, --verbose increment verbosity by one (default: 3, range: 0..5)
-q, --quiet decrement verbosity by one
--debug {classify} enable additional specific logging (requires -v)
--case {strict,local,remote}
how to handle file names that only differ in casing (default: stop if ambigous files are encountered)
-n, --dry-run just simulate and log results, but don't change anything
--progress show progress info, even if redirected or verbose < 3
--no-color prevent use of ansi terminal color codes
--ftp-active use Active FTP mode instead of passive
--migrate replace meta data files from different pyftpsync versions with current format. Existing data will be
discarded.
--no-verify-host-keys
do not check SFTP connection against `~/.ssh/known_hosts`
-m MATCH, --match MATCH
wildcard for file names (but not directories) using fnmatch syntax (default: match all, separate
multiple values with ',')
-x EXCLUDE, --exclude EXCLUDE
wildcard of files and directories to exclude (applied after --match, default:
'.DS_Store,.git,.hg,.svn,#recycle')
--prompt always prompt for password
--no-prompt prevent prompting for invalid credentials
--no-keyring prevent use of the system keyring service for credential lookup
--no-netrc prevent use of .netrc file for credential lookup
--store-password save password to keyring if login succeeds
--resolve {old,new,local,remote,skip,ask}
conflict resolving strategy (default: 'ask')
--create-folder Create remote folder if missing
--report-problems return exit code 10 if any conflict was skipped, a copy error occurred, etc.
$
Example: Synchronize Folders¶
Two-way synchronization of a local folder with an SFTP server:
$ pyftpsync sync --store-password --resolve=ask ~/temp sftp://example.com/target/folder
Note that sftp:
protocol was specified to enable SFTP.
Verbosity Level¶
The verbosity level can have a value from 0 to 6:
Verbosity |
Option |
Log level |
Remarks |
---|---|---|---|
0 |
-qqq |
CRITICAL |
quiet |
1 |
ERROR |
show errors only |
|
2 |
-q |
WARN |
show conflicts and 1 line summary only |
3 |
INFO |
show write operations |
|
4 |
-v |
DEBUG |
show equal files |
5 |
-vv |
DEBUG |
diff-info and benchmark summary |
6 |
-vvv |
DEBUG |
show FTP commands |
Exit Codes¶
The CLI returns those exit codes:
0: OK
1: Error (network, internal, ...)
2: CLI syntax error
3: Aborted by user
10: Unresolved conflicts remaining (with option --report-problems)