TOC
选择一台能访问外网的机器,通过下面命令安装rclone:
# curl https://rclone.org/install.sh | sudo bash
# rclone version
rclone v1.50.0
- os/arch: linux/amd64
- go version: go1.13.3
可以通过rclone config
来配置访问ftp,如下:
# rclone config
2019/10/31 03:20:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> ftp
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
...
11 / FTP Connection
\ "ftp"
...
选择11
选项:
Storage> 11
** See help for ftp backend at: https://rclone.org/ftp/ **
FTP host to connect to
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 / Connect to ftp.example.com
\ "ftp.example.com"
输入ftp server
地址:
host> myftpserver.com
FTP username, leave blank for current username, root
Enter a string value. Press Enter for the default ("").
输入已有的ftp user
,ftp port
信息:
user> ftpuser
FTP port, leave blank to use default (21)
Enter a string value. Press Enter for the default ("").
port>
输入ftp user
的password
:
FTP password
y) Yes type in my own password
g) Generate random password
y/g> y
Enter the password:
password:
Confirm the password:
password:
配置其他信息,很多都可以直接使用默认:
Use FTP over TLS (Implicit)
Enter a boolean value (true or false). Press Enter for the default ("false").
tls>
Edit advanced config? (y/n)
y) Yes
n) No
y/n> n
Remote config
--------------------
[ftp]
type = ftp
host = myftpserver.com
user = ftpuser
pass = *** ENCRYPTED ***
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y
Current remotes:
Name Type
==== ====
ftp ftp
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
使用rclone ls/lsf
命令访问ftp,确认输出结果正确:
# rclone lsf ftp:
hadoop-deploy.sh
storage/
v4.20.17.log
# rclone ls ftp:
4548 hadoop-deploy.sh
24094 v4.20.17.log
12022 storage/rook-ceph/1-operator.yaml
4280 storage/rook-ceph/2-cluster.yaml
416 storage/ceph-tools/mon-add.sh
197 storage/ceph-tools/mon-remove.sh
...
与ftp配置一样,同样可以通过rclone config
来配置cos,但我们也可以通过直接修改配置文件的方式来配置cos:
# vim ~/.config/rclone/rclone.conf
...
[cos] ## 命名为:cos,也可以为其他名字
type = s3 ## 指定存储类型为:s3
provider = Other ## 指定提供商为:Other
env_auth = false ## 不从环境变量中获取密钥
access_key_id = AKIDygxxxxxxxxxxxzunx4T ## 配置为cos的 secret_id
secret_access_key = tXu06xxxxxxxxxxFQP9 ## 配置为cos的 secret_key
endpoint = cos.ap-chengdu.myqcloud.com ## 根据不同的region配置,这里我用的是:ap-chengdu
使用rclone ls/lsf
命令访问cos,确认输出结果正确:
# rclone lsf cos:
mybackup1-1253766168/
mybackup2-1253766168/
pictures-cd-1253766168/
# rclone ls cos:pictures-cd-1253766168/
43385 cos-test.jpg
rclone支持copy命令,可以把源数据迁移到目的,命令格式如下:
# rclone -h
...
copy Copy files from source to dest, skipping already copied
...
copy
模式只拷贝新增和变化的文件,多次执行并不会重复迁移相同的数据。
如下命令,把ftp上的storage
目录下的文件都迁移到cos:
# rclone copy ftp:storage/ cos:mybackup1-1253766168/
# rclone ls cos:mybackup1-1253766168/
416 ceph-tools/mon-add.sh
197 ceph-tools/mon-remove.sh
12022 rook-ceph/1-operator.yaml
4280 rook-ceph/2-cluster.yaml
...
rclone copy
会copy指定源目录下的所有文件,目的里并不会包含源目录名。
如果想包含目录名,在cos的目的路径下添加源目录名即可:
# rclone copy ftp:storage/ cos:mybackup1-1253766168/storage
# rclone ls cos:mybackup1-1253766168/
416 storage/ceph-tools/mon-add.sh
197 storage/ceph-tools/mon-remove.sh
12022 storage/rook-ceph/1-operator.yaml
4280 storage/rook-ceph/2-cluster.yaml
...
rclone copy
支持参数实时显示传输统计 -P
/--progress
:
# rclone copy ftp:storage cos:anybackup-api-1253766168/storage -P
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 0
Checks: 37 / 37, 100%
Transferred: 0 / 0, -
Elapsed time: 0s
如上,如果数据之前copy过,再执行一遍并不会重复传输。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。