vmbackup和vmrestore是何方神圣？
vmbackup和vmrestore是何方神圣？
没有故事的陈师傅
发布于 2022-02-09 00:43:31
发布于 2022-02-09 00:43:31
代码可运行
运行总次数：0
代码可运行
背景
在之前的文章中，给大家介绍了vmstorage、vminsert、vmselect、vmui、vmagent，分别代表了数据的存储、添加、查询、可视化以及数据的采集。想想是不是还差点什么？没错！还缺少数据的备份和还原。那么今天，它们来了！接下来，请让我为大家一一介绍。
vmbackup
vmbackup是一个数据备份的工具，可以从vmstorage即时的快照中去备份vm数据，同时支持将备份数据存储到存储系统，如：
- GCS. Example: gs:///<path/to/backup>
- S3. Example: s3:///<path/to/backup>
- Any S3-compatible storage such as MinIO, Ceph or Swift. See these docs for details
- Local filesystem. Example: fs://</absolute/path/to/backup>
vmbackup支持增量备份和全备。当备份数据存储路径已经有了之前的备份数据，增量备份将自动进行；而全备则可以通过-origin指定相同的远端存储去加速备份。因为在这种情况下，可以在远端存储进行新备份与已经存在的备份之间进行共有数据的复制，从而节省了时间和其他花销。备份进程可以在任何时间被打断，但当你使用vmbackup使用同样的参数进行再次备份时，备份将从打断处自动恢复。同时，通过vmbackup备份的数据能够被vmrestore用来恢复数据。
vmrestore
vmrestore被用于从通过vmbackup备份的数据中进行恢复数据，跟vmbackup备份进程一样，恢复数据进程也可以在任何时候被打断；当再次使用相同的命令和参数去恢复数据时，将从断点处自动恢复。原vmstorage数据存储的的数据将被恢复的数据所替换。
实操
数据备份
1、创建快照
上面提到过，vmbackup需要从即时的快照去备份数据，因此想要备份，我们需要先创建快照。vmstorage给我们提供了API去创建快照。使用以下的方式：
# request
curl http://<victoriametrics-addr>:8428/snapshot/create # 需要根据实际部署修改地址和端口
# response
{"status":"ok","snapshot":"<snapshot-name>"}
下面来实操一下：
[root@test ~]# curl http://172.16.47.122:8482/snapshot/create
{"status":"ok","snapshot":"20220119051931-16CA0D126A870F8B"}
我们再看看vmstorage的日志：
2022-01-19T05:19:31.767Z info VictoriaMetrics/lib/storage/storage.go:304 creating Storage snapshot for "/vm-data"...
2022-01-19T05:19:31.771Z info VictoriaMetrics/lib/storage/table.go:143 creating table snapshot of "/vm-data/data"...
2022-01-19T05:19:31.773Z info VictoriaMetrics/lib/storage/partition.go:1549 creating partition snapshot of "/vm-data/data/small/2022_01" and "/vm-data/data/big/2022_01"...
2022-01-19T05:19:31.920Z info VictoriaMetrics/lib/storage/partition.go:1570 created partition snapshot of "/vm-data/data/small/2022_01" and "/vm-data/data/big/2022_01" at "/vm-data/data/small/snapshots/20220119051931-16CA0D126A870F8B/2022_01" and "/vm-data/data/big/snapshots/20220119051931-16CA0D126A870F8B/2022_01" in 0.146 seconds
2022-01-19T05:19:31.920Z info VictoriaMetrics/lib/storage/table.go:171 created table snapshot for "/vm-data/data" at ("/vm-data/data/small/snapshots/20220119051931-16CA0D126A870F8B", "/vm-data/data/big/snapshots/20220119051931-16CA0D126A870F8B") in 0.149 seconds
2022-01-19T05:19:31.921Z info VictoriaMetrics/lib/mergeset/table.go:1161 creating Table snapshot of "/vm-data/indexdb/16CA0D126A874608"...
2022-01-19T05:19:31.963Z info VictoriaMetrics/lib/mergeset/table.go:1230 created Table snapshot of "/vm-data/indexdb/16CA0D126A874608" at "/vm-data/indexdb/snapshots/20220119051931-16CA0D126A870F8B/16CA0D126A874608" in 0.041 seconds
2022-01-19T05:19:31.963Z info VictoriaMetrics/lib/mergeset/table.go:1161 creating Table snapshot of "/vm-data/indexdb/16CA0D126A874607"...
2022-01-19T05:19:31.966Z info VictoriaMetrics/lib/mergeset/table.go:1230 created Table snapshot of "/vm-data/indexdb/16CA0D126A874607" at "/vm-data/indexdb/snapshots/20220119051931-16CA0D126A870F8B/16CA0D126A874607" in 0.003 seconds
2022-01-19T05:19:31.971Z info VictoriaMetrics/lib/storage/storage.go:361 created Storage snapshot for "/vm-data" at "/vm-data/snapshots/20220119051931-16CA0D126A870F8B" in 0.204 seconds
最后去vmstorage的存储路径下看看文件：
# 我这里是直接去我的nfs下vmstorage对应的存储path下查看的。
# 可以很清楚看到我们之前的快照
[root@test ~]# cd /nfs_data/pvc-b06968ca-f31e-4d93-b16d-cf66a10c64d9/
[root@test pvc-b06968ca-f31e-4d93-b16d-cf66a10c64d9]# ls -al
total 24
drwxr-xr-x 6 root root 4096 Jan 14 13:49 .
drwxr-xrwx 6 root root 4096 Jan 14 13:49 ..
drwxr-xr-x 4 root root 4096 Jan 14 13:49 data
-rw-r--r-- 1 root root 0 Jan 14 13:49 flock.lock
drwxr-xr-x 5 root root 4096 Jan 14 13:49 indexdb
drwxr-xr-x 2 root root 4096 Jan 14 13:49 metadata
drwxr-xr-x 3 root root 4096 Jan 19 13:19 snapshots
[root@test pvc-b06968ca-f31e-4d93-b16d-cf66a10c64d9]# cd snapshots/
[root@test snapshots]# ls -al
total 12
drwxr-xr-x 3 root root 4096 Jan 19 13:19 .
drwxr-xr-x 6 root root 4096 Jan 14 13:49 ..
drwxr-xr-x 4 root root 4096 Jan 19 13:19 20220119051931-16CA0D126A870F8B
[root@test snapshots]# cd 20220119051931-16CA0D126A870F8B/
[root@test 20220119051931-16CA0D126A870F8B]# ls -al
total 16
drwxr-xr-x 4 root root 4096 Jan 19 13:19 .
drwxr-xr-x 3 root root 4096 Jan 19 13:19 ..
drwxr-xr-x 2 root root 4096 Jan 19 13:19 data
lrwxrwxrwx 1 root root 55 Jan 19 13:19 indexdb -> ../../indexdb/snapshots/20220119051931-16CA0D126A870F8B
drwxr-xr-x 2 root root 4096 Jan 19 13:19 metadata
其实也可以通过vmstorage提供的API去查看我们刚刚创建的快照
# 到这里我们的快照已经准备就绪了
[root@test ~]# curl http://172.16.47.122:8482/snapshot/list
{"status":"ok","snapshots":[
"20220119051931-16CA0D126A870F8B"
2、数据备份
有了上面的快照，下面我们来进行备份,我这边没有s3、gcs，直接用local filesystem，将其备份到本地/opt目录下
# 通过刚刚创建的快照进行备份
./vmbackup-prod -storageDataPath=/vm-data -snapshotName=20220119051931-16CA0D126A870F8B -dst=fs:///opt
# 输出如下
2022-01-19T05:45:29.709Z info VictoriaMetrics/lib/logger/flag.go:12 build version: vmbackup-20210715-111047-tags-v1.63.0-0-g61cc13c16
2022-01-19T05:45:29.709Z info VictoriaMetrics/lib/logger/flag.go:13 command line flags
2022-01-19T05:45:29.709Z info VictoriaMetrics/lib/logger/flag.go:28 flag "concurrency"="10" (is_set=false)
2022-01-19T05:45:29.709Z info VictoriaMetrics/lib/logger/flag.go:28 flag "configFilePath"="" (is_set=false)
2022-01-19T05:45:29.709Z info VictoriaMetrics/lib/logger/flag.go:28 flag "configProfile"="" (is_set=false)
2022-01-19T05:45:29.709Z info VictoriaMetrics/lib/logger/flag.go:28 flag "credsFilePath"="" (is_set=false)
2022-01-19T05:45:29.709Z info VictoriaMetrics/lib/logger/flag.go:28 flag "customS3Endpoint"="" (is_set=false)
2022-01-19T05:45:29.709Z info VictoriaMetrics/lib/logger/flag.go:28 flag "dst"="fs:///opt" (is_set=true)
...省略...
00_22197_20220119051721.456_20220119051724.766_16CA152B5133B4AD/timestamps.bin", file_size: 0, offset: 0, size: 0} from src fslocal "/vm-data/snapshots/20220119051931-16CA0D126A870F8B" to dst fsremote "/opt"
2022-01-19T05:45:30.374Z info VictoriaMetrics/lib/backup/actions/backup.go:164 uploaded 125373885 out of 125373885 bytes from src fslocal "/vm-data/snapshots/20220119051931-16CA0D126A870F8B" to dst fsremote "/opt" in 627.488079ms
2022-01-19T05:45:30.374Z info VictoriaMetrics/lib/backup/actions/backup.go:171 backup from src fslocal "/vm-data/snapshots/20220119051931-16CA0D126A870F8B" to dst fsremote "/opt" with origin fsnil is complete; backed up 125373885 bytes in 0.664 seconds; deleted 0 bytes; server-side copied 0 bytes; uploaded 125373885 bytes
# 再去/opt目录下确保数据备份正确
/vm-data # cd /opt/
/opt # ls -al
total 24
drwxr-xr-x 1 root root 4096 Jan 19 05:45 .
drwxr-xr-x 1 root root 4096 Jan 14 05:49 ..
-rw------- 1 root root 2 Jan 19 05:45 backup_complete.ignore
drwx------ 4 root root 4096 Jan 19 05:45 data
drwx------ 4 root root 4096 Jan 19 05:45 indexdb
drwx------ 3 root root 4096 Jan 19 05:45 metadata
3、创建第二个快照
[root@test /]# curl http://172.16.47.122:8482/snapshot/create
{"status":"ok","snapshot":"20220119055254-16CA0D126A870F8C"}
4、增量备份
# 备份
./vmbackup-prod -storageDataPath=/vm-data -snapshotName=20220119055254-16CA0D126A870F8C -dst=fs:///opt
# 日志输出
...省略...
2022-01-19T05:55:18.548Z info VictoriaMetrics/lib/backup/actions/backup.go:171 backup from src fslocal "/vm-data/snapshots/20220119055254-16CA0D126A870F8C" to dst fsremote "/opt" with origin fsnil is complete; backed up 127597222 bytes in 0.214 seconds; deleted 2714902 bytes; server-side copied 0 bytes; uploaded 4938239 bytes
从日志上不难看出，增量备份时，delete file 的值不为0，这里删除的是第一次快照中存在，但在第二次快照中不会存在的数据；另外
花费的时间也明显小于咱们第一次全备。
5、再做一个快照，测试server-side copy
# 创建快照
[root@test ~]# curl http://172.16.47.122:8482/snapshot/create
{"status":"ok","snapshot":"20220119073821-16CA0D126A870F8D"}
6、全备，备份时添加-origin参数
# 备份，添加-origin指定之前备份数据所在目录
./vmbackup-prod -storageDataPath=/vm-data -snapshotName=20220119073821-16CA0D126A870F8D -dst=fs:///ss -origin=fs:///opt
# 从命令输出来看，确定有进行server-side copy
...省略...
2022-01-19T07:44:56.657Z info VictoriaMetrics/lib/backup/actions/backup.go:171 backup from src fslocal "/vm-data/snapshots/20220119073821-16CA0D126A870F8D" to dst fsremote "/ss" with origin fsremote "/opt" is complete; backed up 131747249 bytes in 0.190 seconds; deleted 0 bytes; server-side copied 121316947 bytes; uploaded 10430302 bytes
数据恢复
上面我们创建了多个快照，也进行了多次备份。这里我选择从fs:///ss目录下的备份进行恢复到/tt目录下
# 进行恢复
./vmrestore-prod -src=fs:///ss -storageDataPath=/tt
# vm停止，再进行数据恢复；恢复后，vm再次重启时-storageDataPath参数需要指定为/tt目录，即恢复的数据所在的目录
# 控制台输出
...省略...
2022-01-19T07:58:32.117Z info VictoriaMetrics/lib/logger/flag.go:28 flag "src"="fs:///ss" (is_set=true)
2022-01-19T07:58:32.117Z info VictoriaMetrics/lib/logger/flag.go:28 flag "storageDataPath"="/tt" (is_set=true)
...省略...
2022-01-19T07:58:32.751Z info VictoriaMetrics/lib/backup/actions/restore.go:185 downloaded 131747249 out of 131747249 bytes from fsremote "/ss" to fslocal "/tt" in 624.892637ms
2022-01-19T07:58:32.751Z info VictoriaMetrics/lib/backup/actions/restore.go:192 restored 131747249 bytes from backup in 0.635 seconds; deleted 0 bytes; downloaded 131747249 bytes
# 查看/tt目录下的恢复的数据
/vm-data # ls -al /tt/
total 20
drwxr-xr-x 5 root root 4096 Jan 19 07:58 .
drwxr-xr-x 1 root root 4096 Jan 19 07:44 ..
drwx------ 4 root root 4096 Jan 19 07:58 data
-rw-r--r-- 1 root root 0 Jan 19 07:58 flock.lock
drwx------ 4 root root 4096 Jan 19 07:58 indexdb
drwx------ 2 root root 4096 Jan 19 07:58 metadata
注：在进行恢复数据期间，vm应该停止。
Troubleshooting
- 当发现备份数据时很慢，可以通过设置-concurrency参数来设置并发来加快上传数据至存储系统
- 如果vmbackup/vmrestore使用了太多的带宽，可以通过maxBytesPerSecond参数进行限制
- 如果备份/恢复数据时被打断，使用相同的命令参数重新备份/恢复即可，将在断点处自动恢复
- 单节点vm备份的数据不能被用于恢复数据至集群版本vm
总结
以上就是本次我对vmbackup和vmrestore的简单介绍了，更多内容请查看官网。如果你觉得vm不错，动手试试看吧！
参考
- https://docs.victoriametrics.com
本文参与 腾讯云自媒体同步曝光计划，分享自微信公众号。
原始发表：2022-01-26，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
暂无评论
推荐阅读
推荐阅读
相关推荐
移植一个抖音贴纸组件到Flutter
更多 >
腾讯云开发者
