操作系统:Ubuntu
运行环境:Docker
运行工具:DockerCompose
LPG是现代云原生日志管理解决方案的简称,由三个核心组件组成:
Loki:日志存储和查询引擎
Promtail:日志收集代理
Grafana:日志可视化和仪表板
本文主要记录遇到该问题后的解决方案,并不是搭建教程。
使用Docker搭建日志系统LPG(Loki+Promtail+Grafana)时Loki容器报错:creating WAL folder at "/wal": mkdir wal: permission
通过Docker Compose启动服务时发现Loki容器异常退出。
使用docker log命令查看该容器的日志:
docker logs --tail 20 lpg-loki
日志输出如下所示:
......
creating WAL folder at "/wal": mkdir wal: permission
......
通过日志的字面意思,分析是权限问题导致。Loki想在根目录下创建"wal"文件夹但没有权限创建。
WAL是Loki的预写日志机制,用于数据持久化和故障恢复。
通过检查权限,发现已经通过以下配置赋予了权限:
privileged: true
检查挂载点,Loki容器只挂载了配置文件目录,而数据目录依赖容器内部。
到这里基本找到问题,尝试解决。
在Loki默认配置文件中加入以下内容:
ingester:
wal:
dir: /loki/.cache/loki/wal/
loki.yml
:auth_enabled: false
server:
http_listen_port: 3100
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 1h # Any chunk not receiving new logs in this time will be flushed
max_chunk_age: 1h # All chunks will be flushed when they hit this age, default is 1h
chunk_target_size: 1048576 # Loki will attempt to build chunks up to 1.5MB, flushing first if chunk_idle_period or max_chunk_age is reached first
chunk_retain_period: 30s # Must be greater than index read cache TTL if using an index cache (Default index read cache TTL is 5m)
max_transfer_retries: 0 # Chunk transfers disabled
wal:
dir: /loki/.cache/loki/wal/
schema_config:
configs:
- from: 2020-10-24
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
storage_config:
boltdb_shipper:
active_index_directory: /loki/boltdb-shipper-active
cache_location: /loki/boltdb-shipper-cache
cache_ttl: 24h # Can be increased for faster performance over longer query periods, uses more disk space
shared_store: filesystem
filesystem:
directory: /loki/chunks
compactor:
working_directory: /loki/boltdb-shipper-compactor
shared_store: filesystem
limits_config:
reject_old_samples: true
reject_old_samples_max_age: 168h
chunk_store_config:
max_look_back_period: 0s
table_manager:
retention_deletes_enabled: false
retention_period: 0s
ruler:
storage:
type: local
local:
directory: /loki/rules
rule_path: /loki/rules-temp
alertmanager_url: http://localhost:9093
ring:
kvstore:
store: inmemory
enable_api: true
docker-compose.yml
:version: "3"
services:
# 日志存储和解析
loki:
image: grafana/loki
container_name: lpg-loki
privileged: true
volumes:
- D:\erfan\app\LPG\loki:/etc/loki/
# 修改loki默认配置文件路径
command: -config.file=/etc/loki/loki.yml
ports:
- 3100:3100
# 日志收集器
promtail:
image: grafana/promtail
container_name: lpg-promtail
privileged: true
volumes:
# 将需要收集的日志所在目录挂载到promtail容器中
- D:\erfan\app\LPG\logs\:/var/log/
- D:\erfan\app\LPG\promtail\:/etc/promtail/
# 修改promtail默认配置文件路径
command: -config.file=/etc/promtail/promtail.yml
# 日志可视化
grafana:
image: grafana/grafana
container_name: lpg-grafana
privileged: true
ports:
- 3000:3000
promtail.yml
:server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: http://loki:3100/loki/api/v1/push
scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log
【总结】
调整配置后,重启LPG三个模块,都正常运行。
当遇到Dokcer容器问题后,先查看日志,不要自己猜测,日志会给我们圈一个比较小的范围,解决问题能事半功倍。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。
原创声明:本文系作者授权腾讯云开发者社区发表,未经许可,不得转载。
如有侵权,请联系 cloudcommunity@tencent.com 删除。