linux-error-no-space-left-on-device

CentOS 使用 service nginx restart 重启 nginx 服务报错: Error:No space left on device

发现问题

在执行 service nginx restart 重启 nginx 服务报错: Error:No space left on device

排查问题

1.查看磁盘空间实际占用情况 无异常
2.查看磁盘inodes占用情况 无异常

经过排查

shell

By default, Linux only allocates 8192 watches for inotify, which is ridiculously low. And when it runs out, the error is also No space left on device, which may be confusing if you aren't explicitly looking for this issue.

查看 CentOS 默认 inotify 的 max_user_watches 值

shell

sysctl fs.inotify
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 20000

查看 CentOS 当前使用量

shell

find /proc/*/fd -user "$USER" -lname anon_inode:inotify -printf '%hinfo/%f\n' 2>/dev/null | xargs cat | grep -c '^inotify'
20009

发现已经超过了默认的最大值

解决问题

调整 max_user_watches 大小

shell

vim /etc/sysctl.conf 
fs.inotify.max_user_watches = 80000

刷新配置

shell

sysctl -p

重启服务

shell

service nginx restart