Lsof

Lately I’ve learned the beauty of lsof. At my company, when oncall, often you get disk space alerts. We’ve taken care of a lot of it with log rotation but we still get a few disk space alerts across all environments per day.

I recently learned about sudo lsof | grep deleted to find files held open by processes that have been deleted. Once you have this list sometimes you can find a large file and zero it out with > /proc/<process-id>/fd/<# of file handler> i.e as root run > /proc/22508/fd/3

You can find further notes on this on my github cheat sheet for disk space: https://github.com/wsoula/commands/blob/master/disk-space-check-for-process-with-deleted-files-open.md