There might be times when you find that your Linux machines disk seems to be full and you can’t find the reason for it. You try and find the culprit with the du (disk usage) command, but with no success. The numbers just don’t add up.  In that case actually the problem might be that you have some deleted files that are still open by some program. It can actually happen with faulty logrotate configurations where you don’t tell the program that is writing the log to release the file. Or that you manually deleted a file that some program was writing to.

In such cases the “lsof” command comes to the rescue. Basically, it does what the name says, lists open files – even if it has been deleted and is still in use.  Here is an example of a command that I sometimes use to find if there are deleted files that are still open:

lsof | grep deleted|awk '{$7=$7/1048576 "MB"; print}'

The output of the previous command would list you the open deleted files, the process that is still writing to them and the size of the files. This is some random example output from when I last had to look for missing space:

java 32511 32646 tom 1w REG 12980.00024128MB 19510390447 6341662 /var/log/tomcat/log/catalina.out (deleted)
java 32511 32646 tom 2w REG 0.00024128MB 19510390447 6341662 /var/log/tomcat/log/catalina.out (deleted)

To reclaim the disk space, you just simply need to kill/restart the program that is writing to the deleted file.