cancel
Showing results for 
Search instead for 
Did you mean: 

no space left on the device

Piotr Kirklewski
Super Advisor

no space left on the device

Hi there
I'm using one of my machines for backup purposes only. There is one directory that holds the backups, scripts and logs. I called it /files. This directory is only 36GB in size so I don't get where is the missing 22GB ?

backup:/files# du -sh /files/
36G /files/


df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda1 3842048 78875 3763173 3% /
tmpfs 64537 6 64531 1% /lib/init/rw
udev 64537 1069 63468 2% /dev
tmpfs 64537 1 64536 1% /dev/shm
//10.15.3.1/data 0 0 0 - /mnt/PDM
//10.15.3.6/backup 0 0 0 - /mnt/bac001bri
//GDSBS/BACKUP/ 0 0 0 - /mnt/BACKUPgdsbs
//10.15.3.5/os43 rt 0 0 0 - /mnt/QUBE




df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 58G 58G 0 100% /
tmpfs 253M 0 253M 0% /lib/init/rw
udev 10M 568K 9.5M 6% /dev
tmpfs 253M 0 253M 0% /dev/shm
//10.15.3.1/data 256G 62G 195G 25% /mnt/PDM
//10.15.3.6/backup 2.8T 1.2T 1.6T 45% /mnt/bac001bri
//GDSBS/BACKUP/ 137G 35G 103G 26% /mnt/BACKUPgdsbs
//10.15.3.5/os43 rt 256G 16G 240G 7% /mnt/QUBE

backup:/# du -h |grep '^[0-9]*G'
...
35G /files/Projects
36G /files
Jesus is the King
10 REPLIES
Ivan Ferreira
Honored Contributor

Re: no space left on the device

I don't know where do you see 22GB missing.

According to your output, /files is under the / filesystem (sda1) and it's 100% used.
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
Piotr Kirklewski
Super Advisor

Re: no space left on the device

sda1 = /dev/sda1 58G 58G 0 100% /
/files = 36G /files

So 58GB - 36GB = 22GB

backup:/# ls |sort
bin
boot
cdrom
dev
etc
files
home
initrd.img
lib
lost+found
media
mnt
opt
proc
root
save
sbin
selinux
srv
sys
tmp
usr
var
vmlinuz




du -sh /bin/
4.1M /bin/

backup:/# du -sh /boot
8.6M /boot

backup:/# du -sh /dev/
568K /dev/

backup:/# du -sh /etc/
5.0M /etc/

backup:/# du -sh /files
36G /files

backup:/# du -sh /home
36K /home

backup:/# du -sh /initrd.img
0 /initrd.img

backup:/# du -sh /lib/
68M /lib/

backup:/# du -sh /lost+found/
16K /lost+found/

backup:/# du -sh /media/
12K /media/

backup:/# du -sh /opt/
4.0K /opt/

backup:/# du -sh /proc/
du: cannot access `/proc/13909/task/13909/fd/4': No such file or directory
du: cannot access `/proc/13909/task/13909/fdinfo/4': No such file or directory
du: cannot access `/proc/13909/fd/4': No such file or directory
du: cannot access `/proc/13909/fdinfo/4': No such file or directory
0 /proc/

backup:/# du -sh /root/
76K /root/

backup:/# du -sh /save/
8.0K /save/

backup:/# du -sh /sbin/
3.6M /sbin/

backup:/# du -sh /selinux/
4.0K /selinux/

backup:/# du -sh /srv/
4.0K /srv/

backup:/# du -sh /sys/
0 /sys/

backup:/# du -sh /tmp/
20K /tmp/

backup:/# du -sh /usr/
549M /usr/

backup:/# du -sh /var/
213M /var/

backup:/# du -sh /vmlinuz
0 /vmlinuz



Jesus is the King
Ivan Ferreira
Honored Contributor

Re: no space left on the device

Maybe your missing space is under some of your directories acting as mount point. You should umount all filesystems and check to see if there is any files under the mounted filesystems. Remove the files from the directories acting as mount point.
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
macosta
Trusted Contributor

Re: no space left on the device

Also look for processes that are still holding open file descriptors on a file that has since been deleted.

Piotr Kirklewski
Super Advisor

Re: no space left on the device

How ?
Jesus is the King
Randy Jones_3
Trusted Contributor

Re: no space left on the device

Do a "touch /forcefsck" then restart the system. fsck should run at startup and that should recover any space that was somehow only partially deallocated.
macosta
Trusted Contributor

Re: no space left on the device

I'm not certain a fsck is necessary, but shouldn't hurt anything.

The issue, if I recall correctly, is that a file can be held open, but deleted underneath the open() syscall. While the link count is dropped to 0, the links to the blocks haven't been cleared in the filesystem yet. This explains the discrepancy between 'df' which reads the filesystem metadata to give you a count of blocks, while 'du' actually crawls the filesystem.
Ivan Ferreira
Honored Contributor

Re: no space left on the device

It's true. You can use the lsof command to identify files deleted that was not released. I have found these kind of files on the /tmp file system sometimes.
Por que hacerlo dificil si es posible hacerlo facil? - Why do it the hard way, when you can do it the easy way?
Goran Koruga
Honored Contributor

Re: no space left on the device

Hello.

ls -l /proc/*/fd | grep '(deleted)'

I leave it up to you to expand on this (i.e. pipe to awk/sed/whatever and print filename only, then pipe to "xargs du ...").

Regards,
Goran
Piotr Kirklewski
Super Advisor

Re: no space left on the device

Thanks everyone.
Jesus is the King