Backup File Proliferation

17 Nov 2019

Table of Contents


The following compress and purge approaches handle log files (in this case, from Tomcat) - they are two variations on the same theme.

Compress and Purge

Bash
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
#!/bin/bash  

del_days=365  
zip_days=60  

logDir=${CATALINA_BASE}/logs  

# First zip old files that have not yet been zipped.  
set -o noglob  
list='*.txt *.log'  
for item in ${list}  
do  
  find $logDir -maxdepth 1 -type f -not -name '*.gz' -name ${item} \  
    -mtime +${zip_days} -print0 | sudo xargs -0r gzip -fq  
done  
set +o noglob  

# Now remove really old files.  
find $logDir -maxdepth 1 -type f -name '*' -mtime +${del_days} \  
  -print0 | sudo xargs -0r rm -f

The r in xargs -0r ensures empty input pipes are ignored. Or use --no-run-if-empty.

The 0 in both -print0 and xargs -0r is a safeguard against files names containing spaces.  Any lists (of files, in this case) use the null character as their item separator.

The script needs to be scheduled in crontab.

Logrotate

The logrotate file /etc/logrotate.d/tomcat contains this:

/opt/tomcat-inst/logs/catalina.out {  
    copytruncate  
    daily  
    rotate 180  
    compress  
    missingok  
    size 10K  
}

180 means 180 old files will be kept.

The copytruncate directive allows a process to continue writing to a rotated file.

Make sure logrotate is actually being executed. Various ways to do this. See notes here.

“For most daemon processes, logs should be rotated by the root user. In most cases, logrotate is invoked from a script in the /etc/cron.daily/ directory. If one does not exist, create a script that resembles the following…”

Bash
1
2
#!/bin/bash  
logrotate /etc/logrotate.conf