Apache2 - pipe logs and cron?

Codewalkers Forums Sponsor:

Warn Jester · View Jester's Warnings · View your Warnings · #1

August 11th, 2003, 01:58 PM

Jester

Join Date: Apr 2007

Location: England

Posts: 271

Time spent in forums: < 1 sec
Reputation Power: 7

Apache2 - pipe logs and cron?

Hi,

I'm looking at my log configuration set-up within Apache2. Currently I am utilising the piped logging facility in the following way:

# The format for logging the three different custom logs: common, referer and agent
LogFormat "%h,%l,%u,%t,"%r",%>s,%b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

# Put all logs on a piped logging rotate routine

# General Apche error log on a 1 week rotate, saved to /etc/apache2/logs/error.YYYYMMDD.log
ErrorLog "|/etc/apache2/bin/rotatelogs /etc/apache2/logs/error.%Y%m%d.log 604800"

# Custom logs on a 1 week rotate, saved to /etc/apache2/logs/[log name].YYYYMMDD.log
CustomLog "|/etc/apache2/bin/rotatelogs /etc/apache2/logs/access.%Y%m%d.log 604800" common
CustomLog "|/etc/apache2/bin/rotatelogs /etc/apache2/logs/referer.%Y%m%d.log 604800" referer
CustomLog "|/etc/apache2/bin/rotatelogs /etc/apache2/logs/agent.%Y%m%d.log 604800" agent

This works well, and within my log folder in the web space I get a new log each week. However, what I would like to do is just keep the newest two (i.e. the current log, and the previous log). Any other logs should be gzipped and move to a sub folder within the log folder.

Has anyone any idea on the best way to do this? I know that you can use cron for jobs like this, and I have looked at the tutorial on this site, but am still unsure on how to start. There are also specific folders on my Suse 8.2 installation like /etc/cron.weekly,is there some sort of script that I could drop in there?

Any ideas or pointers in the right direction would be brill,

Thanks, J

Warn postalcow · View postalcow's Warnings · View your Warnings · #2

August 11th, 2003, 11:59 PM

postalcow

Codewalkers Beginner (1000 - 1499 posts)

Join Date: Apr 2007

Location: Ford CIty, PA USA

Posts: 1,267

Time spent in forums: < 1 sec
Reputation Power: 8

RE: Apache2 - pipe logs and cron?

I read through your post a couple of times. I am having a little trouble getting a grasp on what you want. So your OK with your logs being written you just want to tarball/gzip all but the two newest ones correct?

If correct I do something similar. I wrote a PHP script that does a backup everyday gzips um then deletes the the ones more than 5 days old. This script could easily be modified to do what you want if I understand correctly

Warn Jester · View Jester's Warnings · View your Warnings · #3

August 12th, 2003, 08:50 AM

Jester

Join Date: Apr 2007

Location: England

Posts: 271

Time spent in forums: < 1 sec
Reputation Power: 7

RE: Apache2 - pipe logs and cron?

Yep - that's about the long and short of it! Thanks for your time.

Thought I would have to use cron and some weird coding - but much rather use PHP and trigger with cron!

Could you point me in the direction of the code?

J

Warn postalcow · View postalcow's Warnings · View your Warnings · #4

August 12th, 2003, 04:58 PM

postalcow

Join Date: Apr 2007

Location: Ford CIty, PA USA

Posts: 1,267

Time spent in forums: < 1 sec
Reputation Power: 8

RE: Apache2 - pipe logs and cron?

Are you going to rewrite my script yourself?

Warn Jester · View Jester's Warnings · View your Warnings · #5

August 13th, 2003, 08:41 AM

Jester

Join Date: Apr 2007

Location: England

Posts: 271

Time spent in forums: < 1 sec
Reputation Power: 7

RE: Apache2 - pipe logs and cron?

Sorry postalcow - I just assumed that you wouldn't mind! If it's copyrited etc then no prob, just post the pseudo code for me and I'll try to work it out! (I assume that it's phpcl that you trigger by cron? Never done this so do you think you could also show me how to get the script to run each week?!)

I hardly ever just blindly use other's code though - I always use it as a base, research what each part does (if I don't already know) and then write my own.

That is the way I learn (can't just look and have it stay in my head - gotta do it myself to remember :laugh: !!!)

J

Warn Jester · View Jester's Warnings · View your Warnings · #6

August 18th, 2003, 08:54 AM

Jester

Join Date: Apr 2007

Location: England

Posts: 271

Time spent in forums: < 1 sec
Reputation Power: 7

RE: Apache2 - pipe logs and cron?

Sorry for the bump postalcow - but is there any chance of seeing that code? Pretty please?!!!

J

Warn postalcow · View postalcow's Warnings · View your Warnings · #7

August 18th, 2003, 11:33 AM

postalcow

Join Date: Apr 2007

Location: Ford CIty, PA USA

Posts: 1,267

Time spent in forums: < 1 sec
Reputation Power: 8

RE: Apache2 - pipe logs and cron?

No problem on the bump. I am the one that forgot, sorry. Is something like this what you wanted? Just put the script in a location like /usr/local/util and fire it daily by cron.

php Code:

Original - php Code

#!/usr/bin/php -q
 
<?
// Above set path to php 
 
// I use epoch there are many ways to do time calculations. Some are better some are worst
// This is how I do it. Also there are PHP commands for copy and mkdir I just use the unix 
// commands here. This is a hack way to do it but it does work. Others may post better ways
// of doing this
// I seperated everything so it is easier to follow what I am doing
 
$epoch = date(U);
$today = date("Ymd", $epoch);
$backup_path="/backup/path";
$calc_del = ($epoch -432000); // Number of days in seconds to delete 432000 is 5 days
 
$delete = date("Ymd", $calc_del);
 
// If you need to create a DIR for backups
 
exec("mkdir $backup_path/$today");
 
// you could also pre note the dir it creates like this
// exec("mkdir $backup_path/logs$today");
// this will give you a dir of logs20030818
 
// Tarball the file in your backup dir
// * is all in that dir you can set specific files
// I put them in a directory w/todays date formatted so they remain sequential
 
// Add as many tarball's as you like
 
exec("tar -cvf $backup_path/$today/somefile.tar /var/log/httpd/*");
 
 
// gzip all the files
exec("gzip $backup_path/$today/*");
 
// this will delete directorys older that 5 days you can delete anything you want
 
exec("rm -rf $backup_path/$delete");
 
// This is a simple way of doing things but it does work. Is this what you wanted?
 
 
?>

Warn Jester · View Jester's Warnings · View your Warnings · #8

August 19th, 2003, 09:34 AM

Jester

Join Date: Apr 2007

Location: England

Posts: 271

Time spent in forums: < 1 sec
Reputation Power: 7

RE: Apache2 - pipe logs and cron?

More than I expected - it's nice to see someone taking the time to comment rather than pasting up 'black art' code!

Comments allow people to take the 'idea' of the code and apply it in thier own style and situation, rather than copy and paste without any learning or understanding - thank you and long may this ethos continue on codewalkers!

J