File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Linux / UNIX and the fly likes Compressed file system Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » Linux / UNIX
Bookmark "Compressed file system" Watch "Compressed file system" New topic
Author

Compressed file system

David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

Is there any point to this?

Yesterday we had an amusing problem where one database server failed, and the application servers using that database began logging the failure up to the point that they filled their own drives and fell over too.

Preferably the logs would be written to a dedicated (local) partition but that isn't going to happen any time soon. I was however thinking that since the logs are text and compress well that we could create a ZSF partition for them.

Anyone used this and have an opinion?
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16055
    
  21

If ALL of your application servers fill their drives, it sounds like either they're underbudgeted for local storage or maybe you need smarter logging and some way to downcycle the apps until they can get access to the database again. In other words, go into degraded mode and refuse transactions until a probe of the database indicates its running reliably again.

It's common these days for Linux systems to maintain a set of several generations of logs and to compress all but the active one. I imagine that the logrotate facility is taking care of this. That saves on long-term log space, but for short-term situations, probably the best thing to do is simply not log each and every error at a fine-grained level.

Another possibility is to send the messages to a remote logging host attached to a SAN, but in this case, you'd probably have had a network implosion as well.


Customer surveys are for companies who didn't pay proper attention to begin with.
David O'Meara
Rancher

Joined: Mar 06, 2001
Posts: 13459

You are quite correct that

1) the servers don't maintain state locally except for logs, so the drives are small
2) the log settings are defined 'global' and apply to all app server instances on all servers. It is something I would like to address.
3) the default logging level is more verbose than I like

Our logs in other places get rotated and zipped, but this is a nightly rotation so it doesn't help that we had several log files in the order of 5G+ on each server. Rotation and zipping wasn't going to help at this stage.
Tim Holloway
Saloon Keeper

Joined: Jun 25, 2001
Posts: 16055
    
  21

Whoops, lost the reply when the servers were acting up. Anyway, I think I'd recommend you set up to send all your database logs to a remote log host with a nice big drive.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Compressed file system