Monday, 9 March 2015

DropBox and Server logs

If, like me, you run a few servers then your log files are vital to determining the health of your systems. The logs generated may be huge. Some are daily. Some are weekly. Some are only generated when a server encounters a problem and resets. Some are boring. Some are so damn useful in tracking down that annoying bug.

But all of them should be kept and analysed.

I've lost track of the number of times I've seen websites under attack. Some of these attacks are insidious in the way they are undertaken. The attacks themselves are spread out over time so that it's almost to distinguish the attacks from regular usage. Other attacks simply bombard your server with a high frequency that sends your server into fits of 404's and 500's.

But one thing you really need is to get those logs into a place where they can be assessed and archived. Now I have servers all over the place. But the servers which generate those logs are not accessible except via remote control which is time consuming and laborious to setup. So how can I get those logs into a central location?

Cue DropBox.

If you can install DropBox on the server then setting your log files to be dropped into a folder that is synchronised will allow those files to be sent to you whenever they change. It's simple and effective. But if you have many servers then this can become a bit of a pain. Having the same account might work for all your servers, but generally log files can grow to large sizes. DropBox can cope with this by simply ignoring the files that sent it over the limit. On your workstation you can then move the logs out into local storage clearing up space so that DB can then send down the remaining logs. DB is good like that.

But there is a better technique. One which can give you better overall control, but it does take a lot of organisation in the first place.

You will need an email address and account for all servers, plus a control (unless you wish to use your personal account). First register a control user which will act as your central "hub" for receiving your log files. Then send "invites" to all of the servers which you then wish to manage the logs of. Install DB on all of your servers using the invited emails. DB will give you a little bonus space for your servers, as well as a large bonus space for your control account. Once you have your accounts installed on your servers, you then need to create a shared folder for your log files. This is then shared with your control account.

Using this method gives you multiple benefits. Firstly, each server only has their own logs. you're not distributing the logs across all servers which is better for security. Secondly your control account now has an increased capacity for handling the logs. Always useful considering how many you will probably need to manage. Thirdly you have a better indication if any issues arise. A simple script can see which servers haven't yet communicated back their logs and you can then go and check out those servers for issues. That is unless you haven't been smart enough to set up other alerts for your systems in the first place.

No comments:

Post a Comment