SysAdmin Valley

Find is one of my favorite little tools under linux.  It helps me “find” almost anything, I can find files older than a certain date, newer than a certain date, modified on a certain date.  I can find files that have a certain name, or match a part of a name, file extension.  Once I’ve found what I’ve been looking for I can have find do something with those files like delete them or gzip them.

My latest “find” with the find command came about because on one of my JBoss servers I wrote a simple script that looks for log files older than 15 days and deletes them and looks for other log files older than 61 minutes and compresses them with gzip.

#!/bin/bash
LOGS=/usr/local/jboss/server/all/log/
#delete all logs older than 37 days
find $LOGS -mtime +15 | xargs rm -rf
# gzip files last modify at least 1 hour ago
find $LOGS -mmin +61 | xargs gzip

Our JBoss setup automatically writes new logs to server.info.log and server.error.log, then every every hour it renames the INFO and ERROR log to the current date + hour, so server.info.log would be changed to server.info.log.2010-03-09-13 for today at 2pm to roll out the 1pm logs.

The problem I came across in my script was with my server.error.log file.  If an error hasn’t been written to the server.error.log file during that hour, it wasn’t going to rotate an empty error log.  Since the file hadn’t been touched/updated/modified in over 61 minutes, my script came along and gzipped it, at this point JBoss then had a problem because the error log was missing and didn’t create a new one.

So what I needed to do was to find all the files that matched the criteria, but exclude the server.info.log and server.error.log and here is my final script.

#!/bin/bash
LOGS=/usr/local/jboss/server/all/log/
INFOLOG=”server.info.log”
ERRORLOG=”server.error.log”
#delete all logs older than 37 days
find $LOGS -mtime +15 -not -name “$INFOLOG” -not -name “$ERRORLOG” | xargs rm -rf
# gzip files last modify at least 1 hour ago
find $LOGS -mmin +61 -not -name “$INFOLOG” -not -name “$ERRORLOG” | xargs gzip

While working on some MySQL stuff today I came across some interesting projects.  About 2 years ago I attended the Boston MySQL Meetup group which had a guest speaker (Patrick Galbraith) and he spoke about setting up MySQL in a Multi-Master setup.  This is where you have two MySQL database servers and each one is a slave of the other.  Today I came across two projects that look promising, the first is Multi-Master Replication Manager for MySQL (or MMM) and the second is Flipper.

MMM is a set of scripts that perform monitoring/failover and management of MySQL master-master replication.  Flipper is also a set of tools that manage which server in a Multi-Master setup is writable and which is readable by moving IP addresses based on the server’s role.  Both look very promising and hopefully soon I’ll have some free time to play around with them.

I’ve started using Percona’s version of MySQL 5.1 and have run into a few issues trying to get other tools such as mytop or maatkit to install but have been having problems with RPM dependency’s.  I found the solution on this guy’s blog.  Basically, if you install the MySQL-client-percona, MySQL-percona, MySQL-server-percona, MySQL-shared-percona and Percona-XtraDB, instead of installing MySQL-shared-percona, you should download and force upgrade (rpm -Uvh –force packagename) the MySQL-shared-compat library directly from MySQL.  Just make sure you get the same version from MySQL that you’re using of the Percona MySQL.

Do you have multiple computers? Maybe one at work and one at home, or maybe you have two at work. Have you ever said I would love to have a way to keep my documents on multiple computers at the same time and not have to worry about copying the files back and forth. Or maybe you’re just looking for a simple way to backup your files in case of emergencies. Well, you should check out DropBox.

DropBox is a new way to manage your files. First you create an account one their website, then you install a piece of software on all your computers. Then when you add a file into your DropBox on one of your computers, immediately all your other computers download the new file. If you make a change to that document on any of the computers, all the others immediately pickup the change. No more copying your documents to CD/DVD/USB and then over to your other computer. Now all your documents are synchronized immediately.

Here’s another benefit.  If you have multiple computers that have different operating systems, this doesn’t matter.  You can automatically synchronize files between Windows, Mac and Linux too.

Another great feature of DropBox, have you ever accidentally deleted or changed a file and wished that you had a backup copy of it. Well now you do. Now you can recover deleted files or get back previous versions of your documents.

To get started, just go to DropBox and click Download DropBox.

Here’s some tools that are great for managing Amazon’s EC2 servers.

ElasticFox
ElastDream

I’ve been doing quite a bit of work with the Amazon Cloud and here are a few tools I’ve found useful for managing the Amazon Simple Storage Service (S3).

S3Fox
Simple Access for S3 & AWS
Amazon S3 Filesystem for Windows
S3Sync for Ruby
S3cmd for Perl
S3fs for Linux

Edit:
CloudBerry Explorer for S3

Today I had the new to generate some new certs for a customer.  I wanted to see what I used previously for the values when I generated the CSR (Certificate Signing Request).  To view the details I used the following:

openssl req -noout -text -in server.csr

One of the many sites I run is a ColdFusion site.  Not by choice, but because the developer that created the site was a CF developer.  When I acquired the site, the original developer would just run ColdFusion Server as a standalone web and application server.  For most cases this is fine, but as the site has grown, so has the traffic.  When you think of all the traffic that is going to the ColdFusion application server, not just dynamic CFM file, but lots of static content like images, CSS, and plain HTML files.  There is no reason why we need to put the extra stress on the ColdFusion server, when a simple webserver could handle the static content, and pass all the requests for dynamic content to ColdFusion.

The original setup was just ColdFusion running on port 80 which is the standard port when going to http://www.mydomain.com.  So first I needed to change ColdFusion’s webserver to run on a different port.  To do that you need to edit the jrun.xml file which on most Linux/Unix systems is located in cf_root/runtime/servers/coldfusion/SERVER-INF.  cf_root is the main directory where you installed ColdFusion.  In the jrun.xml file, you are looking for the section that says:

<!-- ================================================================== -->
 <!-- This is the built-in JRun Web Server                               -->
 <!-- ================================================================== -->
 <service name="WebService">
 <attribute name="port">80</attribute>
 <attribute name="interface">*</attribute>
 <attribute name="deactivated">false</attribute>
 <attribute name="activeHandlerThreads">100</attribute>
 <attribute name="minHandlerThreads">1</attribute>
 <attribute name="maxHandlerThreads">1000</attribute>
 <attribute name="mapCheck">0</attribute>
 <attribute name="threadWaitTimeout">300</attribute>
 <attribute name="backlog">500</attribute>
 <attribute name="timeout">300</attribute>
 </service>

Where is says port 80, you want to change that to another port like 8080.  Next we need to configure Nginx.  On most RedHat/CentOS based systems the Nginx config’s are in /etc/nginx and we’re looking for the nginx.conf.  Here is the basic Nginx config to server your website on port 80, then pass all the requests for ColdFusion files to ColdFusion on port 8080.  This config has some of the basic Nginx settings stripped out and just gives you the server section, this also assumes that you have installed ColdFusion in the /usr/local/coldfusionmx7 directory, and /usr/local/coldfusionmx7/wwwroot is where all your content is.

server {
  listen       80;
  server_name  _;
  access_log  /var/log/nginx/host.access.log  main;

  location / {
    root   /usr/local/coldfusionmx7/wwwroot;
    index  index.cfm index.html index.htm;
    proxy_pass          http://127.0.0.1:8080/;
    proxy_redirect      off;
    proxy_set_header    Host            $host;
    proxy_set_header    X-Real-IP       $remote_addr;
    proxy_set_header    X-Forwarded_For $proxy_add_x_forwarded_for;         
  }

  location ~* ^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|
exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js)$ {
    root /usr/local/coldfusionmx7/wwwroot;
  }
}

At this point, all you need to do is restart ColdFusion, you should be able to verify it still works by going to http://www.mydomain.com:8080. Then you can start Nginx, and go to http://www.mydomain.com.

I love bash and scripting!!!  There’s almost nothing you can’t do with a shell script that would take me forever if I had to write an app to do the same.  So here’s another quick tip.  To convert text in a file from uppercase to lowercase use the following:

cat FILENAME | tr “[:upper:]” “[:lower:]“

Or to go from lowercase to uppercase:

cat FILENAME | tr “[:lower:]” “[:upper:]“

Another quick hint.  While working on a server I needed to bring up a whole lot of IP addresses (200 to be exact).  If I really wanted to I could bring them all up like this:

ifconfig eth0:2 1.1.1.2 netmask 255.255.255.0 up
ifconfig eth0:3 1.1.1.3 netmask 255.255.255.0 up
and so on to…..
ifconfig eth0:200 1.1.1.200 netmask 255.255.255.0 up

Well, I’m always looking for an easier way, so I turned to my friend BASH and a tool called SEQ.  SEQ will give you a sequence of numbers.  For example if you just wanted 10 numbers you could do the following:

[matt@localhost ~]$ seq 5
1
2
3
4
5

So for this task I needed to bring up IP addresses from 2 through 254.  Running “seq 2 254″ will give me a sequence from 2 to 254, I need more than just to have a list of numbers, I actually need to use them, so here’s the syntax I used to use the numbers to bring up each of the interfaces

for i in $(seq 2 254)
do
ifconfig eth0:$i 1.1.1.$i netmask 255.255.255.0 up
done

Obviously in the above example, you would substitute the sequence you want to use and the IP subnet you want to use.  Also, this syntax would put 1.1.1.2 on sub-interface eth0:2, 1.1.1.3 on sub-interface eth0:3, etc.