SysAdmin Valley

wget is a great tool for grabbing web pages from the command line, but one issue is it downloads the file and saves to the local directory.  Some times you just want to see the output of what is retrieved.  To do this, use the following command:

wget -qO- http://www.mywebsite.com/file.htm

In the past I’ve needed a way to backup an entire cPanel account on my servers so the following has come in real handy.  Just run the following command from the shell and substitute [username] for the username you want to backup (also remove the []).  At the end of the backup process it will tell you where the backup file is located, usually it’s in the /home/ directory on most servers and is named cpmove-username.tar.gz

/scripts/pkgacct [username]

Find is one of my favorite little tools under linux.  It helps me “find” almost anything, I can find files older than a certain date, newer than a certain date, modified on a certain date.  I can find files that have a certain name, or match a part of a name, file extension.  Once I’ve found what I’ve been looking for I can have find do something with those files like delete them or gzip them.

My latest “find” with the find command came about because on one of my JBoss servers I wrote a simple script that looks for log files older than 15 days and deletes them and looks for other log files older than 61 minutes and compresses them with gzip.

#!/bin/bash
LOGS=/usr/local/jboss/server/all/log/
#delete all logs older than 37 days
find $LOGS -mtime +15 | xargs rm -rf
# gzip files last modify at least 1 hour ago
find $LOGS -mmin +61 | xargs gzip

Our JBoss setup automatically writes new logs to server.info.log and server.error.log, then every every hour it renames the INFO and ERROR log to the current date + hour, so server.info.log would be changed to server.info.log.2010-03-09-13 for today at 2pm to roll out the 1pm logs.

The problem I came across in my script was with my server.error.log file.  If an error hasn’t been written to the server.error.log file during that hour, it wasn’t going to rotate an empty error log.  Since the file hadn’t been touched/updated/modified in over 61 minutes, my script came along and gzipped it, at this point JBoss then had a problem because the error log was missing and didn’t create a new one.

So what I needed to do was to find all the files that matched the criteria, but exclude the server.info.log and server.error.log and here is my final script.

#!/bin/bash
LOGS=/usr/local/jboss/server/all/log/
INFOLOG=”server.info.log”
ERRORLOG=”server.error.log”
#delete all logs older than 37 days
find $LOGS -mtime +15 -not -name “$INFOLOG” -not -name “$ERRORLOG” | xargs rm -rf
# gzip files last modify at least 1 hour ago
find $LOGS -mmin +61 -not -name “$INFOLOG” -not -name “$ERRORLOG” | xargs gzip

Do you have multiple computers? Maybe one at work and one at home, or maybe you have two at work. Have you ever said I would love to have a way to keep my documents on multiple computers at the same time and not have to worry about copying the files back and forth. Or maybe you’re just looking for a simple way to backup your files in case of emergencies. Well, you should check out DropBox.

DropBox is a new way to manage your files. First you create an account one their website, then you install a piece of software on all your computers. Then when you add a file into your DropBox on one of your computers, immediately all your other computers download the new file. If you make a change to that document on any of the computers, all the others immediately pickup the change. No more copying your documents to CD/DVD/USB and then over to your other computer. Now all your documents are synchronized immediately.

Here’s another benefit.  If you have multiple computers that have different operating systems, this doesn’t matter.  You can automatically synchronize files between Windows, Mac and Linux too.

Another great feature of DropBox, have you ever accidentally deleted or changed a file and wished that you had a backup copy of it. Well now you do. Now you can recover deleted files or get back previous versions of your documents.

To get started, just go to DropBox and click Download DropBox.

I love bash and scripting!!!  There’s almost nothing you can’t do with a shell script that would take me forever if I had to write an app to do the same.  So here’s another quick tip.  To convert text in a file from uppercase to lowercase use the following:

cat FILENAME | tr “[:upper:]” “[:lower:]“

Or to go from lowercase to uppercase:

cat FILENAME | tr “[:lower:]” “[:upper:]“

Another quick hint.  While working on a server I needed to bring up a whole lot of IP addresses (200 to be exact).  If I really wanted to I could bring them all up like this:

ifconfig eth0:2 1.1.1.2 netmask 255.255.255.0 up
ifconfig eth0:3 1.1.1.3 netmask 255.255.255.0 up
and so on to…..
ifconfig eth0:200 1.1.1.200 netmask 255.255.255.0 up

Well, I’m always looking for an easier way, so I turned to my friend BASH and a tool called SEQ.  SEQ will give you a sequence of numbers.  For example if you just wanted 10 numbers you could do the following:

[matt@localhost ~]$ seq 5
1
2
3
4
5

So for this task I needed to bring up IP addresses from 2 through 254.  Running “seq 2 254″ will give me a sequence from 2 to 254, I need more than just to have a list of numbers, I actually need to use them, so here’s the syntax I used to use the numbers to bring up each of the interfaces

for i in $(seq 2 254)
do
ifconfig eth0:$i 1.1.1.$i netmask 255.255.255.0 up
done

Obviously in the above example, you would substitute the sequence you want to use and the IP subnet you want to use.  Also, this syntax would put 1.1.1.2 on sub-interface eth0:2, 1.1.1.3 on sub-interface eth0:3, etc.

From time to time I need to use Curl to test websites and sometimes I need to make the request come from a different IP address, other than the server’s default IP.  Note, the IP address you use must be active on your server.  To do this, use the following syntax:

curl --interface xxx.xxx.xxx.xxx -s http://www.sysadminvalley.com

Make sure that you substitute xxx.xxx.xxx.xxx with the IP address you want to use.

There are times when you want to make changes to your website and you do not want your visitors to see the site before you have finished deploying and testing the website.  Here is an example using Apache, mod_rewrite and a cookie set by a PHP page.

First, create your maintenance webpage called maintenance.html.  Second, create a file called set_cookie.php with the following contents

<?php
setcookie(“testing”, “testing”, time()+36000);  /* expire in 600 minutes */
?>

Next, create a file called .htaccess in your main web directory with the following contents

RewriteEngine on
RewriteCond %{HTTP_COOKIE} !testing
RewriteCond %{REQUEST_URI} !/maintenance.html$
RewriteCond %{REQUEST_URI} !/set_cookie.php$
RewriteCond %{REQUEST_URI} !/logo\.jpg$
RewriteRule ^(.*) /maintenance.html [NC,L]

Lastly, in your web browser, go to http://www.yourdomain.com/set_cookie.php.  From that point on, you will be able to browse your website, but your visitors will be redirected to your maintenance.html webpage

For this you will need the openssl package.  First we want to start by generating a private key.

root@localhost# openssl genrsa -out www.mydomain.com.key 1024
Generating RSA private key, 1024 bit long modulus
………….++++++
………………..++++++
e is 65537 (0×10001)

Then, we need to generate the certificate request and fill in the appropriate information.  Make sure that the “Common Name” matches the domain you want to protect via SSL, so if you domain was www.mydomain.com, use that.  If you wanted to protect mydomain.com (without the www.) then use that.

root@localhost# openssl req -new -key www.mydomain.com.key -out www.mydomain.com.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter ‘.’, the field will be left blank.
—–
Country Name (2 letter code) [GB]:
State or Province Name (full name) [Berkshire]:
Locality Name (eg, city) [Newbury]:
Organization Name (eg, company) [My Company Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, your name or your server’s hostname) []:www.mydomain.com
Email Address []:email@mydomain.com

Please enter the following ‘extra’ attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Next, generate the self-signed certificate. You can specify the number of days the cert is valid for.

root@localhost# openssl x509 -req -days 365 -in www.mydomain.com.csr -signkey www.mydomain.com.key -out www.mydomain.com.crt
Signature ok
subject=/C=/ST=/L=/O=/CN=www.mydomain.com/emailAddress=email@mydomain.com
Getting Private key

Next, move the certificate and keyfile into apache’s SSL directory.

mv www.mydomain.com.key /etc/httpd/conf/ssl.key/
mv www.mydomain.com.crt /etc/httpd/conf/ssl.crt/

Finally, we configure our SSL virtual host in Apache.  The simplest way to do this is to copy the virtual host for the site you want to make SSL, then add/change the following bits.

<VirtualHost 192.168.1.100:443>
…
SSLEngine on
SSLCertificateFile /etc/httpd/conf/ssl.key/www.mydomain.com.key
SSLCertificateKeyFile /etc/httpd/conf/ssl.crt/www.mydomain.com.crt
…
</VirtualHost>

Here is a quick tip to help doing search and replace in multiple files.    You may need to change to fit your needs.

for file in `ls *.php`
do
sed -e ‘s/Copyright 2008/Copyright 2009/’ “$file” > tmp_file
mv -f tmp_file “$file”
done

What this does is get’s a list of all php files in the current directory, puts them in a loop with a variable called FILE, then does a sed search and replace calling the new file tmp_file and moving the temp file back in place.  If you wanted to do it for all files including in subdirectories, you could substitute:

ls *.php

with:

find ./ -name “*.php”