30. Apr. 2009

Lets assume you have enabled the monitoring capabilities of your tomcat.
If you do not know how to achieve this see my earlier posting here about.

Now it is cool to monitor your application on a regular basis.
Because I’m used to shell scripting in LINUX I present you here a scripting solution within a bash shell. But I’m sure you can do the same in Windows.
The idea is to set up a cron job ( for windows use the scheduler) and get the statuspage from your tomcat every – lets say – five minutes.
Get some of the interesting values out of this page and save them in a csv – File.
Create a new file every day. Perhaps you want to use this files as input for a spreadsheet program to create some nice charts.

But to be honest – this is a very special script – you have to adopt it to your needs.
This are the basic prerequisites:

  • you need linux to run it (this script is developed on a SUSE – Distribution)
  • eventually you need to install some tools (I think the w3m tool is not bundled with every linux distribution)
  • this script only! works correct with a tomcat 6.0 version

#!/bin/bash
# -configure below-

HTTPPORT=”8080″    # The http Port of your tomcat
JKPORT=”8709″      # The Modjk Port of your tomcat (if your tomcat should be reachable from a apache webserver)
SERVER=”localhost:$HTTPPORT” # you can even monitor a tomcat on another server!
USER=”tomcat-oper” # the account and
PW=”IwantItAll”    # password you have defined in tomcat-users.xml
APPL=”CoolAppl”    # The name of your application
# -configure above-
TODAY=`date ‘+%m-%d-%y’`
TIME=`date ‘+%H:%M:%S’`
OUTFILE=”tomcat-stats-${TODAY}.csv”
#
#set -x   # uncomment if you want to improve the script or if you are searching for a bug
#
# # If there is no csv-file create one and put a headerline in
if !(test -e $OUTFILE) then
  echo “Time,FreeMemory,UsedMemory,MaxMemory,HttpThreadsMax,
  HttpThreadsBusy,ModJKThreads,ModJKThreadsBusy,Sessions” > $OUTFILE
fi
# # get the statuspage from tomcat – filter it for the wanted informations – and create a temporary file
  w3m -dump http://${USER}:${PW}@${SERVER}/manager/status/all| \
  egrep “(memory|thread|Active sessions|
  ^JVM|$HTTPPORT|$JKPORT|${APPL})” >/tmp/tomcat-status
#
#
# put the values in variables / uncomment to get a better understanding of the process
#
while read line
   do
      case $line in
           JVM) read line
                JVM_VALUE=`echo $line | awk ‘{print($3 “,” $7 “,” $11)}’`
#               uncomment below if you want to debug
#               echo “JVM_VALUE: <$JVM_VALUE>”

                ;;
            *${HTTPPORT}) read line
                HTTP_MAX_THREADS=`echo $line |awk ‘{print(“,” $3)}’`
#               uncomment if you want to debug
#               echo “HTTP_MAX_THREADS: <$HTTP_MAX_THREADS>”
                HTTP_BSY_THREADS=`echo $line |awk ‘{print(“,” $11)}’`
#               uncomment
below if you want to debug
#               echo “HTTP_BSY_THREADS: <$HTTP_BSY_THREADS>”
                ;;
            *${JKPORT}) read line
                MODJK_MAX_THREADS=`echo $line |awk ‘{print(“,” $3)}’`
#               uncomment below if you want to debug
#               echo “MODJK_MAX_THREADS: <$MODJK_MAX_THREADS>”
                MODJK_BSY_THREADS=`echo $line |awk ‘{print(“,” $11)}’`
#               uncomment below if you want to debug
#               echo “MODJK_BSY_THREADS: <$MODJK_BSY_THREADS>”
                ;;
            *${APPL}) read line
                SESSION_CNT=`echo $line |awk ‘{print(“,” $3)}’`
#               uncomment below if you want to debug
#               echo “SESSION_CNT: <$SESSION_CNT>”
                ;;
            *) ;;
      esac
    done < /tmp/tomcat-status
# # Now is the time to write the data to the csv-file
  echo “${TIME},${JVM_VALUE}${HTTP_MAX_THREADS}${HTTP_BSY_THREADS}
  ${MODJK_MAX_THREADS}${MODJK_BSY_THREADS}${SESSION_CNT}” >>$OUTFILE
exit

*) be aware that based on word wrapping this script might not work instantly

Create a scriptfile (e.g. tomcat-check.sh) and apply the script to your tomcat. Then edit your crontab.

There should be a line like:
“*/5 * * * * cd ~/statistics; ./tomcat-check.sh 1>tomcat-check.log 2>&1”

And then the script should create the csv-files. I’m not used to excel – but even I managed to create a more or less interesting diagram from this data. Note how regular the memory behaves while there is no user session.

Tomcat Monitoring Graphic 

14. Dec. 2008

Sometimes we have to place content on our webserver we want to protect and only accessed by privileged users.

A basic security can be established by a password restricted folder access, enforced by the Apache Webserver itself.

This can be done by some simple instructions added to our .htaccess file:
(In case you haven’t already a .htaccess file, just create a new file calling it .htaccess and locating it in the folder you want to protect.)

AuthType Basic
AuthName “Protected Area”
AuthUserFile FILE
require valid-user

These commands will tell the Apache webserver that the folder in question (and all below) are now protected and it needs a valid combination of username/password to access it.

  • AuthName is some free form text you can specify, telling the visitor what you are asking for and Apache Webserver will show in it’s dialog box.
  • AuthUserFile is the reference to the stored .htpasswd file containing all valid username/password combinations to access the folder in question (ususally and best located out of the protected directory tree).

Having that done we now need to create the necessary .htpasswd file which we can do interactive on our server with the command:

htpasswd2 -c FILE USER

Assuming html is your webroot, a valid example, for protecting a folder called images below our webroot and using demouser/demopassword as user credentials, would be:

.htaccess:
(Locate this file within the folder images and be aware to use an absolute path for AuthUserFile, as for www.domain.com would be wrong!)

AuthUserFile /home/username/html/.htpasswd
AuthType Basic
AuthName “Login for Secure Area”
require valid-user

.htpasswd
(Create a new file called .htpasswd and locate this file within the folder you’ve specified within AuthUserFile path.)

demouser:$1$OHq3K5v3$pvFrGsHjWwYjpkZhc70T9.

10. Dec. 2008

Since Oracle Version 10G ist is possible to transfer tablespaces no longer only between platforms of the same architecture. Now in 10G it also possible to transport from any platform to any platform, provided the platforms belong to the list below:

SQL>  select * from v$transportable_platform;

   PLATFORM_ID PLATFORM_NAME                    ENDIAN_FORMAT
  
             1 Solaris[tm] OE (32-bit)          Big
             2 Solaris[tm] OE (64-bit)          Big
             7 Microsoft Windows NT             Little
            10 Linux IA (32-bit)                Little
             6 AIX-Based Systems (64-bit)       Big
             3 HP-UX (64-bit)                   Big
             5 HP Tru64 UNIX                    Little
             4 HP-UX IA (64-bit)                Big
            11 Linux IA (64-bit)                Little
            15 HP Open VMS                      Little
             8 Microsoft Windows IA (64-bit)    Little
             9 IBM zSeries Based Linux          Big
            13 Linux 64-bit for AMD             Little
            16 Apple Mac OS                     Big
            12 Microsoft Windows 64-bit for AMD Little

But be aware that depending on which plattforms we deal with there might be some converting necessary or not!

To help you with that, I have carried together the little cook book about “How to do” below:

1. Set the tablespace to READ ONLY

SQL> alter tablespace USERS read only;
Tablespace altered.
 

2. Export metadata

exp userid=\’/ as sysdba\’ transport_tablespace=y
tablespaces=users
file=users_tts.dmp log=exp_users_tts.log
statistics=none

3. Converting tablespace if necessary

Depending on we have to convert the file between endianness Little and endianness Big (forward or backward) we can do that with RMAN:

rman target=/

Recovery Manager: Release 10.2.0.3.0 – Production on Wed Dec 10 13:10:57 2008
connected to target database: orav10 (DBID=1953618181)

RMAN> convert tablespace ‘USERS’
2>  to platform=”Linux IA (32-bit)”
3>  db_file_name_convert=’/database/orav10/datafiles/users.dbf’,
4>  ‘/tmp/users.dbf’;

4. Move datafiles and export dump file

$ftp users_tts.dmp

and /database/orav10/datafiles/users.dbf (no conversion)
or /tmp/users.dbf (converted file if conversion had been required)

5. Import metadata

imp userid=\’/ as sysdba\’ TRANSPORT_TABLESPACE=Y
datafiles=
/database/orav10/datafiles/users.dbf
(or
/tmp/users.dbf )
file=users_tts.dmp log=imp_users_tts.log

6. Set the tablespace to READ WRITE

SQL> alter tablespace USERS read write;
Tablespace altered.

next »