Thursday, May 24, 2007

Usefull Shell commands

Replacing a parameter

x=/home/beyondm/account
src="/"
dis="\/"
result=`echo $x | sed s\}$src\}$dis\}\g`

echo $result will result
/home/beyondm/account

Thursday, May 10, 2007

Exciting Open Source ETL - Pentaho Data Integration: KETTLE

Pentaho Data Integration delivers powerful Extraction, Transformation and Loading (ETL) capabilities using an innovative, metadata-driven approach. The intuitive, drag-and-drop design of Pentaho Data Integration increases productivity and it’s extensible, standards-based architecture ensures that you will never be forced to adopt proprietary methodologies in your ETL solutions.

What is Kellte, spoon, pan and Kitchen?

Kettle is an acronym for “Kettle E.T.T.L. Environment”. This means it has been designed to help you with
your ETTL needs: the Extraction, Transformation, Transportation and Loading of data.

Spoon is a graphical user interface that allows you to design transformations and jobs that can be run with the Kettle tools Pan (transformations) and Kitchen(jobs). Pan is a data transformation engine that is capable of performing a multitude of functions such as reading, manipulating and writing data to and from various data sources. Kitchen is a program that can execute jobs designed by Chef in XML or in a database repository. Usually jobs are scheduled in batch mode to be run automatically at regular intervals.

Pentaho Data Integration 2.5 Now Available!

This exciting new release features ehanced MySQL support, advanced error handling, support for the Apache Virtual File System, and numerous usability improvements.

Project URL : http://www.pentaho.com/index.phpDownload : https://sourceforge.net/project/showfiles.php?group_id=140317&package_id=186321

Friday, May 4, 2007

Important MySQL commands

Restoring MySQL dump

Option 1 - Having a DB Dump with insert into statements
* Go to the corresponding directory and run following commad
* mysql -u user -ppassword -D dst_temp -h 192.168.0.57 -e "source LAP_BONUS_EVENT";


Option2 - Having backup with comma-separated format
* Copy the file to Mysql data directory (eg: /usr/local/mysql/data/dst_temp)
* Go to the Mysql data directory and loggin to the mysql prompt (mysql -uuser -ppassword) and run the following command
* LOAD DATA INFILE 'filename' INTO TABLE VOICE_CALL_EVENT FIELDS TERMINATED BY ',';



Change the MySQL grant privileges
* GRANT ALL PRIVILEGES ON *.* TO user@"%" IDENTIFIED BY 'password' WITH GRANT OPTION;


Get a MySQL dump
* Go to MySQL home (eg: /usr/local/mysql)
* execute ./bin/mysqldump -u user -ppassword dst_report > dst_report_08_04_2007.txt


To get no of mysql connections acquired
* mysql> show processlist; (in mysql prompt)

Did you ever encounter a java.lang.OutOfMemoryError: PermGen

Did you ever encounter a java.lang.OutOfMemoryError: PermGen space error when you redeployed your application to an application server?

What is PermGen space anyways? The memory in the Virtual Machine is divided into a number of regions. One of these regions is PermGen. It's an area of memory that is used to (among other things) load class files. The size of this memory region is fixed, i.e. it does not change when the VM is running.

You can specify the size of this region with a commandline switch: -XX:MaxPermSize . The default is 64 Mb on the Sun VMs.

Check interesting link : http://blogs.sun.com/fkieviet/entry/classloader_leaks_the_dreaded_java