Luarocks and Moonscript Installation

* Get luarocks distribution -

# wget http://luarocks.org/releases/luarocks-2.0.6.tar.gz

# tar xvzf luarocks-2.0.6.tar.gz 
# cd luarocks-2.0.6

* Find lua path and configure with lua

# find / -name lua.h

# ./configure --with-lua-include=/opt/lua/include/ --with-lua=/opt/lua/

# make
# make install

* Now Install moonscript

# luarocks install moonscript


Thanks

Installing redis server

Hi, folks
I am gonna show you how to install 'redis' server and setup master slave on a linux
box. Redis is a key-value-store database and is an open source. Here I'll set up two redis instance one acting as a master and other slave - location /opt and /var

Steps for the Redis Configuration :

Step 1:

Redis Download :

 # wget http://redis.googlecode.com/files/redis-2.6.10.tar.gz

Step 2:

Then extract it :

# tar -zxvf redis-2.6.10.tar.gz# cd redis-2.6.10

Step 3 :

Run make

Step 4 :

make test

Step 5 :  Master configuration

a. ) /opt/redis-2.6.10/redis.conf

b. ) daemonize yes

c. ) port 6379 ( you can change it)

Step 6: Client configuration

a. ) /var/redis-2.6.10/redis.conf

b. ) daemonize yes

c. ) port 6380

d. ) slaveof Master IP 6379

Syntax : slaveof <Master Ip> <Master port no>

  Step 7 :

Now go to master server and type :

# cd /opt/redis-2.6.10/src

Step 8 :

# ./redis-server /opt/redis-2.6.10/redis.conf

Step 9 :

# ./redis-cli -p 6379

Step 10 :Then switch to slave terminal and type :

# cd /var/redis-2.6.10/src

Step 11:

# ./redis-server /var/redis-2.6.10/redis.conf

# ./redis-cli -p 6380

Step 12 :

then to check replication type :

info

Output:

Replication
role:slave
master_host:<Master IP >
master_port:6379
master_link_status:up
master_last_io_seconds_ago:3
master_sync_in_progress:0
slave_priority:100
slave_read_only:1
connected_slaves:0

So, this is how we setup redis server master- slave.
Looking forward your comments and suggestions. Thanks !    

How to encrypt and decrypt a file

In Linux there is a utility 'gpg' which is used for encrypting and decrypting of  files.
Syntax -

$ gpg -c filename

[ It will then prompt to enter a pass phrase and it will create a file 'filename.gpg' ] 

Enter passphrase:<YOUR-PASSWORD>
Repeat passphrase:<YOUR-PASSWORD>

To decrypt -
 $ gpg filename.gpg        

[ Now give the same pass phrase to decrypt the file ]

gpg filenme.gpg
gpg: CAST5 encrypted data
Enter passphrase:<YOUR-PASSWORD>

Nginx Openresty

                                                   Lua nginx openresty configuration


* OS - Centos 6, 64bit
* Dependencies - pcre.x86_64, pcre-devel.x86_64
* With module geoip

Building from source distribution - ngx_openresty-1.2.6.6.tar.tgz , GeoIP-1.4.8.tar.gz

Extract and copy to  /opt/

# cd /root/openresty/GeoIP-1.4.8
# ./configure ( if libtool error occure install below)
# make
# make install

# cd /opt/ngx_openresty-1.2.6.6/
# ./configure --with-luajit
# make 
# make install

Testing 

# mkdir /root/luatest
# cd /root/luatest
# mkdir conf logs

Edit nginx.conf and paste following

#
worker_processes  1;
error_log logs/error.log;
events {
            worker_connections 1024;
}
http {
            server {
                            listen 8080;
                                    location / {
                                                        default_type text/html;
                                                                    content_by_lua '
                                                                                    ngx.say("<p>hello, world</p>")
                                                                                                ';
                                                                                                        }
                                                                                                            }
}

#

Start nginx -

# /usr/local/openresty/nginx/sbin/nginx -p /root/luatest -c /root/luatest/conf/nginx.conf

If error comes for lib, execute following and then again start nginx

# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

# curl http://127.0.0.1:8080/  



Thanks

Shell Script : How to use an array to get n number of similar set for different values

In this example we are going to obtain below set (used in nagios config to define a host) using array -

define host{
              use                     linux-server
              host_name              <host>
              alias                        <host>
              address                    <ip>
            }

I have two files, one contains hostname  and second has their IPs.

host.txt           ip.txt 
host1               x.1
host2               x.2
host3               x.3
host4               x.4

Code :

#!/bin/bash

#### Storing the host names in array 'arr', reading from host.txt

j=1
for i in `cat host.txt`
do
  arr[$j]=$i

  j=`expr $j + 1`
done

#### Storing the IP's names in array 'arrp', reading from ip.txt

k=1
for m in `cat ip.txt`
do
  arrp[$k]=$m

  k=`expr $k + 1`
done

#### Taking count of the records

count=`cat host | wc -l`

#### Printing the values stored in the two arrays for making the set

for x in `seq 1 $count`
do
echo "define host{
              use                     linux-server
               host_name           ${arr[$x]}
              alias                      ${arr[$x]}
              address                 ${arrp[$x]}
            }"  
done
##############

Output : 

define host{
              use                     linux-server
               host_name        host1
              alias                   host1
              address              x.1
            }
define host{
              use                     linux-server
               host_name        host2
              alias                   host2
              address               x.2
            }
define host{
              use                     linux-server
               host_name         host3
              alias                    host3
              address                x.3
            }
define host{
              use                     linux-server
               host_name          host4
              alias                     host4
              address                x.4
            }

 Like above we can use array to make similar n number of sets for different values instead of  writing multiple times :)  

Shell Script : How to move all empty files using a shell script

 Here I am giving a simple bash shell script to move all the empty files (zero byte) from current directory to another location-

1. Go to that path where you want to move the files from.
2. Make a text file and edit the following lines (accordingly) -

 #!/bin/bash

for i in `ls`

do 

if [ -s $i ]

then

echo "The file $i has data"

else 

mv $i  <Path where you want to move the empty files>

fi

done

3. Save the file, give execute permission (chmod +x filename) and run.

Hope you've got the desired result.. : ) 

How To Replace A String Recursively

To replace a string in all files recursively with your matching string globally, The following perl utility is very helpful -

Go to the directory and run -

# perl -p -i -e 's/string/replace-with/g' `grep -rl string *`



If your string contains '/' for example you have to replace '/home/sandy' with '/home/mandy' the use it like -

# perl -p -i -e 's/\/home\/sandy/\/home\/mandy/g' `grep -rl sandy *`


If you have any query, please post your comments.
Thanks !

Shell Script : Bash Shell CGI

Shell script with cgi allows you to run linux commands/scripts from web. It can be used to build a monitoring tool for your linux machines/servers or managing your servers from web.

For this you should have cgi integrated with your web service nginx/apache and the user should  have access the cgi document root and  permission to run cgi script.

Here I'm giving an example how you can have some system information by running linux commands/shell script code -


#!/bin/bash

echo "Content-type: text/html"
echo ""
echo "<html><body><head><title>Bash Shell CGI"
echo '</title></head><body bgcolor="#3EA99F">'

### Getting information into veriables

load=`uptime | awk '{print $8,$9,$10}'`

Date=`date +'%Y-%m-%d'`


#### Load Test

### Getting current load avg integer value

la=`uptime | awk '{print $8}' | awk -F '.' '{print $1}'`

## Doing a test

if [ $la -gt 10 ]
then
stat="Critical"
elif [ $la -le 10 ]
then
stat="OK"
fi

####### Printing the information


echo "<h1><font color="blue"><u>System Information</u></font></h1><br><br>"
echo "<font color="black"><h3>Hostname : <font color="white">`hostname`</font></h2></font><br>"
echo "<font color="black"><h3>Load Average : <font color="white">$load - $stat</font></h2></font><br>"
echo "<font color="black"><h3>Date : <font color="white">$Date</font></h2></font><br>"
echo "</body></html>"
 
##### 


Put this file to your cgi-bin directory and run in browser using -


http://your-domain/cgi-bin/filename.cgi


Output : 

Thanks...! If you have any query or suggestions, please comment.
  

How To Setup Voltdb Cluster on AWS Cloud (Part - 2/2)

VoltDb Setup :  (3 nodes cluster ) 

===========================  

Download the voltdb community edition, extract it and move to /opt as /opt/voltb

wget http://voltdb.com/downloads/technologies/server/LINUX-voltdb-3.0.tar.gz

Download and install the java jdk-6u41-linux-x64.bin

http://www.oracle.com/technetwork/java/javase/downloads/jdk6downloads-1902814.html  

Copy to /var , give execute permission and run 

# ./jdk-6u41-linux-x64.bin

Install php-5.3.X

yum install php

Set path variable for java and voltdb (Also put in ~/bashrc)

# export PATH="$PATH:/opt/voltdb/bin" 

# export JAVA_HOME=/var/jdk1.6.0_41 

Node-1 (Leader) -

1. Create a dir /opt/testvoltdb and create a helloworld.sql in it and edit below lines -

CREATE TABLE HELLOWORLD (

HELLO VARCHAR(15),

WORLD VARCHAR(15),

DIALECT VARCHAR(15) NOT NULL,

PRIMARY KEY (DIALECT)

);

[ It will create a table HELLOWORLD ]

2. compile the sql file to create the new catalog

# voltdb compile -o helloworld.jar helloworld.sql

3. Create the deployment.xml file containing the info about how many nodes and partition needs to be used -

# vi deployment.xml

<?xml version="1.0"?>

<deployment>

<cluster hostcount="3"

sitesperhost="4"

kfactor="2"

/>

<paths>

<voltdbroot path="/opt/testvoltdb" />

<snapshots path="/opt/testvoltdb/snapshot" />

</paths>

</deployment>

4. Create and start the database

# voltdb create host <node1-hostname> catalog helloworld.jar deployment deployment.xml &

# sqlcmd

SQL Command :: localhost:21212

1> INSERT INTO HELLOWORLD VALUES(

2> 'You','Are', 'Welcome!');

(1 row(s) affected)

3> SELECT * FROM HELLOWORLD ;

HELLO WORLD DIALECT

------ ------ --------

You Are Welcome!

(1 row(s) affected)

5. Copy the runtime catalog (i.e helloworld.jar ) and deployment.xml to the other nodes on same path /opt/testvoltdb

Node 2 & 3

6. start the database and join the nodes to main cluster node (node-1)

# voltdb create host <node1-hostname> catalog helloworld.jar deployment deployment.xml &
# sqlcmd
1 > SELECT * FROM HELLOWORLD ;
HELLO WORLD DIALECT
------ ------ --------
You Are Welcome!
(1 row(s) affected)   
Thanks....!

How To Setup Voltdb Cluster on AWS Cloud (Part - 1/2)

What is VoltDb:

VoltDB is focused on specific workloads. Most existing RDBMSs are designed to be general purpose, one-size-fits-all systems.

VoltDB was designed to be the most scalable transaction processing system out there, often making compromises unsuitable for other workloads. For non-OLTP workloads, VoltDB is built to work in concert with other specialized systems.

Why Voltdb:

We believe that a set of specialized data management tools can replace the functionality of one-size-fits-all systems, increasing performance, scalability and fault tolerance dramatically.

Instalation Requirement :

1.CentOS 5.8 or later or 6.3 or later and Ubuntu 10.4 or 12.4 (64 Bit )

2.Memory 4 GB

3.sun jdk 6 or higher

4.NTP

For Clustering we require to choose : Cloud Cluster Instance in Amazon.

While creating the instance we Create a Palcement group , Security Group .

Note . Cc2.x large or greater Image to be chosen.

( Note: Make sure to open the port 3021 on all the nodes for voltdb connection and 123 for NTP)

5. PHP 5.3

Performance BENEFITS:
VoltDB is based on 3 big concepts. While none of these concepts are new ideas individually, VoltDB builds all three into the core of the product.
Concept 1: Exploit repeatable workloads.
VoltDB exclusively uses a stored procedure interface. It expects an application’s complete set of procedures to be known in advance. This allows it to pre-optimize execution paths for incoming transactions. Since most OLTP applications perform the same set of operations over and over, this model is a good fit for OLTP.
That’s not to say VoltDB is inflexible. When the application changes, the set of stored procedures can be amended or updated. VoltDB also supports some ad-hoc access for administrative tasks.
Concept 2: Partition data to horizontally scale.
VoltDB divides data among a set of machines (or nodes) in a cluster to achieve parallelization of work and near linear scale-out. Unlike custom sharding or partitioning solutions, this functionality is a native and fundamental feature. VoltDB manages consistency and distribution across cluster nodes and replicates data to multiple nodes to ensure smooth operation during many types of failure.
Concept 3: Build a SQL executor that’s specialized for the problem you’re trying to solve.
OLTP means high throughput of write-heavy transactions, each operating on a relatively small subset of data returning relatively small result sets. Unlike analytical workloads, few transactions scan more a few thousand rows, and most scan only a handful. Since our stored procedures a) never wait for disk access, b) never wait for external input in the middle of a transaction and c) never read or modify huge amounts of data, they can be executed on modern processors in microseconds. If they take microseconds, why interleave their execution with a complex system of row and table locks and thread synchronization? It’s much faster and simpler just to execute work serially.
In order to run single-threaded in a world of multi-core, VoltDB partitions data by core, not by cluster node. A three node cluster with 8 cores / node will run about 24 lean mean single-threaded SQL machines.
Bonus: Leverage the 3 concepts together.
Because workloads are known in advance, replication can be done by performing the same procedure twice in two places. This is simpler and faster than log-shipping changed tuples.
Because data is replicated synchronously, don’t bother with a write-ahead-log at each SQL executor to maintain consistency on a single node.