Solution : Yum breaks after python upgrade

 Use this method to make both python 3 & 2 work without breaking yum.  


*   Link your default python path to use python3

# ln -s /usr/bin/python3.6 /usr/bin/python

*   Link python2.x as python2

 # ln -s /usr/bin/python2.7 /usr/bin/python2

*   Edit yum binary  and change the python path to the following -

# vi /usr/bin/yum
#!/usr/bin/python2


Test :

-bash-4.3$ /usr/bin/python -V
Python 3.6.4

-bash-4.3$ /usr/bin/python2 -V
Python 2.7.14

-bash-4.3$ yum update # or anything

Note that the first two you can link as per the default you want to use. Keeping yum separate by changing python path is making sure that it uses python 2x always regardless your default setting.  




 

 

Python way to query a database and execute a shell command

The following example shows how to connect to a database, query the database and use the result values to perform some operation using python.

In this script, there's a local database which has some employee inventory, We will query some record and send the result as an email notification.


# Including shebang and title 

#!/usr/bin/python
# Python script to query a database and send mail alert using shell command

# Importing modules and Setup mysql connection

# Import Modules

import MySQLdb
import sys
import subprocess

# Setting up DB Connection

connection = MySQLdb.connect (host = "localhost", user = "bingo", passwd = "kurkure", db = "emp_db")

# Cursor Object
cursor = connection.cursor ()

cursor.execute ("select ID,emp_email,manager_email where keyvalue='somevalue';")

# Reading rows and storing to variables 

# Fetching results
data = cursor.fetchall ()
for row in data :
        ID = row[0]
        emp_email = row[1]
        manager_email = row[2]
        print hash,user_email,manager_email;

# Sending email notification by running a shell script

        runn = subprocess.check_call("/<path-to-script>/mail %s %s %s" % (str(row[0]), str(row[1]), str(row[2])), shell=True)

cursor.close ()
connection.close ()

sys.exit()

######## /<path-to-script>/mail script will look like -

'''
echo -e "Hello, \n\n\n Employee notification for - $1) $2  \n\n\n Thanks" | mail -r "do-not-reply@247-inc.net"  -s $"New Record - $2" $3

'''

Open TSDB

OpenTSDB A Time Series Database

It has TSD (time series daemon) and many command line utility. TSD uses Hbase store and retrieve data. It supports LZO, Snappy, hadoop-lzo compression method.We can communicate with the TSD via telnet protocol, HTTP API, Built in GUI

Installation Requirements -

•HBase 0.92 or later
•GnuPlot 4.2 or later
•Java Development Kit 1.6 or later
•Autotools
•Make
•Python
•Git


H/w Requirements -

•CPU cores: Max (Limit to 50% of your available CPU resources)
•RAM: Min 16 GB
•Disk 1 - OS: 10 GB - Thin provisioned
•Disk 2 - Data: 100 GB - Thin provisioned


Performance & HA

Reliability

•Failure modes :

The TSD becomes unhealthy when HBase itself is down or broken for an extended period of time.
TSD will discard incoming data points when it is unable to reach HBase and unable to flush them to HBase because of the buffer exhaust.
Another case when TSDB unable to keep up with load when Tcollector running on hundreds machines and bombarding TSD.

•Points of Failure of OpenTSDB :
OpenTSDB itself doesn't have any specific SPoF as there is no central component and you can run multiple TSDs on different machines.

Bash Shell Script Template Guide - Part (1/2)

Hi Guys,

Want to write a bash script and don;t know the right structure ? This could help you to frame your script in standard way.  It's not exactly what it should be like but somewhat more appropriate way which makes it easy to use, more accurate, enhance readability,  and makes it look nice :)

• The very first thing it is supposed to have 'shebang' 

          #/bin/bash 

⇉  Tips: Better to use #!/bin/env bash instead
  

•  Next you can include script title,description,author,version etc.                                             

           # title                 : Myscript
           # description      : Blowing up the servers
           #
           # author             : Mahendra Shrivastava
           # approval          : John Do
           # date                 : 20170130
           # version            : 1.0.2

• Display Usage  - It's better to have a usage display, defining how your script is supposed to be executed, how many 'arguments' it needs and if it has any dependencies on other file or programs etc. 

          usage()
         {
         cat << EOF

         usage: sh $0 options

         This script requires user file to pass
         eg. sh $0 -f /<Path-to-file>/User-details

         OPTIONS:
                  -f [ master file ]  File containing user details

                  -h Help

         EOF
         }
         null=''"
        if [ "$1" = $null ] then
            usage; exit ; fi

⇉Tips: If you are familiar  with 'getopts', include it to have more accuracy.  

•  Path & Definitions    - Set Environment PATH and all  required definitions  
  

           ## Path

           PATH=$PATH:/usr/local/bin:/usr/bin; export PATH

           ## Definitions

          TIME=`date +"%Y-%m-%d_%T"`
          CMD_PWD=$(pwd)
          BACKUP=$CMD_PWD/backup
          LOG="/var/log/script.log"
      

          ## Hosts Details

          CONFIG_HOST='xyz1.host.com'

                                                                                                                  

Shell Script to Generate Random Numbers

Using 'shuf' you can generate random numbers between a given range.

Here is an exmaple showing how to generate randon number between the range 1 to 20 -

#] for x in `seq 1 10`
#] do
#] ran=`shuf -i 1-20 -n 1`
#] echo $ran
#] done

Sample Output -

18
15
17
11
20
2
11
14
15
3

Install sshpass

'sshpass' command can be used to provide password runtime while logging into a server over ssh.

                                                Installation 

O/S - Centos 6.5

Download rpm -

wget http://pkgs.repoforge.org/sshpass/sshpass-1.05-1.el6.rf.x86_64.rpm

Install -

rpm -ivh sshpass-1.05-1.el6.rf.x86_64.rpm

Run -

sshpass -p 'XXXXX'  ssh username@server.test.com

Linux sort by column number

Sample data : 

# cat flle.txt
1  field1  11
2  field2  40
3  field3  09
4  field4  22


Sort by column 3  ( lowest to highest ) - 

# cat file.txt | sort  -k3
3  field3  09
1  field1  11
4  field4  22
2  field2  40

Sort by column 3  ( highest to lowest ) - 

# cat file.txt | sort  -r -k3
2  field2  40
4  field4  22
1  field1  11
3  field3  09

Thanks

Nodejs Installation Centos

O/s- Centos
Prerequisite - Python

Download and Extract the tar -

# cd /usr/local/src
# wget http://nodejs.org/dist/node-latest.tar.gz
# tar zxvf node-latest.tar.gz
# cd node-v0.10.17/

Configure -

 # python2.6 configure
 # make
 # make install

Test -
# /usr/local/bin/node --version


#####################

Splunk Installation on Linux

O/s - Centos 6

Splunk Server Setup  -

Package - splunk-4.3.2-123586-linux-2.6-x86_64.rpm

# rpm -ivh splunk-4.3.2-123586-linux-2.6-x86_64.rpm
# /opt/splunk/bin/splunk start
# netstat -tunlp | grep 8090


Splunk Client (Forwarder) Setup -

Package -  splunkforwarder-4.2.4-110225-linux-2.6-x86_64.rpm

# rpm -ivh splunkforwarder-4.2.4-110225-linux-2.6-x86_64.rpm
# /opt/splunkforwarder/bin/splunk start
# ps aux| grep splunk
# cd /opt/splunkforwarder/

 # /opt/splunkforwarder/bin/splunk enable boot-start

Adding Source  -

# /opt/splunkforwarder/bin/splunk add monitor  </path/logfile1>
# /opt/splunkforwarder/bin/splunk add monitor  </path/logfile2>

Contacting Splunk Server -

# /opt/splunkforwarder/bin/splunk add <serverip>:8090
# /opt/splunkforwarder/bin/splunk add forward-server <serverip>:8090

Aeroproxy Installation Guide

                                                    Aeroproxy installation 


OS - centos 6.5

Packages required  -

yum install libevent-devel.x86_64 libevent.x86_64  cmake.x86_64 openssl-devel.x86_64 openssl.x86_64
yum groupinstall "Development Tools"

1. libevhtp

wget https://sourceforge.net/projects/levent/files/libevent/libevent-2.0/libevent-2.0.22-stable.tar.gz --no-check-certificate
tar -zxvf libevent-2.0.22-stable.tar.gz
./configure
make
make install
git clone https://github.com/ellzey/libevhtp.git
cd libevhtp/
cmake . -DEVHTP_DISABLE_SSL=ON
make
make install

2. Protobuf -c
wget http://ftpmirror.gnu.org/autoconf/autoconf-2.65.tar.gz
cd autoconf-2.65
./configure
make
make install

wget https://protobuf.googlecode.com/svn/rc/protobuf-2.6.0.tar.gz
cd protobuf-2.6.0
./configure
make
make install

git clone https://github.com/protobuf-c/protobuf-c.git
cd protobuf-c/
./autogen.sh
./configure
make
make install


3. C libevent Client Library - Linux SDK from Aerospike

tar -zxvf citrusleaf_client_libevent2_2.1.41.tgz
cd citrusleaf_client_libevent2_2.1.41
vi Makefile and change following

        $(MAKE) -C example
        $(MAKE) -C example2
        $(MAKE) -C example3
        $(MAKE) -C example4
        $(MAKE) -C example6
        $(MAKE) -C tests/loop_c_ev2

to

        #$(MAKE) -C example
        #$(MAKE) -C example2
        #$(MAKE) -C example3
        #$(MAKE) -C example4
        #$(MAKE) -C example6
        #$(MAKE) -C tests/loop_c_ev2

make
make install

cd lib
cp lib/libev2citrusleaf.a ../aeroproxy/libs/

cd aeroproxy

vim aeroproxy_common.h +20
and change following

include "aerospike/as_integer.h"
include "aerospike/as_string.h"
include "aerospike/as_arraylist.h"
include "aerospike/as_pair.h"
include "aerospike/as_hashmap.h"
include "aerospike/as_hashmap_iterator.h"
include "aerospike/as_stringmap.h"

to

//#include "aerospike/as_integer.h"
//#include "aerospike/as_string.h"
//#include "aerospike/as_arraylist.h"
//#include "aerospike/as_pair.h"
//#include "aerospike/as_hashmap.h"
//#include "aerospike/as_hashmap_iterator.h"
//#include "aerospike/as_stringmap.h"

vim aeroproxy.h +19

and change following

int as_write_data_map(evbase_t *evbase, const char *key, const char *bin_name, const as_map *map, void *udata);
 to
//int as_write_data_map(evbase_t *evbase, const char *key, const char *bin_name, const as_map *map, void *udata);

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib

make

make will generate aeroproxy bin file , copy that in aeroproxy folder of aerospike server.

##########