Category Archives: DevOps

BigData Technology Landscape

BigData Technology Landscape

Big data is data that exceeds the processing capacity of conventional database systems. The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures. To gain value from this data, you must choose an alternative way to process it.

Big data analytics is the process of examining big data to uncover hidden patterns, unknown correlations and other useful information that can be used to make better decisions.

BigData Technology Landscape

9 ways to tell if you’re really a DevOps organization

9 ways to tell if you’re really a DevOps organization
Many people believe that their organizations use a DevOps approach, but the “best practices” they’ve adopted aren’t consistent with a DevOps methodology. You can call what you do at your organization “DevOps,” but if you don’t follow the basic software methodology principles, it’s not DevOps. Period.

Here’s a set of objective criteria you can use to determine whether what you’re doing is truly DevOps. You’re really doing DevOps if…

1. You have frequent, rapid release cycles.
2. Deployment to production is fully automated, and you can automatically roll back.
3. You do continuous integration with automated testing at every check-in.
4. You have far more resources devoted to automated testing than manual testing.
5. Your developers, testers, and operations engineers work together.
6. Your developers, testers, and operations engineers have common, business-oriented goals that cover the whole value chain.
7. You’re paying back your technical debt.
8. Management gives the team the authority to make the changes it needs to make—and the team makes those changes.
9. You have feedback mechanisms in place to ensure continuous assessment—and act on that feedback to ensure continuous improvement.

Step by Step Procedure To Setup Cassandra Cluster On Any Linux Distribution

Step by Step procedure to set-up Cassandra Cluster On Any Linux Distribution

1. What is Cassandra : Cassandra is an open source distributed database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure.

2. Cassandra Cluster: A cluster is arranged as a ring of nodes. Clients send read/write requests to any node in the ring; that node takes on the role of coordinator node, and forwards the request to the node responsible for servicing it. A partitioner decides which nodes store which rows. The recommended partitioners assign rows to nodes based on a hash value of the row key. Nodes are assigned tokens that evenly divide the full range of possible hash values.

3. Prerequisites :
Setup iptables, allow below Cassandra ports.
For Centos:
iptables -A INPUT -p tcp -m tcp –dport 9160 -j ACCEPT
iptables -A INPUT -p tcp -m tcp –dport 7000 -j ACCEPT
iptables -A INPUT -p tcp -m tcp –dport 7001 -j ACCEPT
service iptables restart

For Ubuntu:
iptables -A INPUT -p tcp -m tcp –dport 9160 -j ACCEPT
iptables -A INPUT -p tcp -m tcp –dport 7000 -j ACCEPT
iptables -A INPUT -p tcp -m tcp –dport 7001 -j ACCEPT
sudo sh -c “iptables-save > /etc/iptables.rules”
sudo iptables-apply /etc/iptables.rules
Applying new iptables rules from ‘/etc/iptables.rules’… done.
Can you establish NEW connections to the machine? (y/N) y
… then my job is done. See you next time.

4. Installing Cassandra and Configuring: Login to Remote machine, where you want to install Cassandra.
Below are the steps to install and configure Cassandra Cluster.

Get Cassandra tar-ball
wget “http://downloads.datastax.com/community/dsc-cassandra-2.1.7-bin.tar.gz” -P /usr/share/
Setup Cassandra environment under /usr/share
tar -xvzf /usr/share/dsc-cassandra-2.1.7-bin.tar.gz -C /usr/share/

Delete tar file
rm -rf /usr/share/dsc-cassandra-2.1.7-bin.tar.gz

Cassendra configuration change cluster_name : change clustername with your Cluster
sed -i ‘s/Test Cluster/{{clustername}}/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change listen_address : hostname=serverip
sed -i ‘s/listen_address\:\ localhost/listen_address\:\ {{hostname}}/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change rpc_address
sed -i ‘s/rpc_address\:\ localhost/rpc_address\:\ 0.0.0.0/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change endpoint_snitch
sed -i ‘s/endpoint_snitch\:\ SimpleSnitch/endpoint_snitch\:\ GossipingPropertyFileSnitch/g’ /usr/share/dsc-
cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change broadcast_rpc_address hostname=serverip
sed -i ‘s/^# broadcast_rpc_address\:\ 1.2.3.4/broadcast_rpc_address\:\ {{hostname}}/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change seeds hostname=serverip seeds=seedsip
sed -i ‘s/- seeds\:\ “127.0.0.1”/- seeds\:\ “{{seeds}},{{hostname}}”/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change data_file_directories
sed -i ‘s/# data_file_directories/data_file_directories/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change /var/lib/cassandra/data
sed -i” ‘107 s/^#//’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change commitlog_directory
sed -i ‘s/# commitlog_directory/commitlog_directory/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change saved_caches_directory
sed -i ‘s/# saved_caches_directory/saved_caches_directory/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change thrift_framed_transport_size_in_mb
sed -i ‘s/thrift_framed_transport_size_in_mb\:\ 15/thrift_framed_transport_size_in_mb\:\ 100/g’ /usr/share/dsc-
cassandra-2.1.7/conf/cassandra.yaml

Cassendra configuration change phi_convict_threshold
sed -i ‘s/# phi_convict_threshold\:\ 8/phi_convict_threshold\:\ 10/g’ /usr/share/dsc-cassandra-2.1.7/conf/cassandra.yaml

Start Cassandra service
nohup sh /usr/share/dsc-cassandra-2.1.7/bin/cassandra

5. Command to check Cassandra cluster:
/usr/share/dsc-cassandra-2.1.7/bin/nodetool status
Address Status State Load Owns Token
113427455640312821154458202477256070484
10.1.1.7 Up Normal 459.27 MB 33.33% 0
10.1.1.8 Up Normal 382.53 MB 33.33% 56713727820156410577229101238628035242
10.1.1.6 Up Normal 511.34 MB 33.33% 113427455640312821154458202477256070484