Wednesday, August 13, 2014

Change password without root on a list of servers

Some companies are strict on passwords and force you to to change your password every 30 days, no repeating passwords, the passwords have to be long, contain special characters, etc.  Here is a script that you can add as a cronjob assuming you have your ssh keys setup.

You'd need a file "list.txt" with the list of IPs and "oldpass.txt" file with the current password.  You can just set a cronjob to do reset password on a regular basis (every 25 days for example).


#!/bin/bash
# Generate random new password 24 characters long
newpass=`mkpasswd -l 30 -d 3 -C 5 -s 3`
# Get old password from oldpass file
oldpass=`cat oldpass.txt`
echo Old: $oldpass
echo New: $newpass

# ssh to server and reset password
for server in `cat list.txt`
  do
    echo "Changing $server"
    ssh -t $server 'passwd <<EOF
'$oldpass'
'$newpass'
'$newpass'
EOF'
  done

Sort IP addresses

cat list.txt |sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n

Friday, June 27, 2014

Enable port on Arista switch

Similar to Cisco, enabling port 35 on switch1 for example.

switch1>en
switch1#show interfaces Ethernet 35
switch1#configure
switch1(config)#interface Ethernet 35
switch1(config-if-Et35)#no shutdown
switch1(config-if-Et35)#end

Save your changes:
switch1#copy running-config startup-config

Tuesday, June 17, 2014

Bash - cross-reference two files

I have two files: list_to_remove and current_list.  I want to remove all servers from list_to_remove from the current list.

for i in $(cat list_to_remove); do grep -v "$i.company.com" current_list > temp && mv temp current_list; done

Thursday, June 05, 2014

How to replace a section of text file bash script

Goal:
I want to replace a section of hibernate.cfg.xml with new servers.  I need to remove servers between
                <property name="hibernate.memcached.servers">
server1 server2 server3
                </property>

and replace them with mem1 mem2 mem3

cat replace.sh

#!/bin/bash awk 'BEGIN {A = 1};//{A=0; print "\t\t\n\t\t\t\ mem1.atl.company.com:11212 mem2.atl.company:11212 mem3.atl.company.com:11212 mem4.atl.company.com:11212 mem5.atl.company.com:11212 mem6.atl.company:11212 mem148.atl.company.com:11212 mem149.atl.company.com:11212 mem150.atl.company.com:11212\ \n\t\t"};/.*/ { if ( A == 1) print $0};/<\/property>/{A=1}; ' $1

Run script against any xml file

./replace.sh hibernate.cfg.xml > hibernate.cfg.xml.new

Thursday, May 29, 2014

Puppet manifest for multiple servers

Puppet supports regular expressions.

E.g.
Run puppet manifest on all the web servers:
node /^web.*$/ {
  class { 'server_web': }

}

Run on a few (db0-9)
node /^db[0-9]\..*$/ {
  class { 'server_db': }
}

Wednesday, May 14, 2014

bash script sum

Bash script exercise.
Add variable to the result of the sum.  E.g.
1 1 2 3 5 8 13 21, etc.


#!/bin/bash
x=1
y=1
ans=0

while [ $ans -lt 100 ]
#while true
do
  ans=$(( x + y ))
  echo $x + $y = $ans
  x=$y
  y=$ans
done

Monday, May 12, 2014

Bash "for loop"
How to ping multiple selected servers in selected multiple racks.

for rack in 1 3 4 6 9 {14..27}; do echo $rack; for server in {46..48}; do fping "serv"$server"rack"$rack |grep unreachable |tee -a /tmp/down; done; done

Friday, May 02, 2014

Number of hits from server in bash

Let's say I have a list of servers and I want to calculate how many hits did I have from "client.com" on April 28, 2014

#!/bin/bash
total=0

for server in $(cat server_list)
  do
    echo $server
    echo "Current count = "$total
    servercount=$(ssh $server "zcat /var/log/httpd/access.2014-04-28.log.gz |grep client.com |wc -l")
    echo "Server count ="$servercount
    total=$(($total + $servercount))
done


echo "Total count = "$total

Monday, April 21, 2014

Multiple DNS master servers

It is perfectly fine to have multiple master DNS servers as long as you keep them all in sync.  Here is an example how to make changes to DNS zones in GIT and push the changes to Bind servers.


DNS push script uses dedicated "sysadmin" account.  SSH keys have to be in place.


#!/bin/bash
# OPS branch have to be checked out to ~/git/ops/ to use this script
# I assume you have sudo privileges


usage(){
        echo -e Usage:\\n$0 datacenter
        exit 1
}

if [[ -z "$1" ]]
then
  usage
  exit 1
fi

datacenter=$1

cd ~/git/ops/config/sjc/named/var/named/data/
# Build reverse DNS zones
/usr/bin/mkrdns -root ~/git/ops/config/sjc/named/ ~/git/ops/config/sjc/named/etc/named.conf
# Checking syntax
for zone in `ls`
  do
    test=$(named-checkzone $zone $zone |grep OK)
    if [ "$test" != 'OK' ]
      then
      echo "Zone $zone syntax is wrong!"
      exit 1
    else
      echo "Syntax is fine"
    fi
done

echo "pushing to $datacenter"

# Execute remote dns change
sudo su - sysadmin -c "ssh -t -o StrictHostKeyChecking=no mgr1.$datacenter.company.com \"~/git/ops/scripts/dnspull.sh\""
echo "======================================================"
echo "mgr1.$datacenter.company.com is done"
echo "======================================================"
sudo su - sysadmin -c "ssh -t -o StrictHostKeyChecking=no mgr2.$datacenter.company.com \"~/git/ops/scripts/dnspull.sh\""
echo "======================================================"
echo "mgr2.$datacenter.company.com is done"
echo "======================================================"


DNS pull script will pull your changes from GIT, apply the changes, restart named, test if resolution is working properly (in this case "mgrclust1" server), and push the change to the next server.

#!/bin/bash
# Run it on the actual DNS (mgr1,2) server as user "sysadmin"


# I'm relying on resolv to get DC
DC=$(cat /etc/resolv.conf |grep search |awk '{print $2}' |awk -F. '{print $1}')
echo "=============================================="
echo "Pulling zones for $DC"
echo "=============================================="

# Pull from GITolite
cd ~/git/ops/
git pull origin master

# Copy zones over
rsync -av config/$DC/named/etc/ /var/named/chroot/etc/
rsync -av --delete config/$DC/named/var/named/ /var/named/chroot/var/named/

# Fix permissions
chown sysadmin:named /var/named/chroot/etc/named.conf
chown -R sysadmin:named /var/named/chroot/var/named/

# Restart named, look for OK
named_test=$(sudo /etc/init.d/named restart |grep Starting |awk '{print $4}')
echo $named_test

if [ "$named_test" != 'OK' ]
  then
    echo "Named restart failed!"
    exit 1
  else
    echo "Named restarted"
fi

# Try to resolve mgrclust1
dns_test=$(host mgrclust1 localhost |grep mgrclust1 |awk '{print $4}' |awk -F. '{print $4}')
if [ "$dns_test" != '1' ]
  then
  echo "I can't resolve mgrclust1!"
  exit 1
fi

Monday, December 02, 2013

Kickstart

Menu config file example


/tftpboot/pxelinux.cfg/default

default vesamenu.c32
Menu Background sm_bk.png
Menu Title Boot Menu

label install
   menu label ^Ubuntu-10.04
   kernel ubuntu-installer10/amd64/linux
   append vga=normal initrd=ubuntu-installer10/amd64/initrd.gz ks=http://10.10.4.58/ks10.cfg

label install
   menu label ^Ubuntu-12.04
   menu default
   kernel ubuntu-installer/amd64/linux
   append vga=normal initrd=ubuntu-installer/amd64/initrd.gz ks=http://10.10.4.58/ks12.cfg

LABEL Centos-5.8
    MENU LABEL ^Centos-5.8
    KERNEL centos58/vmlinuz
    APPEND initrd=centos58/initrd.img ramdisk_size=100000 ip=dhcp ksdevice=bootif ks=http://10.10.23.201/ks/ks.cfg
    IPAPPEND 2

LABEL Centos-6
    MENU LABEL ^Centos-6
    KERNEL centos6/vmlinuz
    APPEND initrd=centos6/initrd.img ip=dhcp ksdevice=bootif ks=http://10.10.23.201/ks/ks_co6.cfg

LABEL Centos-6-p1
    MENU LABEL ^Centos-6 p1p1
    KERNEL centos6/vmlinuz
    APPEND initrd=centos6/initrd.img ip=dhcp ksdevice=bootif ks=http://10.10.23.201/ks/ks_co6p1.cfg

LABEL Centos-6-em1
    MENU LABEL ^Centos-6 em1
    KERNEL centos6/vmlinuz
    APPEND initrd=centos6/initrd.img ip=dhcp ksdevice=bootif ks=http://10.10.23.201/ks/ks_co6em1.cfg

label expert
   menu label ^Expert install
   kernel ubuntu-installer/amd64/linux
   append priority=low vga=normal initrd=ubuntu-installer/amd64/initrd.gz --

label cli-expert
   menu label Command-^line expert install
   kernel ubuntu-installer/amd64/linux
   append tasks=standard pkgsel/language-pack-patterns= pkgsel/install-language-support=false priority=low vga=normal initrd=ubuntu-installer/amd64/initrd.gz --

label rescue
   menu label ^Rescue mode
   kernel ubuntu-installer/amd64/linux
   append vga=normal initrd=ubuntu-installer/amd64/initrd.gz rescue/enable=true -- quiet

label Local_drive
   localboot 0
   menu label ^Local Drive

prompt 0
timeout 60


Kickstart config file example:


/srv/kickstart/cs6ks.cfg

#platform=x86, AMD64, or Intel EM64T
install
key --skip
lang en_US.UTF-8
# Forces the cmdline installer to be used (debugging)
#cmdline
# System authorization information
auth  --useshadow  --enablemd5
# System bootloader configuration
bootloader --location=mbr
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel
#ignoredisk --only-use=sda,sdc,sdd
# Use text mode install
text
# Firewall configuration
firewall --disabled
# Run the Setup Agent on first boot
firstboot --disable
# System keyboard
keyboard us
# System language
#lang en_US
# Installation logging level
logging --level=info
# Use network installation
url --url=http://10.10.23.201/centos/6/os/x86_64/
# Network information
network --bootproto=dhcp --device=eth0 --onboot=on
# Reboot after installation
reboot
#Root password
rootpw --iscrypted $1...1

# SELinux configuration
selinux --disabled
# Do not configure the X Window System
skipx
# System timezone
timezone  America/Los_Angeles
# Install OS instead of upgrade
install
# Disk partitioning information
part /boot --bytes-per-inode=4096 --fstype="ext3" --ondisk=sda --size=500

# Web servers use 8G of swap
#part swap --bytes-per-inode=4096 --fstype="swap" --ondisk=sda --size=8000
# Changing swap for testing
part swap --bytes-per-inode=4096 --fstype="swap" --ondisk=sda --size=1000
# The rest goes to root
part / --bytes-per-inode=4096 --fstype="ext3" --grow --ondisk=sda --size=1

# Use WF Repo
repo --name=sm_base --baseurl=http://10.10.23.201/centos/6/os/x86_64/
repo --name=sm_updates --baseurl=http://10.10.23.201/centos/6/updates/x86_64/

%packages
@core
ntp
openssh-clients
openssh-server
strace
oprofile
screen
sysstat
lynx
-bluez-gnome
-bluez-libs
-bluez-utils
-ccid
-coolkey
-conman
-cups
-dosfstools
-ifd-egate
-irda-utils
-NetworkManager
-oddjob
-oddjob-libs
-pcsc-lite
-pcsc-lite-libs
-ppp
-redhat-lsb
-rp-pppoe
-stunnel
-wpa_supplicant
-ypbind
-yp-tool

%post --log=/root/my-post-log
# Remove all i386/i686 packages, we only want x86_64 packages.
rpm -e $(rpm -qa --queryformat='%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n' | grep '\.i[3456]86$')

# XFS support
yum -y install kmod-xfs xfs*

# Disable services typically not needed.
for service in iptables ip6tables rpcgssd rpcidmapd iscsi iscsid ; do
        echo "Disabling some unneeded services."
        chkconfig $service off
        echo "Services disabled."
done

echo "exclude=kernel* *.i?86" >> /etc/yum.conf
rm -rf /etc/yum.repos.d/*
wget http://vmks.corp.example.com/sm6.repo
mv sm6.repo /etc/yum.repos.d/sm.repo
ntpdate -u ntp1
ntpdate -u ntp1
ntpdate -u ntp1
/etc/init.d/ntpd start
chkconfig ntpd on

# Converting from DHCP to static IPs
KSDEVICE=`LANG=C /sbin/route -n | awk '/^0.0.0.0/ { print $8 }'`
IP=`LANG=C /sbin/ifconfig $KSDEVICE | /bin/awk '/inet/ && !/inet6/{sub(/addr:/, ""); print $2}'`
HOSTNAME=`LANG=C /usr/bin/host $IP | /bin/awk '{sub(/\.$/, ""); print $5}' |cut -d"." -f1`
NETMASK=`LANG=C /sbin/ifconfig $KSDEVICE | /bin/awk '/inet/ && !/inet6/ {sub(/Mask:/, ""); print $4}'`
NETWORK=`LANG=C /bin/ipcalc $IP -n $NETMASK | /bin/cut -d\= -f2`
GATEWAY=`LANG=C echo $NETWORK | awk -F'.' '{print $1"."$2"."$3"."$4+1}'`
HWADDR=`LANG=C /sbin/ifconfig $KSDEVICE | /bin/awk '/HWaddr/ { print $5 }'`

cat << EOF > /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=$HOSTNAME
GATEWAY=$GATEWAY
EOF

cat << EOF > /etc/sysconfig/network-scripts/ifcfg-$KSDEVICE
DEVICE=$KSDEVICE
BOOTPROTO=static
IPADDR=$IP
NETMASK=$NETMASK
ONBOOT=yes
HWADDR=$HWADDR
EOF

# chef-client

if [ ! -e /etc/chef ]; then
        mkdir /etc/chef
fi

cat > /etc/chef/client.rb << EOCLRB
log_level        :info
log_location     STDOUT
chef_server_url  "http://chefserver:4000"
validation_client_name "chef-validator"
# Using default node name (fqdn)
EOCLRB
chmod 600 /etc/chef/client.rb

cat > /etc/chef/validation.pem << EOVALPEM
-----BEGIN RSA PRIVATE KEY-----
MIIE...Wg==


-----END RSA PRIVATE KEY-----
EOVALPEM
chmod 600 /etc/chef/validation.pem

# Install Chef packages
yum -y install rubygem-chef
chkconfig chef-client on
yum -y update

Monday, November 25, 2013

Apache Active Directory authentication

If your Linux server is setup with AD authentiction you can add Apache AD auth as well.


AD LDAP authentication

Install OpenLDAP packages:
yum -y install openldap-clients openldap-devel

Comment everything from /etc/openldap/ldap.conf except the following line:

TLS_REQCERT never

Make sure that you can run a simple LDAP query against domaincontroller.yourdomain.com

ldapsearch -x -LLL -D 'Igor Grinkin' -H ldaps://domaincontroller.yourdomain.com -W -b "CN=Domain Admins,CN=Users,DC=corp,DC=yourdomain,DC=com" -L cn=*


Modify vhost.


<Directory "/usr/share/icinga/">
   Options None
   AllowOverride All
   Order allow,deny
   Allow from all
   AuthName "Please enter your Corp AD credentials"
   AuthType Basic
   AuthLDAPURL "ldap://domaincontroller.yourdomain.com:389/OU=YourDomain,DC=corp,DC=domaincontroller,DC=com?samAccountName?sub?(objectCategory=person)"
   AuthLDAPBindDN "cn=joinaccount,CN=Users,DC=corp,DC=yourdomain,DC=com"
   AuthLDAPBindPassword "joinaccount password"
   AuthBasicProvider ldap
   AuthzLDAPAuthoritative off
   Require valid-user
</Directory>

Monday, November 04, 2013

Chef OS versions fork

On CentOS-5.8 the service is portmap. In CentOS-6.4 it got obsolete and replaced by rpcbind.  Here is the case statement for Chef.

# Start portmap or rpcbind
service "port" do
  case node["platform_version"]
    when "5.8"
      service_name "portmap"
    when "6.4"
      service_name "rpcbind"
  end
  action [ :start, :enable ]
end

Sunday, October 06, 2013

Convert MS Word document to HTML

Convert a long Word doc with simple formatting to html.

1.  Save the doc as Web page, filtered with html extenstion

Some special characters will not look right, like ' and "

2.  Substitute those with vi editor

:1,$s/\%x92/'/g
:1,$s/\%x93/"/g
To insert Copyright Symbol  , insert
&copy;

Friday, September 20, 2013

Linux sed example - find a replace string in /etc/fstab.  Changing "sdb1" to "VolGroup00/local"

for i in `cat lists/dblist`; do ssh -t $i "sudo sed -i 's/sdb1/VolGroup00\/local/' /etc/fstab"; done

Linux parted from command line.  Remove partition number 1 on sdb.
parted -s /dev/sdb -- rm 1


Friday, August 16, 2013

MySQL cheatsheet

Login to the database called "testdb" as "root" with "coolpass" password:
# mysql -u root -pcoolpass testdb

Let's give user "testuser" all privileges with password "coolpass"
mysql> GRANT ALL ON *.* to testuser@'localhost' IDENTIFIED BY 'coolpass';
mysql> FLUSH PRIVILEGES;

Show me all the recent users:
mysql> select * from appusers order by UserId desc limit 2;

Show me all messages that were sent today:
mysql> select * from messagecenter where DateTime like '2013-08-16%' order by DateTime desc;

My SQL logging.  Let's see what mysql database is doing.
mysql> show processlist;
The number of rows will give you an idea how busy is the db.

If you want to watch history, you can send output to a log file.
First let's find out where the log is:
mysql> SHOW VARIABLES LIKE "general_log%";
+------------------+------------------------+
| Variable_name    | Value                  |
+------------------+------------------------+
| general_log      | OFF                    |
| general_log_file | /var/lib/mysql/db2.log |
+------------------+------------------------+
2 rows in set (0.00 sec)

Let's enable logging:
mysql> SET GLOBAL general_log = 'ON';

Let's see what it's doing:
tail -f /var/lib/mysql/db2.log

Turn off loggin:
mysql> SET GLOBAL general_log = 'OFF';

Tuesday, August 13, 2013

Extending partitions on Linux VMware virtual machines

Extending partitions on Linux VMware virtual machines

 I had to increase the hard drive size on one of the VMs and read this article:

In the comments people say that you can't do that if you are trying to expand primary root partition.  Not true.  Here is how:

  1. Shut down VM and increase the hard drive size
  2. If that Option is disabled consolidate your snapshots (delete them except the last one) 
  3. Download Linux rescue CD iso
  4. Mount iso as CDROM and boot your VM from it.
  5. Use gparted utility that comes with Linux rescue CD to resize partition to max size
  6. Reboot VM

Friday, November 16, 2012

Ubuntu-12.04 "Packages was corrupt"

When you install Ubuntu-12.04.1-server-amd64 from Kickstart server unattended (and probably ubuntu-12.04-server-amd64 as well) you'll get an error message that the "Packages file was corrupt" and the installation stops.  If you hit continue button it goes on.

Here is the file in Ubuntu distro that causes the error:
ubuntu-12.04/dists/precise/restricted/binary-amd64/Packages.gz

It's a bug.  Few people reported it to Ubuntu team but Ubuntu guys (like Fabio Marconi)  don't seem to care and just closed the tickets.
https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/1023069

Solution:

1.  Unzip that file manually in your Kickstart installation directory.  E.g.:
/srv/kickstart/ubuntu-12.04/dists/precise/restricted/binary-amd64# gunzip Packages.gz

2.  Copy the original file from your iso image directory to the same location:
/srv/kickstart/ubuntu-12.04/dists/precise/restricted/binary-amd64/

You should have 3 files in there:

-r--r--r-- 1 root root   0 Nov 15 17:30 Packages
-r--r--r-- 1 root root  20 Nov 15 17:31 Packages.gz
-rw-r--r-- 1 root root 103 Nov 15 17:30 Release

That's it - now the installer will be happy.



Reason as far as I understand it.
The installer unzips those file during the installation.  After Packages.gz gets unzipped, the size of the new file shows 0 (because it's an emtpy file to start with).  The installer "thinks" that the file is corrupt and stops the installation process.  When you give it unzipped version of the same empty file, it happily continues the installation.

Friday, November 09, 2012

How to redirect non-www URLs to www URLs in Apache


Below is the answer on how redirect your traffic to www URLs.

E.g.
http://domain.com -> http://www.domain.com
https://domain.com -> https://www.domain.com

Q:  Why would you need it?
A:  Search engines will regard those as different pages with duplicate content.
http://www.thesitewizard.com/apache/redirect-domain-www-subdomain.shtml

You can either put this code in .htaccess file or inside your site configuration under
/etc/httpd/conf/sites-enabled


Non-SSL redirection goes within <VirtualHost *:80> </VirtualHost>:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^domain\.com
RewriteRule ^(.*)$ http://www.domain.com$1 [R=permanent,L]

SSL redirection goes within <VirtualHost *:443> </VirtualHost>:

RewriteEngine on
RewriteCond %{HTTP_HOST} ^domain\.com
RewriteRule ^(.*)$ https://www.domain.com$1 [R=permanent,L]

Monday, November 05, 2012

Chef server

How to install Chef server on Ubuntu.


Opscode documentation is extremely confusing.  The following simple instructions tell you how to install Chef server version 10.16.2 on Ubuntu-12.04.

Keep in mind that Opscode guys changed their versioning.  Starting with version 0.10.12 they call it 10.12.0.  So the newest version as it is right now is 10.16.2.

To install it become root:
sudo su -
and run the following:

apt-get install sudo wget lsb-release

echo "deb http://apt.opscode.com/ `lsb_release -cs`-0.10 main" | sudo tee /etc/apt/sources.list.d/opscode.list

mkdir -p /etc/apt/trusted.gpg.d

gpg --keyserver keys.gnupg.net --recv-keys 83EF826A

gpg --export packages@opscode.com | sudo tee 

/etc/apt/trusted.gpg.d/opscode-keyring.gpg > /dev/null

apt-get update

apt-get install opscode-keyring

apt-get upgrade

apt-get install chef chef-server

Check the version:

chef-server -v
Chef Server (API) Version: 10.16.2




GIT repository

The next thing you probably need is GIT repo where you keep all the cookbooks, etc.

git config --global user.name "Igor Grinkin"
git config --global user.email igor@yourdomain.com
git clone git://code.yourrepo.com/chef_server.git
cd chef_server
git pull origin master