Saturday 16 November 2013

Running a PowerCLI Script

I'm a Linux User and fond some interesting scripts for VMWare on the Internet, the problem was I didn't knew how to run them.
Now I finally learned how to run a Power Shell script, so here's how I've done it:

Save the script to a file with a filetype of .ps1
Open the PowerCLI prompt on your PC
Connect to the vCenter with the Connect-VICenter cmdlet
dot-source the .ps1 file. That way the function is know in your PowerCLI session
. ./yourfile.ps1
Call the function implemented by the scipt.

Possibly Related Posts

Tuesday 29 October 2013

Export everpad notes to HTML files

If you use Everpad you can use the following script to export your notes to html files. The script will write the notes into ~/exported_notes and the notes will be sorted under folders with their notebook names:
import os
import sys
import datetime
import sqlite3
export_path = os.getenv("HOME") + '/exported_notes'
#Create directory if it does not exist
if not os.path.exists(export_path):
    os.makedirs(export_path)
# Create a connection to the database.
filename = os.getenv("HOME") + '/.everpad/everpad.5.db'
conn = sqlite3.connect(filename, detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
# Create a cursor object to do the interacting.
c = conn.cursor()
# Grab the columns we need.
sql = 'SELECT notebooks.name, notes.title, notes.content FROM notes'
sql += ' inner join notebooks on notes.notebook_id = notebooks.id'
rows = c.execute(sql)
# Iterate over the result.
for row in rows:
    note_path = export_path + '/' + row[0].replace("/", "_")
    note_path = note_path.replace(" ", "_");
    if not os.path.exists(note_path):
        os.makedirs(note_path)
    note_path += '/' + row[1].replace("/", "_") + '.html'
    note_path = note_path.replace(" ", "_");
    note = open(note_path, 'a+')
    note.write('<h1>'+row[1].encode('utf-8')+'</h1></br>')
    note.write(row[2].encode('utf-8'))
    note.close()
# Commit the changes and close everything.
conn.commit()
c.close()
conn.close()


Possibly Related Posts

Friday 16 August 2013

302 redirects behind SSL reverse proxy

Problem

You have a web server running plain http behind an Apache reverse proxy running https. Your application uses 302 redirects to announce new URLs or whatever the reason is for doing so.

The client will be redirected a plain http url.

Solution

You can create a Vhost listenig for plain http and all that it does is redirect the clients to https, like this:
NameVirtualHost *:80
<VirtualHost *:80>
   ServerName www.example.com
   Redirect permanent / https://secure.example.com/
</VirtualHost>
This way, the client is redirected to the plain http by the proxied server and then redirected back to https by the proxy server.

Another solution is to use mod_rewrite:
RewriteEngine On
# This will enable the Rewrite capabilities
RewriteCond %{HTTPS} !=on
# This checks to make sure the connection is not already HTTPS
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
# This rule will redirect users from their original location, to the same location but using HTTPS.
# i.e.  http://www.example.com/foo/ to https://www.example.com/foo/
# The leading slash is made optional so that this will work either in httpd.conf
# or .htaccess context
This does the same as the previous solution.
But what if there is some firewall or load balancer that prevents plain http packets from reaching your server?
Since Apache version 2.2.4 mod_headers is able to rewrite response headers. Just add the following to your https vhost that is serving as reverse proxy to your application:
Header edit Location ^http://(.*)$ https://$1
This configuration statement will solve your problem. Redirects triggered by your back end web servers will be re-rewritten to comply with your SSL terminating reverse proxy/load balancer.

Example:
Listen a.b.c.d:443
<VirtualHost a.b.c.d:443>
    ServerName example.org
    # …SSL configuration…
    ProxyRequests off
    ProxyPass / http://localhost:3000/
    ProxyPassReverse / http://localhost:3000/
    RequestHeader set X-Forwarded-Protocol "https"
    Header edit Location ^http://(.*)$ https://$1
</VirtualHost>
<VirtualHost a.b.c.d:80>
    ServerName example.org
    ProxyRequests off
    ProxyPass / http://localhost:3000/
    ProxyPassReverse / http://localhost:3000/
    RequestHeader set X-Forwarded-Protocol "http"
</VirtualHost>

Possibly Related Posts

Monday 29 July 2013

Iptables blacklists

Many of you already use online blacklists to fight spam. Recently I've dicovered http://www.openbl.org/ and started using their lists on my firewall to prevent attacks from hosts that are known to be preforming attacks. It works in a very similar way to all the spam blacklists out there, and this is how I've implemented them on my Firewall.

First of all you'll need to  have some packages installed:
sudo apt-get install iptables ipset wget
now create an ipset to store all the abusing IP addresses and use iptables to block them:
#!/bin/bash
BLOCKDB="block.txt"
WORKDIR="/tmp"
pwd=$(pwd)
cd $WORKDIR
#List of ips to block
ipset --create blackips iphash
## Obtain List of badguys from openbl.org
wget -q -c --output-document=$BLOCKDB http://www.openbl.org/lists/base.txt
if [ -f $BLOCKDB ]; then
    IPList=$(grep -Ev "^#" $BLOCKDB | sort -u)
    for i in $IPList
    do
        ipset --add blackips $i
    done
fi
rm $BLOCKDB
## Obtain List of badguys from ciarmy.com
wget -q -c --output-document=$BLOCKDB http://www.ciarmy.com/list/ci-badguys.txt
if [ -f $BLOCKDB ]; then
    IPList=$(grep -Ev "^#" $BLOCKDB | sort -u)
    for i in $IPList
    do
        ipset --add blackips $i
    done
fi
rm $BLOCKDB
## Obtain List of badguys from dshield.org
wget -q -c --output-document=$BLOCKDB http://feeds.dshield.org/top10-2.txt
if [ -f $BLOCKDB ]; then
    IPList=$(grep -E "^[1-9]" $BLOCKDB | cut -f1 | sort -u)
    for i in $IPList
    do
        ipset --add blackips $i
    done
fi
rm $BLOCKDB
#List of networks to block
ipset --create blacknets nethash
## Obtain List of badguys from dshield.org
wget -q -c --output-document=$BLOCKDB http://feeds.dshield.org/block.txt
if [ -f $BLOCKDB ]; then
    IPList=$(grep -E "^[1-9]" $BLOCKDB | cut -f1,3 | sed "s/\t/\//g" | sort -u)
    for i in $IPList
    do
        ipset --add blacknets $i
    done
fi
rm $BLOCKDB
## Obtain List of badguys from spamhaus.org
    wget -q -c --output-document=$BLOCKDB http://www.spamhaus.org/drop/drop.lasso
    if [ -f $BLOCKDB ]; then
      IPList=$(grep -E "^[1-9]" $BLOCKDB | cut -d" " -f1 | sort -u)
      for i in $IPList
      do
        ipset --add blacknets $i
      done
    fi
    rm $BLOCKDB
#Drop blacklisted ips
iptables -A FORWARD -m set --match-set blackips src -j DROP
iptables -A FORWARD -m set --match-set blacknets src -j DROP
cd $pwd
In the above script I've used two ipsets, one for storing IP addresses and another to store network addresses, you can add this scritp to your existing firewall and start taking advantage of the blacklists.
OpenBL is accepting donations http://www.openbl.org/donations.html if you can you should help.

Possibly Related Posts

Wednesday 19 June 2013

NetFLOW over IPSEC VPN


I had the following NetFlow configuration on my router:
 ip flow-cache timeout active 1
 ip flow-export source FastEthernet0/1.1
 ip flow-export version 5
 ip flow-export destination 10.39.30.5 9996
!
interface FastEthernet0/0.2
 ip flow ingress
interface FastEthernet0/1.12
 ip flow ingress
interface FastEthernet0/1.40
 ip flow ingress

Witch worked fine on my other routers. But in this particular case, the NetFLOW server was only accecible through an IPSEC VPN and the flows weren't getting there.
The solution to this was to use "Flexible Netflow" configuration. This allows for the NetFlow export to be sent down the standard IPSEC VPN tunnel.
An example of the NetFlow config is as follows:
flow exporter FLOW_EXPORTER
 destination 10.39.30.5
 source FastEthernet0/1.1
 output-features
 transport udp 9996
 export-protocol netflow-v5
!
!
flow monitor FLOW_MONITOR
 record netflow-original
 exporter FLOW_EXPORTER
 cache timeout active 1
!
interface FastEthernet0/0.2
ip flow monitor FLOW_MONITOR input
!
interface FastEthernet0/1.12
ip flow monitor FLOW_MONITOR input
!
interface FastEthernet0/1.40
ip flow monitor FLOW_MONITOR input
!

Possibly Related Posts

Thursday 14 March 2013

Disable IPv6 on Ubuntu

Edit your /etc/sysctl.conf file and add the following to the bottom:
#disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
Or you can use the following script:
echo "#disable ipv6" | sudo tee -a /etc/sysctl.conf
echo "net.ipv6.conf.all.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf
echo "net.ipv6.conf.lo.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf
For this changes to take effect you must reboot your system.
After rebooting you can check if IPv6 has been disabled with the following command:
cat /proc/sys/net/ipv6/conf/all/disable_ipv6
0 means it's enabled and 1 - disabled.

Possibly Related Posts

Linux Stress tests

Consume CPU:

Fork bomb:

:(){ :|:& };:
The next one will load four CPU cores at 100%:
for i in `seq 1 4` ; do while : ; do : ; done & ; done
Or:
for i in `seq 1 4` ; do cat /dev/zero > /dev/null & ; done
Or:
#!/bin/bash
duration=120 # seconds
instances=4 # cpus
endtime=$(($(date +%s) + $duration))
for ((i=0; i<instances; i++))
do
while (($(date +%s) < $endtime)); do : ; done &
done
Using the stress tool:
stress --cpu 3

Consume RAM:

Create a 30gb ramdisk and fills it with file full of zeroes:
sudo mount -t tmpfs -o size=30G tmpfs /mnt
dd if=/dev/zero of=/mnt/tmp bs=10240 count=30720MB

Create a giant virable:
x="x" ; while : ; do x=$x$x ; echo -n "." ; done

Consume Disk:

dd if=/dev/zero of=bigfile bs=10240 count=30720MB

Simulate packet loss:

For randomly dropping 10% of incoming packets:
iptables -A INPUT -m statistic --mode random --probability 0.1 -j DROP
and for dropping 10% of outgoing packets:
iptables -A OUTPUT -m statistic --mode random --probability 0.1 -j DROP


Possibly Related Posts

Saturday 26 January 2013

How to reset the root password of VMware ESXi 4.1 and 5.0

You can't recover your old password but by following this steps you can set it to blank and then set a new one.

First you'll need a Linux live CD, any one will do.

After booting to a live session of Linux you must look for a file named state.tgz on your vmware host's hard drive. To do so I used:
parted -l
To list all available partitions and mounted every VFAT partition to look inside, I found it was on /dev/sda5, in your case it might be on a different one, you can mount the partitions with:
mount /dev/sda1 /mnt
(replace sda with your device's name and 1 with your partition numver)
then check inside if the file state.tgz exists:
ls /mnt/
After finding the state.tzg file you must uncompress it using:
cd /tmp
tar xzf /mnt/Hypervisor3/state.tgz
this will get you a local.tgz file witch you have to extract using:
tar xzf local.tgz
now edit the file /tmp/etc/shadow
vi etc/shadow
Inside locate the root account and just remove it's hash (everything between the first and the second colon) and login to the service console as root with no password at all.

Finally re-pack the files and move the modified state.tgz back to the VFAT partition. Probably it is a good idea to make a backup copy of the original state.tgz in case something goes wrong:
mv /mnt/state.tgz /mnt/state.tgz.bak
rm local.tgz
tar czf local.tgz etc
tar czf state.tgz local.tgz
mv state.tgz /mnt/
Reboot back into ESXi and you're done.

Possibly Related Posts

Friday 11 January 2013

Calculating total disk usage by files with specific extension

For example if you want to check how much space is being used by log files on your entire system, you can use the following:

find / -type f -name "*.log*" -exec du -b {} \; | awk '{ sum += $1 } END { kb = sum / 1024; mb = kb / 1024; gb = mb / 1024; printf "%.0f MB (%.2fGB) disk space used\n", mb, gb}'
Just replace "*.log*" with the file extension you want to search for and the above will give you the disk used by the sum of all the files with that extension.

Possibly Related Posts

Saturday 5 January 2013

PostgreSQL cluster using DRBD and hot standby

Cluster Configuration:

First install all the necessary packages:
yum install gfs2-utils cman fence-virtd-checkpoint lvm2-cluster perl-Net-Telnet rgmanager device-mapper-multipath ipvsadm piranha luci modcluster cluster-snmp ricci
yum groupinstall "High Availability"
yum install postgresql-server
chkconfig --level 123456 ricci on
chkconfig --level 123456 luci on
chkconfig --level 123456 cman on
chkconfig --level 123456 iptables off
chkconfig --level 123456 ip6tables off
chkconfig postgresql on
chkconfig cman on
chkconfig rgmanager on
Now edit the cluster configuration file:
vi vi /etc/cluster/cluster.conf
Make it look like this:
<?xml version="1.0"?>
<cluster config_version="7" name="pgcluster">
<clusternodes>
<clusternode name="10.39.30.7" votes="1" nodeid="1">
<fence/>
</clusternode>
<clusternode name="10.39.30.8" votes="1" nodeid="2">
<fence/>
</clusternode>
</clusternodes>
<rm>
<failoverdomains>
<failoverdomain name="PGSQL" nofailback="0" ordered="0" restricted="0">
<failoverdomainnode name="10.39.30.7"/>
<failoverdomainnode name="10.39.30.8"/>
</failoverdomain>
</failoverdomains>
<resources>
<ip address="10.39.30.6" monitor_link="on" sleeptime="10"/>
<postgres-8 config_file="/var/lib/pgsql/data/postgresql.conf" name="pgsql" shutdown_wait="5" />
</resources>
<service autostart="1" exclusive="0" domain="PGSQL" name="pgsql" recovery="relocate">
<drbd name="drdb-postgres" resource="r0">
<fs device="/dev/drbd0" fsid="6202" fstype="ext3" mountpoint="/var/lib/pgsql" name="pgsql" options="noatime"/>
</drbd>
<ip ref="10.39.30.6"/>
<postgres-8 ref="pgsql"/>
</service>
</rm>
<cman expected_votes="1" two_node="1"/>
<fence_daemon clean_start="1" post_fail_delay="0" post_join_delay="3"/>
</cluster>

DRDB Configuration:

Install the necessary files:
yum install gcc flex make libxslt rpm-build redhat-rpm-config kernel-devel
You need to download and install DRBD manually
wget http://oss.linbit.com/drbd/8.4/drbd-8.4.1.tar.gz 
 the following commands will generate DRBD RPM packages:
tar -xvf *.tar.gz
mkdir -p /root/rpmbuild/SOURCES/
cp drbd*.tar.gz /root/rpmbuild/SOURCES/
cd drbd-8.4.1
./configure --with-rgmanager --enable-spec --with-km
make tgz
rpmbuild --bb drbd.spec --without xen --without heartbeat --without udev --without pacemaker --with rgmanager
rpmbuild --bb drbd-kernel.spec
rpmbuild --bb drbd-km.spec
Now install the newly created packages:
cd /root/rpmbuild/RPMS/x86_64
rpm -i drbd-utils-8.4.1-1.el6.x86_64.rpm drbd-bash-completion-8.4.1-1.el6.x86_64.rpm drbd-8.4.1-1.el6.x86_64.rpm drbd-rgmanager-8.4.1-1.el6.x86_64.rpm drbd-km-2.6.32_279.14.1.el6.x86_64-8.4.1-1.el6.x86_64.rpm
Add your nodes IP addresses to the hosts file on both machines:
vi /etc/hosts
10.39.30.7 RHPG1
10.39.30.8 RHPG2
Create a DRBD configuration file:
vi /etc/drbd.d/r0.res
resource r0 {
   device /dev/drbd0;
   meta-disk internal;
   on RHPG1 {
      address 10.39.30.7:7789;
      disk /dev/sdb1;
   }
   on RHPG2 {
      address 10.39.30.8:7789;
      disk /dev/sdb1;
   }
}
Create the partion /dev/sdb1 but do not format it:
fdisk /dev/sdb
Run on both machines:
drbdadm create-md r0
modprobe drbd
drbdadm up r0
Run on one of the machines to create the file system:
drbdadm -- --overwrite-data-of-peer primary r0
Check the sync status on any of the hosts with:
service drbd status
Create the file system on /dev/drbd0
mkfs.ext3 /dev/drbd0
and move over the PostgreSQL data
mkdir /tmp/pgdata
mount /dev/drbd0 /tmp/pgdata
cp -r /var/lib/pgsql /tmp/pgdata
Wait until the data is synced over the two hosts check the status with:
service drbd status
Unmount the drbd device:
umount /dev/drbd0
and then, on both hosts do:
rm -rf /var/lib/pgsql/*
Restart the drbd service
service drbd restart
the status from:
service drbd status
should show that both hosts are in secondary:
0:r0 Connected Secondary/Secondary UpToDate/UpToDate C
And ready to be managed by rgmanager.

Possibly Related Posts