302 redirects behind SSL reverse proxy

Problem

You have a web server running plain http behind an Apache reverse proxy running https. Your application uses 302 redirects to announce new URLs or whatever the reason is for doing so.

The client will be redirected a plain http url.

Solution

You can create a Vhost listenig for plain http and all that it does is redirect the clients to https, like this:

NameVirtualHost *:80
<VirtualHost *:80>
   ServerName www.example.com
   Redirect permanent / https://secure.example.com/
</VirtualHost>

This way, the client is redirected to the plain http by the proxied server and then redirected back to https by the proxy server.

Another solution is to use mod_rewrite:

RewriteEngine On
# This will enable the Rewrite capabilities
RewriteCond %{HTTPS} !=on
# This checks to make sure the connection is not already HTTPS
RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
# This rule will redirect users from their original location, to the same location but using HTTPS.
# i.e.  http://www.example.com/foo/ to https://www.example.com/foo/
# The leading slash is made optional so that this will work either in httpd.conf
# or .htaccess context

This does the same as the previous solution.

But what if there is some firewall or load balancer that prevents plain http packets from reaching your server?

Since Apache version 2.2.4 mod_headers is able to rewrite response headers. Just add the following to your https vhost that is serving as reverse proxy to your application:

Header edit Location ^http://(.*)$ https://$1

This configuration statement will solve your problem. Redirects triggered by your back end web servers will be re-rewritten to comply with your SSL terminating reverse proxy/load balancer.

Example:

Listen a.b.c.d:443
<VirtualHost a.b.c.d:443>
    ServerName example.org
    # …SSL configuration…
    ProxyRequests off
    ProxyPass / http://localhost:3000/
    ProxyPassReverse / http://localhost:3000/
    RequestHeader set X-Forwarded-Protocol "https"
    Header edit Location ^http://(.*)$ https://$1
</VirtualHost>
<VirtualHost a.b.c.d:80>
    ServerName example.org
    ProxyRequests off
    ProxyPass / http://localhost:3000/
    ProxyPassReverse / http://localhost:3000/
    RequestHeader set X-Forwarded-Protocol "http"
</VirtualHost>

Iptables blacklists

Many of you already use online blacklists to fight spam. Recently I've dicovered http://www.openbl.org/ and started using their lists on my firewall to prevent attacks from hosts that are known to be preforming attacks. It works in a very similar way to all the spam blacklists out there, and this is how I've implemented them on my Firewall.

First of all you'll need to  have some packages installed:

sudo apt-get install iptables ipset wget

now create an ipset to store all the abusing IP addresses and use iptables to block them:

#!/bin/bash
BLOCKDB="block.txt"
WORKDIR="/tmp"
pwd=$(pwd)
cd $WORKDIR
#List of ips to block
ipset --create blackips iphash
## Obtain List of badguys from openbl.org
wget -q -c --output-document=$BLOCKDB http://www.openbl.org/lists/base.txt
if [ -f $BLOCKDB ]; then
    IPList=$(grep -Ev "^#" $BLOCKDB | sort -u)
    for i in $IPList
    do
        ipset --add blackips $i
    done
fi
rm $BLOCKDB
## Obtain List of badguys from ciarmy.com
wget -q -c --output-document=$BLOCKDB http://www.ciarmy.com/list/ci-badguys.txt
if [ -f $BLOCKDB ]; then
    IPList=$(grep -Ev "^#" $BLOCKDB | sort -u)
    for i in $IPList
    do
        ipset --add blackips $i
    done
fi
rm $BLOCKDB
## Obtain List of badguys from dshield.org
wget -q -c --output-document=$BLOCKDB http://feeds.dshield.org/top10-2.txt
if [ -f $BLOCKDB ]; then
    IPList=$(grep -E "^[1-9]" $BLOCKDB | cut -f1 | sort -u)
    for i in $IPList
    do
        ipset --add blackips $i
    done
fi
rm $BLOCKDB
#List of networks to block
ipset --create blacknets nethash
## Obtain List of badguys from dshield.org
wget -q -c --output-document=$BLOCKDB http://feeds.dshield.org/block.txt
if [ -f $BLOCKDB ]; then
    IPList=$(grep -E "^[1-9]" $BLOCKDB | cut -f1,3 | sed "s/\t/\//g" | sort -u)
    for i in $IPList
    do
        ipset --add blacknets $i
    done
fi
rm $BLOCKDB
## Obtain List of badguys from spamhaus.org
    wget -q -c --output-document=$BLOCKDB http://www.spamhaus.org/drop/drop.lasso
    if [ -f $BLOCKDB ]; then
      IPList=$(grep -E "^[1-9]" $BLOCKDB | cut -d" " -f1 | sort -u)
      for i in $IPList
      do
        ipset --add blacknets $i
      done
    fi
    rm $BLOCKDB
#Drop blacklisted ips
iptables -A FORWARD -m set --match-set blackips src -j DROP
iptables -A FORWARD -m set --match-set blacknets src -j DROP
cd $pwd

In the above script I've used two ipsets, one for storing IP addresses and another to store network addresses, you can add this scritp to your existing firewall and start taking advantage of the blacklists.
OpenBL is accepting donations http://www.openbl.org/donations.html if you can you should help.

NetFLOW over IPSEC VPN

I had the following NetFlow configuration on my router:

 ip flow-cache timeout active 1
 ip flow-export source FastEthernet0/1.1
 ip flow-export version 5
 ip flow-export destination 10.39.30.5 9996
!
interface FastEthernet0/0.2
 ip flow ingress
interface FastEthernet0/1.12
 ip flow ingress
interface FastEthernet0/1.40
 ip flow ingress

Witch worked fine on my other routers. But in this particular case, the NetFLOW server was only accecible through an IPSEC VPN and the flows weren't getting there.

The solution to this was to use "Flexible Netflow" configuration. This allows for the NetFlow export to be sent down the standard IPSEC VPN tunnel.

An example of the NetFlow config is as follows:

flow exporter FLOW_EXPORTER
 destination 10.39.30.5
 source FastEthernet0/1.1
 output-features
 transport udp 9996
 export-protocol netflow-v5
!
!
flow monitor FLOW_MONITOR
 record netflow-original
 exporter FLOW_EXPORTER
 cache timeout active 1
!
interface FastEthernet0/0.2
ip flow monitor FLOW_MONITOR input
!
interface FastEthernet0/1.12
ip flow monitor FLOW_MONITOR input
!
interface FastEthernet0/1.40
ip flow monitor FLOW_MONITOR input
!

Disable IPv6 on Ubuntu

Edit your /etc/sysctl.conf file and add the following to the bottom:

#disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

Or you can use the following script:

echo "#disable ipv6" | sudo tee -a /etc/sysctl.conf
echo "net.ipv6.conf.all.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf
echo "net.ipv6.conf.lo.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf

For this changes to take effect you must reboot your system.
After rebooting you can check if IPv6 has been disabled with the following command:

cat /proc/sys/net/ipv6/conf/all/disable_ipv6

0 means it's enabled and 1 - disabled.

Linux Stress tests

Consume CPU:

Fork bomb:

:(){ :|:& };:

The next one will load four CPU cores at 100%:

for i in `seq 1 4` ; do while : ; do : ; done & ; done

Or:

for i in `seq 1 4` ; do cat /dev/zero > /dev/null & ; done

Or:

#!/bin/bash
duration=120 # seconds
instances=4 # cpus
endtime=$(($(date +%s) + $duration))
for ((i=0; i<instances; i++))
do
while (($(date +%s) < $endtime)); do : ; done &
done

Using the stress tool:

stress --cpu 3

Consume RAM:

Create a 30gb ramdisk and fills it with file full of zeroes:

sudo mount -t tmpfs -o size=30G tmpfs /mnt
dd if=/dev/zero of=/mnt/tmp bs=10240 count=30720MB

Create a giant virable:

x="x" ; while : ; do x=$x$x ; echo -n "." ; done

Consume Disk:

dd if=/dev/zero of=bigfile bs=10240 count=30720MB

Simulate packet loss:

For randomly dropping 10% of incoming packets:

iptables -A INPUT -m statistic --mode random --probability 0.1 -j DROP

and for dropping 10% of outgoing packets:

iptables -A OUTPUT -m statistic --mode random --probability 0.1 -j DROP

How to reset the root password of VMware ESXi 4.1 and 5.0

You can't recover your old password but by following this steps you can set it to blank and then set a new one.

First you'll need a Linux live CD, any one will do.

After booting to a live session of Linux you must look for a file named state.tgz on your vmware host's hard drive. To do so I used:

parted -l

To list all available partitions and mounted every VFAT partition to look inside, I found it was on /dev/sda5, in your case it might be on a different one, you can mount the partitions with:

mount /dev/sda1 /mnt

(replace sda with your device's name and 1 with your partition numver)

then check inside if the file state.tgz exists:

ls /mnt/

After finding the state.tzg file you must uncompress it using:

cd /tmp
tar xzf /mnt/Hypervisor3/state.tgz

this will get you a local.tgz file witch you have to extract using:

tar xzf local.tgz

now edit the file /tmp/etc/shadow

vi etc/shadow

Inside locate the root account and just remove it's hash (everything between the first and the second colon) and login to the service console as root with no password at all.

Finally re-pack the files and move the modified state.tgz back to the VFAT partition. Probably it is a good idea to make a backup copy of the original state.tgz in case something goes wrong:

mv /mnt/state.tgz /mnt/state.tgz.bak
rm local.tgz
tar czf local.tgz etc
tar czf state.tgz local.tgz
mv state.tgz /mnt/

Reboot back into ESXi and you're done.

Calculating total disk usage by files with specific extension

For example if you want to check how much space is being used by log files on your entire system, you can use the following:

find / -type f -name "*.log*" -exec du -b {} \; | awk '{ sum += $1 } END { kb = sum / 1024; mb = kb / 1024; gb = mb / 1024; printf "%.0f MB (%.2fGB) disk space used\n", mb, gb}'

Just replace "*.log*" with the file extension you want to search for and the above will give you the disk used by the sum of all the files with that extension.