SQUID Load Balancing For HTTP-AUTH Applications

Recently I was working with SQUID Load Balancing server for one of the PHP Based Web Application.

The app uses HTTP-AUTH for one of its protected directory. It uses Apache .htaccess with .htpasswd Unfortunately the login was completely failing in the live environment but not in test environment.

The difference found was the live environment had a SQUID Load balancing which was not in TEST (Something wrong should not be the case, both environments should resemble similar).

Then it was observed that the User name and password sent from the client is not reaching the real-application server. It is chopped at the SQUID.

Why SQUID is not passing the information?
SQUID has features to do proxy / load balancing with authentication, where SQUID assumes that the AUTH header is for SQUID and not for the web application so it never forwards the AUTH Header.

How to forward the AUTH Header?
Looking in to the squid.conf

cache_peer IP.ADDRESS parent 80 0 no-query originserver login=PASS

The last suffix login=PASS fixed the problem.

The login=PASS forwards the HTTP-AUTH credentials to the destination server.

SQUID Clearing cache in a load balancer / caching server

Are you running a SQUID Caching server before your web server to boost up the performance?
If yes and you face issues when the content of the site changes. It might be due to the squid's cache still having the old content. The following steps will help to refresh the cache.

Why we need to clear the cache?
In most cases the content in the cache is out dated with the live data.

What is the normal way to clear the cache?
We can clear the cache by removing the files in the cache directory.

Where is the cache directory?
The cache directory changes from system to system based on the configuration file settings.
We can find the cache directory by looking for the cache_dir property in the /etc/squid/squid.conf file.
Steps to clear the cache:
1. Login as privileged user.
2. Shutdown squid.
Eg: Fedora / Redhat / CentOS
# service squid stop
3. Remove the cache files.
The directory is the path specified in cache_dir
#rm -rf /var/spool/squid/*
4. Start the squid again
# service squid start
5. We should be able to see a message cache created in cache_dir directory.

Make Dynamic VirtualHost in Apache

Are you working in Apache? Are you configuring VritualHost often?
Here is a cool solution that avoids us configuring the VirtualHost directive in Apache often.
Any name based apache virtual host will be automatically mapped with some predefined directory path. Which reduces the time of configuring the apache vhosts.

The below example config change in apache will do the following

1. Configures all virtual host to the /srv/www directory
2. All VirtualHost by name should have a directory with the domain name. Ex: Vhost test.example.com will have a directory /srv/www/test.example.com
3. The Document root directory will be htdocs by default
4. The error log will be added to the specific domain file and all access logs are added to common file.
   

Example Configuration to add in httpd.conf
# this log format can be split per-virtual-host based on the first field
LogFormat "%V %h %l %u %t \"%r\" %s %b" vcommon
CustomLog logs/access_log vcommon
#ErrorLog logs/%0_error_log

# include the server name in the filenames used to satisfy requests
VirtualDocumentRoot /srv/www/%0/htdocs
VirtualScriptAlias /srv/www/%0/cgi-bin

Easy SVN Web Administration

In the last month blog we were looking into "How to install and configure an SVN server".
The blog gives a basic SVN server configuration with command line and managing it through the command line. When we end up with more projects / users we need more repositories and managing the users authentication details becomes a nightmare. To simplify the user creation and repository management we can go for a web based SVN solution.

;-) Ohh don't think that we are going to do the Apache setup for each repository and more. We have a better solution.

Here comes a better SVN with USVN (User friendly SVN)

What do we need for this?
Requirements:

• PHP 5 (5.1.2 <= ver)
• apache2
• mod_dav_svn enable
• mod_rewrite enable
• subversion
• mod_svn enable
• mod_authz_svn enable

Steps to setup.
1. Download USVN from http://www.usvn.info/download
2. Extract the zip to the root directory of apache web directory
3. Access the page via web to start the installation and proceed through the installation.

Sample configurations followed in Success Factory.
1. Apache Config

###########################################
# Vhost: svn.successfactory.local #
# Note: we need proper DNS setup to #
# make the URL work #
###########################################
<VirtualHost *:80>
DocumentRoot /srv/www/svn.successfactory.local/htdocs
ServerName svn.successfactory.local
<Directory />
AllowOverride All
</Directory>
<Location /repository/>
ErrorDocument 404 default
DAV svn
Require valid-user
SVNParentPath /srv/svn
SVNListParentPath off
AuthType Basic
AuthName "USVN"
AuthUserFile /srv/svn/htpasswd
AuthzSVNAccessFile /srv/svn/authz
</Location>
</VirtualHost>

The SVN repository resides in /srv/svn as explained in SVN installation post.
An SQLite DB is selected to maintain the SVN management informations.

After installation we can manage the SVN through web like
http://svn.successfactory.local

You will come across a login page as shown below.




We can manage new projects / users / groups through the simple web panel as shown below.





Hope this USVN brings a peace of mind in administring multiple SVN repositories.

Apache RewriteMap with RewriteLock

Recently I was working with a image gallery site. The site was developed with PHP on Apache which stores images and renders it out.

The photos where stored in a similar path stated below.
/images/photo_id/photos_style/photo_id.jpg

Example:
/images/200/portrait/200.jpg

The requirement was not to show the original path in URL and it should resemble like the below
/<photo_id>/<photo_id>_<_style>.jpg

Example:
/200/200_portrait.jpg

The logic had more complexity than explained here which required a math calculation to get the complete path. To achieve the calculation a perl rewrite rule was introduced.

RewriteMap prg MapType:/path/to/rewrite_rule.pl

The perl script was something similar to below with more logic

#!/usr/bin/perl
$| = 1;
while () {
    # ...put here any transformations or lookups...
    print $_;
}

The script started working well by redirecting to original directory (internally) with output like
/images/200/portrait/200.jpg

But when the server got loaded heavily with more requests. The output scrambled like
/mages/200/portit/200.jpg
/ramages/200/portrait/200.jpg

etc...

Which was due to the perl script not in sync with Apache. The problem was solved when a RewriteLock was introduced. But still a surprise how this solved it immediately... ;-)

RewriteLock "/path/to/empty/lock/file"
in the global section of httpd.conf

Linux: Setting UP DNS Cache server

1. The DNS servers in a network may be with huge traffic or it might have more downtime resulting in failure of resolving domain names.

2. May be a dialup machine has very slow internet connection where resolving a DNS query might take more time.

The solution for both the problems is to have a caching DNS server. Installing a dnsmasq and running it as a service on local host will resolve the issue.

Steps to Setup DNS Cache Server
(Following lines works good in Fedora / Redhat / CentOS)

Install dnsmasq
$ yum install dnsmasq

Make dnsmasq start on boot
$ chkconfig dnsmasq on

Start dnsmasq immediately
$ service dnsmasq start

Change the network setting to work through this cache server


Open the network settings

Add the Primary DNS as localhost by adding 127.0.0.1
Move the primary and secondary to secondary and tertiary.
Click File->Save

Restart the network
$ service network restart

Test the network DNS resolving speed after the first time access to the site. It will be much faster as it comes from local.

Install & Configure SVN Server

There are many tutorials available to work with SVN and the best of all is the
svnbook.read-bean.com, This article is about making a quick SVN server with very few steps without much issues.

Server Environment: Redhat / CentOS / Fedora

Install SVN
# yum install subversion

Create SVN Directory

# mkdir -p /srv/svn

Start SVN Server
# svnserve -d /srv/svn

Create SVN Repository
# svnadmin create /srv/svn/myproject

Create Users in /srv/svn/myproject/conf/passwd (Add the following lines)

admin = adM!nPassw0rd
developer = password

Grant User Permissions in /srv/svn/myproject/conf/auth (Add the following lines)

[/]
admin = rw
developer = rw

Test the setup
Checkout from different macine or in same machine with different directory

192.168.1.1 is assumed IP of the SVN server

# svn co http://192.168.1.1/myproject/ myproject

Test SVN Commit

# cd myproject
# echo "Hello World" > test.txt
# svn add test.txt
# svn commit -m "Test Commit"

You can checkout the same project from any machine but now you will find the test.txt.

More details on how to backup and restore svn and structuring to use are explained step by step in TechysPage(SVN-Revision-Control)
It would be good to go through the article in TechysPage.

The recent POST on User friendly SVN will help you to configure SVN with Admin panel