|
This document describes load balanced Web/mail cluster scheme for
generic Linux NFS NAS.
Load balanced cluster in CP
Distribution of requests across load balanced cluster
H-Sphere regards Web or mail load balanced cluster as a single server. Requests are passed to external IPs
routed by the load balancer.
Load balancer distributes requests evenly across the master and slave servers (internal IPs corresponding to external IP).
Shared content
Master and slave servers share the same /hsphere directory mounted by NFS to the
/filer/<cluster_type>_<cluster_id>/ directory on the NAS where load balanced cluster content is actually stored.
Here, <cluster_type> is mail or web, and <cluster_id> is cluster id -
there may be multiple load balanced clusters mounted to the NAS: 01, 02, ...
(See the illustration for 2 Web and 2 mail clusters.)
All user content, scripts, and the majority of H-Sphere related binaries are
installed into the /hsphere directory and shared by master and all slaves.
Follow the Adding Load Balanced Clusters on Shared Linux NFS instructions to learn how to correctly
mount the shared storage on the NAS to the master and slave servers.
Specific master/slave content
Along with the common shared storage, master and slave servers have their own
H-Sphere specific (Apache, FTP) and IP-dependent (network) logs and configuration:
- Both master and slave servers have unique Apache logs stored locally in the /var/log/hsphere/httpd directory
instead of the default /hsphere/local/var/httpd/logs directory.
- On the master server, Apache, FTP and network configuration is located in the /hsphere directory
(which is a mountpoint to the NAS and is common for the master and the slave servers):
/hsphere/local/config/httpd/
/hsphere/local/config/ftpd/
/hsphere/local/network/
On slave servers, however, this data is unique and is stored locally in the /etc/hsphere directory:
/etc/hsphere/httpd/
/etc/hsphere/ftpd/
/etc/hsphere/network/
H-Sphere updater running with the hspackages slaves=web|mail|all option creates the
/etc/hsphere directory data on slave servers.
Synchronization between master and slave servers
The special cron script /hsphere/shared/scripts/cron/lb_sync.sh runs each minute on each slave server to
synchronize data on master and slave servers. It parses and synchronizes:
- User Apache/ProFTPd config files /etc/hsphere/[httpd|ftpd]/sites/*.conf,
and /etc/hsphere/network/ips;
- the /etc/passwd, /etc/shadow, /etc/group files.
Traffic calculation
User logs are located in the /hsphere/local/home/<user>/logs/domain.com/ directory. On master server,
log filenames look like:
{domain.name}
referrer_log
access_log
error_log
Slave servers write the following logs:
{domain.name}_SRV-N
referrer_log_SRV-N
access_log_SRV-N
error_log_SRV-N
where N is slave server id: 1, 2, ...
The /hsphere/shared/scripts/cron/cron-rotate.pl script
has been adapted to calculate statistics on load balanced cluster.
It runs on master and slave servers by the following scheme:
- on slave servers (e.g., 1:30am):
- on master server (e.g., 2:00am):
- rotates logs on the master
- restarts Apache
- merges master and slave logs
- launches log analyzers (Webalizer, AWStats, ModLogAn)
Load balanced cluster map
The following two files construct load balanced cluster map.
At the moment, they need to be created manually on master and slave servers:
- /hsphere/local/config/lb.map - created on the master server and has the following format:
<Master_IP>|<Slave1_IP>|...|<SlaveN_IP>
The lines of the same format should be also added for each dedicated IP bound on the cluster:
<Master_Dedicated_IP>|<Slave1_Dedicated_IP>|...|<SlaveN_Dedicated_IP>
- /etc/hsphere/lb.id - created on both the master and slave servers and contains the following line:
<CLUSTER_TYPE>|<SERVER_ID>
where <CLUSTER_TYPE> is mail or web; <SERVER_ID> is LB server id:
0 for master, 1 for the first slave, 2 for the second slave, etc.
For example, for slave server with <Slave2_IP> in LB Web cluster the lb.id file will look like:
web|2
NAT configuration for load balanced clusters
To configure load balanced Web/mail cluster with NAT, you must have NAT turned on in H-Sphere and
put external Web/mail server IP routed by the Load Balancer into correspondence with the master server's internal IP.
For example, for a load balanced Web cluster with one master and 4 slave servers,
where the master Web server's internal IP 192.168.0.100 corresponds to the external IP 12.34.56.100
bound to the Load Balancer.
- In the ~cpanel/shiva/psoft_config/ips-map.xml file on the CP server there should be the following record:
<ips>
. . .
<ip ext="12.34.56.100" int="192.168.0.100"/>
. . .
</ips>
-
All dedicated IPs on the master server must be also associated with corresponding IPs on the Load Balancer
and similar records must be added to the ip-map.xml file:
<ip ext="LB_Dedicated_IP" int="Master_Dedicated_IP"/>
- Also, you should have external IP in the E.Manager -> DNS Manager -> Service Zone menu in admin CP.
For example:
www.test.com 3600 IN A 12.34.56.100
mail.test.com 3600 IN MX 12.34.56.111
|
