OpenLSD ENOpenLSD
 
High Available HTTP Compression
Home OpenLSD and Co Tools BBMAP TriDenT CoLUMBO Other Works CV-Resume

High Available HTTP Compression

 

In order to answer the constraints of network bandwith, I set up a network compression system for web services entirely autonomous and powerful. It is based on various components:

  • Blade Center with a minimum of 4 blades (2 blades for the function of distribution and high availability and 2 blades for the functions of compression and caching - optional -). It is possible nevertheless to function only with 2 servers.
  • Apache 2.2.X to manage compression on the flight
  • Apache 2.0.X to manage the functions of caching (as long as version 2.2 is not completely stabilized)
  • HAProxy and Keepalived for the functions of distribution and high availability.

The use of a Blade Center is not an obligation, it is indeed completely possible to carry out this brick with only 2 small Intel servers.

 

The benefits are as follows:

  • The bandwidth is divided overall by two (compression up to 90% on the text elements, images and other pdf not being compressed).
  • The cache relieves the Web servers by not re-asking static contents. For the moment, this mask is limited to subtree structures (functions of Apache 2). It would be possible to replace this piece by another (such as Squid).
  • The time response from the user point of vue is 1 against 10 before, i.e. without this solution, some of our users (low bandwith) had an impression of about 10 seconds to see the Web page, against 1 second with this device. This impression was confirmed into the access logs of our applications.
  • The high availability of the service is ensured by the presence of at least 2 Intel servers ensuring compression on the flight. The distribution between these two servers is carried out via a service equivalent to an Alteon (Nortel switch box of level 7). This distribution is ensured via the free software HAProxy.
  • The high availability of the distribution is ensured via Keeaplived. It is possible to place the service Keepalived and HAProxy on the servers running the services of compression in order to decrease the number of Intel servers, even if that brings more flexibility of administration by separating them.

It is possible to use only one physical component to manage multiple web services components (we had 4 distinct web services who used the same technical component of compression).

 

The cost of this solution is limited to the servers(4 Intel servers in a Blade Center for our case is approximately 40.000 euros compared with approximately 400.000 euros for 2 boxes of the Netscaler type which presents at the same time the high availability, the load distribution and compression).

 

We did not have any problem of performances and this in spite of the 140.000 potential users and the 10 million pages a day.