Apache Tuning

The goal of this page is to give some details about experiments for Apache tuning in Nebulous use context. When many requests are sent to Nebulous, the Apache server displays messages in its log looking like:

[Thu Mar 10 09:11:09 2011] [info] server seems busy, (you may need to increase StartServers,
                           or Min/MaxSpareServers), spawning 8 children, there are 0 idle, 
                           and 128 total children
[Thu Mar 10 09:11:10 2011] [notice] child pid 6790 exit signal Segmentation fault (11)
[Thu Mar 10 09:11:10 2011] [notice] child pid 6791 exit signal Segmentation fault (11)
[Thu Mar 10 09:11:10 2011] [notice] child pid 6792 exit signal Segmentation fault (11)
[Thu Mar 10 09:11:10 2011] [notice] child pid 6793 exit signal Segmentation fault (11)

Such messages are seen when the server is overwhelmed by requests. The goal of the experiments is to determine the Apache configuration where such messages could be avoided.

Introduction

Apache Configuration

For convenience, the Apache log level has been set to debug (on ippdb00, it is set to warn).

The original mpm_prefork_module configuration for Apache is the following:

<IfModule mpm_prefork_module>
        StartServers            5
        MinSpareServers         5
        MaxSpareServers         10
        MaxClients              150
        MaxRequestsPerChild     10000
</IfModule>

Such a configuration will be written: [5, 5, 10, 150] (to avoid any confusion, if the MinSpareServers value was 6, the configuration would be denoted [5, 6, 10, 150]).

Constraints on the configuration can be seen in the ​prefork module configuration documentation.

Experiment

I wrote a small utility script that starts nbProcesses processes. Each process attempts to create nbFiles files in Nebulous using neb-touch. To ensure that files do not previously exist in Nebulous, each instance of the utility script is called with an index argument (let's call it experimentNb). Each experiment is run this way:

  python startNebFilesCreation.py <experimentNb> <nbProcesses> <nbFiles>

and creates <nbProcesses>*<nbFiles> files in Nebulous named:

  neb:///<a path to have a total path length of 200 bytes>/<experimentNb>/<processId>/<fileIdWithProcess>

The path length is adjusted through the <a path to have a total path length of 200 bytes> which is basically a sequence of "1234567890/" strings to have a total length of about 200 bytes which is the average size of a Nebulous instance entry.

Since an experiment depends on the Apache configuration, an experiment is fully described by (<experimentNb>, <nbProcesses>, <nbFiles>, <Apache Configuration>) where <Apache Configuration> is the 5-uple shown in previous paragraph.

Operating Conditions

The experiments are performed on ippc01 where a local Nebulous database has been installed. To avoid any conflict with existing tools, a local neb-touch was used (URL: ​https://svn.pan-starrs.ifa.hawaii.edu/repo/ipp/tags/ipp-20110218/Nebulous/bin/neb-touch; Revision: 30846).