Back to index

moin  1.9.0~rc2
Public Member Functions | Private Member Functions | Private Attributes
MoinMoin.support.flup.server.preforkserver.PreforkServer Class Reference

List of all members.

Public Member Functions

def __init__
def run

Private Member Functions

def _cleanupChildren
def _reapChildren
def _spawnChild
def _isClientAllowed
def _notifyParent
def _child
def _hupHandler
def _intHandler
def _chldHandler
def _usr1Handler
def _installSignalHandlers
def _restoreSignalHandlers

Private Attributes

 _minSpare
 _maxSpare
 _maxChildren
 _maxRequests
 _jobClass
 _jobArgs
 _children
 _children_to_purge
 _last_purge
 _keepGoing
 _hupReceived
 _oldSIGs

Detailed Description

A preforked server model conceptually similar to Apache httpd(2). At
any given time, ensures there are at least minSpare children ready to
process new requests (up to a maximum of maxChildren children total).
If the number of idle children is ever above maxSpare, the extra
children are killed.

If maxRequests is positive, each child will only handle that many
requests in its lifetime before exiting.

jobClass should be a class whose constructor takes at least two
arguments: the client socket and client address. jobArgs, which
must be a list or tuple, is any additional (static) arguments you
wish to pass to the constructor.

jobClass should have a run() method (taking no arguments) that does
the actual work. When run() returns, the request is considered
complete and the child process moves to idle state.

Definition at line 67 of file preforkserver.py.


Constructor & Destructor Documentation

def MoinMoin.support.flup.server.preforkserver.PreforkServer.__init__ (   self,
  minSpare = 1,
  maxSpare = 5,
  maxChildren = 50,
  maxRequests = 0,
  jobClass = None,
  jobArgs = () 
)

Definition at line 88 of file preforkserver.py.

00088 
00089                  maxRequests=0, jobClass=None, jobArgs=()):
00090         self._minSpare = minSpare
00091         self._maxSpare = maxSpare
00092         self._maxChildren = max(maxSpare, maxChildren)
00093         self._maxRequests = maxRequests
00094         self._jobClass = jobClass
00095         self._jobArgs = jobArgs
00096 
00097         # Internal state of children. Maps pids to dictionaries with two
00098         # members: 'file' and 'avail'. 'file' is the socket to that
00099         # individidual child and 'avail' is whether or not the child is
00100         # free to process requests.
00101         self._children = {}
00102 
00103         self._children_to_purge = []
00104         self._last_purge = 0
00105 
00106         if minSpare < 1:
00107             raise ValueError("minSpare must be at least 1!")
00108         if maxSpare < minSpare:
00109             raise ValueError("maxSpare must be greater than, or equal to, minSpare!")


Member Function Documentation

def MoinMoin.support.flup.server.preforkserver.PreforkServer._child (   self,
  sock,
  parent 
) [private]
Main loop for children.

Definition at line 355 of file preforkserver.py.

00355 
00356     def _child(self, sock, parent):
00357         """Main loop for children."""
00358         requestCount = 0
00359 
00360         # Re-seed random module
00361         preseed = ''
00362         # urandom only exists in Python >= 2.4
00363         if hasattr(os, 'urandom'):
00364             try:
00365                 preseed = os.urandom(16)
00366             except NotImplementedError:
00367                 pass
00368         # Have doubts about this. random.seed will just hash the string
00369         random.seed('%s%s%s' % (preseed, os.getpid(), time.time()))
00370         del preseed
00371 
00372         while True:
00373             # Wait for any activity on the main socket or parent socket.
00374             r, w, e = select.select([sock, parent], [], [])
00375 
00376             for f in r:
00377                 # If there's any activity on the parent socket, it
00378                 # means the parent wants us to die or has died itself.
00379                 # Either way, exit.
00380                 if f is parent:
00381                     return
00382 
00383             # Otherwise, there's activity on the main socket...
00384             try:
00385                 clientSock, addr = sock.accept()
00386             except socket.error, e:
00387                 if e[0] == errno.EAGAIN:
00388                     # Or maybe not.
00389                     continue
00390                 raise
00391 
00392             setCloseOnExec(clientSock)
00393             
00394             # Check if this client is allowed.
00395             if not self._isClientAllowed(addr):
00396                 clientSock.close()
00397                 continue
00398 
00399             # Notify parent we're no longer available.
00400             self._notifyParent(parent, '\x00')
00401 
00402             # Do the job.
00403             self._jobClass(clientSock, addr, *self._jobArgs).run()
00404 
00405             # If we've serviced the maximum number of requests, exit.
00406             if self._maxRequests > 0:
00407                 requestCount += 1
00408                 if requestCount >= self._maxRequests:
00409                     break
00410                 
00411             # Tell parent we're free again.
00412             if not self._notifyParent(parent, '\xff'):
00413                 return # Parent is gone.

Here is the call graph for this function:

Here is the caller graph for this function:

def MoinMoin.support.flup.server.preforkserver.PreforkServer._chldHandler (   self,
  signum,
  frame 
) [private]

Definition at line 423 of file preforkserver.py.

00423 
00424     def _chldHandler(self, signum, frame):
00425         # Do nothing (breaks us out of select and allows us to reap children).
00426         pass

Closes all child sockets (letting those that are available know
that it's time to exit). Sends SIGINT to those that are currently
processing (and hopes that it finishses ASAP).

Any children remaining after 10 seconds is SIGKILLed.

Definition at line 227 of file preforkserver.py.

00227 
00228     def _cleanupChildren(self):
00229         """
00230         Closes all child sockets (letting those that are available know
00231         that it's time to exit). Sends SIGINT to those that are currently
00232         processing (and hopes that it finishses ASAP).
00233 
00234         Any children remaining after 10 seconds is SIGKILLed.
00235         """
00236         # Let all children know it's time to go.
00237         for pid,d in self._children.items():
00238             if d['file'] is not None:
00239                 d['file'].close()
00240                 d['file'] = None
00241             if not d['avail']:
00242                 # Child is unavailable. SIGINT it.
00243                 try:
00244                     os.kill(pid, signal.SIGINT)
00245                 except OSError, e:
00246                     if e[0] != errno.ESRCH:
00247                         raise
00248 
00249         def alrmHandler(signum, frame):
00250             pass
00251 
00252         # Set up alarm to wake us up after 10 seconds.
00253         oldSIGALRM = signal.getsignal(signal.SIGALRM)
00254         signal.signal(signal.SIGALRM, alrmHandler)
00255         signal.alarm(10)
00256 
00257         # Wait for all children to die.
00258         while len(self._children):
00259             try:
00260                 pid, status = os.wait()
00261             except OSError, e:
00262                 if e[0] in (errno.ECHILD, errno.EINTR):
00263                     break
00264             if self._children.has_key(pid):
00265                 del self._children[pid]
00266 
00267         signal.signal(signal.SIGALRM, oldSIGALRM)
00268 
00269         # Forcefully kill any remaining children.
00270         for pid in self._children.keys():
00271             try:
00272                 os.kill(pid, signal.SIGKILL)
00273             except OSError, e:
00274                 if e[0] != errno.ESRCH:
00275                     raise

def MoinMoin.support.flup.server.preforkserver.PreforkServer._hupHandler (   self,
  signum,
  frame 
) [private]

Definition at line 416 of file preforkserver.py.

00416 
00417     def _hupHandler(self, signum, frame):
00418         self._keepGoing = False
00419         self._hupReceived = True

Definition at line 431 of file preforkserver.py.

00431 
00432     def _installSignalHandlers(self):
00433         supportedSignals = [signal.SIGINT, signal.SIGTERM]
00434         if hasattr(signal, 'SIGHUP'):
00435             supportedSignals.append(signal.SIGHUP)
00436         if hasattr(signal, 'SIGUSR1'):
00437             supportedSignals.append(signal.SIGUSR1)
00438 
00439         self._oldSIGs = [(x,signal.getsignal(x)) for x in supportedSignals]
00440 
00441         for sig in supportedSignals:
00442             if hasattr(signal, 'SIGHUP') and sig == signal.SIGHUP:
00443                 signal.signal(sig, self._hupHandler)
00444             elif hasattr(signal, 'SIGUSR1') and sig == signal.SIGUSR1:
00445                 signal.signal(sig, self._usr1Handler)
00446             else:
00447                 signal.signal(sig, self._intHandler)

def MoinMoin.support.flup.server.preforkserver.PreforkServer._intHandler (   self,
  signum,
  frame 
) [private]

Definition at line 420 of file preforkserver.py.

00420 
00421     def _intHandler(self, signum, frame):
00422         self._keepGoing = False

Override to provide access control.

Definition at line 336 of file preforkserver.py.

00336 
00337     def _isClientAllowed(self, addr):
00338         """Override to provide access control."""
00339         return True

Here is the caller graph for this function:

def MoinMoin.support.flup.server.preforkserver.PreforkServer._notifyParent (   self,
  parent,
  msg 
) [private]
Send message to parent, ignoring EPIPE and retrying on EAGAIN

Definition at line 340 of file preforkserver.py.

00340 
00341     def _notifyParent(self, parent, msg):
00342         """Send message to parent, ignoring EPIPE and retrying on EAGAIN"""
00343         while True:
00344             try:
00345                 parent.send(msg)
00346                 return True
00347             except socket.error, e:
00348                 if e[0] == errno.EPIPE:
00349                     return False # Parent is gone
00350                 if e[0] == errno.EAGAIN:
00351                     # Wait for socket change before sending again
00352                     select.select([], [parent], [])
00353                 else:
00354                     raise
                

Here is the caller graph for this function:

Cleans up self._children whenever children die.

Definition at line 276 of file preforkserver.py.

00276 
00277     def _reapChildren(self):
00278         """Cleans up self._children whenever children die."""
00279         while True:
00280             try:
00281                 pid, status = os.waitpid(-1, os.WNOHANG)
00282             except OSError, e:
00283                 if e[0] == errno.ECHILD:
00284                     break
00285                 raise
00286             if pid <= 0:
00287                 break
00288             if self._children.has_key(pid): # Sanity check.
00289                 if self._children[pid]['file'] is not None:
00290                     self._children[pid]['file'].close()
00291                 del self._children[pid]

Restores previous signal handlers.

Definition at line 448 of file preforkserver.py.

00448 
00449     def _restoreSignalHandlers(self):
00450         """Restores previous signal handlers."""
00451         for signum,handler in self._oldSIGs:
00452             signal.signal(signum, handler)

Here is the caller graph for this function:

Spawn a single child. Returns True if successful, False otherwise.

Definition at line 292 of file preforkserver.py.

00292 
00293     def _spawnChild(self, sock):
00294         """
00295         Spawn a single child. Returns True if successful, False otherwise.
00296         """
00297         # This socket pair is used for very simple communication between
00298         # the parent and its children.
00299         parent, child = socket.socketpair()
00300         parent.setblocking(0)
00301         setCloseOnExec(parent)
00302         child.setblocking(0)
00303         setCloseOnExec(child)
00304         try:
00305             pid = os.fork()
00306         except OSError, e:
00307             if e[0] in (errno.EAGAIN, errno.ENOMEM):
00308                 return False # Can't fork anymore.
00309             raise
00310         if not pid:
00311             # Child
00312             child.close()
00313             # Put child into its own process group.
00314             pid = os.getpid()
00315             os.setpgid(pid, pid)
00316             # Restore signal handlers.
00317             self._restoreSignalHandlers()
00318             # Close copies of child sockets.
00319             for f in [x['file'] for x in self._children.values()
00320                       if x['file'] is not None]:
00321                 f.close()
00322             self._children = {}
00323             try:
00324                 # Enter main loop.
00325                 self._child(sock, parent)
00326             except KeyboardInterrupt:
00327                 pass
00328             sys.exit(0)
00329         else:
00330             # Parent
00331             parent.close()
00332             d = self._children[pid] = {}
00333             d['file'] = child
00334             d['avail'] = True
00335             return True

Here is the call graph for this function:

def MoinMoin.support.flup.server.preforkserver.PreforkServer._usr1Handler (   self,
  signum,
  frame 
) [private]

Definition at line 427 of file preforkserver.py.

00427 
00428     def _usr1Handler(self, signum, frame):
00429         self._children_to_purge = [x['file'] for x in self._children.values()
00430                                    if x['file'] is not None]

The main loop. Pass a socket that is ready to accept() client
connections. Return value will be True or False indiciating whether
or not the loop was exited due to SIGHUP.

Definition at line 110 of file preforkserver.py.

00110 
00111     def run(self, sock):
00112         """
00113         The main loop. Pass a socket that is ready to accept() client
00114         connections. Return value will be True or False indiciating whether
00115         or not the loop was exited due to SIGHUP.
00116         """
00117         # Set up signal handlers.
00118         self._keepGoing = True
00119         self._hupReceived = False
00120         self._installSignalHandlers()
00121 
00122         # Don't want operations on main socket to block.
00123         sock.setblocking(0)
00124 
00125         # Set close-on-exec
00126         setCloseOnExec(sock)
00127         
00128         # Main loop.
00129         while self._keepGoing:
00130             # Maintain minimum number of children. Note that we are checking
00131             # the absolute number of children, not the number of "available"
00132             # children. We explicitly test against _maxSpare to maintain
00133             # an *optimistic* absolute minimum. The number of children will
00134             # always be in the range [_maxSpare, _maxChildren].
00135             while len(self._children) < self._maxSpare:
00136                 if not self._spawnChild(sock): break
00137 
00138             # Wait on any socket activity from live children.
00139             r = [x['file'] for x in self._children.values()
00140                  if x['file'] is not None]
00141 
00142             if len(r) == len(self._children) and not self._children_to_purge:
00143                 timeout = None
00144             else:
00145                 # There are dead children that need to be reaped, ensure
00146                 # that they are by timing out, if necessary. Or there are some
00147                 # children that need to die.
00148                 timeout = 2
00149 
00150             w = (time.time() > self._last_purge + 10) and self._children_to_purge or []
00151             try:
00152                 r, w, e = select.select(r, w, [], timeout)
00153             except select.error, e:
00154                 if e[0] != errno.EINTR:
00155                     raise
00156 
00157             # Scan child sockets and tend to those that need attention.
00158             for child in r:
00159                 # Receive status byte.
00160                 try:
00161                     state = child.recv(1)
00162                 except socket.error, e:
00163                     if e[0] in (errno.EAGAIN, errno.EINTR):
00164                         # Guess it really didn't need attention?
00165                         continue
00166                     raise
00167                 # Try to match it with a child. (Do we need a reverse map?)
00168                 for pid,d in self._children.items():
00169                     if child is d['file']:
00170                         if state:
00171                             # Set availability status accordingly.
00172                             self._children[pid]['avail'] = state != '\x00'
00173                         else:
00174                             # Didn't receive anything. Child is most likely
00175                             # dead.
00176                             d = self._children[pid]
00177                             d['file'].close()
00178                             d['file'] = None
00179                             d['avail'] = False
00180 
00181             for child in w:
00182                 # purging child
00183                 child.send('bye, bye')
00184                 del self._children_to_purge[self._children_to_purge.index(child)]
00185                 self._last_purge = time.time()
00186 
00187                 # Try to match it with a child. (Do we need a reverse map?)
00188                 for pid,d in self._children.items():
00189                     if child is d['file']:
00190                         d['file'].close()
00191                         d['file'] = None
00192                         d['avail'] = False
00193                 break
00194 
00195             # Reap children.
00196             self._reapChildren()
00197 
00198             # See who and how many children are available.
00199             availList = filter(lambda x: x[1]['avail'], self._children.items())
00200             avail = len(availList)
00201 
00202             if avail < self._minSpare:
00203                 # Need to spawn more children.
00204                 while avail < self._minSpare and \
00205                       len(self._children) < self._maxChildren:
00206                     if not self._spawnChild(sock): break
00207                     avail += 1
00208             elif avail > self._maxSpare:
00209                 # Too many spares, kill off the extras.
00210                 pids = [x[0] for x in availList]
00211                 pids.sort()
00212                 pids = pids[self._maxSpare:]
00213                 for pid in pids:
00214                     d = self._children[pid]
00215                     d['file'].close()
00216                     d['file'] = None
00217                     d['avail'] = False
00218 
00219         # Clean up all child processes.
00220         self._cleanupChildren()
00221 
00222         # Restore signal handlers.
00223         self._restoreSignalHandlers()
00224 
00225         # Return bool based on whether or not SIGHUP was received.
00226         return self._hupReceived

Here is the caller graph for this function:


Member Data Documentation

Definition at line 100 of file preforkserver.py.

Definition at line 102 of file preforkserver.py.

Definition at line 118 of file preforkserver.py.

Definition at line 94 of file preforkserver.py.

Definition at line 93 of file preforkserver.py.

Definition at line 117 of file preforkserver.py.

Definition at line 103 of file preforkserver.py.

Definition at line 91 of file preforkserver.py.

Definition at line 92 of file preforkserver.py.

Definition at line 90 of file preforkserver.py.

Definition at line 89 of file preforkserver.py.

Definition at line 438 of file preforkserver.py.


The documentation for this class was generated from the following file: