Back to index

python-biopython  1.60
Public Member Functions | Public Attributes
Bio.PopGen.FDist.Async.SplitFDist Class Reference

List of all members.

Public Member Functions

def __init__
def monitor
def acquire
def release
def run_fdist

Public Attributes

 async
 report_fun
 split_size
 parts
 data_dir

Detailed Description

Splits a FDist run.

   The idea is to split a certain number of simulations in smaller
   numbers (e.g. 30.000 sims split in 30 packets of 1.000). This
   allows to run simulations in parallel, thus taking advantage
   of multi-core CPUs.

   Each SplitFDist object can only be used to run a single FDist
   simulation.

Definition at line 66 of file Async.py.


Constructor & Destructor Documentation

def Bio.PopGen.FDist.Async.SplitFDist.__init__ (   self,
  report_fun = None,
  num_thr = 2,
  split_size = 1000,
  fdist_dir = '',
  ext = None 
)
Constructor.

   Parameters:
   report_fun - Function that is called when a single packet is
       run, it should have a single parameter: Fst.
   num_thr - Number of desired threads, typically the number
       of cores.
   split_size - Size that a full simulation will be split in.
   ext - Binary extension name (e.g. nothing on Unix, '.exe' on
       Windows).

Definition at line 78 of file Async.py.

00078 
00079         num_thr = 2, split_size = 1000, fdist_dir = '', ext = None):
00080         """Constructor.
00081 
00082            Parameters:
00083            report_fun - Function that is called when a single packet is
00084                run, it should have a single parameter: Fst.
00085            num_thr - Number of desired threads, typically the number
00086                of cores.
00087            split_size - Size that a full simulation will be split in.
00088            ext - Binary extension name (e.g. nothing on Unix, '.exe' on
00089                Windows).
00090         """
00091         self.async = Local.Local(num_thr)
00092         self.async.hooks['fdist'] = FDistAsync(fdist_dir, ext)
00093         self.report_fun = report_fun
00094         self.split_size = split_size


Member Function Documentation

Allows the external acquisition of the lock.

Definition at line 136 of file Async.py.

00136 
00137     def acquire(self):
00138         """Allows the external acquisition of the lock.
00139         """
00140         self.async.access_ds.acquire()

Monitors and reports (using report_fun) execution.

   Every time a partial simulation ends, calls report_fun.
   IMPORTANT: monitor calls can be concurrent with other
   events, ie, a tasks might end while report_fun is being
   called. This means that report_fun should be consider that
   other events might be happening while it is running (it
   can call acquire/release if necessary).

Definition at line 96 of file Async.py.

00096 
00097     def monitor(self):
00098         """Monitors and reports (using report_fun) execution.
00099 
00100            Every time a partial simulation ends, calls report_fun.
00101            IMPORTANT: monitor calls can be concurrent with other
00102            events, ie, a tasks might end while report_fun is being
00103            called. This means that report_fun should be consider that
00104            other events might be happening while it is running (it
00105            can call acquire/release if necessary).
00106         """
00107         while(True):
00108             sleep(1)
00109             self.async.access_ds.acquire()
00110             keys =  self.async.done.keys()[:]
00111             self.async.access_ds.release()
00112             for done in keys:
00113                 self.async.access_ds.acquire()
00114                 fst, files = self.async.done[done]
00115                 del self.async.done[done]
00116                 out_dat = files['out.dat']
00117                 f = open(self.data_dir + os.sep + 'out.dat','a')
00118                 f.writelines(out_dat.readlines())
00119                 f.close()
00120                 out_dat.close()
00121                 self.async.access_ds.release()
00122                 for file in os.listdir(self.parts[done]):
00123                     os.remove (self.parts[done] + os.sep + file)
00124                 os.rmdir(self.parts[done])
00125                 #print fst, out_dat
00126                 if self.report_fun:
00127                     self.report_fun(fst)
00128             self.async.access_ds.acquire()
00129             if len(self.async.waiting) == 0 and len(self.async.running) == 0 \
00130                and len(self.async.done) == 0:
00131                 break
00132             self.async.access_ds.release()
00133             #print 'R', self.async.running
00134             #print 'W', self.async.waiting
00135             #print 'R', self.async.running

Here is the call graph for this function:

Allows the external release of the lock.

Definition at line 141 of file Async.py.

00141 
00142     def release(self):
00143         """Allows the external release of the lock.
00144         """
00145         self.async.access_ds.release()

Here is the call graph for this function:

def Bio.PopGen.FDist.Async.SplitFDist.run_fdist (   self,
  npops,
  nsamples,
  fst,
  sample_size,
  mut = 0,
  num_sims = 20000,
  data_dir = '.',
  is_dominant = False,
  theta = 0.06,
  beta = (0.25, 0.25,
  max_freq = 0.99 
)
Runs FDist.

   Parameters can be seen on FDistController.run_fdist.

   It will split a single execution in several parts and
   create separated data directories.

Definition at line 150 of file Async.py.

00150 
00151         max_freq = 0.99):
00152         """Runs FDist.
00153 
00154            Parameters can be seen on FDistController.run_fdist.
00155 
00156            It will split a single execution in several parts and
00157            create separated data directories.
00158         """
00159         num_parts = num_sims/self.split_size
00160         self.parts = {}
00161         self.data_dir = data_dir
00162         for directory in range(num_parts):
00163            full_path = data_dir + os.sep + str(directory)
00164            try:
00165                os.mkdir(full_path)
00166            except OSError:
00167                pass #Its ok, if it is already there
00168            if "ss_file" in os.listdir(data_dir):
00169                shutil.copy(data_dir + os.sep + "ss_file", full_path)
00170            id = self.async.run_program('fdist', {
00171                'npops'       : npops,
00172                'nsamples'    : nsamples,
00173                'fst'         : fst,
00174                'sample_size' : sample_size,
00175                'mut'         : mut,
00176                'num_sims'    : self.split_size,
00177                'data_dir'    : full_path,
00178                'is_dominant' : is_dominant,
00179                'theta'       : theta,
00180                'beta'        : beta,
00181                'max_freq'    : max_freq 
00182            }, {})
00183            self.parts[id] = full_path
00184         thread.start_new_thread(self.monitor, ())

Here is the caller graph for this function:


Member Data Documentation

Definition at line 90 of file Async.py.

Definition at line 160 of file Async.py.

Definition at line 159 of file Async.py.

Definition at line 92 of file Async.py.

Definition at line 93 of file Async.py.


The documentation for this class was generated from the following file: