Back to index

nordugrid-arc-nox  1.1.0~rc6
Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | Static Private Attributes
Arc::SubmitterBES Class Reference

#include <SubmitterBES.h>

Inheritance diagram for Arc::SubmitterBES:
Inheritance graph
[legend]
Collaboration diagram for Arc::SubmitterBES:
Collaboration graph
[legend]

List of all members.

Public Member Functions

virtual URL Submit (const JobDescription &jobdesc, const ExecutionTarget &et) const
 This virtual method should be overridden by plugins which should be capable of submitting jobs, defined in the JobDescription jobdesc, to the ExecutionTarget et.
virtual URL Migrate (const URL &jobid, const JobDescription &jobdesc, const ExecutionTarget &et, bool forcemigration) const
 This virtual method should be overridden by plugins which should be capable of migrating jobs.
virtual bool ModifyJobDescription (JobDescription &jobdesc, const ExecutionTarget &et) const
std::string GetCksum (const std::string &file) const

Static Public Member Functions

static PluginInstance (PluginArgument *arg)
static std::string GetCksum (const std::string &file, const UserConfig &usercfg)

Protected Member Functions

bool PutFiles (const JobDescription &jobdesc, const URL &url) const
void AddJob (const JobDescription &job, const URL &jobid, const URL &cluster, const URL &infoendpoint) const
void AddJob (const JobDescription &job, const URL &jobid, const URL &cluster, const URL &infoendpoint, const std::map< std::string, std::string > &additionalInfo) const

Protected Attributes

const std::string flavour
const UserConfigusercfg

Private Member Functions

 SubmitterBES (const UserConfig &usercfg)
 ~SubmitterBES ()

Static Private Attributes

static Logger logger

Detailed Description

Definition at line 16 of file SubmitterBES.h.


Constructor & Destructor Documentation

Arc::SubmitterBES::SubmitterBES ( const UserConfig usercfg) [private]

Definition at line 42 of file SubmitterBES.cpp.

    : Submitter(usercfg, "BES") {}

Here is the caller graph for this function:

Definition at line 45 of file SubmitterBES.cpp.

{}

Member Function Documentation

void Arc::Submitter::AddJob ( const JobDescription job,
const URL jobid,
const URL cluster,
const URL infoendpoint 
) const [inline, protected, inherited]

Definition at line 69 of file Submitter.h.

                                               {
      std::map<std::string, std::string> additionalInfo;
      AddJob(job, jobid, cluster, infoendpoint, additionalInfo);
    }

Here is the caller graph for this function:

void Arc::Submitter::AddJob ( const JobDescription job,
const URL jobid,
const URL cluster,
const URL infoendpoint,
const std::map< std::string, std::string > &  additionalInfo 
) const [protected, inherited]

Definition at line 72 of file Submitter.cpp.

                                                                                   {
    NS ns;
    XMLNode info(ns, "Job");
    info.NewChild("JobID") = jobid.str();
    if (!job.Identification.JobName.empty())
      info.NewChild("Name") = job.Identification.JobName;
    info.NewChild("Flavour") = flavour;
    info.NewChild("Cluster") = cluster.str();
    info.NewChild("InfoEndpoint") = infoendpoint.str();
    info.NewChild("LocalSubmissionTime") = (std::string)Arc::Time();

    for (std::map<std::string, std::string>::const_iterator it = additionalInfo.begin();
         it != additionalInfo.end(); it++)
      info.NewChild(it->first) = it->second;

    for (std::list<std::string>::const_iterator
           it = job.Identification.ActivityOldId.begin();
         it != job.Identification.ActivityOldId.end(); it++)
      info.NewChild("OldJobID") = *it;

    std::string rep = job.UnParse("arcjsdl");
    info.NewChild("JobDescription") = (std::string)rep;

    for (std::list<FileType>::const_iterator it = job.DataStaging.File.begin();
         it != job.DataStaging.File.end(); it++)
      if (!it->Source.empty())
        if (it->Source.begin()->URI.Protocol() == "file") {
          if (!info["LocalInputFiles"])
            info.NewChild("LocalInputFiles");
          XMLNode File = info["LocalInputFiles"].NewChild("File");
          File.NewChild("Source") = it->Name;
          File.NewChild("CheckSum") = GetCksum(it->Source.begin()->URI.Path());
        }

    FileLock lock(usercfg.JobListFile());
    Config jobstorage;
    jobstorage.ReadFromFile(usercfg.JobListFile());
    jobstorage.NewChild(info);
    jobstorage.SaveToFile(usercfg.JobListFile());
  }

Here is the call graph for this function:

std::string Arc::Submitter::GetCksum ( const std::string &  file) const [inline, inherited]

Definition at line 65 of file Submitter.h.

{ return GetCksum(file, usercfg); }

Here is the call graph for this function:

Here is the caller graph for this function:

std::string Arc::Submitter::GetCksum ( const std::string &  file,
const UserConfig usercfg 
) [static, inherited]

Definition at line 116 of file Submitter.cpp.

                                                                                {
    DataHandle source(file, usercfg);
    DataBuffer buffer;

    MD5Sum md5sum;
    buffer.set(&md5sum);

    if (!source->StartReading(buffer))
      return "";

    int handle;
    unsigned int length;
    unsigned long long int offset;

    while (buffer.for_write() || !buffer.eof_read())
      if (buffer.for_write(handle, length, offset, true))
        buffer.is_written(handle);

    if (!source->StopReading())
      return "";

    if (!buffer.checksum_valid())
      return "";

    char buf[100];
    md5sum.print(buf, 100);
    return buf;
  }

Here is the call graph for this function:

Definition at line 47 of file SubmitterBES.cpp.

                                                    {
    SubmitterPluginArgument *subarg =
      dynamic_cast<SubmitterPluginArgument*>(arg);
    if (!subarg)
      return NULL;
    return new SubmitterBES(*subarg);
  }

Here is the call graph for this function:

URL Arc::SubmitterBES::Migrate ( const URL jobid,
const JobDescription jobdesc,
const ExecutionTarget et,
bool  forcemigration 
) const [virtual]

This virtual method should be overridden by plugins which should be capable of migrating jobs.

The active job which should be migrated is pointed to by the URL jobid, and is represented by the JobDescription jobdesc. The forcemigration boolean specifies if the migration should succeed if the active job cannot be terminated. The protected method AddJob can be used to save job information. This method should return the URL of the migrated job. In case migration fails an empty URL should be returned.

Implements Arc::Submitter.

Definition at line 83 of file SubmitterBES.cpp.

                                                        {
    logger.msg(INFO, "Trying to migrate to %s: Migration to a BES cluster is not supported.", et.url.str());
    return URL();
  }

Here is the call graph for this function:

bool Arc::SubmitterBES::ModifyJobDescription ( JobDescription jobdesc,
const ExecutionTarget et 
) const [virtual]

Implements Arc::Submitter.

Definition at line 90 of file SubmitterBES.cpp.

                                                                                                  {
    return true;
  }
bool Arc::Submitter::PutFiles ( const JobDescription jobdesc,
const URL url 
) const [protected, inherited]

Definition at line 35 of file Submitter.cpp.

                                                                          {

    FileCache cache;
    DataMover mover;
    mover.retry(true);
    mover.secure(false);
    mover.passive(true);
    mover.verbose(false);

    for (std::list<FileType>::const_iterator it = job.DataStaging.File.begin();
         it != job.DataStaging.File.end(); it++)
      if (!it->Source.empty()) {
        const URL& src = it->Source.begin()->URI;
        if (src.Protocol() == "file") {
          URL dst(std::string(url.str() + '/' + it->Name));
          DataHandle source(src, usercfg);
          DataHandle destination(dst, usercfg);
          DataStatus res =
            mover.Transfer(*source, *destination, cache, URLMap(), 0, 0, 0,
                           usercfg.Timeout());
          if (!res.Passed()) {
            if (!res.GetDesc().empty())
              logger.msg(ERROR, "Failed uploading file: %s - %s",
                         std::string(res), res.GetDesc());
            else
              logger.msg(ERROR, "Failed uploading file: %s", std::string(res));
            return false;
          }
        }
      }

    return true;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

URL Arc::SubmitterBES::Submit ( const JobDescription jobdesc,
const ExecutionTarget et 
) const [virtual]

This virtual method should be overridden by plugins which should be capable of submitting jobs, defined in the JobDescription jobdesc, to the ExecutionTarget et.

The protected convenience method AddJob can be used to save job information. This method should return the URL of the submitted job. In case submission fails an empty URL should be returned.

Implements Arc::Submitter.

Definition at line 55 of file SubmitterBES.cpp.

                                                             {
    MCCConfig cfg;
    usercfg.ApplyToConfig(cfg);
    AREXClient ac(et.url, cfg, usercfg.Timeout(), false);

    std::string jobid;
    // !! TODO: ordinary JSDL is needed - keeping ARCJSDL so far
    if (!ac.submit(jobdesc.UnParse("ARCJSDL"), jobid, et.url.Protocol() == "https"))
      return URL();

    if (jobid.empty()) {
      logger.msg(INFO, "No job identifier returned by BES service");
      return URL();
    }

    XMLNode jobidx(jobid);

    JobDescription job(jobdesc);

    // Unfortunately Job handling framework somewhy want to have job
    // URL instead of identifier we have to invent one. So we disguise
    // XML blob inside URL path.
    URL jobid_url(disguise_id_into_url(et.url,jobid));
    AddJob(job, jobid_url, et.Cluster, et.url);
    return et.url;
  }

Here is the call graph for this function:


Member Data Documentation

const std::string Arc::Submitter::flavour [protected, inherited]

Definition at line 82 of file Submitter.h.

Logger Arc::SubmitterBES::logger [static, private]

Reimplemented from Arc::Submitter.

Definition at line 20 of file SubmitterBES.h.

const UserConfig& Arc::Submitter::usercfg [protected, inherited]

Definition at line 83 of file Submitter.h.


The documentation for this class was generated from the following files: