Back to index

nordugrid-arc-nox  1.1.0~rc6
Public Member Functions | Static Public Member Functions | Protected Attributes | Private Member Functions | Private Attributes | Static Private Attributes
Arc::DataPointARC Class Reference

#include <DataPointARC.h>

Inheritance diagram for Arc::DataPointARC:
Inheritance graph
[legend]
Collaboration diagram for Arc::DataPointARC:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 DataPointARC (const URL &url, const UserConfig &usercfg)
virtual ~DataPointARC ()
DataStatus Check ()
 Query the DataPoint to check if object is accessible.
DataStatus Remove ()
 Remove/delete object at URL.
DataStatus ListFiles (std::list< FileInfo > &files, bool long_list=false, bool resolve=false, bool metadata=false)
 List file(s).
DataStatus StartReading (DataBuffer &buffer)
 Start reading data from URL.
DataStatus StartWriting (DataBuffer &buffer, DataCallback *space_cb=NULL)
 Start writing data to URL.
DataStatus StopReading ()
 Stop reading.
DataStatus StopWriting ()
 Stop writing.
virtual bool IsIndex () const
 Check if URL is an Indexing Service.
virtual long long int BufSize () const
 Get suggested buffer size for transfers.
virtual int BufNum () const
 Get suggested number of buffers for transfers.
virtual bool Local () const
 Returns true if file is local, e.g. file:// urls.
virtual bool ReadOnly () const
virtual void ReadOutOfOrder (bool v)
 Allow/disallow DataPoint to produce scattered data during reading operation.
virtual bool WriteOutOfOrder ()
 Returns true if URL can accept scattered data for writing operation.
virtual void SetAdditionalChecks (bool v)
 Allow/disallow additional checks.
virtual bool GetAdditionalChecks () const
 Check if additional checks before will be performed.
virtual void SetSecure (bool v)
 Allow/disallow heavy security during data transfer.
virtual bool GetSecure () const
 Check if heavy security during data transfer is allowed.
virtual void Passive (bool v)
 Request passive transfers for FTP-like protocols.
virtual void Range (unsigned long long int start=0, unsigned long long int end=0)
 Set range of bytes to retrieve.
virtual DataStatus Resolve (bool source)
 Resolves index service URL into list of ordinary URLs.
virtual bool Registered () const
 Check if file is registered in Indexing Service.
virtual DataStatus PreRegister (bool replication, bool force=false)
 Index service preregistration.
virtual DataStatus PostRegister (bool replication)
 Index Service postregistration.
virtual DataStatus PreUnregister (bool replication)
 Index Service preunregistration.
virtual DataStatus Unregister (bool all)
 Index Service unregistration.
virtual bool AcceptsMeta ()
 If endpoint can have any use from meta information.
virtual bool ProvidesMeta ()
 If endpoint can provide at least some meta information directly.
virtual const URLCurrentLocation () const
 Returns current (resolved) URL.
virtual const std::string & CurrentLocationMetadata () const
 Returns meta information used to create current URL.
virtual DataStatus CompareLocationMetadata () const
 Compare metadata of DataPoint and current location.
virtual bool NextLocation ()
 Switch to next location in list of URLs.
virtual bool LocationValid () const
 Returns false if out of retries.
virtual bool HaveLocations () const
 Returns true if number of resolved URLs is not 0.
virtual DataStatus AddLocation (const URL &url, const std::string &meta)
 Add URL to list.
virtual DataStatus RemoveLocation ()
 Remove current URL from list.
virtual DataStatus RemoveLocations (const DataPoint &p)
 Remove locations present in another DataPoint object.
virtual const URLGetURL () const
 Returns the URL that was passed to the constructor.
virtual const UserConfigGetUserConfig () const
 Returns the UserConfig that was passed to the constructor.
virtual std::string str () const
 Returns a string representation of the DataPoint.
virtual operator bool () const
 Is DataPoint valid?
virtual bool operator! () const
 Is DataPoint valid?
virtual DataStatus GetFailureReason (void) const
 Returns reason of transfer failure, as reported by callbacks. This could be different from the failure returned by the methods themselves.
virtual bool CheckSize () const
 Check if meta-information 'size' is available.
virtual void SetSize (const unsigned long long int val)
 Set value of meta-information 'size'.
virtual unsigned long long int GetSize () const
 Get value of meta-information 'size'.
virtual bool CheckCheckSum () const
 Check if meta-information 'checksum' is available.
virtual void SetCheckSum (const std::string &val)
 Set value of meta-information 'checksum'.
virtual const std::string & GetCheckSum () const
 Get value of meta-information 'checksum'.
virtual const std::string DefaultCheckSum () const
 Default checksum type.
virtual bool CheckCreated () const
 Check if meta-information 'creation/modification time' is available.
virtual void SetCreated (const Time &val)
 Set value of meta-information 'creation/modification time'.
virtual const TimeGetCreated () const
 Get value of meta-information 'creation/modification time'.
virtual bool CheckValid () const
 Check if meta-information 'validity time' is available.
virtual void SetValid (const Time &val)
 Set value of meta-information 'validity time'.
virtual const TimeGetValid () const
 Get value of meta-information 'validity time'.
virtual bool Cache () const
 Returns true if file is cacheable.
virtual int GetTries () const
 Returns number of retries left.
virtual void SetTries (const int n)
 Set number of retries.
virtual void NextTry (void)
 Decrease number of retries left.
virtual void SetMeta (const DataPoint &p)
 Copy meta information from another object.
virtual bool CompareMeta (const DataPoint &p) const
 Compare meta information from another object.

Static Public Member Functions

static PluginInstance (PluginArgument *arg)

Protected Attributes

DataBufferbuffer
long long int bufsize
int bufnum
bool local
bool readonly
bool linkable
bool is_secure
bool force_secure
bool force_passive
bool additional_checks
bool allow_out_of_order
unsigned long long int range_start
unsigned long long int range_end
const URLurl
const UserConfigusercfg
unsigned long long int size
std::string checksum
Time created
Time valid
int triesleft
DataStatus failure_code
bool cache
std::list< std::string > valid_url_options
 Subclasses should add their own specific options to this list.

Private Member Functions

bool checkBartenderURL (const URL &bartender_url)

Private Attributes

DataHandletransfer
bool reading
bool writing
URL bartender_url
URL turl
MD5Summd5sum
int chksum_index

Static Private Attributes

static Logger logger

Detailed Description

Definition at line 16 of file DataPointARC.h.


Constructor & Destructor Documentation

Arc::DataPointARC::DataPointARC ( const URL url,
const UserConfig usercfg 
)

Definition at line 80 of file DataPointARC.cpp.

    : DataPointDirect(url, usercfg),
      transfer(NULL),
      md5sum(NULL),
      reading(false),
      writing(false),
      bartender_url(url.HTTPOption("BartenderURL")) {
    if (!bartender_url) {
      if (!usercfg.Bartender().empty()){
        std::vector<int> shuffledKeys;
        for (int i = 0; i < usercfg.Bartender().size(); i++)
          shuffledKeys.push_back(i);
        std::random_shuffle(shuffledKeys.begin(), shuffledKeys.end());

        // pick random bartender url:
        for (int i = 0; i < shuffledKeys.size(); i++) {
          if (checkBartenderURL(usercfg.Bartender()[shuffledKeys[i]])) {
            bartender_url = usercfg.Bartender()[shuffledKeys[i]];
            break;
          }
        }
      }
      if (!bartender_url)
        bartender_url = URL("http://localhost:60000/Bartender");
    }

    md5sum = new MD5Sum();
  }

Here is the call graph for this function:

Here is the caller graph for this function:

Definition at line 109 of file DataPointARC.cpp.

                              {
    StopReading();
    StopWriting();
    if (md5sum) {
      delete md5sum;
      md5sum = NULL;
    }
    if (transfer){
      delete transfer;
      transfer = NULL;
    }
  }

Here is the call graph for this function:


Member Function Documentation

bool Arc::DataPointDirect::AcceptsMeta ( ) [virtual, inherited]

If endpoint can have any use from meta information.

Implements Arc::DataPoint.

Definition at line 122 of file DataPointDirect.cpp.

                                    {
    return false;
  }
DataStatus Arc::DataPointDirect::AddLocation ( const URL url,
const std::string &  meta 
) [virtual, inherited]

Add URL to list.

Parameters:
urlLocation URL to add.
metaLocation meta information.

Implements Arc::DataPoint.

Definition at line 161 of file DataPointDirect.cpp.

int Arc::DataPointDirect::BufNum ( ) const [virtual, inherited]

Get suggested number of buffers for transfers.

Implements Arc::DataPoint.

Definition at line 56 of file DataPointDirect.cpp.

                                    {
    return bufnum;
  }
long long int Arc::DataPointDirect::BufSize ( ) const [virtual, inherited]

Get suggested buffer size for transfers.

Implements Arc::DataPoint.

Definition at line 52 of file DataPointDirect.cpp.

                                               {
    return bufsize;
  }
bool Arc::DataPoint::Cache ( ) const [virtual, inherited]

Returns true if file is cacheable.

Definition at line 86 of file DataPoint.cpp.

                              {
    return cache;
  }

Here is the caller graph for this function:

Query the DataPoint to check if object is accessible.

If possible this method will also try to provide meta information about the object.

Implements Arc::DataPoint.

Definition at line 482 of file DataPointARC.cpp.

                                 {
    if (!url.Host().empty()){
      logger.msg(ERROR, "Hostname is not implemented for arc protocol");
      return DataStatus::CheckError;
    }
    return DataStatus::Success;
  }

Here is the call graph for this function:

bool Arc::DataPointARC::checkBartenderURL ( const URL bartender_url) [private]

Definition at line 40 of file DataPointARC.cpp.

                                                              {
    MCCConfig cfg;
    usercfg.ApplyToConfig(cfg);

    ClientSOAP client(cfg, bartender_url, usercfg.Timeout());
    std::string xml;

    NS ns("bar", "http://www.nordugrid.org/schemas/bartender");
    PayloadSOAP request(ns);
    request.NewChild("bar:list").NewChild("bar:listRequestList").NewChild("bar:listRequestElement").NewChild("bar:requestID") = "0";
    request["bar:list"]["bar:listRequestList"]["bar:listRequestElement"].NewChild("bar:LN") = bartender_url.Path();
    request["bar:list"].NewChild("bar:neededMetadataList").NewChild("bar:neededMetadataElement").NewChild("bar:section") = "entry";
    request["bar:list"]["bar:neededMetadatList"]["bar:neededMetadataElement"].NewChild("bar:property") = "";
    request.GetXML(xml, true);

    PayloadSOAP *response = NULL;

    MCC_Status status;
    try{
      status = client.process(&request, &response);
    }
    catch(std::exception e){
      logger.msg(WARNING, bartender_url.Path(), "did not work");
    }

    bool ret = true;
    if(!response)
      ret = false;
    else{
      response->Child().GetXML(xml, true);
      logger.msg(VERBOSE, "checingBartenderURL: Response:\n%s", xml);
      if(xml.find("Failed to send SOAP message") != std::string::npos)
        ret = false;
    }
    if(!status)
      ret = false;
    delete response;
    return ret;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

bool Arc::DataPoint::CheckCheckSum ( ) const [virtual, inherited]

Check if meta-information 'checksum' is available.

Definition at line 102 of file DataPoint.cpp.

                                      {
    return (!checksum.empty());
  }

Here is the caller graph for this function:

bool Arc::DataPoint::CheckCreated ( ) const [virtual, inherited]

Check if meta-information 'creation/modification time' is available.

Definition at line 118 of file DataPoint.cpp.

                                     {
    return (created != -1);
  }

Here is the caller graph for this function:

bool Arc::DataPoint::CheckSize ( ) const [virtual, inherited]

Check if meta-information 'size' is available.

Definition at line 90 of file DataPoint.cpp.

                                  {
    return (size != (unsigned long long int)(-1));
  }

Here is the caller graph for this function:

bool Arc::DataPoint::CheckValid ( ) const [virtual, inherited]

Check if meta-information 'validity time' is available.

Definition at line 130 of file DataPoint.cpp.

                                   {
    return (valid != -1);
  }

Here is the caller graph for this function:

Compare metadata of DataPoint and current location.

Returns inconsistency error or error encountered during operation, or success

Implements Arc::DataPoint.

Definition at line 143 of file DataPointDirect.cpp.

bool Arc::DataPoint::CompareMeta ( const DataPoint p) const [virtual, inherited]

Compare meta information from another object.

Undefined values are not used for comparison.

Parameters:
pobject to which to compare.

Definition at line 165 of file DataPoint.cpp.

                                                      {
    if (CheckSize() && p.CheckSize())
      if (GetSize() != p.GetSize())
        return false;
    if (CheckCheckSum() && p.CheckCheckSum())
      // TODO: compare checksums properly
      if (strcasecmp(GetCheckSum().c_str(), p.GetCheckSum().c_str()))
        return false;
    if (CheckValid() && p.CheckValid())
      if (GetValid() != p.GetValid())
        return false;
    return true;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

const URL & Arc::DataPointDirect::CurrentLocation ( ) const [virtual, inherited]

Returns current (resolved) URL.

Implements Arc::DataPoint.

Definition at line 134 of file DataPointDirect.cpp.

                                                    {
    return url;
  }

Here is the caller graph for this function:

const std::string & Arc::DataPointDirect::CurrentLocationMetadata ( ) const [virtual, inherited]

Returns meta information used to create current URL.

Usage differs between different indexing services.

Implements Arc::DataPoint.

Definition at line 138 of file DataPointDirect.cpp.

                                                                  {
    static const std::string empty;
    return empty;
  }
const std::string Arc::DataPoint::DefaultCheckSum ( ) const [virtual, inherited]

Default checksum type.

Reimplemented in Arc::DataPointSRM, and Arc::DataPointLFC.

Definition at line 114 of file DataPoint.cpp.

                                                   {
    return std::string("cksum");
  }

Here is the call graph for this function:

Here is the caller graph for this function:

bool Arc::DataPointDirect::GetAdditionalChecks ( ) const [virtual, inherited]

Check if additional checks before will be performed.

Implements Arc::DataPoint.

Definition at line 80 of file DataPointDirect.cpp.

                                                  {
    return additional_checks;
  }
const std::string & Arc::DataPoint::GetCheckSum ( ) const [virtual, inherited]

Get value of meta-information 'checksum'.

Definition at line 110 of file DataPoint.cpp.

                                                {
    return checksum;
  }

Here is the caller graph for this function:

const Time & Arc::DataPoint::GetCreated ( ) const [virtual, inherited]

Get value of meta-information 'creation/modification time'.

Definition at line 126 of file DataPoint.cpp.

                                          {
    return created;
  }

Here is the caller graph for this function:

DataStatus Arc::DataPoint::GetFailureReason ( void  ) const [virtual, inherited]

Returns reason of transfer failure, as reported by callbacks. This could be different from the failure returned by the methods themselves.

Definition at line 82 of file DataPoint.cpp.

                                               {
    return failure_code;
  }

Here is the caller graph for this function:

bool Arc::DataPointDirect::GetSecure ( ) const [virtual, inherited]

Check if heavy security during data transfer is allowed.

Implements Arc::DataPoint.

Definition at line 88 of file DataPointDirect.cpp.

                                        {
    return is_secure;
  }
unsigned long long int Arc::DataPoint::GetSize ( ) const [virtual, inherited]

Get value of meta-information 'size'.

Definition at line 98 of file DataPoint.cpp.

                                                  {
    return size;
  }

Here is the caller graph for this function:

int Arc::DataPoint::GetTries ( ) const [virtual, inherited]

Returns number of retries left.

Definition at line 142 of file DataPoint.cpp.

                                {
    return triesleft;
  }

Here is the caller graph for this function:

const URL & Arc::DataPoint::GetURL ( ) const [virtual, inherited]

Returns the URL that was passed to the constructor.

Definition at line 45 of file DataPoint.cpp.

                                     {
    return url;
  }

Here is the caller graph for this function:

const UserConfig & Arc::DataPoint::GetUserConfig ( ) const [virtual, inherited]

Returns the UserConfig that was passed to the constructor.

Definition at line 49 of file DataPoint.cpp.

                                                   {
    return usercfg;
  }

Here is the caller graph for this function:

const Time & Arc::DataPoint::GetValid ( ) const [virtual, inherited]

Get value of meta-information 'validity time'.

Definition at line 138 of file DataPoint.cpp.

                                        {
    return valid;
  }

Here is the caller graph for this function:

bool Arc::DataPointDirect::HaveLocations ( ) const [virtual, inherited]

Returns true if number of resolved URLs is not 0.

Implements Arc::DataPoint.

Definition at line 157 of file DataPointDirect.cpp.

                                            {
    return true;
  }

Definition at line 122 of file DataPointARC.cpp.

                                                    {
    DataPointPluginArgument *dmcarg = dynamic_cast<DataPointPluginArgument*>(arg);
    if (!dmcarg)
      return NULL;
    if (((const URL&)(*dmcarg)).Protocol() != "arc")
      return NULL;
    return new DataPointARC(*dmcarg, *dmcarg);
  }

Here is the call graph for this function:

bool Arc::DataPointDirect::IsIndex ( ) const [virtual, inherited]

Check if URL is an Indexing Service.

Implements Arc::DataPoint.

Definition at line 48 of file DataPointDirect.cpp.

                                      {
    return false;
  }
DataStatus Arc::DataPointARC::ListFiles ( std::list< FileInfo > &  files,
bool  long_list = false,
bool  resolve = false,
bool  metadata = false 
) [virtual]

List file(s).

If the DataPoint represents a directory its contents will be listed.

Parameters:
fileswill contain list of file names and optionally their attributes.
long)listif true, list additional properties of each file.
resolveif true, resolve physical locations (relevant for indexing services only).

Implements Arc::DataPoint.

Definition at line 131 of file DataPointARC.cpp.

                                                                               {
    if (!url.Host().empty()){
      logger.msg(ERROR, "Hostname is not implemented for arc protocol");
      return DataStatus::UnimplementedError;
    }
    MCCConfig cfg;
    usercfg.ApplyToConfig(cfg);

    ClientSOAP client(cfg, bartender_url, usercfg.Timeout());
    std::string xml;

    NS ns("bar", "http://www.nordugrid.org/schemas/bartender");
    PayloadSOAP request(ns);
    request.NewChild("bar:list").NewChild("bar:listRequestList").NewChild("bar:listRequestElement").NewChild("bar:requestID") = "0";
    request["bar:list"]["bar:listRequestList"]["bar:listRequestElement"].NewChild("bar:LN") = url.Path();
    request["bar:list"].NewChild("bar:neededMetadataList").NewChild("bar:neededMetadataElement").NewChild("bar:section") = "entry";
    request["bar:list"]["bar:neededMetadatList"]["bar:neededMetadataElement"].NewChild("bar:property") = "";
    request.GetXML(xml, true);
    logger.msg(INFO, "Request:\n%s", xml);

    PayloadSOAP *response = NULL;

    MCC_Status status = client.process(&request, &response);

    if (!status) {
      logger.msg(ERROR, (std::string)status);
      if (response)
        delete response;
      return DataStatus::ListError;
    }

    if (!response) {
      logger.msg(ERROR, "No SOAP response");
      return DataStatus::ListError;
    }

    response->Child().GetXML(xml, true);
    logger.msg(INFO, "Response:\n%s", xml);

    XMLNode nd = (*response).Child()["listResponseList"]["listResponseElement"];
    nd.GetXML(xml, true);

    logger.msg(INFO, "nd:\n%s", xml);

    if (nd["status"] == "not found")
      return DataStatus::ListError;

    if (nd["status"] == "found")
      for (int i = 0;; i++) {
        XMLNode cnd = nd["entries"]["entry"][i];
        if (!cnd)
          break;
        std::string file_name = cnd["name"];
        std::string type;
        for (int j = 0;; j++) {
          XMLNode ccnd = cnd["metadataList"]["metadata"][j];
          if (!ccnd)
            break;
          if (ccnd["property"] == "type")
            type = (std::string)ccnd["value"];
        }
        logger.msg(INFO, "cnd:\n%s is a %s", file_name, type);
        std::list<FileInfo>::iterator f = files.insert(files.end(), FileInfo(file_name.c_str()));
        if (type == "collection")
          f->SetType(FileInfo::file_type_dir);
        else
          f->SetType(FileInfo::file_type_file);
      }
    else {
      // its a file or something
      // we know it exists so we use file name from url
      std::string path = url.Path();
      std::string::size_type i = path.rfind(G_DIR_SEPARATOR, path.length());
      std::string file_name = path.substr((i != std::string::npos)?(i+1):0);
      std::list<FileInfo>::iterator f = files.insert(files.end(), FileInfo(file_name.c_str()));
      f->SetType(FileInfo::file_type_file);
    }

    std::string answer = (std::string)((*response).Child().Name());

    delete response;

    logger.msg(INFO, answer);

    return DataStatus::Success;
  }

Here is the call graph for this function:

bool Arc::DataPointDirect::Local ( ) const [virtual, inherited]

Returns true if file is local, e.g. file:// urls.

Implements Arc::DataPoint.

Definition at line 60 of file DataPointDirect.cpp.

                                    {
    return local;
  }
bool Arc::DataPointDirect::LocationValid ( ) const [virtual, inherited]

Returns false if out of retries.

Implements Arc::DataPoint.

Definition at line 153 of file DataPointDirect.cpp.

                                            {
    return (triesleft > 0);
  }
bool Arc::DataPointDirect::NextLocation ( ) [virtual, inherited]

Switch to next location in list of URLs.

At last location switch to first if number of allowed retries is not exceeded. Returns false if no retries left.

Implements Arc::DataPoint.

Definition at line 147 of file DataPointDirect.cpp.

                                     {
    if (triesleft > 0)
      --triesleft;
    return (triesleft > 0);
  }
void Arc::DataPoint::NextTry ( void  ) [virtual, inherited]

Decrease number of retries left.

Definition at line 150 of file DataPoint.cpp.

                          {
    if(triesleft) --triesleft;
  }

Here is the caller graph for this function:

Arc::DataPoint::operator bool ( void  ) const [virtual, inherited]

Is DataPoint valid?

Definition at line 57 of file DataPoint.cpp.

                                 {

    if (!url)
      return false;
      
    // URL option validation. Subclasses which do not want to validate
    // URL options should override this method.
    std::map<std::string, std::string> options = url.Options();
    for (std::map<std::string, std::string>::iterator i = options.begin(); i != options.end(); i++) {
      bool valid = false; 
      for (std::list<std::string>::const_iterator j = valid_url_options.begin(); j != valid_url_options.end(); j++) {
        if (i->first == *j) valid = true;
      }
      if (!valid) {
        logger.msg(ERROR, "Invalid URL option: %s", i->first);
        return false;
      }
    }
    return true;
  }

Here is the call graph for this function:

bool Arc::DataPoint::operator! ( void  ) const [virtual, inherited]

Is DataPoint valid?

Definition at line 78 of file DataPoint.cpp.

                                  {
    return !((bool)*this);
  }
void Arc::DataPointDirect::Passive ( bool  v) [virtual, inherited]

Request passive transfers for FTP-like protocols.

Parameters:
trueto request.

Implements Arc::DataPoint.

Definition at line 92 of file DataPointDirect.cpp.

                                        {
    force_passive = val;
  }
DataStatus Arc::DataPointDirect::PostRegister ( bool  replication) [virtual, inherited]

Index Service postregistration.

Used for same purpose as PreRegister. Should be called after actual transfer of file successfully finished.

Parameters:
replicationif true, the file is being replicated between two locations registered in Indexing Service under same name.

Implements Arc::DataPoint.

Definition at line 110 of file DataPointDirect.cpp.

DataStatus Arc::DataPointDirect::PreRegister ( bool  replication,
bool  force = false 
) [virtual, inherited]

Index service preregistration.

This function registers the physical location of a file into an indexing service. It should be called before the actual transfer to that location happens.

Parameters:
replicationif true, the file is being replicated between two locations registered in the indexing service under same name.
forceif true, perform registration of a new file even if it already exists. Should be used to fix failures in Indexing Service.

Implements Arc::DataPoint.

Definition at line 106 of file DataPointDirect.cpp.

DataStatus Arc::DataPointDirect::PreUnregister ( bool  replication) [virtual, inherited]

Index Service preunregistration.

Should be called if file transfer failed. It removes changes made by PreRegister.

Parameters:
replicationif true, the file is being replicated between two locations registered in Indexing Service under same name.

Implements Arc::DataPoint.

Definition at line 114 of file DataPointDirect.cpp.

bool Arc::DataPointDirect::ProvidesMeta ( ) [virtual, inherited]

If endpoint can provide at least some meta information directly.

Implements Arc::DataPoint.

Reimplemented in Arc::DataPointGridFTP.

Definition at line 126 of file DataPointDirect.cpp.

                                     {
    return false;
  }
void Arc::DataPointDirect::Range ( unsigned long long int  start = 0,
unsigned long long int  end = 0 
) [virtual, inherited]

Set range of bytes to retrieve.

Default values correspond to whole file.

Implements Arc::DataPoint.

Definition at line 96 of file DataPointDirect.cpp.

                                                          {
    range_start = start;
    range_end = end;
  }
bool Arc::DataPointDirect::ReadOnly ( ) const [virtual, inherited]

Implements Arc::DataPoint.

Definition at line 64 of file DataPointDirect.cpp.

                                       {
    return readonly;
  }
void Arc::DataPointDirect::ReadOutOfOrder ( bool  v) [virtual, inherited]

Allow/disallow DataPoint to produce scattered data during reading operation.

Parameters:
vtrue if allowed (default is false).

Implements Arc::DataPoint.

Definition at line 68 of file DataPointDirect.cpp.

                                               {
    allow_out_of_order = val;
  }
bool Arc::DataPointDirect::Registered ( ) const [virtual, inherited]

Check if file is registered in Indexing Service.

Proper value is obtainable only after Resolve.

Implements Arc::DataPoint.

Definition at line 130 of file DataPointDirect.cpp.

                                         {
    return false;
  }

Remove/delete object at URL.

Implements Arc::DataPoint.

Definition at line 490 of file DataPointARC.cpp.

                                  {
    std::string host = url.Host();
    if (!url.Host().empty()){
      logger.msg(ERROR, "Hostname is not implemented for arc protocol");
      return DataStatus::UnimplementedError;
    }
    MCCConfig cfg;
    usercfg.ApplyToConfig(cfg);

    ClientSOAP client(cfg, bartender_url, usercfg.Timeout());
    std::string xml;

    NS ns("bar", "http://www.nordugrid.org/schemas/bartender");
    PayloadSOAP request(ns);
    request.NewChild("bar:delFile").NewChild("bar:delFileRequestList").NewChild("bar:delFileRequestElement").NewChild("bar:requestID") = "0";
    request["bar:delFile"]["bar:delFileRequestList"]["bar:delFileRequestElement"].NewChild("bar:LN") = url.Path();

    request.GetXML(xml, true);
    logger.msg(INFO, "Request:\n%s", xml);

    PayloadSOAP *response = NULL;

    MCC_Status status = client.process(&request, &response);

    if (!status) {
      logger.msg(ERROR, (std::string)status);
      if (response)
        delete response;
      return DataStatus::DeleteError;
    }

    if (!response) {
      logger.msg(ERROR, "No SOAP response");
      return DataStatus::DeleteError;
    }

    response->Child().GetXML(xml, true);
    logger.msg(INFO, "Response:\n%s", xml);

    XMLNode nd = (*response).Child()["delFileResponseList"]["delFileResponseElement"];

    if (nd["success"] == "deleted")
      logger.msg(INFO, "Deleted %s", url.Path());
    delete response;
    return DataStatus::Success;
  }

Here is the call graph for this function:

Remove current URL from list.

Implements Arc::DataPoint.

Definition at line 165 of file DataPointDirect.cpp.

DataStatus Arc::DataPointDirect::RemoveLocations ( const DataPoint p) [virtual, inherited]

Remove locations present in another DataPoint object.

Implements Arc::DataPoint.

Definition at line 169 of file DataPointDirect.cpp.

DataStatus Arc::DataPointDirect::Resolve ( bool  source) [virtual, inherited]

Resolves index service URL into list of ordinary URLs.

Also obtains meta information about the file.

Parameters:
sourcetrue if DataPoint object represents source of information.

Implements Arc::DataPoint.

Definition at line 102 of file DataPointDirect.cpp.

void Arc::DataPointDirect::SetAdditionalChecks ( bool  v) [virtual, inherited]

Allow/disallow additional checks.

Check for existance of remote file (and probably other checks too) before initiating reading and writing operations.

Parameters:
vtrue if allowed (default is true).

Implements Arc::DataPoint.

Definition at line 76 of file DataPointDirect.cpp.

                                                    {
    additional_checks = val;
  }
void Arc::DataPoint::SetCheckSum ( const std::string &  val) [virtual, inherited]

Set value of meta-information 'checksum'.

Definition at line 106 of file DataPoint.cpp.

                                                  {
    checksum = val;
  }

Here is the caller graph for this function:

void Arc::DataPoint::SetCreated ( const Time val) [virtual, inherited]

Set value of meta-information 'creation/modification time'.

Definition at line 122 of file DataPoint.cpp.

                                            {
    created = val;
  }

Here is the caller graph for this function:

void Arc::DataPoint::SetMeta ( const DataPoint p) [virtual, inherited]

Copy meta information from another object.

Already defined values are not overwritten.

Parameters:
pobject from which information is taken.

Definition at line 154 of file DataPoint.cpp.

                                            {
    if (!CheckSize())
      SetSize(p.GetSize());
    if (!CheckCheckSum())
      SetCheckSum(p.GetCheckSum());
    if (!CheckCreated())
      SetCreated(p.GetCreated());
    if (!CheckValid())
      SetValid(p.GetValid());
  }

Here is the call graph for this function:

Here is the caller graph for this function:

void Arc::DataPointDirect::SetSecure ( bool  v) [virtual, inherited]

Allow/disallow heavy security during data transfer.

Parameters:
vtrue if allowed (default depends on protocol).

Implements Arc::DataPoint.

Definition at line 84 of file DataPointDirect.cpp.

                                          {
    force_secure = val;
  }
void Arc::DataPoint::SetSize ( const unsigned long long int  val) [virtual, inherited]

Set value of meta-information 'size'.

Definition at line 94 of file DataPoint.cpp.

                                                          {
    size = val;
  }

Here is the caller graph for this function:

void Arc::DataPoint::SetTries ( const int  n) [virtual, inherited]

Set number of retries.

Reimplemented in Arc::DataPointIndex.

Definition at line 146 of file DataPoint.cpp.

                                      {
    triesleft = std::max(0, n);
  }

Here is the caller graph for this function:

void Arc::DataPoint::SetValid ( const Time val) [virtual, inherited]

Set value of meta-information 'validity time'.

Definition at line 134 of file DataPoint.cpp.

                                          {
    valid = val;
  }

Here is the caller graph for this function:

Start reading data from URL.

Separate thread to transfer data will be created. No other operation can be performed while reading is in progress.

Parameters:
bufferoperation will use this buffer to put information into. Should not be destroyed before stop_reading was called and returned.

Implements Arc::DataPoint.

Definition at line 218 of file DataPointARC.cpp.

                                                       {
    if (!url.Host().empty()){
      logger.msg(ERROR, "Hostname is not implemented for arc protocol");
      return DataStatus::UnimplementedError;
    }

    logger.msg(VERBOSE, "StartReading");
    if (reading)
      return DataStatus::IsReadingError;
    if (writing)
      return DataStatus::IsWritingError;

    reading = true;
    buffer = &buf;
    MCCConfig cfg;
    usercfg.ApplyToConfig(cfg);

    // get TURL from bartender
    ClientSOAP client(cfg, bartender_url, usercfg.Timeout());
    std::string xml;

    NS ns("bar", "http://www.nordugrid.org/schemas/bartender");
    PayloadSOAP request(ns);
    request.NewChild("bar:getFile").NewChild("bar:getFileRequestList").NewChild("bar:getFileRequestElement").NewChild("bar:requestID") = "0";
    request["bar:getFile"]["bar:getFileRequestList"]["bar:getFileRequestElement"].NewChild("bar:LN") = url.Path();
    // only supports http protocol:
    request["bar:getFile"]["bar:getFileRequestList"]["bar:getFileRequestElement"].NewChild("bar:protocol") = "http";
    request.GetXML(xml, true);
    logger.msg(INFO, "Request:\n%s", xml);

    PayloadSOAP *response = NULL;

    MCC_Status status = client.process(&request, &response);

    if (!status) {
      logger.msg(ERROR, (std::string)status);
      if (response)
        delete response;
      return DataStatus::ReadError;
    }

    if (!response) {
      logger.msg(ERROR, "No SOAP response");
      return DataStatus::ReadError;
    }

    response->Child().GetXML(xml, true);
    logger.msg(INFO, "Response:\n%s", xml);

    XMLNode nd = (*response).Child()["getFileResponseList"]["getFileResponseElement"];
    nd.GetXML(xml, true);

    logger.msg(INFO, "nd:\n%s", xml);

    if (nd["success"] != "done" || !nd["TURL"]) {
      delete response;
      return DataStatus::ReadError;
    }

    logger.msg(INFO, "Recieved transfer URL: %s", (std::string)nd["TURL"]);

    turl = (std::string) nd["TURL"];
    delete response;
    // redirect actual reading to http dmc
    if (transfer){ 
      delete transfer;
      transfer = NULL;
    }
    transfer = new DataHandle(turl, usercfg);
    if (!(*transfer)->StartReading(buf)) {
      if (transfer) {
        delete transfer;
        transfer = NULL;
      }
      reading = false;
      return DataStatus::ReadError;
    }

    return DataStatus::Success;
  }

Here is the call graph for this function:

DataStatus Arc::DataPointARC::StartWriting ( DataBuffer buffer,
DataCallback space_cb = NULL 
) [virtual]

Start writing data to URL.

Separate thread to transfer data will be created. No other operation can be performed while writing is in progress.

Parameters:
bufferoperation will use this buffer to get information from. Should not be destroyed before stop_writing was called and returned.
space_cbcallback which is called if there is not enough space to store data. May not implemented for all protocols.

Implements Arc::DataPoint.

Definition at line 311 of file DataPointARC.cpp.

                                                                {
    if (!url.Host().empty()){
      logger.msg(ERROR, "Hostname is not implemented for arc protocol");
      return DataStatus::UnimplementedError;
    }
    logger.msg(VERBOSE, "StartWriting");
    if (reading)
      return DataStatus::IsReadingError;
    if (writing)
      return DataStatus::IsWritingError;

    writing = true;
    buffer = &buf;
    chksum_index = buffer->add(md5sum);
    MCCConfig cfg;
    usercfg.ApplyToConfig(cfg);

    // get TURL from bartender
    ClientSOAP client(cfg, bartender_url, usercfg.Timeout());
    std::string xml;
    std::stringstream out;
    out << this->GetSize();
    std::string size_str = out.str();
    NS ns("bar", "http://www.nordugrid.org/schemas/bartender");
    PayloadSOAP request(ns);
    request.NewChild("bar:putFile").NewChild("bar:putFileRequestList").NewChild("bar:putFileRequestElement").NewChild("bar:requestID") = "0";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"].NewChild("bar:LN") = url.Path();
    // only supports http protocol:
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"].NewChild("bar:protocol") = "http";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"].NewChild("bar:metadataList").NewChild("bar:metadata").NewChild("bar:section") = "states";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"]["bar:metadata"].NewChild("bar:property") = "checksum";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"]["bar:metadata"].NewChild("bar:value") = "";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"].NewChild("bar:metadata").NewChild("bar:section") = "states";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"]["bar:metadata"][1].NewChild("bar:property") = "checksumType";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"]["bar:metadata"][1].NewChild("bar:value") = "md5";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"].NewChild("bar:metadata").NewChild("bar:section") = "states";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"]["bar:metadata"][2].NewChild("bar:property") = "neededReplicas";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"]["bar:metadata"][2].NewChild("bar:value") = "3";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"].NewChild("bar:metadata").NewChild("bar:section") = "states";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"]["bar:metadata"][3].NewChild("bar:property") = "size";
    request["bar:putFile"]["bar:putFileRequestList"]["bar:putFileRequestElement"]["bar:metadataList"]["bar:metadata"][3].NewChild("bar:value") = size_str;
    request.GetXML(xml, true);
    logger.msg(INFO, "Request:\n%s", xml);

    PayloadSOAP *response = NULL;

    MCC_Status status = client.process(&request, &response);

    if (!status) {
      logger.msg(ERROR, (std::string)status);
      if (response)
        delete response;
      return DataStatus::WriteError;
    }

    if (!response) {
      logger.msg(ERROR, "No SOAP response");
      return DataStatus::WriteError;
    }

    response->Child().GetXML(xml, true);
    logger.msg(INFO, "Response:\n%s", xml);

    XMLNode nd = (*response).Child()["putFileResponseList"]["putFileResponseElement"];
    nd.GetXML(xml, true);

    logger.msg(INFO, "nd:\n%s", xml);

    if (nd["success"] != "done" || !nd["TURL"]) {
      delete response;
      return DataStatus::WriteError;
    }

    logger.msg(INFO, "Recieved transfer URL: %s", (std::string)nd["TURL"]);

    turl = (std::string) nd["TURL"];
    // redirect actual writing to http dmc
    if (transfer){
      delete transfer;
      transfer = NULL;
    }
    transfer = new DataHandle(turl, usercfg);
    delete response;
    if (!(*transfer)->StartWriting(buf, callback)) {
      if (transfer) {
        delete transfer;
        transfer = NULL;
      }
      writing = false;
      return DataStatus::WriteError;
    }
    return DataStatus::Success;

  }

Here is the call graph for this function:

Stop reading.

Must be called after corresponding start_reading method, either after all data is transferred or to cancel transfer. Use buffer object to find out when data is transferred. Must return failure if any happened during transfer.

Implements Arc::DataPoint.

Definition at line 299 of file DataPointARC.cpp.

                                       {
    if (!reading)
      return DataStatus::ReadStopError;
    reading = false;
    if (!transfer)
      return DataStatus::Success;
    DataStatus ret = (*transfer)->StopReading();
    delete transfer;
    transfer = NULL;
    return ret;
  }

Here is the caller graph for this function:

Stop writing.

Must be called after corresponding start_writing method, either after all data is transferred or to cancel transfer. Use buffer object to find out when data is transferred. Must return failure if any happened during transfer.

Implements Arc::DataPoint.

Definition at line 407 of file DataPointARC.cpp.

                                       {
    if (!writing)
      return DataStatus::WriteStopError;
    writing = false;
    if (!transfer)
      return DataStatus::Success;
    // update checksum and size
    DataStatus ret = (*transfer)->StopWriting();
    buffer->wait_read();
    // md5res_u memory is handled by md5sum
    unsigned char *md5res_u;
    unsigned int length;
    md5sum->result(md5res_u, length);
    std::string md5str = "";
    for (int i = 0; i < length; i++) {
      char tmpChar[3];
      snprintf(tmpChar, sizeof(tmpChar), "%.2x", md5res_u[i]);
      md5str += tmpChar;
    }
    logger.msg(VERBOSE, "Calculated checksum: %s", md5str);

    MCCConfig cfg;
    usercfg.ApplyToConfig(cfg);

    // get TURL from bartender
    ClientSOAP client(cfg, bartender_url, usercfg.Timeout());
    std::string xml;
    std::stringstream out;
    out << this->GetSize();
    std::string size_str = out.str();
    NS ns("bar", "http://www.nordugrid.org/schemas/bartender");
    PayloadSOAP request(ns);
    request.NewChild("bar:modify").NewChild("bar:modifyRequestList").NewChild("bar:modifyRequestElement").NewChild("bar:changeID") = "0";
    request["bar:modify"]["bar:modifyRequestList"]["bar:modifyRequestElement"].NewChild("bar:LN") = url.Path();
    request["bar:modify"]["bar:modifyRequestList"]["bar:modifyRequestElement"].NewChild("bar:changeType") = "set";
    request["bar:modify"]["bar:modifyRequestList"]["bar:modifyRequestElement"].NewChild("bar:section") = "states";
    request["bar:modify"]["bar:modifyRequestList"]["bar:modifyRequestElement"].NewChild("bar:property") = "checksum";
    request["bar:modify"]["bar:modifyRequestList"]["bar:modifyRequestElement"].NewChild("bar:value") = md5str;
    request.GetXML(xml, true);
    logger.msg(INFO, "Request:\n%s", xml);

    PayloadSOAP *response = NULL;

    MCC_Status status = client.process(&request, &response);

    if (!status) {
      logger.msg(ERROR, (std::string)status);
      if (response)
        delete response;
      return DataStatus::WriteError;
    }

    if (!response) {
      logger.msg(ERROR, "No SOAP response");
      return DataStatus::WriteError;
    }

    response->Child().GetXML(xml, true);
    logger.msg(INFO, "Response:\n%s", xml);

    XMLNode nd = (*response).Child()["modifyResponseList"]["modifyResponseElement"];
    nd.GetXML(xml, true);

    logger.msg(INFO, "nd:\n%s", xml);

    if (nd["success"] != "set")
      return DataStatus::WriteError;

    delete md5sum;
    md5sum = NULL;
    delete transfer;
    transfer = NULL;
    return ret;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

std::string Arc::DataPoint::str ( ) const [virtual, inherited]

Returns a string representation of the DataPoint.

Definition at line 53 of file DataPoint.cpp.

                                 {
    return url.str();
  }

Here is the call graph for this function:

Here is the caller graph for this function:

DataStatus Arc::DataPointDirect::Unregister ( bool  all) [virtual, inherited]

Index Service unregistration.

Remove information about file registered in Indexing Service.

Parameters:
allif true, information about file itself is (LFN) is removed. Otherwise only particular physical instance is unregistered.

Implements Arc::DataPoint.

Definition at line 118 of file DataPointDirect.cpp.

bool Arc::DataPointDirect::WriteOutOfOrder ( ) [virtual, inherited]

Returns true if URL can accept scattered data for writing operation.

Implements Arc::DataPoint.

Reimplemented in Arc::DataPointGridFTP, and Arc::DataPointFile.

Definition at line 72 of file DataPointDirect.cpp.

                                        {
    return false;
  }

Member Data Documentation

bool Arc::DataPointDirect::additional_checks [protected, inherited]

Definition at line 84 of file DataPointDirect.h.

bool Arc::DataPointDirect::allow_out_of_order [protected, inherited]

Definition at line 85 of file DataPointDirect.h.

Definition at line 23 of file DataPointARC.h.

DataBuffer* Arc::DataPointDirect::buffer [protected, inherited]

Definition at line 75 of file DataPointDirect.h.

int Arc::DataPointDirect::bufnum [protected, inherited]

Definition at line 77 of file DataPointDirect.h.

long long int Arc::DataPointDirect::bufsize [protected, inherited]

Definition at line 76 of file DataPointDirect.h.

bool Arc::DataPoint::cache [protected, inherited]

Definition at line 325 of file DataPoint.h.

std::string Arc::DataPoint::checksum [protected, inherited]

Definition at line 320 of file DataPoint.h.

Definition at line 26 of file DataPointARC.h.

Time Arc::DataPoint::created [protected, inherited]

Definition at line 321 of file DataPoint.h.

DataStatus Arc::DataPoint::failure_code [protected, inherited]

Definition at line 324 of file DataPoint.h.

bool Arc::DataPointDirect::force_passive [protected, inherited]

Definition at line 83 of file DataPointDirect.h.

bool Arc::DataPointDirect::force_secure [protected, inherited]

Definition at line 82 of file DataPointDirect.h.

bool Arc::DataPointDirect::is_secure [protected, inherited]

Definition at line 81 of file DataPointDirect.h.

bool Arc::DataPointDirect::linkable [protected, inherited]

Definition at line 80 of file DataPointDirect.h.

bool Arc::DataPointDirect::local [protected, inherited]

Definition at line 78 of file DataPointDirect.h.

Logger Arc::DataPointARC::logger [static, private]

Reimplemented from Arc::DataPoint.

Definition at line 19 of file DataPointARC.h.

Definition at line 25 of file DataPointARC.h.

unsigned long long int Arc::DataPointDirect::range_end [protected, inherited]

Definition at line 87 of file DataPointDirect.h.

unsigned long long int Arc::DataPointDirect::range_start [protected, inherited]

Definition at line 86 of file DataPointDirect.h.

Definition at line 21 of file DataPointARC.h.

bool Arc::DataPointDirect::readonly [protected, inherited]

Definition at line 79 of file DataPointDirect.h.

unsigned long long int Arc::DataPoint::size [protected, inherited]

Definition at line 319 of file DataPoint.h.

Definition at line 20 of file DataPointARC.h.

int Arc::DataPoint::triesleft [protected, inherited]

Definition at line 323 of file DataPoint.h.

Definition at line 24 of file DataPointARC.h.

const URL& Arc::DataPoint::url [protected, inherited]

Definition at line 315 of file DataPoint.h.

const UserConfig& Arc::DataPoint::usercfg [protected, inherited]

Definition at line 316 of file DataPoint.h.

Time Arc::DataPoint::valid [protected, inherited]

Definition at line 322 of file DataPoint.h.

std::list<std::string> Arc::DataPoint::valid_url_options [protected, inherited]

Subclasses should add their own specific options to this list.

Definition at line 327 of file DataPoint.h.

Definition at line 22 of file DataPointARC.h.


The documentation for this class was generated from the following files: