Back to index

nordugrid-arc-nox  1.1.0~rc6
Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected Attributes | Private Member Functions | Private Attributes | Static Private Attributes
Arc::DataPointRLS Class Reference

#include <DataPointRLS.h>

Inheritance diagram for Arc::DataPointRLS:
Inheritance graph
[legend]
Collaboration diagram for Arc::DataPointRLS:
Collaboration graph
[legend]

List of all members.

Public Member Functions

 DataPointRLS (const URL &url, const UserConfig &usercfg)
 ~DataPointRLS ()
virtual DataStatus Resolve (bool source)
 Resolves index service URL into list of ordinary URLs.
virtual DataStatus PreRegister (bool replication, bool force=false)
 Index service preregistration.
virtual DataStatus PostRegister (bool replication)
 Index Service postregistration.
virtual DataStatus PreUnregister (bool replication)
 Index Service preunregistration.
virtual DataStatus Unregister (bool all)
 Index Service unregistration.
virtual DataStatus ListFiles (std::list< FileInfo > &files, bool long_list=false, bool resolve=false, bool metadata=false)
 List file(s).
bool ResolveCallback (globus_rls_handle_t *h, const URL &url, void *arg)
bool ListFilesCallback (globus_rls_handle_t *h, const URL &url, void *arg)
bool UnregisterCallback (globus_rls_handle_t *h, const URL &url, void *arg)
virtual const URLCurrentLocation () const
 Returns current (resolved) URL.
virtual const std::string & CurrentLocationMetadata () const
 Returns meta information used to create current URL.
virtual DataStatus CompareLocationMetadata () const
 Compare metadata of DataPoint and current location.
virtual bool NextLocation ()
 Switch to next location in list of URLs.
virtual bool LocationValid () const
 Returns false if out of retries.
virtual bool HaveLocations () const
 Returns true if number of resolved URLs is not 0.
virtual DataStatus RemoveLocation ()
 Remove current URL from list.
virtual DataStatus RemoveLocations (const DataPoint &p)
 Remove locations present in another DataPoint object.
virtual DataStatus AddLocation (const URL &url, const std::string &meta)
 Add URL to list.
virtual bool IsIndex () const
 Check if URL is an Indexing Service.
virtual bool AcceptsMeta ()
 If endpoint can have any use from meta information.
virtual bool ProvidesMeta ()
 If endpoint can provide at least some meta information directly.
virtual bool Registered () const
 Check if file is registered in Indexing Service.
virtual void SetTries (const int n)
 Set number of retries.
virtual long long int BufSize () const
 Get suggested buffer size for transfers.
virtual int BufNum () const
 Get suggested number of buffers for transfers.
virtual bool Local () const
 Returns true if file is local, e.g. file:// urls.
virtual bool ReadOnly () const
virtual DataStatus StartReading (DataBuffer &buffer)
 Start reading data from URL.
virtual DataStatus StartWriting (DataBuffer &buffer, DataCallback *space_cb=NULL)
 Start writing data to URL.
virtual DataStatus StopReading ()
 Stop reading.
virtual DataStatus StopWriting ()
 Stop writing.
virtual DataStatus Check ()
 Query the DataPoint to check if object is accessible.
virtual DataStatus Remove ()
 Remove/delete object at URL.
virtual void ReadOutOfOrder (bool v)
 Allow/disallow DataPoint to produce scattered data during reading operation.
virtual bool WriteOutOfOrder ()
 Returns true if URL can accept scattered data for writing operation.
virtual void SetAdditionalChecks (bool v)
 Allow/disallow additional checks.
virtual bool GetAdditionalChecks () const
 Check if additional checks before will be performed.
virtual void SetSecure (bool v)
 Allow/disallow heavy security during data transfer.
virtual bool GetSecure () const
 Check if heavy security during data transfer is allowed.
virtual void Passive (bool v)
 Request passive transfers for FTP-like protocols.
virtual void Range (unsigned long long int start=0, unsigned long long int end=0)
 Set range of bytes to retrieve.
virtual const URLGetURL () const
 Returns the URL that was passed to the constructor.
virtual const UserConfigGetUserConfig () const
 Returns the UserConfig that was passed to the constructor.
virtual std::string str () const
 Returns a string representation of the DataPoint.
virtual operator bool () const
 Is DataPoint valid?
virtual bool operator! () const
 Is DataPoint valid?
virtual DataStatus GetFailureReason (void) const
 Returns reason of transfer failure, as reported by callbacks. This could be different from the failure returned by the methods themselves.
virtual bool CheckSize () const
 Check if meta-information 'size' is available.
virtual void SetSize (const unsigned long long int val)
 Set value of meta-information 'size'.
virtual unsigned long long int GetSize () const
 Get value of meta-information 'size'.
virtual bool CheckCheckSum () const
 Check if meta-information 'checksum' is available.
virtual void SetCheckSum (const std::string &val)
 Set value of meta-information 'checksum'.
virtual const std::string & GetCheckSum () const
 Get value of meta-information 'checksum'.
virtual const std::string DefaultCheckSum () const
 Default checksum type.
virtual bool CheckCreated () const
 Check if meta-information 'creation/modification time' is available.
virtual void SetCreated (const Time &val)
 Set value of meta-information 'creation/modification time'.
virtual const TimeGetCreated () const
 Get value of meta-information 'creation/modification time'.
virtual bool CheckValid () const
 Check if meta-information 'validity time' is available.
virtual void SetValid (const Time &val)
 Set value of meta-information 'validity time'.
virtual const TimeGetValid () const
 Get value of meta-information 'validity time'.
virtual bool Cache () const
 Returns true if file is cacheable.
virtual int GetTries () const
 Returns number of retries left.
virtual void NextTry (void)
 Decrease number of retries left.
virtual void SetMeta (const DataPoint &p)
 Copy meta information from another object.
virtual bool CompareMeta (const DataPoint &p) const
 Compare meta information from another object.

Static Public Member Functions

static PluginInstance (PluginArgument *arg)

Protected Member Functions

void ClearLocations ()

Protected Attributes

bool resolved
bool registered
const URLurl
const UserConfigusercfg
unsigned long long int size
std::string checksum
Time created
Time valid
int triesleft
DataStatus failure_code
bool cache
std::list< std::string > valid_url_options
 Subclasses should add their own specific options to this list.

Private Member Functions

URL AddPFN (const URL &url, bool source)

Private Attributes

bool guid_enabled
std::string pfn_path

Static Private Attributes

static Logger logger

Detailed Description

Definition at line 19 of file DataPointRLS.h.


Constructor & Destructor Documentation

Arc::DataPointRLS::DataPointRLS ( const URL url,
const UserConfig usercfg 
)

Definition at line 30 of file DataPointRLS.cpp.

    : DataPointIndex(url, usercfg),
      guid_enabled(false) {
    valid_url_options.push_back("guid");
    globus_module_activate(GLOBUS_COMMON_MODULE);
    globus_module_activate(GLOBUS_IO_MODULE);
    globus_module_activate(GLOBUS_RLS_CLIENT_MODULE);
    if (!proxy_initialized)
      proxy_initialized = GlobusRecoverProxyOpenSSL();
    std::string guidopt = url.Option("guid", "no");
    if ((guidopt == "yes") || (guidopt == ""))
      guid_enabled = true;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

Definition at line 44 of file DataPointRLS.cpp.

                              {
    globus_module_deactivate(GLOBUS_RLS_CLIENT_MODULE);
    globus_module_deactivate(GLOBUS_IO_MODULE);
    globus_module_deactivate(GLOBUS_COMMON_MODULE);
  }

Member Function Documentation

bool Arc::DataPointIndex::AcceptsMeta ( ) [virtual, inherited]

If endpoint can have any use from meta information.

Implements Arc::DataPoint.

Definition at line 149 of file DataPointIndex.cpp.

                                   {
    return true;
  }
DataStatus Arc::DataPointIndex::AddLocation ( const URL url,
const std::string &  meta 
) [virtual, inherited]

Add URL to list.

Parameters:
urlLocation URL to add.
metaLocation meta information.

Implements Arc::DataPoint.

Definition at line 120 of file DataPointIndex.cpp.

                                                                {
    logger.msg(DEBUG, "Add location: url: %s", url.str());
    logger.msg(DEBUG, "Add location: metadata: %s", meta);
    for (std::list<URLLocation>::iterator i = locations.begin();
         i != locations.end(); ++i)
      if ((i->Name() == meta) && (url == (*i)))
        return DataStatus::LocationAlreadyExistsError;
    locations.push_back(URLLocation(url, meta));
    if(locations.end() == location) {
      location = locations.begin();
      SetHandle();
    }
    return DataStatus::Success;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

URL Arc::DataPointRLS::AddPFN ( const URL url,
bool  source 
) [private]

Definition at line 125 of file DataPointRLS.cpp.

                                                      {
    if(source) return purl;
    bool se_uses_lfn = false;
    std::string u = purl.fullstr();
    if (purl.Protocol() == "se") {
      u += "?";
      se_uses_lfn = true;
    } else {
      u += "/";
    }
    if (guid_enabled) {
      std::string guid = UUID();
      if ((!se_uses_lfn) && (!pfn_path.empty()))
        u += pfn_path;
      else
        u += guid;
    }
    else if ((!se_uses_lfn) && (!pfn_path.empty()))
      u += pfn_path;
    else
      u += get_path_str(url);
    return URL(u);
  }

Here is the call graph for this function:

Here is the caller graph for this function:

int Arc::DataPointIndex::BufNum ( ) const [virtual, inherited]

Get suggested number of buffers for transfers.

Implements Arc::DataPoint.

Definition at line 198 of file DataPointIndex.cpp.

                                   {
    if (!h || !*h)
      return 1;
    return (*h)->BufNum();
  }
long long int Arc::DataPointIndex::BufSize ( ) const [virtual, inherited]

Get suggested buffer size for transfers.

Implements Arc::DataPoint.

Definition at line 192 of file DataPointIndex.cpp.

                                              {
    if (!h || !*h)
      return -1;
    return (*h)->BufSize();
  }
bool Arc::DataPoint::Cache ( ) const [virtual, inherited]

Returns true if file is cacheable.

Definition at line 86 of file DataPoint.cpp.

                              {
    return cache;
  }

Here is the caller graph for this function:

DataStatus Arc::DataPointIndex::Check ( ) [virtual, inherited]

Query the DataPoint to check if object is accessible.

If possible this method will also try to provide meta information about the object.

Implements Arc::DataPoint.

Definition at line 186 of file DataPointIndex.cpp.

                                   {
    if (!h || !*h)
      return DataStatus::NoLocationError;
    return (*h)->Check();
  }
bool Arc::DataPoint::CheckCheckSum ( ) const [virtual, inherited]

Check if meta-information 'checksum' is available.

Definition at line 102 of file DataPoint.cpp.

                                      {
    return (!checksum.empty());
  }

Here is the caller graph for this function:

bool Arc::DataPoint::CheckCreated ( ) const [virtual, inherited]

Check if meta-information 'creation/modification time' is available.

Definition at line 118 of file DataPoint.cpp.

                                     {
    return (created != -1);
  }

Here is the caller graph for this function:

bool Arc::DataPoint::CheckSize ( ) const [virtual, inherited]

Check if meta-information 'size' is available.

Definition at line 90 of file DataPoint.cpp.

                                  {
    return (size != (unsigned long long int)(-1));
  }

Here is the caller graph for this function:

bool Arc::DataPoint::CheckValid ( ) const [virtual, inherited]

Check if meta-information 'validity time' is available.

Definition at line 130 of file DataPoint.cpp.

                                   {
    return (valid != -1);
  }

Here is the caller graph for this function:

void Arc::DataPointIndex::ClearLocations ( ) [inline, protected, inherited]

Definition at line 73 of file DataPointIndex.h.

                          {
      locations.clear();
      location = locations.end();
      SetHandle();
    };

Here is the call graph for this function:

Here is the caller graph for this function:

Compare metadata of DataPoint and current location.

Returns inconsistency error or error encountered during operation, or success

Implements Arc::DataPoint.

Definition at line 40 of file DataPointIndex.cpp.

                                                           {
    if (h && *h) {
      DataStatus res = (*h)->Check();
      if (!res.Passed())
        return res;
      if (!CompareMeta(*(*h)))
        return DataStatus::InconsistentMetadataError;
    }
    return DataStatus::Success;
  }  

Here is the call graph for this function:

bool Arc::DataPoint::CompareMeta ( const DataPoint p) const [virtual, inherited]

Compare meta information from another object.

Undefined values are not used for comparison.

Parameters:
pobject to which to compare.

Definition at line 165 of file DataPoint.cpp.

                                                      {
    if (CheckSize() && p.CheckSize())
      if (GetSize() != p.GetSize())
        return false;
    if (CheckCheckSum() && p.CheckCheckSum())
      // TODO: compare checksums properly
      if (strcasecmp(GetCheckSum().c_str(), p.GetCheckSum().c_str()))
        return false;
    if (CheckValid() && p.CheckValid())
      if (GetValid() != p.GetValid())
        return false;
    return true;
  }

Here is the call graph for this function:

Here is the caller graph for this function:

const URL & Arc::DataPointIndex::CurrentLocation ( ) const [virtual, inherited]

Returns current (resolved) URL.

Implements Arc::DataPoint.

Definition at line 26 of file DataPointIndex.cpp.

                                                   {
    static const URL empty;
    if (locations.end() == location)
      return empty;
    return *location;
  }

Here is the caller graph for this function:

const std::string & Arc::DataPointIndex::CurrentLocationMetadata ( ) const [virtual, inherited]

Returns meta information used to create current URL.

Usage differs between different indexing services.

Implements Arc::DataPoint.

Definition at line 33 of file DataPointIndex.cpp.

                                                                 {
    static const std::string empty;
    if (locations.end() == location)
      return empty;
    return location->Name();
  }
const std::string Arc::DataPoint::DefaultCheckSum ( ) const [virtual, inherited]

Default checksum type.

Reimplemented in Arc::DataPointSRM, and Arc::DataPointLFC.

Definition at line 114 of file DataPoint.cpp.

                                                   {
    return std::string("cksum");
  }

Here is the call graph for this function:

Here is the caller graph for this function:

bool Arc::DataPointIndex::GetAdditionalChecks ( ) const [virtual, inherited]

Check if additional checks before will be performed.

Implements Arc::DataPoint.

Definition at line 238 of file DataPointIndex.cpp.

                                                 {
    if (!h || !*h)
      return false;
    return (*h)->GetAdditionalChecks();
  }
const std::string & Arc::DataPoint::GetCheckSum ( ) const [virtual, inherited]

Get value of meta-information 'checksum'.

Definition at line 110 of file DataPoint.cpp.

                                                {
    return checksum;
  }

Here is the caller graph for this function:

const Time & Arc::DataPoint::GetCreated ( ) const [virtual, inherited]

Get value of meta-information 'creation/modification time'.

Definition at line 126 of file DataPoint.cpp.

                                          {
    return created;
  }

Here is the caller graph for this function:

DataStatus Arc::DataPoint::GetFailureReason ( void  ) const [virtual, inherited]

Returns reason of transfer failure, as reported by callbacks. This could be different from the failure returned by the methods themselves.

Definition at line 82 of file DataPoint.cpp.

                                               {
    return failure_code;
  }

Here is the caller graph for this function:

bool Arc::DataPointIndex::GetSecure ( ) const [virtual, inherited]

Check if heavy security during data transfer is allowed.

Implements Arc::DataPoint.

Definition at line 249 of file DataPointIndex.cpp.

                                       {
    if (!h || !*h)
      return false;
    return (*h)->GetSecure();
  }
unsigned long long int Arc::DataPoint::GetSize ( ) const [virtual, inherited]

Get value of meta-information 'size'.

Definition at line 98 of file DataPoint.cpp.

                                                  {
    return size;
  }

Here is the caller graph for this function:

int Arc::DataPoint::GetTries ( ) const [virtual, inherited]

Returns number of retries left.

Definition at line 142 of file DataPoint.cpp.

                                {
    return triesleft;
  }

Here is the caller graph for this function:

const URL & Arc::DataPoint::GetURL ( ) const [virtual, inherited]

Returns the URL that was passed to the constructor.

Definition at line 45 of file DataPoint.cpp.

                                     {
    return url;
  }

Here is the caller graph for this function:

const UserConfig & Arc::DataPoint::GetUserConfig ( ) const [virtual, inherited]

Returns the UserConfig that was passed to the constructor.

Definition at line 49 of file DataPoint.cpp.

                                                   {
    return usercfg;
  }

Here is the caller graph for this function:

const Time & Arc::DataPoint::GetValid ( ) const [virtual, inherited]

Get value of meta-information 'validity time'.

Definition at line 138 of file DataPoint.cpp.

                                        {
    return valid;
  }

Here is the caller graph for this function:

bool Arc::DataPointIndex::HaveLocations ( ) const [virtual, inherited]

Returns true if number of resolved URLs is not 0.

Implements Arc::DataPoint.

Definition at line 51 of file DataPointIndex.cpp.

                                           {
    return (locations.size() != 0);
  }

Here is the caller graph for this function:

Definition at line 50 of file DataPointRLS.cpp.

                                                    {
    DataPointPluginArgument *dmcarg =
      dynamic_cast<DataPointPluginArgument*>(arg);
    if (!dmcarg)
      return NULL;
    if (((const URL&)(*dmcarg)).Protocol() != "rls")
      return NULL;
    // Make this code non-unloadable because Globus
    // may have problems with unloading
    Glib::Module* module = dmcarg->get_module();
    PluginsFactory* factory = dmcarg->get_factory();
    if(factory && module) factory->makePersistent(module);
    return new DataPointRLS(*dmcarg, *dmcarg);
  }

Here is the call graph for this function:

bool Arc::DataPointIndex::IsIndex ( ) const [virtual, inherited]

Check if URL is an Indexing Service.

Implements Arc::DataPoint.

Definition at line 145 of file DataPointIndex.cpp.

                                     {
    return true;
  }
DataStatus Arc::DataPointRLS::ListFiles ( std::list< FileInfo > &  files,
bool  long_list = false,
bool  resolve = false,
bool  metadata = false 
) [virtual]

List file(s).

If the DataPoint represents a directory its contents will be listed.

Parameters:
fileswill contain list of file names and optionally their attributes.
long)listif true, list additional properties of each file.
resolveif true, resolve physical locations (relevant for indexing services only).

Implements Arc::DataPoint.

Definition at line 1163 of file DataPointRLS.cpp.

                                                                                                          {
    std::list<URL> rlis;
    std::list<URL> lrcs;
    if (url.Host().empty()) {
      logger.msg(INFO, "RLS URL must contain host");
      return DataStatus::ListError;
    }
    rlis.push_back(url.ConnectionURL());
    lrcs.push_back(url.ConnectionURL());

    list_files_rls_t arg(*this, files, long_list);
    rls_find_lrcs(rlis, lrcs, true, false,
                  &list_files_callback, (void*)&arg);
    return arg.success;
  }

Here is the call graph for this function:

bool Arc::DataPointRLS::ListFilesCallback ( globus_rls_handle_t *  h,
const URL url,
void *  arg 
)

Definition at line 1033 of file DataPointRLS.cpp.

                                                                     {
    std::list<FileInfo>& files(((list_files_rls_t*)arg)->files);
    DataStatus& success(((list_files_rls_t*)arg)->success);
    bool& resolve(((list_files_rls_t*)arg)->resolve);
    std::string& guid(((list_files_rls_t*)arg)->guid);

    int lrc_offset = 0;
    globus_result_t err;
    int errcode;
    char errmsg[MAXERRMSG + 32];
    globus_list_t *pfns = NULL;
    if (guid_enabled && !path_empty(url) && guid.empty()) {
      // looking gor guid only once
      // looking for guid only if lfn specified
      globus_rls_attribute_t opr;
      opr.type = globus_rls_attr_type_str;
      opr.val.s = const_cast<char*>(get_path_str(url));
      int off = 0;
      globus_list_t *guids = NULL;
      err = globus_rls_client_lrc_attr_search(h, const_cast<char*>("lfn"),
                                              globus_rls_obj_lrc_lfn,
                                              globus_rls_attr_op_eq,
                                              &opr, NULL, &off, 1, &guids);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        logger.msg(INFO, "Failed to find GUID for specified LFN in %s: %s",
                   rlsurl.str(), errmsg);
        return true;
      }
      if (!guids) {
        logger.msg(INFO, "There is no GUID for specified LFN in %s",
                   rlsurl.str());
        return true;
      }
      globus_rls_attribute_object_t *obattr =
        (globus_rls_attribute_object_t*)globus_list_first(guids);
      guid = obattr->key;
      globus_rls_client_free_list(guids);
    }
    if (!guid.empty())
      err = globus_rls_client_lrc_get_pfn
              (h, const_cast<char*>(guid.c_str()), &lrc_offset, 1000, &pfns);
    else if (!path_empty(url))
      err = globus_rls_client_lrc_get_pfn
              (h, const_cast<char*>(get_path_str(url)), &lrc_offset, 1000, &pfns);
    else
      err = globus_rls_client_lrc_get_pfn_wc(h, const_cast<char*>("*"),
                                             rls_pattern_unix,
                                             &lrc_offset, 1000, &pfns);
    if (err != GLOBUS_SUCCESS) {
      globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                   GLOBUS_FALSE);
      if (errcode == GLOBUS_RLS_LFN_NEXIST) {
        logger.msg(VERBOSE, "No LFNs found in %s", rlsurl.str());
        success = DataStatus::Success;
        return true;
      }
      logger.msg(INFO, "Failed to retrieve list of LFNs/PFNs from %s",
                 rlsurl.str());
      return true;
    }
    success = DataStatus::Success;
    std::string last_lfn = "";
    std::string last_guid = "";
    for (globus_list_t *p = pfns; p; p = globus_list_rest(p)) {
      globus_rls_string2_t *str2 =
        (globus_rls_string2_t*)globus_list_first(p);
      std::string lfn(str2->s1);
      URL pfn(str2->s2);
      if (guid_enabled) {
        if (lfn != last_guid) {
          last_guid = lfn;
          last_lfn = "";
          // get real lfn
          globus_list_t *lfn_list = NULL;
          err = globus_rls_client_lrc_attr_value_get
                  (h, const_cast<char*>(lfn.c_str()),
                  const_cast<char*>("lfn"), globus_rls_obj_lrc_lfn, &lfn_list);
          if (err != GLOBUS_SUCCESS) {
            globus_rls_client_error_info(err, &errcode, errmsg,
                                         MAXERRMSG + 32, GLOBUS_FALSE);
            continue;
          }
          if (lfn_list == NULL)
            continue;
          globus_rls_attribute_t *attr =
            (globus_rls_attribute_t*)globus_list_first(lfn_list);
          if (attr->type != globus_rls_attr_type_str) {
            globus_rls_client_free_list(lfn_list);
            continue;
          }
          // use only first lfn (TODO: all lfns)
          last_lfn = attr->val.s;
          globus_rls_client_free_list(lfn_list);
        }
        if (!last_lfn.empty()) {
          logger.msg(VERBOSE, "lfn: %s(%s) - %s",
                     last_lfn, last_guid, pfn.str());
          std::list<FileInfo>::iterator f;
          for (f = files.begin(); f != files.end(); ++f)
            if (f->GetName() == last_lfn)
              break;
          if (f == files.end()) {
            f = files.insert(files.end(), FileInfo(last_lfn.c_str()));
            if (resolve)
              get_attributes(h, last_guid, *f);
          }
          f->AddURL(pfn);
        }
      }
      else { // !guid_enabled
        logger.msg(VERBOSE, "lfn: %s - pfn: %s", lfn, pfn.str());
        std::list<FileInfo>::iterator f;
        for (f = files.begin(); f != files.end(); ++f)
          if (f->GetName() == lfn)
            break;
        if (f == files.end()) {
          f = files.insert(files.end(), FileInfo(lfn));
          if (resolve)
            get_attributes(h, lfn, *f);
        }
        f->AddURL(pfn);
      }
    }
    globus_rls_client_free_list(pfns);
    return true;
  }

Here is the call graph for this function:

bool Arc::DataPointIndex::Local ( ) const [virtual, inherited]

Returns true if file is local, e.g. file:// urls.

Implements Arc::DataPoint.

Definition at line 204 of file DataPointIndex.cpp.

                                   {
    if (!h || !*h)
      return false;
    return (*h)->Local();
  }
bool Arc::DataPointIndex::LocationValid ( ) const [virtual, inherited]

Returns false if out of retries.

Implements Arc::DataPoint.

Definition at line 55 of file DataPointIndex.cpp.

                                           {
    if (triesleft <= 0)
      return false;
    if (locations.end() == location)
      return false;
    return true;
  }

Here is the caller graph for this function:

bool Arc::DataPointIndex::NextLocation ( ) [virtual, inherited]

Switch to next location in list of URLs.

At last location switch to first if number of allowed retries is not exceeded. Returns false if no retries left.

Implements Arc::DataPoint.

Definition at line 73 of file DataPointIndex.cpp.

                                    {
    if (!LocationValid()) {
      --triesleft;
      return false;
    }
    ++location;
    if (locations.end() == location)
      if (--triesleft > 0)
        location = locations.begin();
    SetHandle();
    return LocationValid();
  }

Here is the call graph for this function:

void Arc::DataPoint::NextTry ( void  ) [virtual, inherited]

Decrease number of retries left.

Definition at line 150 of file DataPoint.cpp.

                          {
    if(triesleft) --triesleft;
  }

Here is the caller graph for this function:

Arc::DataPoint::operator bool ( void  ) const [virtual, inherited]

Is DataPoint valid?

Definition at line 57 of file DataPoint.cpp.

                                 {

    if (!url)
      return false;
      
    // URL option validation. Subclasses which do not want to validate
    // URL options should override this method.
    std::map<std::string, std::string> options = url.Options();
    for (std::map<std::string, std::string>::iterator i = options.begin(); i != options.end(); i++) {
      bool valid = false; 
      for (std::list<std::string>::const_iterator j = valid_url_options.begin(); j != valid_url_options.end(); j++) {
        if (i->first == *j) valid = true;
      }
      if (!valid) {
        logger.msg(ERROR, "Invalid URL option: %s", i->first);
        return false;
      }
    }
    return true;
  }

Here is the call graph for this function:

bool Arc::DataPoint::operator! ( void  ) const [virtual, inherited]

Is DataPoint valid?

Definition at line 78 of file DataPoint.cpp.

                                  {
    return !((bool)*this);
  }
void Arc::DataPointIndex::Passive ( bool  v) [virtual, inherited]

Request passive transfers for FTP-like protocols.

Parameters:
trueto request.

Implements Arc::DataPoint.

Definition at line 255 of file DataPointIndex.cpp.

                                     {
    if (h && *h)
      (*h)->Passive(v);
  }
DataStatus Arc::DataPointRLS::PostRegister ( bool  replication) [virtual]

Index Service postregistration.

Used for same purpose as PreRegister. Should be called after actual transfer of file successfully finished.

Parameters:
replicationif true, the file is being replicated between two locations registered in Indexing Service under same name.

Implements Arc::DataPoint.

Definition at line 436 of file DataPointRLS.cpp.

                                                        {
    globus_rls_handle_t *h;
    char errmsg[MAXERRMSG + 32];
    globus_result_t err;
    int errcode;

    if (url.Host().empty()) {
      logger.msg(INFO, "RLS URL must contain host");
      return DataStatus::PostRegisterError;
    }
    err = globus_rls_client_connect
            (const_cast<char*>(url.ConnectionURL().c_str()), &h);
    if (err != GLOBUS_SUCCESS) {
      globus_rls_client_error_info(err, NULL, errmsg, MAXERRMSG + 32,
                                   GLOBUS_FALSE);
      logger.msg(INFO, "Failed to connect to RLS server: %s", errmsg);
      return DataStatus::PostRegisterError;
    }
    // assume that is RLI and try to resolve for special/any name

    std::string pfn;
    std::string guid;
    pfn = CurrentLocation().str();
    // it is always better to register pure url
    std::string rls_lfn = get_path_str(url);
    if (!replication)
      if (guid_enabled) {
        for (;;) {
          // generate guid
          guid = UUID();
          // store in LRC
          err = globus_rls_client_lrc_create
                  (h, const_cast<char*>(guid.c_str()),
                  const_cast<char*>(pfn.c_str()));
          if (err != GLOBUS_SUCCESS) {
            err = globus_rls_client_error_info(err, &errcode, NULL, 0,
                                               GLOBUS_TRUE);
            if (errcode == GLOBUS_RLS_LFN_EXIST) {
              globus_rls_free_result(err);
              continue;
            }
          }
          rls_lfn = guid;
          break;
        }
        if (err != GLOBUS_SUCCESS) {
          globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                       GLOBUS_FALSE);
          logger.msg(INFO, "Failed to create GUID in RLS: %s", errmsg);
          globus_rls_client_close(h);
          return DataStatus::PostRegisterError;
        }
        // Check if there is no same LFN
        globus_rls_attribute_t opr;
        opr.type = globus_rls_attr_type_str;
        opr.val.s = const_cast<char*>(get_path_str(url));
        int off = 0;
        globus_list_t *guids = NULL;
        err = globus_rls_client_lrc_attr_search(h, const_cast<char*>("lfn"),
                                                globus_rls_obj_lrc_lfn,
                                                globus_rls_attr_op_eq,
                                                &opr, NULL, &off, 1, &guids);
        if (err != GLOBUS_SUCCESS) {
          globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                       GLOBUS_FALSE);
          if ((errcode != GLOBUS_RLS_LFN_NEXIST) &&
              (errcode != GLOBUS_RLS_ATTR_NEXIST) &&
              (errcode != GLOBUS_RLS_ATTR_VALUE_NEXIST)) {
            logger.msg(INFO, "Failed to check for existing LFN in %s: %s",
                       url.str(), errmsg);
            globus_rls_client_close(h);
            return DataStatus::PostRegisterError;
          }
        }
        if (guids) {
          globus_rls_client_free_list(guids);
          logger.msg(INFO, "There is same LFN in %s", url.str());
          globus_rls_client_close(h);
          return DataStatus::PostRegisterError;
        }
        // add LFN
        globus_rls_attribute_t attr;
        attr.objtype = globus_rls_obj_lrc_lfn;
        attr.type = globus_rls_attr_type_str;
        attr.name = const_cast<char*>("lfn");
        attr.val.s = const_cast<char*>(get_path_str(url));
        err = globus_rls_client_lrc_attr_put
                (h, const_cast<char*>(rls_lfn.c_str()), &attr, 0);
        if (err != GLOBUS_SUCCESS) {
          globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                       GLOBUS_FALSE);
          logger.msg(INFO, "Failed to add LFN-GUID to RLS: %s", errmsg);
          globus_rls_client_close(h);
          return DataStatus::PostRegisterError;
        }
      }
      else {
        err = globus_rls_client_lrc_create
                (h, const_cast<char*>(get_path_str(url)),
                const_cast<char*>(pfn.c_str()));
        if (err != GLOBUS_SUCCESS) {
          err = globus_rls_client_error_info(err, &errcode, NULL, 0,
                                             GLOBUS_TRUE);
          if (errcode == GLOBUS_RLS_LFN_EXIST) {
            globus_rls_free_result(err);
            err = globus_rls_client_lrc_add
                    (h, const_cast<char*>(get_path_str(url)),
                    const_cast<char*>(pfn.c_str()));
          }
        }
      }
    else {
      if (guid_enabled) {
        // get guid
        globus_rls_attribute_t opr;
        opr.type = globus_rls_attr_type_str;
        opr.val.s = const_cast<char*>(get_path_str(url));
        int off = 0;
        globus_list_t *guids = NULL;
        err = globus_rls_client_lrc_attr_search(h, const_cast<char*>("lfn"),
                                                globus_rls_obj_lrc_lfn,
                                                globus_rls_attr_op_eq,
                                                &opr, NULL, &off, 1, &guids);
        if (err != GLOBUS_SUCCESS) {
          globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                       GLOBUS_FALSE);
          logger.msg(INFO, "Failed to find GUID for specified LFN in %s: %s",
                     url.str(), errmsg);
          globus_rls_client_close(h);
          return DataStatus::PostRegisterError;
        }
        if (!guids) {
          logger.msg(INFO, "There is no GUID for specified LFN in %s",
                     url.str());
          globus_rls_client_close(h);
          return DataStatus::PostRegisterError;
        }
        globus_rls_attribute_object_t *obattr =
          (globus_rls_attribute_object_t*)globus_list_first(guids);
        guid = obattr->key;
        globus_rls_client_free_list(guids);
        rls_lfn = guid;
      }
      err = globus_rls_client_lrc_add
              (h, const_cast<char*>(rls_lfn.c_str()),
              const_cast<char*>(pfn.c_str()));
    }
    if (err != GLOBUS_SUCCESS) {
      globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                   GLOBUS_FALSE);
      if (errcode != GLOBUS_RLS_MAPPING_EXIST) {
        logger.msg(INFO, "Failed to create/add LFN-PFN mapping: %s", errmsg);
        globus_rls_client_close(h);
        return DataStatus::PostRegisterError;
      }
    }
    globus_rls_attribute_t attr;
    std::string attr_val;
    attr.objtype = globus_rls_obj_lrc_lfn;
    attr.type = globus_rls_attr_type_str;
    attr.name = const_cast<char*>("filetype");
    attr.val.s = const_cast<char*>("file");
    err = globus_rls_client_lrc_attr_put
            (h, const_cast<char*>(rls_lfn.c_str()), &attr, 0);
    if (err != GLOBUS_SUCCESS) {
      globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                   GLOBUS_FALSE);
      if (errcode != GLOBUS_RLS_ATTR_EXIST)
        logger.msg(INFO, "Warning: failed to add attribute to RLS: %s",
                   errmsg);
    }
    if (CheckSize()) {
      attr.name = const_cast<char*>("size");
      attr_val = tostring(GetSize());
      attr.val.s = const_cast<char*>(attr_val.c_str());
      err = globus_rls_client_lrc_attr_put
              (h, const_cast<char*>(rls_lfn.c_str()), &attr, 0);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        if (errcode != GLOBUS_RLS_ATTR_EXIST)
          logger.msg(INFO, "Warning: failed to add attribute to RLS: %s",
                     errmsg);
      }
    }
    if (CheckCheckSum()) {
      attr.name = const_cast<char*>("filechecksum");
      attr_val = GetCheckSum();
      attr.val.s = const_cast<char*>(attr_val.c_str());
      err = globus_rls_client_lrc_attr_put
              (h, const_cast<char*>(rls_lfn.c_str()), &attr, 0);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        if (errcode != GLOBUS_RLS_ATTR_EXIST)
          logger.msg(INFO, "Warning: failed to add attribute to RLS: %s",
                     errmsg);
      }
    }
    if (CheckCreated()) {
      attr.name = const_cast<char*>("modifytime");
      attr_val = GetCreated();
      attr.val.s = const_cast<char*>(attr_val.c_str());
      err = globus_rls_client_lrc_attr_put
              (h, const_cast<char*>(rls_lfn.c_str()), &attr, 0);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        if (errcode != GLOBUS_RLS_ATTR_EXIST)
          logger.msg(INFO, "Warning: failed to add attribute to RLS: %s",
                     errmsg);
      }
    }
    if (url.Options().size() > 0)
      for (std::map<std::string, std::string>::const_iterator pos =
             url.Options().begin(); pos != url.Options().end(); pos++) {
        attr.name = const_cast<char*>(pos->first.c_str());
        attr.val.s = const_cast<char*>(pos->second.c_str());
        err = globus_rls_client_lrc_attr_put
                (h, const_cast<char*>(rls_lfn.c_str()), &attr, 0);
        if (err != GLOBUS_SUCCESS) {
          globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                       GLOBUS_FALSE);
          if (errcode != GLOBUS_RLS_ATTR_EXIST)
            logger.msg(INFO, "Warning: failed to add attribute to RLS: %s",
                       errmsg);
        }
      }
    globus_rls_client_close(h);
    return DataStatus::Success;
  }

Here is the call graph for this function:

DataStatus Arc::DataPointRLS::PreRegister ( bool  replication,
bool  force = false 
) [virtual]

Index service preregistration.

This function registers the physical location of a file into an indexing service. It should be called before the actual transfer to that location happens.

Parameters:
replicationif true, the file is being replicated between two locations registered in the indexing service under same name.
forceif true, perform registration of a new file even if it already exists. Should be used to fix failures in Indexing Service.

Implements Arc::DataPoint.

Definition at line 415 of file DataPointRLS.cpp.

                                                                   {
    if (url.Host().empty()) {
      logger.msg(INFO, "RLS URL must contain host");
      return DataStatus::PreRegisterError;
    }
    if (replication) { /* replicating inside same lfn */
      if (!registered) { /* for replication it must be there */
        logger.msg(ERROR, "LFN is missing in RLS (needed for replication)");
        return DataStatus::PreRegisterError;
      }
      return DataStatus::Success;
    }
    if (registered)  /* algorithm require this to be new file */
      if (!force) {
        logger.msg(ERROR, "LFN already exists in replica");
        return DataStatus::PreRegisterError;
      }
    /* RLS does not support LFN only in database - hence doing nothing here */
    return DataStatus::Success;
  }

Here is the call graph for this function:

DataStatus Arc::DataPointRLS::PreUnregister ( bool  replication) [virtual]

Index Service preunregistration.

Should be called if file transfer failed. It removes changes made by PreRegister.

Parameters:
replicationif true, the file is being replicated between two locations registered in Indexing Service under same name.

Implements Arc::DataPoint.

Definition at line 668 of file DataPointRLS.cpp.

                                             {
    return DataStatus::Success;
  }
bool Arc::DataPointIndex::ProvidesMeta ( ) [virtual, inherited]

If endpoint can provide at least some meta information directly.

Implements Arc::DataPoint.

Definition at line 153 of file DataPointIndex.cpp.

                                    {
    return true;
  }
void Arc::DataPointIndex::Range ( unsigned long long int  start = 0,
unsigned long long int  end = 0 
) [virtual, inherited]

Set range of bytes to retrieve.

Default values correspond to whole file.

Implements Arc::DataPoint.

Definition at line 260 of file DataPointIndex.cpp.

                                                         {
    if (h && *h)
      (*h)->Range(start, end);
  }
bool Arc::DataPointIndex::ReadOnly ( ) const [virtual, inherited]

Implements Arc::DataPoint.

Definition at line 210 of file DataPointIndex.cpp.

                                      {
    if (!h || !*h)
      return true;
    return (*h)->ReadOnly();
  }
void Arc::DataPointIndex::ReadOutOfOrder ( bool  v) [virtual, inherited]

Allow/disallow DataPoint to produce scattered data during reading operation.

Parameters:
vtrue if allowed (default is false).

Implements Arc::DataPoint.

Definition at line 222 of file DataPointIndex.cpp.

                                            {
    if (h && *h)
      (*h)->ReadOutOfOrder(v);
  }
bool Arc::DataPointIndex::Registered ( ) const [virtual, inherited]

Check if file is registered in Indexing Service.

Proper value is obtainable only after Resolve.

Implements Arc::DataPoint.

Definition at line 157 of file DataPointIndex.cpp.

                                        {
    return registered;
  }
DataStatus Arc::DataPointIndex::Remove ( ) [virtual, inherited]

Remove/delete object at URL.

Implements Arc::DataPoint.

Definition at line 216 of file DataPointIndex.cpp.

                                    {
    if (!h || !*h)
      return DataStatus::NoLocationError;
    return (*h)->Remove();
  }

Remove current URL from list.

Implements Arc::DataPoint.

Definition at line 86 of file DataPointIndex.cpp.

Here is the call graph for this function:

DataStatus Arc::DataPointIndex::RemoveLocations ( const DataPoint p) [virtual, inherited]

Remove locations present in another DataPoint object.

Implements Arc::DataPoint.

Definition at line 96 of file DataPointIndex.cpp.

                                                                {
    if (!p_.IsIndex())
      return DataStatus::Success;
    const DataPointIndex& p = dynamic_cast<const DataPointIndex&>(p_);
    std::list<URLLocation>::iterator p_int;
    std::list<URLLocation>::const_iterator p_ext;
    for (p_ext = p.locations.begin(); p_ext != p.locations.end(); ++p_ext)
      for (p_int = locations.begin(); p_int != locations.end();)
        // Compare URLs
        if (*p_int == *p_ext)
          if (location == p_int) {
            p_int = locations.erase(p_int);
            location = p_int;
          }
          else
            p_int = locations.erase(p_int);
        else
          ++p_int;
    if (locations.end() == location)
      location = locations.begin();
    SetHandle();
    return DataStatus::Success;
  }

Here is the call graph for this function:

DataStatus Arc::DataPointRLS::Resolve ( bool  source) [virtual]

Resolves index service URL into list of ordinary URLs.

Also obtains meta information about the file.

Parameters:
sourcetrue if DataPoint object represents source of information.

Implements Arc::DataPoint.

Definition at line 330 of file DataPointRLS.cpp.

                                              {
    resolved = false;
    registered = false;
    if (url.Host().empty()) {
      logger.msg(INFO, "RLS URL must contain host");
      if (source)
        return DataStatus::ReadResolveError;
      else
        return DataStatus::WriteResolveError;
    }
    if (source) {
      if (get_path_str(url)[0] == 0) {
        logger.msg(INFO, "Source must contain LFN");
        return DataStatus::ReadResolveError;
      }
      std::list<URL> rlis;
      std::list<URL> lrcs;
      rlis.push_back(url.ConnectionURL());
      lrcs.push_back(url.ConnectionURL());
      meta_resolve_rls_t arg(*this, source);
      rls_find_lrcs(rlis, lrcs, true, false,
                    &meta_resolve_callback, (void*)&arg);
      if (!arg.success)
        return arg.success;
    }
    else {
      if (get_path_str(url) == 0) {
        logger.msg(INFO, "Destination must contain LFN");
        return DataStatus::WriteResolveError;
      }
      std::list<URL> rlis;
      std::list<URL> lrcs;
      rlis.push_back(url.ConnectionURL());
      lrcs.push_back(url.ConnectionURL());
      if (url.Locations().size() == 0)
        logger.msg(INFO, "Locations are missing in destination RLS url - "
                   "will use those registered with special name");
      meta_resolve_rls_t arg(*this, source);
      rls_find_lrcs(rlis, lrcs, true, false,
                    &meta_resolve_callback, (void*)&arg);
      if (!arg.success)
        return arg.success;
      if (!HaveLocations()) {
        logger.msg(INFO, "No locations found for destination");
        return DataStatus::WriteResolveError;
      }
/*
      // This part is done directly in callback. The difference
      // is that instead of "arbitrary" LRC the one provided by
      // callback is used
      // Make pfns
      std::list<URL>::iterator lrc_p = lrcs.begin();
      for (std::list<URLLocation>::iterator loc = locations.begin();
           loc != locations.end();) {
        // AddPFN
        if (!loc->Name().empty()) {
          logger.msg(VERBOSE, "Using location: %s - %s",
                     loc->Name(), loc->str());
          ++loc;
        }
        // Use arbitrary lrc
        else if (lrc_p == lrcs.end()) {   // no LRC
          logger.msg(VERBOSE, "Removing location: %s - %s",
                     loc->Name(), loc->str());
          loc = locations.erase(loc);
        }
        else {
          *loc = URLLocation(*loc, lrc_p->str());
          ++lrc_p;
          if (lrc_p == lrcs.end())
            lrc_p = lrcs.begin();
          logger.msg(VERBOSE, "Using location: %s - %s",
                     loc->Name(), loc->str());
          ++loc;
        }
      }
*/
    }
    if (CheckCheckSum()) logger.msg(VERBOSE, "meta_get_data: checksum: %s", GetCheckSum());
    if (CheckSize()) logger.msg(VERBOSE, "meta_get_data: size: %llu", GetSize());
    if (CheckCreated()) logger.msg(VERBOSE, "meta_get_data: created: %s", GetCreated().str());
    resolved = true;
    return DataStatus::Success;
  }

Here is the call graph for this function:

bool Arc::DataPointRLS::ResolveCallback ( globus_rls_handle_t *  h,
const URL url,
void *  arg 
)

Definition at line 150 of file DataPointRLS.cpp.

                                                                   {
    bool& source(((meta_resolve_rls_t*)arg)->source);
    DataStatus& success(((meta_resolve_rls_t*)arg)->success);
    bool& obtained_info(((meta_resolve_rls_t*)arg)->obtained_info);
    std::string& guid(((meta_resolve_rls_t*)arg)->guid);

    char errmsg[MAXERRMSG + 32];
    globus_result_t err;
    int errcode;

    // Ask LRC if it contains file of interest

    if (guid_enabled && source && guid.empty()) {
      // map lfn->guid (only once)
      globus_rls_attribute_t opr;
      opr.type = globus_rls_attr_type_str;
      opr.val.s = const_cast<char*>(get_path_str(url));
      int off = 0;
      globus_list_t *guids = NULL;
      err = globus_rls_client_lrc_attr_search(h, const_cast<char*>("lfn"),
                                              globus_rls_obj_lrc_lfn,
                                              globus_rls_attr_op_eq, &opr,
                                              NULL, &off, 1, &guids);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        logger.msg(INFO, "Failed to find GUID for specified LFN in %s: %s",
                   rlsurl.str(), errmsg);
        return true;
      }
      if (!guids) {
        logger.msg(INFO, "There is no GUID for specified LFN in %s",
                   rlsurl.str());
        return true;
      }
      globus_rls_attribute_object_t *obattr =
        (globus_rls_attribute_object_t*)globus_list_first(guids);
      guid = obattr->key;
      globus_rls_client_free_list(guids);
    }
    globus_list_t *pfns_list = NULL;
    if (source) {
      if (!guid.empty())
        err = globus_rls_client_lrc_get_pfn
                (h, const_cast<char*>(guid.c_str()), 0, 0, &pfns_list);
      else
        err = globus_rls_client_lrc_get_pfn
                (h, const_cast<char*>(get_path_str(url)), 0, 0, &pfns_list);
    } else {
      err = globus_rls_client_lrc_get_pfn
              (h, const_cast<char*>("__storage_service__"), 0, 0, &pfns_list);
    }
    if (err != GLOBUS_SUCCESS) {
      globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                   GLOBUS_FALSE);
      if (errcode == GLOBUS_RLS_INVSERVER) {
        return true;
      } else if (errcode == GLOBUS_RLS_LFN_NEXIST) {
        return true;
      } else { // do not know
        logger.msg(INFO, "Warning: can't get PFNs from server %s: %s",
                   rlsurl.str(), errmsg);
        return true;
      }
    }
    if (!success) {
      success = DataStatus::Success; // got something
      if (source)
        registered = true;
    }
    if (url.Locations().size() == 0) {
      for (globus_list_t *lp = pfns_list; lp; lp = globus_list_rest(lp)) {
        globus_rls_string2_t *str2 =
          (globus_rls_string2_t*)globus_list_first(lp);
        URL pfn(str2->s2);
        for (std::map<std::string, std::string>::const_iterator i =
               url.CommonLocOptions().begin();
             i != url.CommonLocOptions().end(); i++) {
          pfn.AddOption(i->first, i->second, false);
        }
        URL pfn_ = AddPFN(pfn,source);
        logger.msg(VERBOSE, "Adding location: %s - %s", rlsurl.str(), pfn.str());
        AddLocation(pfn_, rlsurl.str());
      }
    } else {
      for (std::list<URLLocation>::const_iterator loc = url.Locations().begin();
           loc != url.Locations().end(); loc++) {
        if(source) {
          // for source find subset of locations registered in RLS
          for (globus_list_t *lp = pfns_list; lp; lp = globus_list_rest(lp)) {
            globus_rls_string2_t *str2 =
              (globus_rls_string2_t*)globus_list_first(lp);
            URL pfn(str2->s2);
            // for RLS URLs are used instead of metanames
            if (pfn == *loc) {
              logger.msg(VERBOSE, "Adding location: %s - %s",
                         rlsurl.str(), pfn.str());
              for (std::map<std::string, std::string>::const_iterator i =
                     url.CommonLocOptions().begin();
                   i != url.CommonLocOptions().end(); i++) {
                pfn.AddOption(i->first, i->second, false);
              }
              URL pfn_ = AddPFN(pfn,source);
              AddLocation(pfn_, rlsurl.str());
              break;
            }
          }
        } else {
          // for destination accept specified locations
          URL pfn_(*loc);
          for (std::map<std::string, std::string>::const_iterator i =
                 url.CommonLocOptions().begin();
            i != url.CommonLocOptions().end(); i++) {
            pfn_.AddOption(i->first, i->second, false);
          }
          pfn_ = AddPFN(pfn_,source);
          AddLocation(pfn_, rlsurl.str());
        }
      }
    }
    globus_rls_client_free_list(pfns_list);
    if (!obtained_info) {
      /* obtain metadata - assume it is same everywhere */
      globus_list_t *attr_list;
      if (!guid.empty())
        err = globus_rls_client_lrc_attr_value_get
                (h, const_cast<char*>(guid.c_str()),
                NULL, globus_rls_obj_lrc_lfn, &attr_list);
      else
        err = globus_rls_client_lrc_attr_value_get
                (h, const_cast<char*>(get_path_str(url)),
                NULL, globus_rls_obj_lrc_lfn, &attr_list);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        if (errcode == GLOBUS_RLS_ATTR_NEXIST)
          return true;
        logger.msg(INFO, "Warning: Failed to obtain attributes from %s: %s",
                   rlsurl.str(), errmsg);
        return true;
      }
      registered = true; // even for destination
      for (globus_list_t *lpa = attr_list; lpa; lpa = globus_list_rest(lpa)) {
        globus_rls_attribute_t *attr =
          (globus_rls_attribute_t*)globus_list_first(lpa);
        if (attr->type != globus_rls_attr_type_str)
          continue;
        logger.msg(VERBOSE, "Attribute: %s - %s", attr->name, attr->val.s);
        if (strcmp(attr->name, "filechecksum") == 0) {
          if (!CheckCheckSum())
            SetCheckSum(attr->val.s);
        }
        else if (strcmp(attr->name, "size") == 0) {
          if (!CheckSize())
            SetSize(stringtoull(attr->val.s));
        }
        else if (strcmp(attr->name, "modifytime") == 0) {
          if (!CheckCreated()) {
            Time created(attr->val.s);
            if (created == -1)
              created.SetTime(stringtoull(attr->val.s));
            SetCreated(created);
          }
        }
        else if (strcmp(attr->name, "created") == 0)
          if (!CheckCreated()) {
            Time created(attr->val.s);
            if (created == -1)
              created.SetTime(stringtoull(attr->val.s));
            SetCreated(created);
          }
      }
      globus_rls_client_free_list(attr_list);
      obtained_info = true;
    }
    return true;
  }

Here is the call graph for this function:

void Arc::DataPointIndex::SetAdditionalChecks ( bool  v) [virtual, inherited]

Allow/disallow additional checks.

Check for existance of remote file (and probably other checks too) before initiating reading and writing operations.

Parameters:
vtrue if allowed (default is true).

Implements Arc::DataPoint.

Definition at line 233 of file DataPointIndex.cpp.

                                                 {
    if (h && *h)
      (*h)->SetAdditionalChecks(v);
  }
void Arc::DataPoint::SetCheckSum ( const std::string &  val) [virtual, inherited]

Set value of meta-information 'checksum'.

Definition at line 106 of file DataPoint.cpp.

                                                  {
    checksum = val;
  }

Here is the caller graph for this function:

void Arc::DataPoint::SetCreated ( const Time val) [virtual, inherited]

Set value of meta-information 'creation/modification time'.

Definition at line 122 of file DataPoint.cpp.

                                            {
    created = val;
  }

Here is the caller graph for this function:

void Arc::DataPoint::SetMeta ( const DataPoint p) [virtual, inherited]

Copy meta information from another object.

Already defined values are not overwritten.

Parameters:
pobject from which information is taken.

Definition at line 154 of file DataPoint.cpp.

                                            {
    if (!CheckSize())
      SetSize(p.GetSize());
    if (!CheckCheckSum())
      SetCheckSum(p.GetCheckSum());
    if (!CheckCreated())
      SetCreated(p.GetCreated());
    if (!CheckValid())
      SetValid(p.GetValid());
  }

Here is the call graph for this function:

Here is the caller graph for this function:

void Arc::DataPointIndex::SetSecure ( bool  v) [virtual, inherited]

Allow/disallow heavy security during data transfer.

Parameters:
vtrue if allowed (default depends on protocol).

Implements Arc::DataPoint.

Definition at line 244 of file DataPointIndex.cpp.

                                       {
    if (h && *h)
      (*h)->SetSecure(v);
  }
void Arc::DataPoint::SetSize ( const unsigned long long int  val) [virtual, inherited]

Set value of meta-information 'size'.

Definition at line 94 of file DataPoint.cpp.

                                                          {
    size = val;
  }

Here is the caller graph for this function:

void Arc::DataPointIndex::SetTries ( const int  n) [virtual, inherited]

Set number of retries.

Reimplemented from Arc::DataPoint.

Definition at line 136 of file DataPointIndex.cpp.

                                           {
    triesleft = std::max(0, n);
    if (triesleft == 0)
      location = locations.end();
    else if (locations.end() == location)
      location = locations.begin();
    SetHandle();
  }

Here is the call graph for this function:

void Arc::DataPoint::SetValid ( const Time val) [virtual, inherited]

Set value of meta-information 'validity time'.

Definition at line 134 of file DataPoint.cpp.

                                          {
    valid = val;
  }

Here is the caller graph for this function:

DataStatus Arc::DataPointIndex::StartReading ( DataBuffer buffer) [virtual, inherited]

Start reading data from URL.

Separate thread to transfer data will be created. No other operation can be performed while reading is in progress.

Parameters:
bufferoperation will use this buffer to put information into. Should not be destroyed before stop_reading was called and returned.

Implements Arc::DataPoint.

Definition at line 161 of file DataPointIndex.cpp.

                                                            {
    if (!h || !*h)
      return DataStatus::NoLocationError;
    return (*h)->StartReading(buffer);
  }
DataStatus Arc::DataPointIndex::StartWriting ( DataBuffer buffer,
DataCallback space_cb = NULL 
) [virtual, inherited]

Start writing data to URL.

Separate thread to transfer data will be created. No other operation can be performed while writing is in progress.

Parameters:
bufferoperation will use this buffer to get information from. Should not be destroyed before stop_writing was called and returned.
space_cbcallback which is called if there is not enough space to store data. May not implemented for all protocols.

Implements Arc::DataPoint.

Definition at line 167 of file DataPointIndex.cpp.

                                                            {
    if (!h || !*h)
      return DataStatus::NoLocationError;
    return (*h)->StartWriting(buffer, cb);
  }
DataStatus Arc::DataPointIndex::StopReading ( ) [virtual, inherited]

Stop reading.

Must be called after corresponding start_reading method, either after all data is transferred or to cancel transfer. Use buffer object to find out when data is transferred. Must return failure if any happened during transfer.

Implements Arc::DataPoint.

Definition at line 174 of file DataPointIndex.cpp.

                                         {
    if (!h || !*h)
      return DataStatus::NoLocationError;
    return (*h)->StopReading();
  }
DataStatus Arc::DataPointIndex::StopWriting ( ) [virtual, inherited]

Stop writing.

Must be called after corresponding start_writing method, either after all data is transferred or to cancel transfer. Use buffer object to find out when data is transferred. Must return failure if any happened during transfer.

Implements Arc::DataPoint.

Definition at line 180 of file DataPointIndex.cpp.

                                         {
    if (!h || !*h)
      return DataStatus::NoLocationError;
    return (*h)->StopWriting();
  }
std::string Arc::DataPoint::str ( ) const [virtual, inherited]

Returns a string representation of the DataPoint.

Definition at line 53 of file DataPoint.cpp.

                                 {
    return url.str();
  }

Here is the call graph for this function:

Here is the caller graph for this function:

DataStatus Arc::DataPointRLS::Unregister ( bool  all) [virtual]

Index Service unregistration.

Remove information about file registered in Indexing Service.

Parameters:
allif true, information about file itself is (LFN) is removed. Otherwise only particular physical instance is unregistered.

Implements Arc::DataPoint.

Definition at line 789 of file DataPointRLS.cpp.

                                              {
    if (url.Host().empty()) {
      logger.msg(INFO, "RLS URL must contain host");
      return DataStatus::UnregisterError;
    }
    if (!all) {
      if (!LocationValid()) {
        logger.msg(ERROR, "Location is missing");
        return DataStatus::UnregisterError;
      }
      if (CurrentLocation().Protocol() == "se") {
        logger.msg(VERBOSE, "SE location will be unregistered automatically");
        return DataStatus::Success;
      }
    }
    if (!guid_enabled) {
      globus_rls_handle_t *h;
      char errmsg[MAXERRMSG + 32];
      globus_result_t err;
      int errcode;
      globus_list_t *pfns_list;

      err = globus_rls_client_connect
              (const_cast<char*>(url.ConnectionURL().c_str()), &h);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, NULL, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        logger.msg(INFO, "Failed to connect to RLS server: %s", errmsg);
        return DataStatus::UnregisterError;
      }
      // first find all LRC servers storing required information
      globus_list_t *lrcs = NULL;
      globus_rls_string2_t lrc_direct;
      globus_bool_t free_lrcs = GLOBUS_FALSE;
      lrc_direct.s1 = const_cast<char*>(get_path_str(url));
      lrc_direct.s2 = NULL; // for current connection
      int lrc_offset = 0;
      int lrc_limit = 0;
      err = globus_rls_client_rli_get_lrc
              (h, const_cast<char*>(get_path_str(url)),
              &lrc_offset, lrc_limit, &lrcs);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        if (errcode == GLOBUS_RLS_LFN_NEXIST) {
          logger.msg(INFO, "LFN must be already deleted, try LRC anyway");
          lrcs = NULL;
        }
        else if (errcode != GLOBUS_RLS_INVSERVER) {
          logger.msg(INFO, "Failed to retrieve LFN/LRC: %s", errmsg);
          globus_rls_client_close(h);
          return DataStatus::UnregisterError;
        }
        // Probably that is LRC server only.
        globus_list_insert(&lrcs, &lrc_direct);
      }
      else
        free_lrcs = GLOBUS_TRUE;
      err = GLOBUS_SUCCESS;
      // TODO: sort by lrc and cache connections
      DataStatus success = DataStatus::Success;
      for (globus_list_t *p = lrcs; p; p = globus_list_rest(p)) {
        globus_rls_string2_t *str2 =
          (globus_rls_string2_t*)globus_list_first(p);
        char *lrc = str2->s2;
        globus_rls_handle_t *h_;
        if (lrc) {
          err = globus_rls_client_connect(lrc, &h_);
          if (err != GLOBUS_SUCCESS) {
            globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                         GLOBUS_FALSE);
            logger.msg(INFO, "Warning: Failed to connect to LRC at %s: %s",
                       lrc, errmsg);
            success = DataStatus::UnregisterError;
            continue;
          }
        }
        else
          h_ = h; // This server is already connected
        if (all) {
          err = globus_rls_client_lrc_get_pfn
                  (h_, const_cast<char*>(get_path_str(url)),
                  &lrc_offset, lrc_limit, &pfns_list);
          if (err != GLOBUS_SUCCESS) {
            globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                         GLOBUS_FALSE);
            if ((errcode != GLOBUS_RLS_MAPPING_NEXIST) &&
                (errcode != GLOBUS_RLS_LFN_NEXIST) &&
                (errcode != GLOBUS_RLS_PFN_NEXIST)) {
              logger.msg(INFO,
                         "Warning: Failed to retrieve LFN/PFNs from %s: %s",
                         lrc ? lrc : url.ConnectionURL(), errmsg);
              if (lrc)
                globus_rls_client_close(h_);
              success = DataStatus::UnregisterError;
              continue;
            }
            // Probably no such LFN - good, less work to do
            pfns_list = NULL;
          }
          for (globus_list_t *lp = pfns_list; lp; lp = globus_list_rest(lp)) {
            globus_rls_string2_t *str2 =
              (globus_rls_string2_t*)globus_list_first(lp);
            URL pfn(str2->s1);
            if (pfn.Protocol() == "se")
              logger.msg(VERBOSE,
                         "SE location will be unregistered automatically");
            else {
              err = globus_rls_client_lrc_delete
                      (h_, const_cast<char*>(get_path_str(url)), str2->s1);
              if (err != GLOBUS_SUCCESS) {
                globus_rls_client_error_info(err, &errcode, errmsg,
                                             MAXERRMSG + 32, GLOBUS_FALSE);
                if ((errcode != GLOBUS_RLS_MAPPING_NEXIST) &&
                    (errcode != GLOBUS_RLS_LFN_NEXIST) &&
                    (errcode != GLOBUS_RLS_PFN_NEXIST)) {
                  logger.msg(INFO,
                             "Warning: Failed to delete LFN/PFN from %s: %s",
                             lrc ? lrc : url.ConnectionURL(), errmsg);
                  if (lrc)
                    globus_rls_client_close(h_);
                  success = DataStatus::UnregisterError;
                  continue;
                }
              }
            }
          }
          if (pfns_list)
            globus_rls_client_free_list(pfns_list);
        }
        else { // ! all
          err = globus_rls_client_lrc_delete
                  (h_, const_cast<char*>(get_path_str(url)),
                  const_cast<char*>(CurrentLocation().str().c_str()));
          if (err != GLOBUS_SUCCESS) {
            globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                         GLOBUS_FALSE);
            if ((errcode != GLOBUS_RLS_MAPPING_NEXIST) &&
                (errcode != GLOBUS_RLS_LFN_NEXIST) &&
                (errcode != GLOBUS_RLS_PFN_NEXIST)) {
              logger.msg(INFO, "Warning: Failed to delete LFN/PFN from %s: %s",
                         lrc, errmsg);
              if (lrc)
                globus_rls_client_close(h_);
              success = DataStatus::UnregisterError;
              continue;
            }
          }
        }
        if (lrc)
          globus_rls_client_close(h_);
      }
      globus_rls_client_close(h);
      if (free_lrcs)
        globus_rls_client_free_list(lrcs);
      else
        globus_list_free(lrcs);
      if (!success) {
        registered = false;
        ClearLocations();
      }
      return success;
    }
    else { // guid_enabled
      std::list<URL> rlis;
      std::list<URL> lrcs;
      rlis.push_back(url.ConnectionURL());
      lrcs.push_back(url.ConnectionURL());
      meta_unregister_rls_t arg(*this, all);
      rls_find_lrcs(rlis, lrcs, true, false,
                    &meta_unregister_callback, (void*)&arg);
      if (!arg.success) {
        registered = false;
        ClearLocations();
      }
      return arg.success;
    }
  }

Here is the call graph for this function:

bool Arc::DataPointRLS::UnregisterCallback ( globus_rls_handle_t *  h,
const URL url,
void *  arg 
)

Definition at line 689 of file DataPointRLS.cpp.

                                                                      {
    bool& all(((meta_unregister_rls_t*)arg)->all);
    DataStatus& success(((meta_unregister_rls_t*)arg)->success);
    std::string& guid(((meta_unregister_rls_t*)arg)->guid);

    int lrc_offset = 0;
    int lrc_limit = 0;
    globus_result_t err;
    int errcode;
    char errmsg[MAXERRMSG + 32];
    globus_list_t *pfns_list;
    if (guid_enabled && guid.empty()) {
      // map lfn->guid (only once)
      globus_rls_attribute_t opr;
      opr.type = globus_rls_attr_type_str;
      opr.val.s = const_cast<char*>(get_path_str(url));
      int off = 0;
      globus_list_t *guids = NULL;
      err = globus_rls_client_lrc_attr_search(h, const_cast<char*>("lfn"),
                                              globus_rls_obj_lrc_lfn,
                                              globus_rls_attr_op_eq,
                                              &opr, NULL, &off, 1, &guids);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        logger.msg(DEBUG, "Failed to find GUID for specified LFN in %s: %s",
                   rlsurl.str(), errmsg);
        return true;
      }
      if (!guids) {
        logger.msg(DEBUG, "There is no GUID for specified LFN in %s",
                   rlsurl.str());
        return true;
      }
      globus_rls_attribute_object_t *obattr =
        (globus_rls_attribute_object_t*)globus_list_first(guids);
      guid = obattr->key;
      globus_rls_client_free_list(guids);
    }
    if (all) {
      if (!guid.empty())
        err = globus_rls_client_lrc_get_pfn
                (h, const_cast<char*>(guid.c_str()),
                &lrc_offset, lrc_limit, &pfns_list);
      else
        err = globus_rls_client_lrc_get_pfn
                (h, const_cast<char*>(get_path_str(url)),
                &lrc_offset, lrc_limit, &pfns_list);
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        logger.msg(INFO, "Warning: Failed to retrieve LFN/PFNs from %s: %s",
                   rlsurl.str(), errmsg);
        success = DataStatus::UnregisterError;
        return true;
      }
      for (globus_list_t *lp = pfns_list; lp; lp = globus_list_rest(lp)) {
        globus_rls_string2_t *str2 =
          (globus_rls_string2_t*)globus_list_first(lp);
        URL pfn(str2->s2);
        if (pfn.Protocol() == "se")
          logger.msg(VERBOSE, "SE location will be unregistered automatically");
        else {
          err = globus_rls_client_lrc_delete(h, str2->s1, str2->s2);
          if (err != GLOBUS_SUCCESS) {
            globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                         GLOBUS_FALSE);
            if ((errcode != GLOBUS_RLS_MAPPING_NEXIST) &&
                (errcode != GLOBUS_RLS_LFN_NEXIST) &&
                (errcode != GLOBUS_RLS_PFN_NEXIST)) {
              logger.msg(INFO, "Warning: Failed to delete LFN/PFN from %s: %s",
                         rlsurl.str(), errmsg);
              success = DataStatus::UnregisterError;
              continue;
            }
          }
        }
      }
      globus_rls_client_free_list(pfns_list);
    }
    else { // ! all
      err = globus_rls_client_lrc_delete
              (h, const_cast<char*>(get_path_str(url)),
              const_cast<char*>(CurrentLocation().str().c_str()));
      if (err != GLOBUS_SUCCESS) {
        globus_rls_client_error_info(err, &errcode, errmsg, MAXERRMSG + 32,
                                     GLOBUS_FALSE);
        if ((errcode != GLOBUS_RLS_MAPPING_NEXIST) &&
            (errcode != GLOBUS_RLS_LFN_NEXIST) &&
            (errcode != GLOBUS_RLS_PFN_NEXIST)) {
          logger.msg(INFO, "Warning: Failed to delete LFN/PFN from %s: %s",
                     rlsurl.str(), errmsg);
          success = DataStatus::UnregisterError;
        }
      }
    }
    return true;
  }

Here is the call graph for this function:

bool Arc::DataPointIndex::WriteOutOfOrder ( ) [virtual, inherited]

Returns true if URL can accept scattered data for writing operation.

Implements Arc::DataPoint.

Definition at line 227 of file DataPointIndex.cpp.

                                       {
    if (!h || !*h)
      return false;
    return (*h)->WriteOutOfOrder();
  }

Member Data Documentation

bool Arc::DataPoint::cache [protected, inherited]

Definition at line 325 of file DataPoint.h.

std::string Arc::DataPoint::checksum [protected, inherited]

Definition at line 320 of file DataPoint.h.

Time Arc::DataPoint::created [protected, inherited]

Definition at line 321 of file DataPoint.h.

DataStatus Arc::DataPoint::failure_code [protected, inherited]

Definition at line 324 of file DataPoint.h.

Definition at line 38 of file DataPointRLS.h.

Logger Arc::DataPointRLS::logger [static, private]

Reimplemented from Arc::DataPoint.

Definition at line 37 of file DataPointRLS.h.

std::string Arc::DataPointRLS::pfn_path [private]

Definition at line 39 of file DataPointRLS.h.

bool Arc::DataPointIndex::registered [protected, inherited]

Definition at line 72 of file DataPointIndex.h.

bool Arc::DataPointIndex::resolved [protected, inherited]

Definition at line 71 of file DataPointIndex.h.

unsigned long long int Arc::DataPoint::size [protected, inherited]

Definition at line 319 of file DataPoint.h.

int Arc::DataPoint::triesleft [protected, inherited]

Definition at line 323 of file DataPoint.h.

const URL& Arc::DataPoint::url [protected, inherited]

Definition at line 315 of file DataPoint.h.

const UserConfig& Arc::DataPoint::usercfg [protected, inherited]

Definition at line 316 of file DataPoint.h.

Time Arc::DataPoint::valid [protected, inherited]

Definition at line 322 of file DataPoint.h.

std::list<std::string> Arc::DataPoint::valid_url_options [protected, inherited]

Subclasses should add their own specific options to this list.

Definition at line 327 of file DataPoint.h.


The documentation for this class was generated from the following files: