Back to index

python3.2  3.2.2
Public Member Functions | Public Attributes
urllib.robotparser.Entry Class Reference

List of all members.

Public Member Functions

def __init__
def __str__
def applies_to
def allowance

Public Attributes

 useragents
 rulelines

Detailed Description

An entry has one or more user-agents and zero or more rulelines

Definition at line 170 of file robotparser.py.


Constructor & Destructor Documentation

Definition at line 172 of file robotparser.py.

00172 
00173     def __init__(self):
00174         self.useragents = []
00175         self.rulelines = []

Here is the caller graph for this function:


Member Function Documentation

Definition at line 176 of file robotparser.py.

00176 
00177     def __str__(self):
00178         ret = []
00179         for agent in self.useragents:
00180             ret.extend(["User-agent: ", agent, "\n"])
00181         for line in self.rulelines:
00182             ret.extend([str(line), "\n"])
00183         return ''.join(ret)

def urllib.robotparser.Entry.allowance (   self,
  filename 
)
Preconditions:
- our agent applies to this entry
- filename is URL decoded

Definition at line 197 of file robotparser.py.

00197 
00198     def allowance(self, filename):
00199         """Preconditions:
00200         - our agent applies to this entry
00201         - filename is URL decoded"""
00202         for line in self.rulelines:
00203             if line.applies_to(filename):
00204                 return line.allowance
00205         return True
def urllib.robotparser.Entry.applies_to (   self,
  useragent 
)
check if this entry applies to the specified agent

Definition at line 184 of file robotparser.py.

00184 
00185     def applies_to(self, useragent):
00186         """check if this entry applies to the specified agent"""
00187         # split the name token and make it lower case
00188         useragent = useragent.split("/")[0].lower()
00189         for agent in self.useragents:
00190             if agent == '*':
00191                 # we have the catch-all agent
00192                 return True
00193             agent = agent.lower()
00194             if agent in useragent:
00195                 return True
00196         return False


Member Data Documentation

Definition at line 174 of file robotparser.py.

Definition at line 173 of file robotparser.py.


The documentation for this class was generated from the following file: