Back to index

python3.2  3.2.2
Classes | Functions | Variables
test.test_robotparser Namespace Reference

Classes

class  RobotTestCase
class  NetworkTestCase

Functions

def RobotTest
def test_main

Variables

tuple tests = unittest.TestSuite()
string doc
list good = ['/','/test.html']
list bad = ['/cyberworld/map/index.html','/tmp/xxx','/foo.html']

Function Documentation

def test.test_robotparser.RobotTest (   index,
  robots_txt,
  good_urls,
  bad_urls,
  agent = "test_robotparser" 
)

Definition at line 37 of file test_robotparser.py.

00037 
00038               agent="test_robotparser"):
00039 
00040     lines = io.StringIO(robots_txt).readlines()
00041     parser = urllib.robotparser.RobotFileParser()
00042     parser.parse(lines)
00043     for url in good_urls:
00044         tests.addTest(RobotTestCase(index, parser, url, 1, agent))
00045     for url in bad_urls:
00046         tests.addTest(RobotTestCase(index, parser, url, 0, agent))
00047 
00048 # Examples from http://www.robotstxt.org/wc/norobots.html (fetched 2002)
00049 
# 1.

Definition at line 272 of file test_robotparser.py.

00272 
00273 def test_main():
00274     support.run_unittest(NetworkTestCase)
00275     support.run_unittest(tests)


Variable Documentation

list test.test_robotparser.bad = ['/cyberworld/map/index.html','/tmp/xxx','/foo.html']

Definition at line 58 of file test_robotparser.py.

Initial value:
00001 """
00002 User-agent: *
00003 Disallow: /cyberworld/map/ # This is an infinite virtual URL space
00004 Disallow: /tmp/ # these will soon disappear
00005 Disallow: /foo.html
00006 """

Definition at line 50 of file test_robotparser.py.

list test.test_robotparser.good = ['/','/test.html']

Definition at line 57 of file test_robotparser.py.

tuple test.test_robotparser.tests = unittest.TestSuite()

Definition at line 34 of file test_robotparser.py.