Back to index

python3.2  3.2.2
Public Member Functions | Public Attributes
lib2to3.pgen2.grammar.Grammar Class Reference
Inheritance diagram for lib2to3.pgen2.grammar.Grammar:
Inheritance graph
[legend]
Collaboration diagram for lib2to3.pgen2.grammar.Grammar:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def __init__
def dump
def load
def copy
def report

Public Attributes

 symbol2number
 number2symbol
 states
 dfas
 labels
 keywords
 tokens
 symbol2label
 start
_PyObject_HEAD_EXTRA Py_ssize_t ob_refcnt
struct _typeobjectob_type

Detailed Description

Pgen parsing tables tables conversion class.

Once initialized, this class supplies the grammar tables for the
parsing engine implemented by parse.py.  The parsing engine
accesses the instance variables directly.  The class here does not
provide initialization of the tables; several subclasses exist to
do this (see the conv and pgen modules).

The load() method reads the tables from a pickle file, which is
much faster than the other ways offered by subclasses.  The pickle
file is written by calling dump() (after loading the grammar
tables using a subclass).  The report() method prints a readable
representation of the tables to stdout, for debugging.

The instance variables are as follows:

symbol2number -- a dict mapping symbol names to numbers.  Symbol
                 numbers are always 256 or higher, to distinguish
                 them from token numbers, which are between 0 and
                 255 (inclusive).

number2symbol -- a dict mapping numbers to symbol names;
                 these two are each other's inverse.

states        -- a list of DFAs, where each DFA is a list of
                 states, each state is is a list of arcs, and each
                 arc is a (i, j) pair where i is a label and j is
                 a state number.  The DFA number is the index into
                 this list.  (This name is slightly confusing.)
                 Final states are represented by a special arc of
                 the form (0, j) where j is its own state number.

dfas          -- a dict mapping symbol numbers to (DFA, first)
                 pairs, where DFA is an item from the states list
                 above, and first is a set of tokens that can
                 begin this grammar rule (represented by a dict
                 whose values are always 1).

labels        -- a list of (x, y) pairs where x is either a token
                 number or a symbol number, and y is either None
                 or a string; the strings are keywords.  The label
                 number is the index in this list; label numbers
                 are used to mark state transitions (arcs) in the
                 DFAs.

start         -- the number of the grammar's start symbol.

keywords      -- a dict mapping keyword strings to arc labels.

tokens        -- a dict mapping token numbers to arc labels.

Definition at line 22 of file grammar.py.


Constructor & Destructor Documentation

Definition at line 76 of file grammar.py.

00076 
00077     def __init__(self):
00078         self.symbol2number = {}
00079         self.number2symbol = {}
00080         self.states = []
00081         self.dfas = {}
00082         self.labels = [(0, "EMPTY")]
00083         self.keywords = {}
00084         self.tokens = {}
00085         self.symbol2label = {}
00086         self.start = 256

Here is the caller graph for this function:


Member Function Documentation

Copy the grammar.

Definition at line 100 of file grammar.py.

00100 
00101     def copy(self):
00102         """
00103         Copy the grammar.
00104         """
00105         new = self.__class__()
00106         for dict_attr in ("symbol2number", "number2symbol", "dfas", "keywords",
00107                           "tokens", "symbol2label"):
00108             setattr(new, dict_attr, getattr(self, dict_attr).copy())
00109         new.labels = self.labels[:]
00110         new.states = self.states[:]
00111         new.start = self.start
00112         return new

Here is the call graph for this function:

def lib2to3.pgen2.grammar.Grammar.dump (   self,
  filename 
)
Dump the grammar tables to a pickle file.

Definition at line 87 of file grammar.py.

00087 
00088     def dump(self, filename):
00089         """Dump the grammar tables to a pickle file."""
00090         f = open(filename, "wb")
00091         pickle.dump(self.__dict__, f, 2)
00092         f.close()

Here is the call graph for this function:

def lib2to3.pgen2.grammar.Grammar.load (   self,
  filename 
)
Load the grammar tables from a pickle file.

Definition at line 93 of file grammar.py.

00093 
00094     def load(self, filename):
00095         """Load the grammar tables from a pickle file."""
00096         f = open(filename, "rb")
00097         d = pickle.load(f)
00098         f.close()
00099         self.__dict__.update(d)

Here is the call graph for this function:

Dump the grammar tables to standard output, for debugging.

Definition at line 113 of file grammar.py.

00113 
00114     def report(self):
00115         """Dump the grammar tables to standard output, for debugging."""
00116         from pprint import pprint
00117         print("s2n")
00118         pprint(self.symbol2number)
00119         print("n2s")
00120         pprint(self.number2symbol)
00121         print("states")
00122         pprint(self.states)
00123         print("dfas")
00124         pprint(self.dfas)
00125         print("labels")
00126         pprint(self.labels)
00127         print("start", self.start)
00128 
00129 
00130 # Map from operator to number (since tokenize doesn't do this)

Here is the caller graph for this function:


Member Data Documentation

Reimplemented in lib2to3.pgen2.conv.Converter.

Definition at line 80 of file grammar.py.

Reimplemented in lib2to3.pgen2.conv.Converter.

Definition at line 82 of file grammar.py.

Reimplemented in lib2to3.pgen2.conv.Converter.

Definition at line 81 of file grammar.py.

Reimplemented in lib2to3.pgen2.conv.Converter.

Definition at line 78 of file grammar.py.

Definition at line 107 of file object.h.

struct _typeobject* _object::ob_type [inherited]

Definition at line 108 of file object.h.

Reimplemented in lib2to3.pgen2.conv.Converter.

Definition at line 85 of file grammar.py.

Reimplemented in lib2to3.pgen2.conv.Converter.

Definition at line 79 of file grammar.py.

Definition at line 84 of file grammar.py.

Reimplemented in lib2to3.pgen2.conv.Converter.

Definition at line 77 of file grammar.py.

Reimplemented in lib2to3.pgen2.conv.Converter.

Definition at line 83 of file grammar.py.


The documentation for this class was generated from the following file: