Back to index

lightning-sunbird  0.9+nobinonly
Public Types | Public Member Functions | Static Public Member Functions | Protected Member Functions | Static Protected Member Functions | Protected Attributes | Friends
CToken Class Reference

Token objects represent sequences of characters as they are consumed from the input stream (URL). More...

#include <nsToken.h>

Inheritance diagram for CToken:
Inheritance graph
[legend]

List of all members.

Public Types

enum  eTokenOrigin { eSource, eResidualStyle }

Public Member Functions

virtual ~CToken ()
 destructor gess5/11/98
void AddRef ()
 Make a note on number of times you have been referenced harishd 08/02/00.
void Release (nsFixedSizeAllocator &aArenaPool)
 Free yourself if no one is holding you.
 CToken (PRInt32 aTag=0)
 Default constructor gess7/21/98.
virtual const nsSubstringGetStringValue (void)=0
 Retrieve string value of the token gess5/11/98.
virtual void GetSource (nsString &anOutputString)
 Get string of full contents, suitable for debug dump.
virtual void AppendSourceTo (nsAString &anOutputString)
 harishd 03/23/00
void SetTypeID (PRInt32 aValue)
 Sets the ordinal value of this token (not currently used) gess5/11/98.
virtual PRInt32 GetTypeID (void)
 Getter which retrieves the current ordinal value for this token gess5/11/98.
virtual PRInt16 GetAttributeCount (void)
 Getter which retrieves the current attribute count for this token gess5/11/98.
virtual nsresult Consume (PRUnichar aChar, nsScanner &aScanner, PRInt32 aMode)
 Causes token to consume data from given scanner.
virtual PRInt32 GetTokenType (void)
 Getter which retrieves type of token gess5/11/98.
virtual PRBool IsWellFormed (void) const
 For tokens who care, this can tell us whether the token is well formed or not.
virtual PRBool IsEmpty (void)
virtual void SetEmpty (PRBool aValue)
 If aValue is TRUE then the token represents a short-hand tag.
PRInt32 GetNewlineCount ()
void SetNewlineCount (PRInt32 aCount)
PRInt32 GetLineNumber ()
void SetLineNumber (PRInt32 aLineNumber)
void SetInError (PRBool aInError)
PRBool IsInError ()
void SetAttributeCount (PRInt16 aValue)
virtual void SelfTest (void)
 perform self test.

Static Public Member Functions

static void Destroy (CToken *aToken, nsFixedSizeAllocator &aArenaPool)
 Destroy a token.
static int GetTokenCount ()

Protected Member Functions

virtual size_t SizeOf () const =0
 Returns the size of the token object.

Static Protected Member Functions

static voidoperator new (size_t aSize, nsFixedSizeAllocator &anArena) CPP_THROW_NEW
 harishd 08/01/00
static void operator delete (void *, size_t)
 Hide operator delete; clients should use Destroy() instead.

Protected Attributes

PRInt32 mTypeID
PRInt32 mUseCount
PRInt32 mNewlineCount
PRUint32 mLineNumber: 31
PRUint32 mInError: 1
PRInt16 mAttrCount

Friends

class nsTokenAllocator

Detailed Description

Token objects represent sequences of characters as they are consumed from the input stream (URL).

While they're pretty general in nature, we use subclasses (found in nsHTMLTokens.h) to define <start>, </end>, <text>, <comment>, <&entity>, <newline>, and <whitespace> tokens.

gess 3/25/98

Definition at line 106 of file nsToken.h.


Member Enumeration Documentation

Enumerator:
eSource 
eResidualStyle 

Definition at line 109 of file nsToken.h.


Constructor & Destructor Documentation

CToken::~CToken ( ) [virtual]

destructor gess5/11/98

Decstructor.

gess 3/25/98

Definition at line 91 of file nsToken.cpp.

                {
  // Tokens are allocated through the arena ( not heap allocated..yay ).
  // We, therefore, don't need this macro anymore..
#ifdef MATCH_CTOR_DTOR 
  MOZ_COUNT_DTOR(CToken);
#endif
  ++DelTokenCount;
  mUseCount=0;
}

Here is the caller graph for this function:

CToken::CToken ( PRInt32  aTag = 0)

Default constructor gess7/21/98.

Default constructor.

gess 7/21/98

Definition at line 63 of file nsToken.cpp.

                           {
  // Tokens are allocated through the arena ( not heap allocated..yay ).
  // We, therefore, don't need this macro anymore..
#ifdef MATCH_CTOR_DTOR 
  MOZ_COUNT_CTOR(CToken);
#endif 
  mAttrCount=0;
  mNewlineCount=0;
  mLineNumber = 0;
  mInError = PR_FALSE;
  mTypeID=aTag;
  // Note that the use count starts with 1 instead of 0. This
  // is because of the assumption that any token created is in
  // use and therefore does not require an explicit addref, or
  // rather IF_HOLD. This, also, will make sure that tokens created 
  // on the stack do not accidently hit the arena recycler.
  mUseCount=1;

#ifdef NS_DEBUG
  ++TokenCount;
#endif
}

Member Function Documentation

void CToken::AddRef ( void  ) [inline]

Make a note on number of times you have been referenced harishd 08/02/00.

Definition at line 154 of file nsToken.h.

{ ++mUseCount; }
void CToken::AppendSourceTo ( nsAString &  anOutputString) [virtual]

harishd 03/23/00

harishd 3/23/00

Returns:
reference to string containing string value

Reimplemented in CAttributeToken, CEntityToken, CCommentToken, CEndToken, and CStartToken.

Definition at line 130 of file nsToken.cpp.

                                                     {
  anOutputString.Append(GetStringValue());
}

Here is the call graph for this function:

Here is the caller graph for this function:

nsresult CToken::Consume ( PRUnichar  aChar,
nsScanner aScanner,
PRInt32  aMode 
) [virtual]

Causes token to consume data from given scanner.

Virtual method used to tell this toke to consume his valid chars.

Note that behavior varies wildly between CToken subclasses. gess5/11/98

Parameters:
aChar-- most recent char consumed
aScanner-- input source where token should get data
Returns:
error code (0 means ok)

gess 3/25/98

Parameters:
aChar-- first char in sequence
aScanner-- object to retrieve data from
Returns:
int error code

Reimplemented in CDoctypeDeclToken, CInstructionToken, CNewlineToken, CAttributeToken, CMarkupDeclToken, CCDATASectionToken, CTextToken, CWhitespaceToken, CEntityToken, CCommentToken, CEndToken, and CStartToken.

Definition at line 111 of file nsToken.cpp.

                                                                          {
  nsresult result=NS_OK;
  return result;
}

Here is the caller graph for this function:

static void CToken::Destroy ( CToken aToken,
nsFixedSizeAllocator &  aArenaPool 
) [inline, static]

Destroy a token.

Definition at line 143 of file nsToken.h.

    {
      size_t sz = aToken->SizeOf();
      aToken->~CToken();
      aArenaPool.Free(aToken, sz);
    }

Here is the call graph for this function:

Here is the caller graph for this function:

Getter which retrieves the current attribute count for this token gess5/11/98.

Retrieves copy of attr count for this token.

Returns:
current attribute count

gess 3/25/98

Returns:
int containing attribute count

Definition at line 151 of file nsToken.cpp.

                                      {
  return mAttrCount;
}

Here is the caller graph for this function:

Definition at line 257 of file nsToken.h.

    { 
      return mLineNumber;
    }

Here is the caller graph for this function:

Definition at line 247 of file nsToken.h.

    { 
      return mNewlineCount; 
    }

Here is the caller graph for this function:

void CToken::GetSource ( nsString anOutputString) [virtual]

Get string of full contents, suitable for debug dump.

It should look exactly like the input source. gess5/11/98

Returns:
reference to string containing string value

Reimplemented in CAttributeToken, CEntityToken, CEndToken, and CStartToken.

Definition at line 122 of file nsToken.cpp.

                                               {
  anOutputString.Assign(GetStringValue());
}

Here is the call graph for this function:

Here is the caller graph for this function:

virtual const nsSubstring& CToken::GetStringValue ( void  ) [pure virtual]

Retrieve string value of the token gess5/11/98.

Returns:
reference to string containing string value

Implemented in CDoctypeDeclToken, CInstructionToken, CNewlineToken, CAttributeToken, CMarkupDeclToken, CCDATASectionToken, CTextToken, CWhitespaceToken, CEntityToken, CCommentToken, CIndirectTextToken, CEndToken, and CStartToken.

Here is the caller graph for this function:

int CToken::GetTokenCount ( ) [static]

Definition at line 49 of file nsToken.cpp.

Getter which retrieves type of token gess5/11/98.

Retrieve type of token.

Returns:
int containing token type

This class returns -1, but subclasses return something more meaningful.

gess 3/25/98

Returns:
int value containing token type.

Reimplemented in CDoctypeDeclToken, CInstructionToken, CNewlineToken, CAttributeToken, CMarkupDeclToken, CCDATASectionToken, CTextToken, CWhitespaceToken, CEntityToken, CCommentToken, CEndToken, and CStartToken.

Definition at line 163 of file nsToken.cpp.

                                 {
  return -1;
}

Here is the caller graph for this function:

PRInt32 CToken::GetTypeID ( void  ) [virtual]

Getter which retrieves the current ordinal value for this token gess5/11/98.

Retrieves copy of internal ordinal value.

Returns:
current ordinal value

This method is deprecated, and will soon be going away.

gess 3/25/98

Returns:
int containing ordinal value

Reimplemented in CEndToken, and CStartToken.

Definition at line 141 of file nsToken.cpp.

                              {
  return mTypeID;
}

Here is the caller graph for this function:

virtual PRBool CToken::IsEmpty ( void  ) [inline, virtual]

Reimplemented in CStartToken.

Definition at line 240 of file nsToken.h.

{ return PR_FALSE; }
PRBool CToken::IsInError ( ) [inline]

Definition at line 272 of file nsToken.h.

    {
      return mInError;
    }

Here is the caller graph for this function:

virtual PRBool CToken::IsWellFormed ( void  ) const [inline, virtual]

For tokens who care, this can tell us whether the token is well formed or not.

gess 8/30/00

Returns:
PR_FALSE; subclasses MUST override if they care.

Reimplemented in CStartToken.

Definition at line 238 of file nsToken.h.

{return PR_FALSE;}

Here is the caller graph for this function:

static void CToken::operator delete ( void ,
size_t   
) [inline, static, protected]

Hide operator delete; clients should use Destroy() instead.

Definition at line 131 of file nsToken.h.

{}
static void* CToken::operator new ( size_t  aSize,
nsFixedSizeAllocator &  anArena 
) [inline, static, protected]

harishd 08/01/00

Parameters:
aSize-
aArena- Allocate memory from this pool.

Definition at line 123 of file nsToken.h.

    {
      return anArena.Alloc(aSize);
    }
void CToken::Release ( nsFixedSizeAllocator &  aArenaPool) [inline]

Free yourself if no one is holding you.

harishd 08/02/00

Definition at line 160 of file nsToken.h.

                                                   {
      if(--mUseCount==0)
        Destroy(this, aArenaPool);
    }

Here is the call graph for this function:

void CToken::SelfTest ( void  ) [virtual]

perform self test.

gess 3/25/98

gess5/11/98

Definition at line 172 of file nsToken.cpp.

                          {
#ifdef _DEBUG
#endif
}
void CToken::SetAttributeCount ( PRInt16  aValue) [inline]

Definition at line 277 of file nsToken.h.

Here is the caller graph for this function:

virtual void CToken::SetEmpty ( PRBool  aValue) [inline, virtual]

If aValue is TRUE then the token represents a short-hand tag.

Reimplemented in CStartToken.

Definition at line 245 of file nsToken.h.

{ return ; }

Here is the call graph for this function:

Here is the caller graph for this function:

void CToken::SetInError ( PRBool  aInError) [inline]

Definition at line 267 of file nsToken.h.

    {
      mInError = aInError;
    }

Here is the caller graph for this function:

void CToken::SetLineNumber ( PRInt32  aLineNumber) [inline]

Definition at line 262 of file nsToken.h.

    { 
      mLineNumber = mLineNumber == 0 ? aLineNumber : mLineNumber;
    }
void CToken::SetNewlineCount ( PRInt32  aCount) [inline]

Definition at line 252 of file nsToken.h.

Here is the caller graph for this function:

void CToken::SetTypeID ( PRInt32  aValue) [inline]

Sets the ordinal value of this token (not currently used) gess5/11/98.

Parameters:
valueis the new ord value for this token

Definition at line 196 of file nsToken.h.

                                   {
      mTypeID = aValue;
    }

Here is the caller graph for this function:

virtual size_t CToken::SizeOf ( ) const [protected, pure virtual]

Returns the size of the token object.

Here is the caller graph for this function:


Friends And Related Function Documentation

friend class nsTokenAllocator [friend]

Definition at line 115 of file nsToken.h.


Member Data Documentation

Definition at line 300 of file nsToken.h.

Definition at line 299 of file nsToken.h.

Definition at line 298 of file nsToken.h.

Definition at line 297 of file nsToken.h.

PRInt32 CToken::mTypeID [protected]

Definition at line 295 of file nsToken.h.

Definition at line 296 of file nsToken.h.


The documentation for this class was generated from the following files: