Back to index

python3.2  3.2.2
Public Member Functions
test.test_tokenize.TestTokenize Class Reference
Inheritance diagram for test.test_tokenize.TestTokenize:
Inheritance graph
[legend]
Collaboration diagram for test.test_tokenize.TestTokenize:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def test_tokenize

Detailed Description

Definition at line 884 of file test_tokenize.py.


Member Function Documentation

Definition at line 886 of file test_tokenize.py.

00886 
00887     def test_tokenize(self):
00888         import tokenize as tokenize_module
00889         encoding = object()
00890         encoding_used = None
00891         def mock_detect_encoding(readline):
00892             return encoding, ['first', 'second']
00893 
00894         def mock__tokenize(readline, encoding):
00895             nonlocal encoding_used
00896             encoding_used = encoding
00897             out = []
00898             while True:
00899                 next_line = readline()
00900                 if next_line:
00901                     out.append(next_line)
00902                     continue
00903                 return out
00904 
00905         counter = 0
00906         def mock_readline():
00907             nonlocal counter
00908             counter += 1
00909             if counter == 5:
00910                 return b''
00911             return counter
00912 
00913         orig_detect_encoding = tokenize_module.detect_encoding
00914         orig__tokenize = tokenize_module._tokenize
00915         tokenize_module.detect_encoding = mock_detect_encoding
00916         tokenize_module._tokenize = mock__tokenize
00917         try:
00918             results = tokenize(mock_readline)
00919             self.assertEqual(list(results), ['first', 'second', 1, 2, 3, 4])
00920         finally:
00921             tokenize_module.detect_encoding = orig_detect_encoding
00922             tokenize_module._tokenize = orig__tokenize
00923 
00924         self.assertTrue(encoding_used, encoding)
00925 

Here is the call graph for this function:


The documentation for this class was generated from the following file: