Back to index

python3.2  3.2.2
Public Member Functions | Public Attributes | Private Member Functions | Private Attributes
_pyio.BufferedReader Class Reference
Inheritance diagram for _pyio.BufferedReader:
Inheritance graph
[legend]
Collaboration diagram for _pyio.BufferedReader:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def __init__
def read
def peek
def read1
def tell
def seek
 Positioning ###.
def truncate
def flush
 Flush and close ###.
def close
def detach
def seekable
 Inquiries ###.
def readable
def writable
def raw
def closed
def name
def mode
def __getstate__
def __repr__
def fileno
 Lower-level APIs ###.
def isatty
def readinto
def write
def __enter__
 Context manager ###.
def __exit__
def readline
 Readline[s] and writelines ###.
def __iter__
def __next__
def readlines
def writelines
def __new__
def register
def __instancecheck__
def __subclasscheck__

Public Attributes

 buffer_size

Private Member Functions

def _reset_read_buf
def _read_unlocked
def _peek_unlocked

Private Attributes

 _read_lock
 _read_buf
 _read_pos

Detailed Description

BufferedReader(raw[, buffer_size])

A buffer for a readable, sequential BaseRawIO object.

The constructor creates a BufferedReader for the given readable raw
stream and buffer_size. If buffer_size is omitted, DEFAULT_BUFFER_SIZE
is used.

Definition at line 897 of file _pyio.py.


Constructor & Destructor Documentation

def _pyio.BufferedReader.__init__ (   self,
  raw,
  buffer_size = DEFAULT_BUFFER_SIZE 
)
Create a new buffered reader using the given readable raw IO object.

Definition at line 908 of file _pyio.py.

00908 
00909     def __init__(self, raw, buffer_size=DEFAULT_BUFFER_SIZE):
00910         """Create a new buffered reader using the given readable raw IO object.
00911         """
00912         if not raw.readable():
00913             raise IOError('"raw" argument must be readable.')
00914 
00915         _BufferedIOMixin.__init__(self, raw)
00916         if buffer_size <= 0:
00917             raise ValueError("invalid buffer size")
00918         self.buffer_size = buffer_size
00919         self._reset_read_buf()
00920         self._read_lock = Lock()

Here is the caller graph for this function:


Member Function Documentation

def _pyio.IOBase.__enter__ (   self) [inherited]

Context manager ###.

Context management protocol.  Returns self (an instance of IOBase).

Definition at line 419 of file _pyio.py.

00419 
00420     def __enter__(self):  # That's a forward reference
00421         """Context management protocol.  Returns self (an instance of IOBase)."""
00422         self._checkClosed()
00423         return self

Here is the call graph for this function:

def _pyio.IOBase.__exit__ (   self,
  args 
) [inherited]
Context management protocol.  Calls close()

Definition at line 424 of file _pyio.py.

00424 
00425     def __exit__(self, *args):
00426         """Context management protocol.  Calls close()"""
00427         self.close()

Here is the call graph for this function:

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.__getstate__ (   self) [inherited]

Definition at line 761 of file _pyio.py.

00761 
00762     def __getstate__(self):
00763         raise TypeError("can not serialize a '{0}' object"
00764                         .format(self.__class__.__name__))

def abc.ABCMeta.__instancecheck__ (   cls,
  instance 
) [inherited]
Override for isinstance(instance, cls).

Definition at line 158 of file abc.py.

00158 
00159     def __instancecheck__(cls, instance):
00160         """Override for isinstance(instance, cls)."""
00161         # Inline the cache checking
00162         subclass = instance.__class__
00163         if subclass in cls._abc_cache:
00164             return True
00165         subtype = type(instance)
00166         if subtype is subclass:
00167             if (cls._abc_negative_cache_version ==
00168                 ABCMeta._abc_invalidation_counter and
00169                 subclass in cls._abc_negative_cache):
00170                 return False
00171             # Fall back to the subclass check.
00172             return cls.__subclasscheck__(subclass)
00173         return any(cls.__subclasscheck__(c) for c in {subclass, subtype})

Here is the call graph for this function:

def _pyio.IOBase.__iter__ (   self) [inherited]

Definition at line 486 of file _pyio.py.

00486 
00487     def __iter__(self):
00488         self._checkClosed()
00489         return self

Here is the call graph for this function:

def abc.ABCMeta.__new__ (   mcls,
  name,
  bases,
  namespace 
) [inherited]

Definition at line 116 of file abc.py.

00116 
00117     def __new__(mcls, name, bases, namespace):
00118         cls = super().__new__(mcls, name, bases, namespace)
00119         # Compute set of abstract method names
00120         abstracts = {name
00121                      for name, value in namespace.items()
00122                      if getattr(value, "__isabstractmethod__", False)}
00123         for base in bases:
00124             for name in getattr(base, "__abstractmethods__", set()):
00125                 value = getattr(cls, name, None)
00126                 if getattr(value, "__isabstractmethod__", False):
00127                     abstracts.add(name)
00128         cls.__abstractmethods__ = frozenset(abstracts)
00129         # Set up inheritance registry
00130         cls._abc_registry = WeakSet()
00131         cls._abc_cache = WeakSet()
00132         cls._abc_negative_cache = WeakSet()
00133         cls._abc_negative_cache_version = ABCMeta._abc_invalidation_counter
00134         return cls

Here is the call graph for this function:

def _pyio.IOBase.__next__ (   self) [inherited]

Reimplemented in _pyio.TextIOWrapper.

Definition at line 490 of file _pyio.py.

00490 
00491     def __next__(self):
00492         line = self.readline()
00493         if not line:
00494             raise StopIteration
00495         return line

Here is the call graph for this function:

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.__repr__ (   self) [inherited]

Definition at line 765 of file _pyio.py.

00765 
00766     def __repr__(self):
00767         clsname = self.__class__.__name__
00768         try:
00769             name = self.name
00770         except AttributeError:
00771             return "<_pyio.{0}>".format(clsname)
00772         else:
00773             return "<_pyio.{0} name={1!r}>".format(clsname, name)

Here is the call graph for this function:

def abc.ABCMeta.__subclasscheck__ (   cls,
  subclass 
) [inherited]
Override for issubclass(subclass, cls).

Definition at line 174 of file abc.py.

00174 
00175     def __subclasscheck__(cls, subclass):
00176         """Override for issubclass(subclass, cls)."""
00177         # Check cache
00178         if subclass in cls._abc_cache:
00179             return True
00180         # Check negative cache; may have to invalidate
00181         if cls._abc_negative_cache_version < ABCMeta._abc_invalidation_counter:
00182             # Invalidate the negative cache
00183             cls._abc_negative_cache = WeakSet()
00184             cls._abc_negative_cache_version = ABCMeta._abc_invalidation_counter
00185         elif subclass in cls._abc_negative_cache:
00186             return False
00187         # Check the subclass hook
00188         ok = cls.__subclasshook__(subclass)
00189         if ok is not NotImplemented:
00190             assert isinstance(ok, bool)
00191             if ok:
00192                 cls._abc_cache.add(subclass)
00193             else:
00194                 cls._abc_negative_cache.add(subclass)
00195             return ok
00196         # Check if it's a direct subclass
00197         if cls in getattr(subclass, '__mro__', ()):
00198             cls._abc_cache.add(subclass)
00199             return True
00200         # Check if it's a subclass of a registered class (recursive)
00201         for rcls in cls._abc_registry:
00202             if issubclass(subclass, rcls):
00203                 cls._abc_cache.add(subclass)
00204                 return True
00205         # Check if it's a subclass of a subclass (recursive)
00206         for scls in cls.__subclasses__():
00207             if issubclass(subclass, scls):
00208                 cls._abc_cache.add(subclass)
00209                 return True
00210         # No dice; update negative cache
00211         cls._abc_negative_cache.add(subclass)
00212         return False

Here is the call graph for this function:

def _pyio.BufferedReader._peek_unlocked (   self,
  n = 0 
) [private]

Definition at line 1004 of file _pyio.py.

01004 
01005     def _peek_unlocked(self, n=0):
01006         want = min(n, self.buffer_size)
01007         have = len(self._read_buf) - self._read_pos
01008         if have < want or have <= 0:
01009             to_read = self.buffer_size - have
01010             while True:
01011                 try:
01012                     current = self.raw.read(to_read)
01013                 except IOError as e:
01014                     if e.errno != EINTR:
01015                         raise
01016                     continue
01017                 break
01018             if current:
01019                 self._read_buf = self._read_buf[self._read_pos:] + current
01020                 self._read_pos = 0
01021         return self._read_buf[self._read_pos:]

def _pyio.BufferedReader._read_unlocked (   self,
  n = None 
) [private]

Definition at line 938 of file _pyio.py.

00938 
00939     def _read_unlocked(self, n=None):
00940         nodata_val = b""
00941         empty_values = (b"", None)
00942         buf = self._read_buf
00943         pos = self._read_pos
00944 
00945         # Special case for when the number of bytes to read is unspecified.
00946         if n is None or n == -1:
00947             self._reset_read_buf()
00948             chunks = [buf[pos:]]  # Strip the consumed bytes.
00949             current_size = 0
00950             while True:
00951                 # Read until EOF or until read() would block.
00952                 try:
00953                     chunk = self.raw.read()
00954                 except IOError as e:
00955                     if e.errno != EINTR:
00956                         raise
00957                     continue
00958                 if chunk in empty_values:
00959                     nodata_val = chunk
00960                     break
00961                 current_size += len(chunk)
00962                 chunks.append(chunk)
00963             return b"".join(chunks) or nodata_val
00964 
00965         # The number of bytes to read is specified, return at most n bytes.
00966         avail = len(buf) - pos  # Length of the available buffered data.
00967         if n <= avail:
00968             # Fast path: the data to read is fully buffered.
00969             self._read_pos += n
00970             return buf[pos:pos+n]
00971         # Slow path: read from the stream until enough bytes are read,
00972         # or until an EOF occurs or until read() would block.
00973         chunks = [buf[pos:]]
00974         wanted = max(self.buffer_size, n)
00975         while avail < n:
00976             try:
00977                 chunk = self.raw.read(wanted)
00978             except IOError as e:
00979                 if e.errno != EINTR:
00980                     raise
00981                 continue
00982             if chunk in empty_values:
00983                 nodata_val = chunk
00984                 break
00985             avail += len(chunk)
00986             chunks.append(chunk)
00987         # n is more then avail only when an EOF occurred or when
00988         # read() would have blocked.
00989         n = min(n, avail)
00990         out = b"".join(chunks)
00991         self._read_buf = out[n:]  # Save the extra data in the buffer.
00992         self._read_pos = 0
00993         return out[:n] if out else nodata_val

Here is the call graph for this function:

def _pyio.BufferedReader._reset_read_buf (   self) [private]

Definition at line 921 of file _pyio.py.

00921 
00922     def _reset_read_buf(self):
00923         self._read_buf = b""
00924         self._read_pos = 0

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.close (   self) [inherited]
Flush and close the IO object.

This method has no effect if the file is already closed.

Reimplemented from _pyio.IOBase.

Definition at line 721 of file _pyio.py.

00721 
00722     def close(self):
00723         if self.raw is not None and not self.closed:
00724             self.flush()
00725             self.raw.close()

Here is the call graph for this function:

def _pyio._BufferedIOMixin.closed (   self) [inherited]
closed: bool.  True iff the file has been closed.

For backwards compatibility, this is a property, not a predicate.

Reimplemented from _pyio.IOBase.

Definition at line 750 of file _pyio.py.

00750 
00751     def closed(self):
00752         return self.raw.closed

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.detach (   self) [inherited]
Separate the underlying raw stream from the buffer and return it.

After the raw stream has been detached, the buffer is in an unusable
state.

Reimplemented from _pyio.BufferedIOBase.

Definition at line 726 of file _pyio.py.

00726 
00727     def detach(self):
00728         if self.raw is None:
00729             raise ValueError("raw stream already detached")
00730         self.flush()
00731         raw = self._raw
00732         self._raw = None
00733         return raw

Here is the call graph for this function:

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.fileno (   self) [inherited]

Lower-level APIs ###.

Reimplemented from _pyio.IOBase.

Definition at line 776 of file _pyio.py.

00776 
00777     def fileno(self):
00778         return self.raw.fileno()

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.flush (   self) [inherited]

Flush and close ###.

Reimplemented from _pyio.IOBase.

Reimplemented in _pyio.BufferedWriter.

Definition at line 716 of file _pyio.py.

00716 
00717     def flush(self):
00718         if self.closed:
00719             raise ValueError("flush of closed file")
00720         self.raw.flush()

Here is the call graph for this function:

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.isatty (   self) [inherited]
Return a bool indicating whether this is an 'interactive' stream.

Return False if it can't be determined.

Reimplemented from _pyio.IOBase.

Definition at line 779 of file _pyio.py.

00779 
00780     def isatty(self):
00781         return self.raw.isatty()
00782 

def _pyio._BufferedIOMixin.mode (   self) [inherited]

Definition at line 758 of file _pyio.py.

00758 
00759     def mode(self):
00760         return self.raw.mode

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.name (   self) [inherited]

Definition at line 754 of file _pyio.py.

00754 
00755     def name(self):
00756         return self.raw.name

Here is the caller graph for this function:

def _pyio.BufferedReader.peek (   self,
  n = 0 
)
Returns buffered bytes without advancing the position.

The argument indicates a desired minimal number of bytes; we
do at most one raw read to satisfy it.  We never return more
than self.buffer_size.

Reimplemented in _pyio.BufferedRandom.

Definition at line 994 of file _pyio.py.

00994 
00995     def peek(self, n=0):
00996         """Returns buffered bytes without advancing the position.
00997 
00998         The argument indicates a desired minimal number of bytes; we
00999         do at most one raw read to satisfy it.  We never return more
01000         than self.buffer_size.
01001         """
01002         with self._read_lock:
01003             return self._peek_unlocked(n)

Here is the call graph for this function:

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.raw (   self) [inherited]

Definition at line 746 of file _pyio.py.

00746 
00747     def raw(self):
00748         return self._raw

Here is the caller graph for this function:

def _pyio.BufferedReader.read (   self,
  n = None 
)
Read n bytes.

Returns exactly n bytes of data unless the underlying raw IO
stream reaches EOF or if the call would block in non-blocking
mode. If n is negative, read until EOF or until read() would
block.

Reimplemented from _pyio.BufferedIOBase.

Reimplemented in _pyio.BufferedRandom.

Definition at line 925 of file _pyio.py.

00925 
00926     def read(self, n=None):
00927         """Read n bytes.
00928 
00929         Returns exactly n bytes of data unless the underlying raw IO
00930         stream reaches EOF or if the call would block in non-blocking
00931         mode. If n is negative, read until EOF or until read() would
00932         block.
00933         """
00934         if n is not None and n < -1:
00935             raise ValueError("invalid number of bytes to read")
00936         with self._read_lock:
00937             return self._read_unlocked(n)

Here is the call graph for this function:

Here is the caller graph for this function:

def _pyio.BufferedReader.read1 (   self,
  n 
)
Reads up to n bytes, with at most one read() system call.

Reimplemented from _pyio.BufferedIOBase.

Reimplemented in _pyio.BufferedRandom.

Definition at line 1022 of file _pyio.py.

01022 
01023     def read1(self, n):
01024         """Reads up to n bytes, with at most one read() system call."""
01025         # Returns up to n bytes.  If at least one byte is buffered, we
01026         # only return buffered bytes.  Otherwise, we do one raw read.
01027         if n < 0:
01028             raise ValueError("number of bytes to read must be positive")
01029         if n == 0:
01030             return b""
01031         with self._read_lock:
01032             self._peek_unlocked(1)
01033             return self._read_unlocked(
01034                 min(n, len(self._read_buf) - self._read_pos))

Here is the call graph for this function:

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.readable (   self) [inherited]
Return a bool indicating whether object was opened for reading.

If False, read() will raise UnsupportedOperation.

Reimplemented from _pyio.IOBase.

Definition at line 739 of file _pyio.py.

00739 
00740     def readable(self):
00741         return self.raw.readable()

def _pyio.BufferedIOBase.readinto (   self,
  b 
) [inherited]
Read up to len(b) bytes into bytearray b.

Like read(), this may issue multiple reads to the underlying raw
stream, unless the latter is 'interactive'.

Returns an int representing the number of bytes read (0 for EOF).

Raises BlockingIOError if the underlying raw stream has no
data at the moment.

Reimplemented in _pyio.BufferedRandom, and _pyio.BufferedRWPair.

Definition at line 630 of file _pyio.py.

00630 
00631     def readinto(self, b):
00632         """Read up to len(b) bytes into bytearray b.
00633 
00634         Like read(), this may issue multiple reads to the underlying raw
00635         stream, unless the latter is 'interactive'.
00636 
00637         Returns an int representing the number of bytes read (0 for EOF).
00638 
00639         Raises BlockingIOError if the underlying raw stream has no
00640         data at the moment.
00641         """
00642         # XXX This ought to work with anything that supports the buffer API
00643         data = self.read(len(b))
00644         n = len(data)
00645         try:
00646             b[:n] = data
00647         except TypeError as err:
00648             import array
00649             if not isinstance(b, array.array):
00650                 raise err
00651             b[:n] = array.array('b', data)
00652         return n

Here is the call graph for this function:

def _pyio.IOBase.readline (   self,
  limit = -1 
) [inherited]

Readline[s] and writelines ###.

Reimplemented in _pyio.TextIOWrapper.

Definition at line 449 of file _pyio.py.

00449 
00450     def readline(self, limit=-1):
00451         r"""Read and return a line of bytes from the stream.
00452 
00453         If limit is specified, at most limit bytes will be read.
00454         Limit should be an int.
00455 
00456         The line terminator is always b'\n' for binary files; for text
00457         files, the newlines argument to open can be used to select the line
00458         terminator(s) recognized.
00459         """
00460         # For backwards compatibility, a (slowish) readline().
00461         if hasattr(self, "peek"):
00462             def nreadahead():
00463                 readahead = self.peek(1)
00464                 if not readahead:
00465                     return 1
00466                 n = (readahead.find(b"\n") + 1) or len(readahead)
00467                 if limit >= 0:
00468                     n = min(n, limit)
00469                 return n
00470         else:
00471             def nreadahead():
00472                 return 1
00473         if limit is None:
00474             limit = -1
00475         elif not isinstance(limit, int):
00476             raise TypeError("limit must be an integer")
00477         res = bytearray()
00478         while limit < 0 or len(res) < limit:
00479             b = self.read(nreadahead())
00480             if not b:
00481                 break
00482             res += b
00483             if res.endswith(b"\n"):
00484                 break
00485         return bytes(res)

Here is the caller graph for this function:

def _pyio.IOBase.readlines (   self,
  hint = None 
) [inherited]
Return a list of lines from the stream.

hint can be specified to control the number of lines read: no more
lines will be read if the total size (in bytes/characters) of all
lines so far exceeds hint.

Definition at line 496 of file _pyio.py.

00496 
00497     def readlines(self, hint=None):
00498         """Return a list of lines from the stream.
00499 
00500         hint can be specified to control the number of lines read: no more
00501         lines will be read if the total size (in bytes/characters) of all
00502         lines so far exceeds hint.
00503         """
00504         if hint is None or hint <= 0:
00505             return list(self)
00506         n = 0
00507         lines = []
00508         for line in self:
00509             lines.append(line)
00510             n += len(line)
00511             if n >= hint:
00512                 break
00513         return lines

def abc.ABCMeta.register (   cls,
  subclass 
) [inherited]
Register a virtual subclass of an ABC.

Definition at line 135 of file abc.py.

00135 
00136     def register(cls, subclass):
00137         """Register a virtual subclass of an ABC."""
00138         if not isinstance(subclass, type):
00139             raise TypeError("Can only register classes")
00140         if issubclass(subclass, cls):
00141             return  # Already a subclass
00142         # Subtle: test for cycles *after* testing for "already a subclass";
00143         # this means we allow X.register(X) and interpret it as a no-op.
00144         if issubclass(cls, subclass):
00145             # This would create a cycle, which is bad for the algorithm below
00146             raise RuntimeError("Refusing to create an inheritance cycle")
00147         cls._abc_registry.add(subclass)
00148         ABCMeta._abc_invalidation_counter += 1  # Invalidate negative cache

Here is the caller graph for this function:

def _pyio.BufferedReader.seek (   self,
  pos,
  whence = 0 
)

Positioning ###.

Reimplemented from _pyio._BufferedIOMixin.

Reimplemented in _pyio.BufferedRandom.

Definition at line 1038 of file _pyio.py.

01038 
01039     def seek(self, pos, whence=0):
01040         if not (0 <= whence <= 2):
01041             raise ValueError("invalid whence value")
01042         with self._read_lock:
01043             if whence == 1:
01044                 pos -= len(self._read_buf) - self._read_pos
01045             pos = _BufferedIOMixin.seek(self, pos, whence)
01046             self._reset_read_buf()
01047             return pos

Here is the call graph for this function:

Here is the caller graph for this function:

def _pyio._BufferedIOMixin.seekable (   self) [inherited]

Inquiries ###.

Reimplemented from _pyio.IOBase.

Definition at line 736 of file _pyio.py.

00736 
00737     def seekable(self):
00738         return self.raw.seekable()

def _pyio.BufferedReader.tell (   self)
Return an int indicating the current stream position.

Reimplemented from _pyio._BufferedIOMixin.

Reimplemented in _pyio.BufferedRandom.

Definition at line 1035 of file _pyio.py.

01035 
01036     def tell(self):
01037         return _BufferedIOMixin.tell(self) - len(self._read_buf) + self._read_pos

def _pyio._BufferedIOMixin.truncate (   self,
  pos = None 
) [inherited]
Truncate file to size bytes.

Size defaults to the current IO position as reported by tell().  Return
the new size.

Reimplemented from _pyio.IOBase.

Reimplemented in _pyio.BufferedRandom, and _pyio.BufferedWriter.

Definition at line 702 of file _pyio.py.

00702 
00703     def truncate(self, pos=None):
00704         # Flush the stream.  We're mixing buffered I/O with lower-level I/O,
00705         # and a flush may be necessary to synch both views of the current
00706         # file state.
00707         self.flush()
00708 
00709         if pos is None:
00710             pos = self.tell()
00711         # XXX: Should seek() be used, instead of passing the position
00712         # XXX  directly to truncate?
00713         return self.raw.truncate(pos)

Here is the call graph for this function:

def _pyio._BufferedIOMixin.writable (   self) [inherited]
Return a bool indicating whether object was opened for writing.

If False, write() and truncate() will raise UnsupportedOperation.

Reimplemented from _pyio.IOBase.

Definition at line 742 of file _pyio.py.

00742 
00743     def writable(self):
00744         return self.raw.writable()

def _pyio.BufferedIOBase.write (   self,
  b 
) [inherited]
Write the given bytes buffer to the IO stream.

Return the number of bytes written, which is never less than
len(b).

Raises BlockingIOError if the buffer is full and the
underlying raw stream cannot accept more data at the moment.

Reimplemented in _pyio.BufferedRandom, _pyio.BufferedRWPair, _pyio.BufferedWriter, and _pyio.BytesIO.

Definition at line 653 of file _pyio.py.

00653 
00654     def write(self, b):
00655         """Write the given bytes buffer to the IO stream.
00656 
00657         Return the number of bytes written, which is never less than
00658         len(b).
00659 
00660         Raises BlockingIOError if the buffer is full and the
00661         underlying raw stream cannot accept more data at the moment.
00662         """
00663         self._unsupported("write")

Here is the call graph for this function:

def _pyio.IOBase.writelines (   self,
  lines 
) [inherited]

Definition at line 514 of file _pyio.py.

00514 
00515     def writelines(self, lines):
00516         self._checkClosed()
00517         for line in lines:
00518             self.write(line)
00519 
00520 io.IOBase.register(IOBase)
00521 


Member Data Documentation

Definition at line 922 of file _pyio.py.

Definition at line 919 of file _pyio.py.

Definition at line 923 of file _pyio.py.

Definition at line 917 of file _pyio.py.


The documentation for this class was generated from the following file: