Skip to content

Commit d4ec0c3

Browse files
committed
Merge with trunk up to revision 45620.
1 parent 13247bf commit d4ec0c3

23 files changed

Lines changed: 141 additions & 115 deletions

Doc/howto/unicode.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -733,6 +733,7 @@ Version 1.02: posted August 16 2005. Corrects factual errors.
733733

734734
.. comment Additional topic: building Python w/ UCS2 or UCS4 support
735735
.. comment Describe obscure -U switch somewhere?
736+
.. comment Describe use of codecs.StreamRecoder and StreamReaderWriter
736737
737738
.. comment
738739
Original outline:

Doc/lib/libcodecs.tex

Lines changed: 32 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -93,21 +93,21 @@ \section{\module{codecs} ---
9393
lookup:
9494

9595
\begin{funcdesc}{getencoder}{encoding}
96-
Lookup up the codec for the given encoding and return its encoder
96+
Look up the codec for the given encoding and return its encoder
9797
function.
9898

9999
Raises a \exception{LookupError} in case the encoding cannot be found.
100100
\end{funcdesc}
101101

102102
\begin{funcdesc}{getdecoder}{encoding}
103-
Lookup up the codec for the given encoding and return its decoder
103+
Look up the codec for the given encoding and return its decoder
104104
function.
105105

106106
Raises a \exception{LookupError} in case the encoding cannot be found.
107107
\end{funcdesc}
108108

109109
\begin{funcdesc}{getincrementalencoder}{encoding}
110-
Lookup up the codec for the given encoding and return its incremental encoder
110+
Look up the codec for the given encoding and return its incremental encoder
111111
class or factory function.
112112

113113
Raises a \exception{LookupError} in case the encoding cannot be found or the
@@ -116,7 +116,7 @@ \section{\module{codecs} ---
116116
\end{funcdesc}
117117

118118
\begin{funcdesc}{getincrementaldecoder}{encoding}
119-
Lookup up the codec for the given encoding and return its incremental decoder
119+
Look up the codec for the given encoding and return its incremental decoder
120120
class or factory function.
121121

122122
Raises a \exception{LookupError} in case the encoding cannot be found or the
@@ -125,14 +125,14 @@ \section{\module{codecs} ---
125125
\end{funcdesc}
126126

127127
\begin{funcdesc}{getreader}{encoding}
128-
Lookup up the codec for the given encoding and return its StreamReader
128+
Look up the codec for the given encoding and return its StreamReader
129129
class or factory function.
130130

131131
Raises a \exception{LookupError} in case the encoding cannot be found.
132132
\end{funcdesc}
133133

134134
\begin{funcdesc}{getwriter}{encoding}
135-
Lookup up the codec for the given encoding and return its StreamWriter
135+
Look up the codec for the given encoding and return its StreamWriter
136136
class or factory function.
137137

138138
Raises a \exception{LookupError} in case the encoding cannot be found.
@@ -353,7 +353,7 @@ \subsubsection{Codec Objects \label{codec-objects}}
353353
the encoding/decoding process during method calls.
354354

355355
The joined output of calls to the \method{encode}/\method{decode} method is the
356-
same as if the all single inputs where joined into one, and this input was
356+
same as if all the single inputs were joined into one, and this input was
357357
encoded/decoded with the stateless encoder/decoder.
358358

359359

@@ -363,7 +363,7 @@ \subsubsection{IncrementalEncoder Objects \label{incremental-encoder-objects}}
363363

364364
The \class{IncrementalEncoder} class is used for encoding an input in multiple
365365
steps. It defines the following methods which every incremental encoder must
366-
define in order to be compatible to the Python codec registry.
366+
define in order to be compatible with the Python codec registry.
367367

368368
\begin{classdesc}{IncrementalEncoder}{\optional{errors}}
369369
Constructor for a \class{IncrementalEncoder} instance.
@@ -410,7 +410,7 @@ \subsubsection{IncrementalDecoder Objects \label{incremental-decoder-objects}}
410410

411411
The \class{IncrementalDecoder} class is used for decoding an input in multiple
412412
steps. It defines the following methods which every incremental decoder must
413-
define in order to be compatible to the Python codec registry.
413+
define in order to be compatible with the Python codec registry.
414414

415415
\begin{classdesc}{IncrementalDecoder}{\optional{errors}}
416416
Constructor for a \class{IncrementalDecoder} instance.
@@ -456,15 +456,15 @@ \subsubsection{IncrementalDecoder Objects \label{incremental-decoder-objects}}
456456

457457
The \class{StreamWriter} and \class{StreamReader} classes provide
458458
generic working interfaces which can be used to implement new
459-
encodings submodules very easily. See \module{encodings.utf_8} for an
460-
example on how this is done.
459+
encoding submodules very easily. See \module{encodings.utf_8} for an
460+
example of how this is done.
461461

462462

463463
\subsubsection{StreamWriter Objects \label{stream-writer-objects}}
464464

465465
The \class{StreamWriter} class is a subclass of \class{Codec} and
466466
defines the following methods which every stream writer must define in
467-
order to be compatible to the Python codec registry.
467+
order to be compatible with the Python codec registry.
468468

469469
\begin{classdesc}{StreamWriter}{stream\optional{, errors}}
470470
Constructor for a \class{StreamWriter} instance.
@@ -473,7 +473,7 @@ \subsubsection{StreamWriter Objects \label{stream-writer-objects}}
473473
free to add additional keyword arguments, but only the ones defined
474474
here are used by the Python codec registry.
475475

476-
\var{stream} must be a file-like object open for writing (binary)
476+
\var{stream} must be a file-like object open for writing binary
477477
data.
478478

479479
The \class{StreamWriter} may implement different error handling
@@ -512,19 +512,19 @@ \subsubsection{StreamWriter Objects \label{stream-writer-objects}}
512512
Flushes and resets the codec buffers used for keeping state.
513513

514514
Calling this method should ensure that the data on the output is put
515-
into a clean state, that allows appending of new fresh data without
515+
into a clean state that allows appending of new fresh data without
516516
having to rescan the whole stream to recover state.
517517
\end{methoddesc}
518518

519519
In addition to the above methods, the \class{StreamWriter} must also
520-
inherit all other methods and attribute from the underlying stream.
520+
inherit all other methods and attributes from the underlying stream.
521521

522522

523523
\subsubsection{StreamReader Objects \label{stream-reader-objects}}
524524

525525
The \class{StreamReader} class is a subclass of \class{Codec} and
526526
defines the following methods which every stream reader must define in
527-
order to be compatible to the Python codec registry.
527+
order to be compatible with the Python codec registry.
528528

529529
\begin{classdesc}{StreamReader}{stream\optional{, errors}}
530530
Constructor for a \class{StreamReader} instance.
@@ -589,20 +589,20 @@ \subsubsection{StreamReader Objects \label{stream-reader-objects}}
589589
\var{size}, if given, is passed as size argument to the stream's
590590
\method{readline()} method.
591591

592-
If \var{keepends} is false lineends will be stripped from the
592+
If \var{keepends} is false line-endings will be stripped from the
593593
lines returned.
594594

595595
\versionchanged[\var{keepends} argument added]{2.4}
596596
\end{methoddesc}
597597

598598
\begin{methoddesc}{readlines}{\optional{sizehint\optional{, keepends}}}
599-
Read all lines available on the input stream and return them as list
599+
Read all lines available on the input stream and return them as a list
600600
of lines.
601601

602-
Line breaks are implemented using the codec's decoder method and are
602+
Line-endings are implemented using the codec's decoder method and are
603603
included in the list entries if \var{keepends} is true.
604604

605-
\var{sizehint}, if given, is passed as \var{size} argument to the
605+
\var{sizehint}, if given, is passed as the \var{size} argument to the
606606
stream's \method{read()} method.
607607
\end{methoddesc}
608608

@@ -614,7 +614,7 @@ \subsubsection{StreamReader Objects \label{stream-reader-objects}}
614614
\end{methoddesc}
615615

616616
In addition to the above methods, the \class{StreamReader} must also
617-
inherit all other methods and attribute from the underlying stream.
617+
inherit all other methods and attributes from the underlying stream.
618618

619619
The next two base classes are included for convenience. They are not
620620
needed by the codec registry, but may provide useful in practice.
@@ -640,7 +640,7 @@ \subsubsection{StreamReaderWriter Objects \label{stream-reader-writer}}
640640

641641
\class{StreamReaderWriter} instances define the combined interfaces of
642642
\class{StreamReader} and \class{StreamWriter} classes. They inherit
643-
all other methods and attribute from the underlying stream.
643+
all other methods and attributes from the underlying stream.
644644

645645

646646
\subsubsection{StreamRecoder Objects \label{stream-recoder-objects}}
@@ -666,14 +666,14 @@ \subsubsection{StreamRecoder Objects \label{stream-recoder-objects}}
666666
\var{stream} must be a file-like object.
667667

668668
\var{encode}, \var{decode} must adhere to the \class{Codec}
669-
interface, \var{Reader}, \var{Writer} must be factory functions or
669+
interface. \var{Reader}, \var{Writer} must be factory functions or
670670
classes providing objects of the \class{StreamReader} and
671671
\class{StreamWriter} interface respectively.
672672

673673
\var{encode} and \var{decode} are needed for the frontend
674674
translation, \var{Reader} and \var{Writer} for the backend
675675
translation. The intermediate format used is determined by the two
676-
sets of codecs, e.g. the Unicode codecs will use Unicode as
676+
sets of codecs, e.g. the Unicode codecs will use Unicode as the
677677
intermediate encoding.
678678

679679
Error handling is done in the same way as defined for the
@@ -682,7 +682,7 @@ \subsubsection{StreamRecoder Objects \label{stream-recoder-objects}}
682682

683683
\class{StreamRecoder} instances define the combined interfaces of
684684
\class{StreamReader} and \class{StreamWriter} classes. They inherit
685-
all other methods and attribute from the underlying stream.
685+
all other methods and attributes from the underlying stream.
686686

687687
\subsection{Encodings and Unicode\label{encodings-overview}}
688688

@@ -695,7 +695,7 @@ \subsection{Encodings and Unicode\label{encodings-overview}}
695695
memory, CPU endianness and how these arrays are stored as bytes become
696696
an issue. Transforming a unicode object into a sequence of bytes is
697697
called encoding and recreating the unicode object from the sequence of
698-
bytes is known as decoding. There are many different methods how this
698+
bytes is known as decoding. There are many different methods for how this
699699
transformation can be done (these methods are also called encodings).
700700
The simplest method is to map the codepoints 0-255 to the bytes
701701
\code{0x0}-\code{0xff}. This means that a unicode object that contains
@@ -742,7 +742,7 @@ \subsection{Encodings and Unicode\label{encodings-overview}}
742742
it's a normal character that will be decoded like any other.
743743

744744
There's another encoding that is able to encoding the full range of
745-
Unicode characters: UTF-8. UTF-8 is an 8bit encoding, which means
745+
Unicode characters: UTF-8. UTF-8 is an 8-bit encoding, which means
746746
there are no issues with byte order in UTF-8. Each byte in a UTF-8
747747
byte sequence consists of two parts: Marker bits (the most significant
748748
bits) and payload bits. The marker bits are a sequence of zero to six
@@ -762,7 +762,7 @@ \subsection{Encodings and Unicode\label{encodings-overview}}
762762
The least significant bit of the Unicode character is the rightmost x
763763
bit.
764764

765-
As UTF-8 is an 8bit encoding no BOM is required and any \code{U+FEFF}
765+
As UTF-8 is an 8-bit encoding no BOM is required and any \code{U+FEFF}
766766
character in the decoded Unicode string (even if it's the first
767767
character) is treated as a \samp{ZERO WIDTH NO-BREAK SPACE}.
768768

@@ -775,7 +775,7 @@ \subsection{Encodings and Unicode\label{encodings-overview}}
775775
variant of UTF-8 (that Python 2.5 calls \code{"utf-8-sig"}) for its Notepad
776776
program: Before any of the Unicode characters is written to the file,
777777
a UTF-8 encoded BOM (which looks like this as a byte sequence: \code{0xef},
778-
\code{0xbb}, \code{0xbf}) is written. As it's rather improbably that any
778+
\code{0xbb}, \code{0xbf}) is written. As it's rather improbable that any
779779
charmap encoded file starts with these byte values (which would e.g. map to
780780

781781
LATIN SMALL LETTER I WITH DIAERESIS \\
@@ -794,8 +794,8 @@ \subsection{Encodings and Unicode\label{encodings-overview}}
794794

795795
\subsection{Standard Encodings\label{standard-encodings}}
796796

797-
Python comes with a number of codecs builtin, either implemented as C
798-
functions, or with dictionaries as mapping tables. The following table
797+
Python comes with a number of codecs built-in, either implemented as C
798+
functions or with dictionaries as mapping tables. The following table
799799
lists the codecs by name, together with a few common aliases, and the
800800
languages for which the encoding is likely used. Neither the list of
801801
aliases nor the list of languages is meant to be exhaustive. Notice
@@ -1337,7 +1337,7 @@ \subsection{\module{encodings.idna} ---
13371337
UTF-8 codec with BOM signature}
13381338
\declaremodule{standard}{encodings.utf-8-sig} % XXX utf_8_sig gives TeX errors
13391339
\modulesynopsis{UTF-8 codec with BOM signature}
1340-
\moduleauthor{Walter D\"orwald}
1340+
\moduleauthor{Walter D\"orwald}{}
13411341

13421342
\versionadded{2.5}
13431343

0 commit comments

Comments
 (0)