diff -r 2b8cd2bc2745 Doc/library/stdtypes.rst --- a/Doc/library/stdtypes.rst Mon Jun 16 19:48:02 2014 +1000 +++ b/Doc/library/stdtypes.rst Mon Jun 16 21:29:42 2014 +1000 @@ -1598,7 +1598,8 @@ .. method:: str.index(sub[, start[, end]]) - Like :meth:`find`, but raise :exc:`ValueError` when the substring is not found. + Like :meth:`~str.find`, but raise :exc:`ValueError` when the substring is + not found. .. method:: str.isalnum() @@ -2198,16 +2199,28 @@ Also see the :ref:`bytes ` built-in. -Since bytes objects are sequences of integers, for a bytes object *b*, -``b[0]`` will be an integer, while ``b[0:1]`` will be a bytes object of -length 1. (This contrasts with text strings, where both indexing and -slicing will produce a string of length 1) +Since 2 hexadecimal digits correspond precisely to a single byte, hexadecimal +numbers are a commonly used format for describing binary data. Accordingly, +the bytes type has an additional class method to read data in that format: + +.. classmethod:: bytes.fromhex(string) + + This :class:`bytes` class method returns a bytes object, decoding the + given string object. The string must contain two hexadecimal digits per + byte, with ASCII spaces being ignored. + + >>> bytes.fromhex('2Ef0 F1f2 ') + b'.\xf0\xf1\xf2' + +Since bytes objects are sequences of integers (akin to a tuple), for a bytes +object *b*, ``b[0]`` will be an integer, while ``b[0:1]`` will be a bytes +object of length 1. (This contrasts with text strings, where both indexing +and slicing will produce a string of length 1) The representation of bytes objects uses the literal format (``b'...'``) since it is often more useful than e.g. ``bytes([46, 46, 46])``. You can always convert a bytes object into a list of integers using ``list(b)``. - .. note:: For Python 2.x users: In the Python 2.x series, a variety of implicit conversions between 8-bit strings (the closest thing 2.x offers to a @@ -2241,6 +2254,29 @@ Also see the :ref:`bytearray ` built-in. +Since 2 hexadecimal digits correspond precisely to a single byte, hexadecimal +numbers are a commonly used format for describing binary data. Accordingly, +the bytearray type has an additional class method to read data in that format: + +.. classmethod:: bytearray.fromhex(string) + + This :class:`bytearray` class method returns bytearray object, decoding + the given string object. The string must contain two hexadecimal digits + per byte, with ASCII spaces being ignored. + + >>> bytearray.fromhex('2Ef0 F1f2 ') + bytearray(b'.\xf0\xf1\xf2') + +Since bytearray objects are sequences of integers (akin to a list), for a +bytearray object *b*, ``b[0]`` will be an integer, while ``b[0:1]`` will be +a bytearray object of length 1. (This contrasts with text strings, where +both indexing and slicing will produce a string of length 1) + +The representation of bytearray objects uses the bytes literal format +(``bytearray(b'...')``) since it is often more useful than e.g. +``bytearray([46, 46, 46])``. You can always convert a bytearray object into +a list of integers using ``list(b)``. + .. _bytes-methods: @@ -2257,20 +2293,6 @@ freely mixed in operations without causing errors. However, the return type of the result may depend on the order of operands. -Due to the common use of ASCII text as the basis for binary protocols, bytes -and bytearray objects provide almost all methods found on text strings, with -the exceptions of: - -* :meth:`str.encode` (which converts text strings to bytes objects) -* :meth:`str.format` and :meth:`str.format_map` (which are used to format - text for display to users) -* :meth:`str.isidentifier`, :meth:`str.isnumeric`, :meth:`str.isdecimal`, - :meth:`str.isprintable` (which are used to check various properties of - text strings which are not typically applicable to binary protocols). - -All other string methods are supported, although sometimes with slight -differences in functionality and semantics (as described below). - .. note:: The methods on bytes and bytearray objects don't accept strings as their @@ -2285,25 +2307,31 @@ a = b"abc" b = a.replace(b"a", b"f") -Whenever a bytes or bytearray method needs to interpret the bytes as -characters (e.g. the :meth:`is...` methods, :meth:`split`, :meth:`strip`), -the ASCII character set is assumed (text strings use Unicode semantics). +Some bytes and bytearray operations assumes the use of ASCII compatible +binary formats, and hence should be avoided when working with arbitrary +binary data. These restrictions are covered below. .. note:: - Using these ASCII based methods to manipulate binary data that is not + Using these ASCII based operations to manipulate binary data that is not stored in an ASCII based format may lead to data corruption. -The search operations (:keyword:`in`, :meth:`count`, :meth:`find`, -:meth:`index`, :meth:`rfind` and :meth:`rindex`) all accept both integers -in the range 0 to 255 (inclusive) as well as bytes and byte array sequences. - -.. versionchanged:: 3.3 - All of the search methods also accept an integer in the range 0 to 255 - (inclusive) as their first argument. - - -Each bytes and bytearray instance provides a :meth:`~bytes.decode` convenience -method that is the inverse of :meth:`str.encode`: +The following methods on bytes and bytearray objects can be used with +arbitrary binary data. + +.. method:: bytes.count(sub[, start[, end]]) + bytearray.count(sub[, start[, end]]) + + Return the number of non-overlapping occurrences of subsequence *sub* in + the range [*start*, *end*]. Optional arguments *start* and *end* are + interpreted as in slice notation. + + The subsequence to search for may be any type that supports the + :ref:`buffer protocol ` or an integer in the range 0 to + 255. + + .. versionchanged:: 3.3 + Also accept an integer in the range 0 to 255 as the subsequence. + .. method:: bytes.decode(encoding="utf-8", errors="strict") bytearray.decode(encoding="utf-8", errors="strict") @@ -2319,34 +2347,174 @@ .. versionchanged:: 3.1 Added support for keyword arguments. -Since 2 hexadecimal digits correspond precisely to a single byte, hexadecimal -numbers are a commonly used format for describing binary data. Accordingly, -the bytes and bytearray types have an additional class method to read data in -that format: - -.. classmethod:: bytes.fromhex(string) - bytearray.fromhex(string) - - This :class:`bytes` class method returns a bytes or bytearray object, - decoding the given string object. The string must contain two hexadecimal - digits per byte, spaces are ignored. - - >>> bytes.fromhex('2Ef0 F1f2 ') - b'.\xf0\xf1\xf2' - - -The maketrans and translate methods differ in semantics from the versions -available on strings: + +.. method:: bytes.endswith(suffix[, start[, end]]) + bytearray.endswith(suffix[, start[, end]]) + + Return ``True`` if the binary data ends with the specified *suffix*, + otherwise return ``False``. *suffix* can also be a tuple of suffixes to + look for. With optional *start*, test beginning at that position. With + optional *end*, stop comparing at that position. + + The suffix(es) to search for may be any type that supports the + :ref:`buffer protocol `. + + +.. method:: bytes.find(sub[, start[, end]]) + bytearray.find(sub[, start[, end]]) + + Return the lowest index in the data where the subsequence *sub* is found, + such that *sub* is contained in the slice ``s[start:end]``. Optional + arguments *start* and *end* are interpreted as in slice notation. Return + ``-1`` if *sub* is not found. + + The subsequence to search for may be any type that supports the + :ref:`buffer protocol ` or an integer in the range 0 to + 255. + + .. note:: + + The :meth:`~bytes.find` method should be used only if you need to know the + position of *sub*. To check if *sub* is a substring or not, use the + :keyword:`in` operator:: + + >>> b'Py' in b'Python' + True + + .. versionchanged:: 3.3 + Also accept an integer in the range 0 to 255 as the subsequence. + + +.. method:: bytes.index(sub[, start[, end]]) + bytearray.index(sub[, start[, end]]) + + Like :meth:`~bytes.find`, but raise :exc:`ValueError` when the + subsequence is not found. + + The subsequence to search for may be any type that supports the + :ref:`buffer protocol ` or an integer in the range 0 to + 255. + + .. versionchanged:: 3.3 + Also accept an integer in the range 0 to 255 as the subsequence. + + +.. method:: bytes.join(iterable) + bytearray.join(iterable) + + Return a bytes or bytearray object which is the concatenation of the + binary data sequences in the :term:`iterable` *iterable*. A + :exc:`TypeError` will be raised if there are any values in *iterable* + that do not support the :ref:`buffer protocol `, including + :class:`str` objects. The separator between elements is the contents + of the bytes or bytearray object providing this method. + + +.. staticmethod:: bytes.maketrans(from, to) + bytearray.maketrans(from, to) + + This static method returns a translation table usable for + :meth:`bytes.translate` that will map each character in *from* into the + character at the same position in *to*; *from* and *to* must both support + the :ref:`buffer protocol ` and have the same length. + + .. versionadded:: 3.1 + + +.. method:: bytes.partition(sep) + bytearray.partition(sep) + + Split the sequence at the first occurrence of *sep*, and return a 3-tuple + containing the part before the separator, the separator, and the part + after the separator. If the separator is not found, return a 3-tuple + containing a copy of the original sequence, followed by two empty bytes or + bytearray objects. + + The separator to search for may be any type that supports the + :ref:`buffer protocol `. + + +.. method:: bytes.replace(old, new[, count]) + bytearray.replace(old, new[, count]) + + Return a copy of the sequence with all occurrences of subsequence *old* + replaced by *new*. If the optional argument *count* is given, only the + first *count* occurrences are replaced. + + The subsequence to search for and its replacement may be any type that + supports the :ref:`buffer protocol `. + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.rfind(sub[, start[, end]]) + bytearray.rfind(sub[, start[, end]]) + + Return the highest index in the sequence where the subsequence *sub* is + found, such that *sub* is contained within ``s[start:end]``. Optional + arguments *start* and *end* are interpreted as in slice notation. Return + ``-1`` on failure. + + The subsequence to search for may be any type that supports the + :ref:`buffer protocol ` or an integer in the range 0 to + 255. + + .. versionchanged:: 3.3 + Also accept an integer in the range 0 to 255 as the subsequence. + + +.. method:: bytes.rindex(sub[, start[, end]]) + bytearray.rindex(sub[, start[, end]]) + + Like :meth:`~bytes.rfind` but raises :exc:`ValueError` when the + subsequence *sub* is not found. + + The subsequence to search for may be any type that supports the + :ref:`buffer protocol ` or an integer in the range 0 to + 255. + + .. versionchanged:: 3.3 + Also accept an integer in the range 0 to 255 as the subsequence. + + +.. method:: bytes.rpartition(sep) + bytearray.rpartition(sep) + + Split the sequence at the last occurrence of *sep*, and return a 3-tuple + containing the part before the separator, the separator, and the part + after the separator. If the separator is not found, return a 3-tuple + containing a copy of the original sequence, followed by two empty bytes or + bytearray objects. + + The separator to search for may be any type that supports the + :ref:`buffer protocol `. + + +.. method:: bytes.startswith(prefix[, start[, end]]) + bytearray.startswith(prefix[, start[, end]]) + + Return ``True`` if the binary data starts with the specified *prefix*, + otherwise return ``False``. *prefix* can also be a tuple of prefixes to + look for. With optional *start*, test beginning at that position. With + optional *end*, stop comparing at that position. + + The prefix(es) to search for may be any type that supports the + :ref:`buffer protocol `. + .. method:: bytes.translate(table[, delete]) bytearray.translate(table[, delete]) Return a copy of the bytes or bytearray object where all bytes occurring in - the optional argument *delete* are removed, and the remaining bytes have been - mapped through the given translation table, which must be a bytes object of - length 256. - - You can use the :func:`bytes.maketrans` method to create a translation table. + the optional argument *delete* are removed, and the remaining bytes have + been mapped through the given translation table, which must be a bytes + object of length 256. + + You can use the :func:`bytes.maketrans` method to create a translation + table. Set the *table* argument to ``None`` for translations that only delete characters:: @@ -2355,15 +2523,356 @@ b'rd ths shrt txt' -.. staticmethod:: bytes.maketrans(from, to) - bytearray.maketrans(from, to) - - This static method returns a translation table usable for - :meth:`bytes.translate` that will map each character in *from* into the - character at the same position in *to*; *from* and *to* must be bytes objects - and have the same length. - - .. versionadded:: 3.1 +The following methods on bytes and bytearray objects have default behaviours +that assume the use of ASCII compatible binary formats, but can still be used +with arbitrary binary data by passing appropriate arguments. + +.. method:: bytes.center(width[, fillchar]) + + Return centered in a string of length *width*. Padding is done using the + specified *fillchar* (default is a space). + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + +.. method:: bytes.ljust(width[, fillchar]) + + Return the string left justified in a string of length *width*. Padding is done + using the specified *fillchar* (default is a space). The original string is + returned if *width* is less than or equal to ``len(s)``. + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + +.. method:: bytes.lstrip([chars]) + + Return a copy of the string with leading characters removed. The *chars* + argument is a string specifying the set of characters to be removed. If omitted + or ``None``, the *chars* argument defaults to removing whitespace. The *chars* + argument is not a prefix; rather, all combinations of its values are stripped: + + >>> ' spacious '.lstrip() + 'spacious ' + >>> 'www.example.com'.lstrip('cmowz.') + 'example.com' + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.rjust(width[, fillchar]) + + Return the string right justified in a string of length *width*. Padding is done + using the specified *fillchar* (default is a space). The original string is + returned if *width* is less than or equal to ``len(s)``. + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.rsplit(sep=None, maxsplit=-1) + + Return a list of the words in the string, using *sep* as the delimiter string. + If *maxsplit* is given, at most *maxsplit* splits are done, the *rightmost* + ones. If *sep* is not specified or ``None``, any whitespace string is a + separator. Except for splitting from the right, :meth:`rsplit` behaves like + :meth:`split` which is described in detail below. + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.rstrip([chars]) + + Return a copy of the string with trailing characters removed. The *chars* + argument is a string specifying the set of characters to be removed. If omitted + or ``None``, the *chars* argument defaults to removing whitespace. The *chars* + argument is not a suffix; rather, all combinations of its values are stripped: + + >>> ' spacious '.rstrip() + ' spacious' + >>> 'mississippi'.rstrip('ipz') + 'mississ' + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.split(sep=None, maxsplit=-1) + + Return a list of the words in the string, using *sep* as the delimiter + string. If *maxsplit* is given, at most *maxsplit* splits are done (thus, + the list will have at most ``maxsplit+1`` elements). If *maxsplit* is not + specified or ``-1``, then there is no limit on the number of splits + (all possible splits are made). + + If *sep* is given, consecutive delimiters are not grouped together and are + deemed to delimit empty strings (for example, ``'1,,2'.split(',')`` returns + ``['1', '', '2']``). The *sep* argument may consist of multiple characters + (for example, ``'123'.split('')`` returns ``['1', '2', '3']``). + Splitting an empty string with a specified separator returns ``['']``. + + If *sep* is not specified or is ``None``, a different splitting algorithm is + applied: runs of consecutive whitespace are regarded as a single separator, + and the result will contain no empty strings at the start or end if the + string has leading or trailing whitespace. Consequently, splitting an empty + string or a string consisting of just whitespace with a ``None`` separator + returns ``[]``. + + For example, ``' 1 2 3 '.split()`` returns ``['1', '2', '3']``, and + ``' 1 2 3 '.split(None, 1)`` returns ``['1', '2 3 ']``. + + +.. method:: bytes.strip([chars]) + + Return a copy of the string with the leading and trailing characters removed. + The *chars* argument is a string specifying the set of characters to be removed. + If omitted or ``None``, the *chars* argument defaults to removing whitespace. + The *chars* argument is not a prefix or suffix; rather, all combinations of its + values are stripped: + + >>> ' spacious '.strip() + 'spacious' + >>> 'www.example.com'.strip('cmowz.') + 'example' + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +The following methods on bytes and bytearray objects assume the use of ASCII +compatible binary formats and should not be applied to arbitrary binary data. +Note that all of the bytearray methods in this section do *not* operate in +place, and instead produce new objects. + +.. method:: bytes.capitalize() + + Return a copy of the string with its first character capitalized and the + rest lowercased. + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.casefold() + + Return a casefolded copy of the string. Casefolded strings may be used for + caseless matching. + + Casefolding is similar to lowercasing but more aggressive because it is + intended to remove all case distinctions in a string. For example, the German + lowercase letter ``'ß'`` is equivalent to ``"ss"``. Since it is already + lowercase, :meth:`lower` would do nothing to ``'ß'``; :meth:`casefold` + converts it to ``"ss"``. + + The casefolding algorithm is described in section 3.13 of the Unicode + Standard. + + .. versionadded:: 3.3 + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.expandtabs(tabsize=8) + + Return a copy of the string where all tab characters are replaced by one or + more spaces, depending on the current column and the given tab size. Tab + positions occur every *tabsize* characters (default is 8, giving tab + positions at columns 0, 8, 16 and so on). To expand the string, the current + column is set to zero and the string is examined character by character. If + the character is a tab (``\t``), one or more space characters are inserted + in the result until the current column is equal to the next tab position. + (The tab character itself is not copied.) If the character is a newline + (``\n``) or return (``\r``), it is copied and the current column is reset to + zero. Any other character is copied unchanged and the current column is + incremented by one regardless of how the character is represented when + printed. + + >>> '01\t012\t0123\t01234'.expandtabs() + '01 012 0123 01234' + >>> '01\t012\t0123\t01234'.expandtabs(4) + '01 012 0123 01234' + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.isalnum() + + Return true if all characters in the string are alphanumeric and there is at + least one character, false otherwise. A character ``c`` is alphanumeric if one + of the following returns ``True``: ``c.isalpha()``, ``c.isdecimal()``, + ``c.isdigit()``, or ``c.isnumeric()``. + + +.. method:: bytes.isalpha() + + Return true if all characters in the string are alphabetic and there is at least + one character, false otherwise. Alphabetic characters are those characters defined + in the Unicode character database as "Letter", i.e., those with general category + property being one of "Lm", "Lt", "Lu", "Ll", or "Lo". Note that this is different + from the "Alphabetic" property defined in the Unicode Standard. + + +.. method:: bytes.isdecimal() + + Return true if all characters in the string are decimal + characters and there is at least one character, false + otherwise. Decimal characters are those from general category "Nd". This category + includes digit characters, and all characters + that can be used to form decimal-radix numbers, e.g. U+0660, + ARABIC-INDIC DIGIT ZERO. + + +.. method:: bytes.isdigit() + + Return true if all characters in the string are digits and there is at least one + character, false otherwise. Digits include decimal characters and digits that need + special handling, such as the compatibility superscript digits. Formally, a digit + is a character that has the property value Numeric_Type=Digit or Numeric_Type=Decimal. + + +.. method:: bytes.islower() + + Return true if all cased characters [4]_ in the string are lowercase and + there is at least one cased character, false otherwise. + + +.. method:: bytes.isspace() + + Return true if there are only whitespace characters in the string and there is + at least one character, false otherwise. Whitespace characters are those + characters defined in the Unicode character database as "Other" or "Separator" + and those with bidirectional property being one of "WS", "B", or "S". + + +.. method:: bytes.istitle() + + Return true if the string is a titlecased string and there is at least one + character, for example uppercase characters may only follow uncased characters + and lowercase characters only cased ones. Return false otherwise. + + +.. method:: bytes.isupper() + + Return true if all cased characters [4]_ in the string are uppercase and + there is at least one cased character, false otherwise. + + +.. method:: bytes.lower() + + Return a copy of the string with all the cased characters [4]_ converted to + lowercase. + + The lowercasing algorithm used is described in section 3.13 of the Unicode + Standard. + + +.. index:: + single: universal newlines; bytes.splitlines method + +.. method:: bytes.splitlines([keepends]) + + Return a list of the lines in the string, breaking at line boundaries. + This method uses the :term:`universal newlines` approach to splitting lines. + Line breaks are not included in the resulting list unless *keepends* is + given and true. + + For example, ``'ab c\n\nde fg\rkl\r\n'.splitlines()`` returns + ``['ab c', '', 'de fg', 'kl']``, while the same call with ``splitlines(True)`` + returns ``['ab c\n', '\n', 'de fg\r', 'kl\r\n']``. + + Unlike :meth:`~bytes.split` when a delimiter string *sep* is given, this + method returns an empty list for the empty string, and a terminal line + break does not result in an extra line. + + +.. method:: bytes.swapcase() + + Return a copy of the string with uppercase characters converted to lowercase and + vice versa. Note that it is not necessarily true that + ``s.swapcase().swapcase() == s``. + + +.. method:: bytes.title() + + Return a titlecased version of the string where words start with an uppercase + character and the remaining characters are lowercase. + + The algorithm uses a simple language-independent definition of a word as + groups of consecutive letters. The definition works in many contexts but + it means that apostrophes in contractions and possessives form word + boundaries, which may not be the desired result:: + + >>> "they're bill's friends from the UK".title() + "They'Re Bill'S Friends From The Uk" + + A workaround for apostrophes can be constructed using regular expressions:: + + >>> import re + >>> def titlecase(s): + ... return re.sub(r"[A-Za-z]+('[A-Za-z]+)?", + ... lambda mo: mo.group(0)[0].upper() + + ... mo.group(0)[1:].lower(), + ... s) + ... + >>> titlecase("they're bill's friends.") + "They're Bill's Friends." + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.upper() + + Return a copy of the string with all the cased characters [4]_ converted to + uppercase. Note that ``bytes.upper().isupper()`` might be ``False`` if ``s`` + contains uncased characters or if the Unicode category of the resulting + character(s) is not "Lu" (Letter, uppercase), but e.g. "Lt" (Letter, + titlecase). + + The uppercasing algorithm used is described in section 3.13 of the Unicode + Standard. + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. + + +.. method:: bytes.zfill(width) + + Return the numeric string left filled with zeros in a string of length + *width*. A sign prefix is handled correctly. The original string is + returned if *width* is less than or equal to ``len(s)``. + + .. note:: + + The bytearray version of this method does *not* operate in place - it + always produces a new object, even if no changes were made. .. _typememoryview: