Changeset 391 for python/trunk/Doc/faq/programming.rst

Timestamp:

Mar 19, 2014, 11:31:01 PM (11 years ago)

Author:

dmik

Message:

python: Merge vendor 2.7.6 to trunk.

Location:

python/trunk

Files:

• . (modified) (1 prop)
• Doc/faq/programming.rst (modified) (38 diffs)

python/trunk/Doc/faq/programming.rst

@@ -5 +5 @@
 ===============
 
-.. contents::
+.. only:: html
+
+   .. contents::
 
 General Questions
@@ -172 +174 @@
 
    L2 = []
-   for i in range[3]:
+   for i in range(3):
        L2.append(L1[i])
 
 it is much shorter and far faster to use ::
 
-   L2 = list(L1[:3]) # "list" is redundant if L1 is a list.
+   L2 = list(L1[:3])  # "list" is redundant if L1 is a list.
 
 Note that the functionally-oriented built-in functions such as :func:`map`,
@@ -184 +186 @@
 together::
 
-   >>> zip([1,2,3], [4,5,6])
+   >>> zip([1, 2, 3], [4, 5, 6])
    [(1, 4), (2, 5), (3, 6)]
 
 or to compute a number of sines::
 
-   >>> map( math.sin, (1,2,3,4))
-   [0.841470984808, 0.909297426826, 0.14112000806,   -0.756802495308]
+   >>> map(math.sin, (1, 2, 3, 4))
+   [0.841470984808, 0.909297426826, 0.14112000806, -0.756802495308]
 
 The operation completes very quickly in such cases.
 
-Other examples include the ``join()`` and ``split()`` methods of string objects.
+Other examples include the ``join()`` and ``split()`` :ref:`methods
+of string objects <string-methods>`.
 For example if s1..s7 are large (10K+) strings then
 ``"".join([s1,s2,s3,s4,s5,s6,s7])`` may be far faster than the more obvious
 ``s1+s2+s3+s4+s5+s6+s7``, since the "summation" will compute many
 subexpressions, whereas ``join()`` does all the copying in one pass.  For
-manipulating strings, use the ``replace()`` method on string objects. Use
-regular expressions only when you're not dealing with constant string patterns.
-Consider using the string formatting operations ``string % tuple`` and ``string
-% dictionary``.
+manipulating strings, use the ``replace()`` and the ``format()`` :ref:`methods
+on string objects <string-methods>`.  Use regular expressions only when you're
+not dealing with constant string patterns.  You may still use :ref:`the old %
+operations <string-formatting>` ``string % tuple`` and ``string % dictionary``.
 
 Be sure to use the :meth:`list.sort` built-in method to do sorting, and see the
@@ -212 +215 @@
 suppose you have a program that runs slowly and you use the profiler to
 determine that a Python function ``ff()`` is being called lots of times.  If you
-notice that ``ff ()``::
+notice that ``ff()``::
 
    def ff(x):
@@ -333 +336 @@
    11
 
-In Python3, you can do a similar thing in a nested scope using the
-:keyword:`nonlocal` keyword:
-
-.. doctest::
-   :options: +SKIP
-
-   >>> def foo():
-   ...    x = 10
-   ...    def bar():
-   ...        nonlocal x
-   ...        print x
-   ...        x += 1
-   ...    bar()
-   ...    print x
-   >>> foo()
-   10
-   11
-
 
 What are the rules for local and global variables in Python?
@@ -370 +355 @@
 
 
+Why do lambdas defined in a loop with different values all return the same result?
+----------------------------------------------------------------------------------
+
+Assume you use a for loop to define a few different lambdas (or even plain
+functions), e.g.::
+
+   >>> squares = []
+   >>> for x in range(5):
+   ...    squares.append(lambda: x**2)
+
+This gives you a list that contains 5 lambdas that calculate ``x**2``.  You
+might expect that, when called, they would return, respectively, ``0``, ``1``,
+``4``, ``9``, and ``16``.  However, when you actually try you will see that
+they all return ``16``::
+
+   >>> squares[2]()
+   16
+   >>> squares[4]()
+   16
+
+This happens because ``x`` is not local to the lambdas, but is defined in
+the outer scope, and it is accessed when the lambda is called --- not when it
+is defined.  At the end of the loop, the value of ``x`` is ``4``, so all the
+functions now return ``4**2``, i.e. ``16``.  You can also verify this by
+changing the value of ``x`` and see how the results of the lambdas change::
+
+   >>> x = 8
+   >>> squares[2]()
+   64
+
+In order to avoid this, you need to save the values in variables local to the
+lambdas, so that they don't rely on the value of the global ``x``::
+
+   >>> squares = []
+   >>> for x in range(5):
+   ...    squares.append(lambda n=x: n**2)
+
+Here, ``n=x`` creates a new variable ``n`` local to the lambda and computed
+when the lambda is defined so that it has the same value that ``x`` had at
+that point in the loop.  This means that the value of ``n`` will be ``0``
+in the first lambda, ``1`` in the second, ``2`` in the third, and so on.
+Therefore each lambda will now return the correct result::
+
+   >>> squares[2]()
+   4
+   >>> squares[4]()
+   16
+
+Note that this behaviour is not peculiar to lambdas, but applies to regular
+functions too.
+
+
 How do I share global variables across modules?
 ------------------------------------------------
@@ -413 +450 @@
 It's good practice if you import modules in the following order:
 
-1. standard library modules -- e.g. ``sys``, ``os``, ``getopt``, ``re``)
+1. standard library modules -- e.g. ``sys``, ``os``, ``getopt``, ``re``
 2. third-party library modules (anything installed in Python's site-packages
    directory) -- e.g. mx.DateTime, ZODB, PIL.Image, etc.
@@ -422 +459 @@
 write ``import m2``, even though it's legal.  Write ``from package.sub import
 m2`` instead.  Relative imports can lead to a module being initialized twice,
-leading to confusing bugs.
+leading to confusing bugs.  See :pep:`328` for details.
 
 It is sometimes necessary to move imports to a function or class to avoid
@@ -487 +524 @@
 
 
+.. index::
+   single: argument; difference from parameter
+   single: parameter; difference from argument
+
+.. _faq-argument-vs-parameter:
+
+What is the difference between arguments and parameters?
+--------------------------------------------------------
+
+:term:`Parameters <parameter>` are defined by the names that appear in a
+function definition, whereas :term:`arguments <argument>` are the values
+actually passed to a function when calling it.  Parameters define what types of
+arguments a function can accept.  For example, given the function definition::
+
+   def func(foo, bar=None, **kwargs):
+       pass
+
+*foo*, *bar* and *kwargs* are parameters of ``func``.  However, when calling
+``func``, for example::
+
+   func(42, bar=314, extra=somevar)
+
+the values ``42``, ``314``, and ``somevar`` are arguments.
+
+
 How do I write a function with output parameters (call by reference)?
 ---------------------------------------------------------------------
@@ -650 +712 @@
    b = a
    print b
-   <__main__.A instance at 016D07CC>
+   <__main__.A instance at 0x16D07CC>
    print a
-   <__main__.A instance at 016D07CC>
+   <__main__.A instance at 0x16D07CC>
 
 Arguably the class has a name: even though it is bound to two names and invoked
@@ -682 +744 @@
 
     >>> "a" in "b", "a"
-    (False, '1')
+    (False, 'a')
 
 Since the comma is not an operator, but a separator between expressions the
 above is evaluated as if you had entered::
 
-    >>> ("a" in "b"), "a"
+    ("a" in "b"), "a"
 
 not::
 
-    >>> "a" in ("5", "a")
+    "a" in ("b", "a")
 
 The same is true of the various assignment operators (``=``, ``+=`` etc).  They
@@ -709 +771 @@
 
 For versions previous to 2.5 the answer would be 'No'.
-
-.. XXX remove rest?
-
-In many cases you can mimic ``a ? b : c`` with ``a and b or c``, but there's a
-flaw: if *b* is zero (or empty, or ``None`` -- anything that tests false) then
-*c* will be selected instead.  In many cases you can prove by looking at the
-code that this can't happen (e.g. because *b* is a constant or has a type that
-can never be false), but in general this can be a problem.
-
-Tim Peters (who wishes it was Steve Majewski) suggested the following solution:
-``(a and [b] or [c])[0]``.  Because ``[b]`` is a singleton list it is never
-false, so the wrong path is never taken; then applying ``[0]`` to the whole
-thing gets the *b* or *c* that you really wanted.  Ugly, but it gets you there
-in the rare cases where it is really inconvenient to rewrite your code using
-'if'.
-
-The best course is usually to write a simple ``if...else`` statement.  Another
-solution is to implement the ``?:`` operator as a function::
-
-   def q(cond, on_true, on_false):
-       if cond:
-           if not isfunction(on_true):
-               return on_true
-           else:
-               return apply(on_true)
-       else:
-           if not isfunction(on_false):
-               return on_false
-           else:
-               return apply(on_false)
-
-In most cases you'll pass b and c directly: ``q(a, b, c)``.  To avoid evaluating
-b or c when they shouldn't be, encapsulate them within a lambda function, e.g.:
-``q(a, lambda: b, lambda: c)``.
-
-It has been asked *why* Python has no if-then-else expression.  There are
-several answers: many languages do just fine without one; it can easily lead to
-less readable code; no sufficiently "Pythonic" syntax has been discovered; a
-search of the standard library found remarkably few places where using an
-if-then-else expression would make the code more understandable.
-
-In 2002, :pep:`308` was written proposing several possible syntaxes and the
-community was asked to vote on the issue.  The vote was inconclusive.  Most
-people liked one of the syntaxes, but also hated other syntaxes; many votes
-implied that people preferred no ternary operator rather than having a syntax
-they hated.
 
 
@@ -768 +784 @@
 
    # First 10 Fibonacci numbers
-   print map(lambda x,f=lambda x,f:(x<=1) or (f(x-1,f)+f(x-2,f)): f(x,f),
+   print map(lambda x,f=lambda x,f:(f(x-1,f)+f(x-2,f)) if x>1 else 1: f(x,f),
    range(10))
 
@@ -794 +810 @@
 ------------------------------------------------
 
-To specify an octal digit, precede the octal value with a zero.  For example, to
-set the variable "a" to the octal value "10" (8 in decimal), type::
-
-   >>> a = 010
+To specify an octal digit, precede the octal value with a zero, and then a lower
+or uppercase "o".  For example, to set the variable "a" to the octal value "10"
+(8 in decimal), type::
+
+   >>> a = 0o10
    >>> a
    8
@@ -813 +830 @@
 
 
-Why does -22 / 10 return -3?
-----------------------------
+Why does -22 // 10 return -3?
+-----------------------------
 
 It's primarily driven by the desire that ``i % j`` have the same sign as ``j``.
 If you want that, and also want::
 
-    i == (i / j) * j + (i % j)
+    i == (i // j) * j + (i % j)
 
 then integer division has to return the floor.  C also requires that identity to
-hold, and then compilers that truncate ``i / j`` need to make ``i % j`` have the
-same sign as ``i``.
+hold, and then compilers that truncate ``i // j`` need to make ``i % j`` have
+the same sign as ``i``.
 
 There are few real use cases for ``i % j`` when ``j`` is negative.  When ``j``
@@ -830 +847 @@
 ago?  ``-190 % 12 == 2`` is useful; ``-190 % 12 == -10`` is a bug waiting to
 bite.
+
+.. note::
+
+   On Python 2, ``a / b`` returns the same as ``a // b`` if
+   ``__future__.division`` is not in effect.  This is also known as "classic"
+   division.
 
 
@@ -862 +885 @@
 To convert, e.g., the number 144 to the string '144', use the built-in type
 constructor :func:`str`.  If you want a hexadecimal or octal representation, use
-the built-in functions ``hex()`` or ``oct()``.  For fancy formatting, use
-:ref:`the % operator <string-formatting>` on strings, e.g. ``"%04d" % 144``
-yields ``'0144'`` and ``"%.3f" % (1/3.0)`` yields ``'0.333'``.  See the library
-reference manual for details.
+the built-in functions :func:`hex` or :func:`oct`.  For fancy formatting, see
+the :ref:`formatstrings` section, e.g. ``"{:04d}".format(144)`` yields
+``'0144'`` and ``"{:.3f}".format(1/3)`` yields ``'0.333'``.  You may also use
+:ref:`the % operator <string-formatting>` on strings.  See the library reference
+manual for details.
 
 
@@ -874 +898 @@
 ability, try converting the string to a list or use the array module::
 
+   >>> import io
    >>> s = "Hello, world"
    >>> a = list(s)
@@ -887 +912 @@
    array('c', 'Hello, world')
    >>> a[0] = 'y' ; print a
-   array('c', 'yello world')
+   array('c', 'yello, world')
    >>> a.tostring()
    'yello, world'
@@ -954 +979 @@
 
 Starting with Python 2.2, you can use ``S.rstrip("\r\n")`` to remove all
-occurences of any line terminator from the end of the string ``S`` without
+occurrences of any line terminator from the end of the string ``S`` without
 removing other trailing whitespace.  If the string ``S`` represents more than
 one line, with several empty lines at the end, the line terminators for all the
@@ -963 +988 @@
    ...          "\r\n")
    >>> lines.rstrip("\n\r")
-   "line 1 "
+   'line 1 '
 
 Since this is typically only desired when reading text one line at a time, using
 ``S.rstrip()`` this way works well.
 
-For older versions of Python, There are two partial substitutes:
+For older versions of Python, there are two partial substitutes:
 
 - If you want to remove all trailing whitespace, use the ``rstrip()`` method of
@@ -989 +1014 @@
 if the line uses something other than whitespace as a separator.
 
-For more complicated input parsing, regular expressions more powerful than C's
-:cfunc:`sscanf` and better suited for the task.
+For more complicated input parsing, regular expressions are more powerful
+than C's :c:func:`sscanf` and better suited for the task.
 
 
@@ -1094 +1119 @@
 list, deleting duplicates as you go::
 
-   if List:
-       List.sort()
-       last = List[-1]
-       for i in range(len(List)-2, -1, -1):
-           if last == List[i]:
-               del List[i]
+   if mylist:
+       mylist.sort()
+       last = mylist[-1]
+       for i in range(len(mylist)-2, -1, -1):
+           if last == mylist[i]:
+               del mylist[i]
            else:
-               last = List[i]
+               last = mylist[i]
 
 If all elements of the list may be used as dictionary keys (i.e. they are all
@@ -1107 +1132 @@
 
    d = {}
-   for x in List:
-       d[x] = x
-   List = d.values()
+   for x in mylist:
+       d[x] = 1
+   mylist = list(d.keys())
 
 In Python 2.5 and later, the following is possible instead::
 
-   List = list(set(List))
+   mylist = list(set(mylist))
 
 This converts the list into a set, thereby removing duplicates, and then back
@@ -1149 +1174 @@
 You probably tried to make a multidimensional array like this::
 
-   A = [[None] * 2] * 3
+   >>> A = [[None] * 2] * 3
 
 This looks correct if you print it::
@@ -1181 +1206 @@
 
 Or, you can use an extension that provides a matrix datatype; `Numeric Python
-<http://numpy.scipy.org/>`_ is the best known.
+<http://www.numpy.org/>`_ is the best known.
 
 
@@ -1189 +1214 @@
 Use a list comprehension::
 
-   result = [obj.method() for obj in List]
+   result = [obj.method() for obj in mylist]
 
 More generically, you can try the following function::
@@ -1200 +1225 @@
 
 
+Why does a_tuple[i] += ['item'] raise an exception when the addition works?
+---------------------------------------------------------------------------
+
+This is because of a combination of the fact that augmented assignment
+operators are *assignment* operators, and the difference between mutable and
+immutable objects in Python.
+
+This discussion applies in general when augmented assignment operators are
+applied to elements of a tuple that point to mutable objects, but we'll use
+a ``list`` and ``+=`` as our exemplar.
+
+If you wrote::
+
+   >>> a_tuple = (1, 2)
+   >>> a_tuple[0] += 1
+   Traceback (most recent call last):
+      ...
+   TypeError: 'tuple' object does not support item assignment
+
+The reason for the exception should be immediately clear: ``1`` is added to the
+object ``a_tuple[0]`` points to (``1``), producing the result object, ``2``,
+but when we attempt to assign the result of the computation, ``2``, to element
+``0`` of the tuple, we get an error because we can't change what an element of
+a tuple points to.
+
+Under the covers, what this augmented assignment statement is doing is
+approximately this::
+
+   >>> result = a_tuple[0] + 1
+   >>> a_tuple[0] = result
+   Traceback (most recent call last):
+     ...
+   TypeError: 'tuple' object does not support item assignment
+
+It is the assignment part of the operation that produces the error, since a
+tuple is immutable.
+
+When you write something like::
+
+   >>> a_tuple = (['foo'], 'bar')
+   >>> a_tuple[0] += ['item']
+   Traceback (most recent call last):
+     ...
+   TypeError: 'tuple' object does not support item assignment
+
+The exception is a bit more surprising, and even more surprising is the fact
+that even though there was an error, the append worked::
+
+    >>> a_tuple[0]
+    ['foo', 'item']
+
+To see why this happens, you need to know that (a) if an object implements an
+``__iadd__`` magic method, it gets called when the ``+=`` augmented assignment
+is executed, and its return value is what gets used in the assignment statement;
+and (b) for lists, ``__iadd__`` is equivalent to calling ``extend`` on the list
+and returning the list.  That's why we say that for lists, ``+=`` is a
+"shorthand" for ``list.extend``::
+
+    >>> a_list = []
+    >>> a_list += [1]
+    >>> a_list
+    [1]
+
+This is equivalent to::
+
+    >>> result = a_list.__iadd__([1])
+    >>> a_list = result
+
+The object pointed to by a_list has been mutated, and the pointer to the
+mutated object is assigned back to ``a_list``.  The end result of the
+assignment is a no-op, since it is a pointer to the same object that ``a_list``
+was previously pointing to, but the assignment still happens.
+
+Thus, in our tuple example what is happening is equivalent to::
+
+   >>> result = a_tuple[0].__iadd__(['item'])
+   >>> a_tuple[0] = result
+   Traceback (most recent call last):
+     ...
+   TypeError: 'tuple' object does not support item assignment
+
+The ``__iadd__`` succeeds, and thus the list is extended, but even though
+``result`` points to the same object that ``a_tuple[0]`` already points to,
+that final assignment still results in an error, because tuples are immutable.
+
+
 Dictionaries
 ============
@@ -1214 +1325 @@
 be presented in order sorted by the key.
 
-A more complicated solution is to subclass ``UserDict.UserDict`` to create a
+A more complicated solution is to subclass ``dict`` to create a
 ``SortedDict`` class that prints itself in a predictable order.  Here's one
 simpleminded implementation of such a class::
 
-   import UserDict, string
-
-   class SortedDict(UserDict.UserDict):
+   class SortedDict(dict):
        def __repr__(self):
-           result = []
-           append = result.append
-           keys = self.data.keys()
-           keys.sort()
-           for k in keys:
-               append("%s: %s" % (`k`, `self.data[k]`))
-           return "{%s}" % string.join(result, ", ")
-
-     __str__ = __repr__
+           keys = sorted(self.keys())
+           result = ("{!r}: {!r}".format(k, self[k]) for k in keys)
+           return "{{{}}}".format(", ".join(result))
+
+       __str__ = __repr__
 
 This will work for many common situations you might encounter, though it's far
@@ -1252 +1357 @@
 strings by their uppercase values::
 
-  tmp1 = [(x.upper(), x) for x in L] # Schwartzian transform
+  tmp1 = [(x.upper(), x) for x in L]  # Schwartzian transform
   tmp1.sort()
   Usorted = [x[1] for x in tmp1]
@@ -1259 +1364 @@
 each string::
 
-  tmp2 = [(int(s[10:15]), s) for s in L] # Schwartzian transform
+  tmp2 = [(int(s[10:15]), s) for s in L]  # Schwartzian transform
   tmp2.sort()
   Isorted = [x[1] for x in tmp2]
@@ -1296 +1401 @@
 An alternative for the last step is::
 
-   result = []
-   for p in pairs: result.append(p[1])
+   >>> result = []
+   >>> for p in pairs: result.append(p[1])
 
 If you find this more legible, you might prefer to use this instead of the final
@@ -1365 +1470 @@
 that does something::
 
-   def search (obj):
+   def search(obj):
        if isinstance(obj, Mailbox):
            # ... code to search a mailbox
@@ -1468 +1573 @@
 -----------------------------------------------------------
 
-Static data (in the sense of C++ or Java) is easy; static methods (again in the
-sense of C++ or Java) are not supported directly.
+Both static data and static methods (in the sense of C++ or Java) are supported
+in Python.
 
 For static data, simply define a class attribute.  To assign a new value to the
@@ -1488 +1593 @@
 
 Caution: within a method of C, an assignment like ``self.count = 42`` creates a
-new and unrelated instance vrbl named "count" in ``self``'s own dict.  Rebinding
-of a class-static data name must always specify the class whether inside a
-method or not::
+new and unrelated instance named "count" in ``self``'s own dict.  Rebinding of a
+class-static data name must always specify the class whether inside a method or
+not::
 
    C.count = 314
@@ -1620 +1725 @@
 
 
+Why does the result of ``id()`` appear to be not unique?
+--------------------------------------------------------
+
+The :func:`id` builtin returns an integer that is guaranteed to be unique during
+the lifetime of the object.  Since in CPython, this is the object's memory
+address, it happens frequently that after an object is deleted from memory, the
+next freshly created object is allocated at the same position in memory.  This
+is illustrated by this example:
+
+>>> id(1000)
+13901272
+>>> id(2000)
+13901272
+
+The two ids belong to different integer objects that are created before, and
+deleted immediately after execution of the ``id()`` call.  To be sure that
+objects whose id you want to examine are still alive, create another reference
+to the object:
+
+>>> a = 1000; b = 2000
+>>> id(a)
+13901272
+>>> id(b)
+13891296
+
+
 Modules
 =======
@@ -1637 +1768 @@
 directory.
 
-Running Python on a top level script is not considered an import and no ``.pyc``
-will be created.  For example, if you have a top-level module ``abc.py`` that
-imports another module ``xyz.py``, when you run abc, ``xyz.pyc`` will be created
-since xyz is imported, but no ``abc.pyc`` file will be created since ``abc.py``
-isn't being imported.
-
-If you need to create abc.pyc -- that is, to create a .pyc file for a module
+Running Python on a top level script is not considered an import and no
+``.pyc`` will be created.  For example, if you have a top-level module
+``foo.py`` that imports another module ``xyz.py``, when you run ``foo``,
+``xyz.pyc`` will be created since ``xyz`` is imported, but no ``foo.pyc`` file
+will be created since ``foo.py`` isn't being imported.
+
+If you need to create ``foo.pyc`` -- that is, to create a ``.pyc`` file for a module
 that is not imported -- you can, using the :mod:`py_compile` and
 :mod:`compileall` modules.
@@ -1651 +1782 @@
 
    >>> import py_compile
-   >>> py_compile.compile('abc.py')
-
-This will write the ``.pyc`` to the same location as ``abc.py`` (or you can
+   >>> py_compile.compile('foo.py')                 # doctest: +SKIP
+
+This will write the ``.pyc`` to the same location as ``foo.py`` (or you can
 override that with the optional parameter ``cfile``).