From 436dec875c0119a93a9f22fad8a800e7b88489d4 Mon Sep 17 00:00:00 2001 From: Marc Poulin Date: Tue, 11 Sep 2018 15:52:31 -0600 Subject: [PATCH 1/6] Issue 796, clarify ways to manipulate a list --- docs/writing/style.rst | 53 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 50 insertions(+), 3 deletions(-) diff --git a/docs/writing/style.rst b/docs/writing/style.rst index 59781c6ee..4de1f28d5 100644 --- a/docs/writing/style.rst +++ b/docs/writing/style.rst @@ -581,6 +581,32 @@ provide a powerful, concise way to work with lists. Also, the :py:func:`map` and :py:func:`filter` functions can perform operations on lists using a different, more concise syntax. +Starting with Python 3.0, the :py:func:`map` and :py:func:`filter` +functions return an iterator instead of a list. If you really need a list, you +should wrap these functions in :py:func`list` like so + +.. code-block:: python + + list(map(...)) + list(filter(...)) + +Filtering a list +~~~~~~~~~~~~~~~~ + +**Very Bad**: + +Never remove items from a list that you are iterating over. +Python will lose track of its current position. + +.. code-block:: python + + # Filter elements greater than 4 + a = [3, 4, 5] + for i in a: + if i > 4: + a.remove(i) + + **Bad**: .. code-block:: python @@ -598,9 +624,13 @@ more concise syntax. a = [3, 4, 5] b = [i for i in a if i > 4] - # Or: + # Or (Python 2.x): b = filter(lambda x: x > 4, a) - + # Or (Python 3.x) + b = list(filter(lambda x: x > 4, a)) + +Modifying the values in a list +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Bad**: .. code-block:: python @@ -616,8 +646,25 @@ more concise syntax. a = [3, 4, 5] a = [i + 3 for i in a] - # Or: + # Or (Python 2.x): a = map(lambda i: i + 3, a) + # Or (Python 3.x) + a = list(map(lambda i: i + 3, a)) + +**Best**: + +Creating a new list instead of modifying the original list will prevent +unexpected side-effects. + +.. code-block:: python + + a = [3, 4, 5] + b = [i + 3 for i in a] + # Or (Python 2.x): + b = map(lambda i: i + 3, a) + # Or (Python 3.x) + b = list(map(lambda i: i + 3, a)) + Use :py:func:`enumerate` keep a count of your place in the list. From e69d7e7657db812c7ac74b6a439d5c7a1290fe5c Mon Sep 17 00:00:00 2001 From: Marc Poulin Date: Thu, 13 Sep 2018 11:00:37 -0600 Subject: [PATCH 2/6] 2.x vs 3.x, lists vs. iterators Describe the differences between 2.x and 3.x and suggest situations when iterators would be more appropriate than lists. --- docs/writing/style.rst | 82 +++++++++++++++++++++++++++++------------- 1 file changed, 57 insertions(+), 25 deletions(-) diff --git a/docs/writing/style.rst b/docs/writing/style.rst index 4de1f28d5..e71ffa34e 100644 --- a/docs/writing/style.rst +++ b/docs/writing/style.rst @@ -581,21 +581,12 @@ provide a powerful, concise way to work with lists. Also, the :py:func:`map` and :py:func:`filter` functions can perform operations on lists using a different, more concise syntax. -Starting with Python 3.0, the :py:func:`map` and :py:func:`filter` -functions return an iterator instead of a list. If you really need a list, you -should wrap these functions in :py:func`list` like so - -.. code-block:: python - - list(map(...)) - list(filter(...)) - Filtering a list ~~~~~~~~~~~~~~~~ -**Very Bad**: +**Bad**: -Never remove items from a list that you are iterating over. +Never remove items from list while you are iterating through it. Python will lose track of its current position. .. code-block:: python @@ -606,28 +597,69 @@ Python will lose track of its current position. if i > 4: a.remove(i) +Python has standard ways of filtering lists, but there are several things you need to consider -**Bad**: +* Python 2.x vs. 3.x +* Lists vs. iterators +* Creating a new list vs. modifying the original list + +Python 2.x vs. 3.x +:::::::::::::::::: + +* Starting with Python 3.0, the :py:func:`map` and :py:func:`filter` +functions return an iterator instead of a list. If you really need a list, you +should wrap these functions in :py:func`list` like so .. code-block:: python - # Filter elements greater than 4 - a = [3, 4, 5] - b = [] - for i in a: - if i > 4: - b.append(i) + list(map(...)) + list(filter(...)) -**Good**: +* List comprehensions and generator expressions work the same in both 2.x and 3.x (except that comprehensions in 2.x "leak" variables into the enclosing namespace) + + * comprehensions create a new list object + * generators iterate over the original list + +* The filter function + + * in 2.x returns a list (use itertools.ifilter if you want an iterator) + * in 3.x returns an iterator + +Lists vs. iterators +::::::::::::::::::: + +Creating a new list requires more work and uses more memory. If you a just going to loop through the new list, consider using an iterator instead. .. code-block:: python - a = [3, 4, 5] - b = [i for i in a if i > 4] - # Or (Python 2.x): - b = filter(lambda x: x > 4, a) - # Or (Python 3.x) - b = list(filter(lambda x: x > 4, a)) + # comprehensions create a new list object + filtered_values = [value for value in sequence if value != x] + # Or (2.x) + filtered_values = filter(lambda i: i != x, sequence) + + # generators are lazy + filtered_values = (value for value in sequence if value != x) + # Or (3.x) + filtered_values = filter(lambda i: i != x, sequence) + # Or (2.x) + filtered_values = itertools.ifilter(lambda i: i != x, sequence) + + + +Creating a new list vs. modifying the original list +::::::::::::::::::::::::::::::::::::::::::::::::::: + +Modifying the original list can be risky if there are other variables referencing it. But you can use *slice assignment* if you really want to do that. + +.. code-block:: python + + # replace the contents of the original list + sequence[::] = [value for value in sequence if value != x] + # Or + sequence[::] = (value for value in sequence if value != x) + # Or + sequence[::] = filter(lambda value: value != x, sequence) + Modifying the values in a list ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 29abc5acf3353ff61dd9d9b85c06f5ebbedab7d2 Mon Sep 17 00:00:00 2001 From: Marc Poulin Date: Thu, 13 Sep 2018 11:28:43 -0600 Subject: [PATCH 3/6] Modifying the values in a list To avoid side effects, don't modify the original list. Create a new list instead. --- docs/writing/style.rst | 29 +++++++++++------------------ 1 file changed, 11 insertions(+), 18 deletions(-) diff --git a/docs/writing/style.rst b/docs/writing/style.rst index e71ffa34e..0766c8a7c 100644 --- a/docs/writing/style.rst +++ b/docs/writing/style.rst @@ -606,9 +606,7 @@ Python has standard ways of filtering lists, but there are several things you ne Python 2.x vs. 3.x :::::::::::::::::: -* Starting with Python 3.0, the :py:func:`map` and :py:func:`filter` -functions return an iterator instead of a list. If you really need a list, you -should wrap these functions in :py:func`list` like so +* Starting with Python 3.0, the :py:func:`map` and :py:func:`filter` functions return an iterator instead of a list. If you really need a list, you should wrap these functions in :py:func`list` like so .. code-block:: python @@ -665,38 +663,33 @@ Modifying the values in a list ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Bad**: +Remember that assignment never creates a new object. If 2 or more variables refer to the same list, changing one of them changes them all. + .. code-block:: python # Add three to all list members. a = [3, 4, 5] + b = a # a and b refer to the same list object + for i in range(len(a)): - a[i] += 3 + a[i] += 3 # b[i] also changes **Good**: +It's safer to create a new list object and leave the original alone. + .. code-block:: python a = [3, 4, 5] + b = a + + # assign the variable "a" to a new list without changing "b" a = [i + 3 for i in a] # Or (Python 2.x): a = map(lambda i: i + 3, a) # Or (Python 3.x) a = list(map(lambda i: i + 3, a)) -**Best**: - -Creating a new list instead of modifying the original list will prevent -unexpected side-effects. - -.. code-block:: python - - a = [3, 4, 5] - b = [i + 3 for i in a] - # Or (Python 2.x): - b = map(lambda i: i + 3, a) - # Or (Python 3.x) - b = list(map(lambda i: i + 3, a)) - Use :py:func:`enumerate` keep a count of your place in the list. From 6d8a602a0c9601a0237d59e765e2d383aac2d20a Mon Sep 17 00:00:00 2001 From: Marc Poulin Date: Fri, 14 Sep 2018 10:46:54 -0600 Subject: [PATCH 4/6] Cleaned up formatting and grammar --- docs/writing/style.rst | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/writing/style.rst b/docs/writing/style.rst index 0766c8a7c..2ead991d9 100644 --- a/docs/writing/style.rst +++ b/docs/writing/style.rst @@ -597,7 +597,8 @@ Python will lose track of its current position. if i > 4: a.remove(i) -Python has standard ways of filtering lists, but there are several things you need to consider +Python has a few standard ways of filtering lists. +You will need to consider * Python 2.x vs. 3.x * Lists vs. iterators @@ -606,19 +607,19 @@ Python has standard ways of filtering lists, but there are several things you ne Python 2.x vs. 3.x :::::::::::::::::: -* Starting with Python 3.0, the :py:func:`map` and :py:func:`filter` functions return an iterator instead of a list. If you really need a list, you should wrap these functions in :py:func`list` like so +Starting with Python 3.0, the :py:func:`filter` function returns an iterator instead of a list. +If you really need a list, you should wrap it in :py:func:`list` like so .. code-block:: python - list(map(...)) list(filter(...)) -* List comprehensions and generator expressions work the same in both 2.x and 3.x (except that comprehensions in 2.x "leak" variables into the enclosing namespace) +List comprehensions and generator expressions work the same in both 2.x and 3.x (except that comprehensions in 2.x "leak" variables into the enclosing namespace) * comprehensions create a new list object * generators iterate over the original list -* The filter function +The :py:func:`filter` function * in 2.x returns a list (use itertools.ifilter if you want an iterator) * in 3.x returns an iterator @@ -663,7 +664,7 @@ Modifying the values in a list ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ **Bad**: -Remember that assignment never creates a new object. If 2 or more variables refer to the same list, changing one of them changes them all. +Remember that assignment never creates a new object. If two or more variables refer to the same list, changing one of them changes them all. .. code-block:: python From dc078e1a66ba3ad3e55f490fdd6f9db2b156c52c Mon Sep 17 00:00:00 2001 From: Marc Poulin Date: Fri, 14 Sep 2018 11:11:16 -0600 Subject: [PATCH 5/6] Cleaned up some wording --- docs/writing/style.rst | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/docs/writing/style.rst b/docs/writing/style.rst index 2ead991d9..1ff838231 100644 --- a/docs/writing/style.rst +++ b/docs/writing/style.rst @@ -586,8 +586,7 @@ Filtering a list **Bad**: -Never remove items from list while you are iterating through it. -Python will lose track of its current position. +Never remove items from a list while you are iterating through it. .. code-block:: python @@ -597,8 +596,17 @@ Python will lose track of its current position. if i > 4: a.remove(i) +Don't make multiple passes through the list. + +.. code-block:: python + + while i in a: + a.remove(i) + +**Good**: + Python has a few standard ways of filtering lists. -You will need to consider +The approach you use depends on * Python 2.x vs. 3.x * Lists vs. iterators @@ -608,7 +616,7 @@ Python 2.x vs. 3.x :::::::::::::::::: Starting with Python 3.0, the :py:func:`filter` function returns an iterator instead of a list. -If you really need a list, you should wrap it in :py:func:`list` like so +Wrap it in :py:func:`list` if you truly need a list. .. code-block:: python From 1d1ebf540d63464da053d4e4f3c0e5d755b2cd82 Mon Sep 17 00:00:00 2001 From: Marc Poulin Date: Fri, 14 Sep 2018 11:20:53 -0600 Subject: [PATCH 6/6] Side effects of changing a list --- docs/writing/style.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/writing/style.rst b/docs/writing/style.rst index 1ff838231..9fde6b918 100644 --- a/docs/writing/style.rst +++ b/docs/writing/style.rst @@ -610,7 +610,7 @@ The approach you use depends on * Python 2.x vs. 3.x * Lists vs. iterators -* Creating a new list vs. modifying the original list +* Possible side effects of modifying the original list Python 2.x vs. 3.x :::::::::::::::::: @@ -644,7 +644,7 @@ Creating a new list requires more work and uses more memory. If you a just going # Or (2.x) filtered_values = filter(lambda i: i != x, sequence) - # generators are lazy + # generators don't create another list filtered_values = (value for value in sequence if value != x) # Or (3.x) filtered_values = filter(lambda i: i != x, sequence) @@ -653,8 +653,8 @@ Creating a new list requires more work and uses more memory. If you a just going -Creating a new list vs. modifying the original list -::::::::::::::::::::::::::::::::::::::::::::::::::: +Possible side effects of modifying the original list +:::::::::::::::::::::::::::::::::::::::::::::::::::: Modifying the original list can be risky if there are other variables referencing it. But you can use *slice assignment* if you really want to do that.