ProgrammingInPython/modules/Closures.html at gh-pages · UWPCE-PythonCert/ProgrammingInPython

590 lines (545 loc) · 51.7 KB
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Closures and Function Currying &mdash; Programming in Python 7.0 documentation</title>
      <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
      <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
  <!--[if lt IE 9]>
    <script src="../_static/js/html5shiv.min.js"></script>
  <![endif]-->
        <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
        <script src="../_static/jquery.js"></script>
        <script src="../_static/underscore.js"></script>
        <script src="../_static/doctools.js"></script>
    <script src="../_static/js/theme.js"></script>
    <link rel="index" title="Index" href="../genindex.html" />
    <link rel="search" title="Search" href="../search.html" />
    <link rel="next" title="lambda and keyword Magic" href="../exercises/lambda_magic.html" />
    <link rel="prev" title="IPython Parallel Quickstart" href="IPythonParallel.html" /> 
<body class="wy-body-for-nav"> 
  <div class="wy-grid-for-nav">
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #4b2e83" >
            <a href="../index.html">
            <img src="../_static/UWPCE_logo_full.png" class="logo" alt="Logo"/>
          </a>
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
        </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
              <p class="caption" role="heading"><span class="caption-text">Topics in the Program</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../topics/01-setting_up/index.html">1. Setting up your Environment</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/02-basic_python/index.html">2. Basic Python</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/03-recursion_booleans/index.html">3. Booleans and Recursion</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/04-sequences_iteration/index.html">4. Sequences and Iteration</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/05-text_handling/index.html">5. Basic Text Handling</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/06-exceptions/index.html">6. Exception Handling</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/07-unit_testing/index.html">7. Unit Testing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/08-dicts_sets/index.html">8. Dictionaries and Sets</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/09-files/index.html">9. File Handling</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/10-modules_packages/index.html">10. Modules and Packages</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/11-argument_passing/index.html">11. Advanced Argument Passing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/12-comprehensions/index.html">12. Comprehensions</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/13-intro_oo/index.html">13. Intro to Object Oriented Programing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/14-magic_methods/index.html">14. Properties and Magic Methods</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/15-subclassing/index.html">15. Subclassing and Inheritance</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/16-multiple_inheritance/index.html">16. Multiple Inheritance</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../topics/17-functional_programming/index.html">17. Introduction to Functional Programming</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="OO_vs_functional.html">Object Oriented vs Functional Programming</a></li>
<li class="toctree-l2"><a class="reference internal" href="Lambda.html">Anonymous Functions: Lambda</a></li>
<li class="toctree-l2"><a class="reference internal" href="MapFilterReduce.html">Map Filter and Reduce</a></li>
<li class="toctree-l2"><a class="reference internal" href="IPythonParallel.html">IPython Parallel Quickstart</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Closures and Function Currying</a></li>
<li class="toctree-l2"><a class="reference internal" href="../exercises/lambda_magic.html">lambda and keyword Magic</a></li>
<li class="toctree-l2"><a class="reference internal" href="../exercises/trapezoid.html">Trapezoidal Rule</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/18-advanced_testing/index.html">18. Advanced Testing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../topics/99-extras/index.html">19. Extra Topics</a></li>
        </div>
      </div>
    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu"  style="background: #4b2e83" >
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../index.html">Programming in Python</a>
      </nav>
      <div class="wy-nav-content">
        <div class="rst-content style-external-links">
          <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../index.html" class="icon icon-home"></a> &raquo;</li>
          <li><a href="../topics/17-functional_programming/index.html"><span class="section-number">17. </span>Introduction to Functional Programming</a> &raquo;</li>
      <li>Closures and Function Currying</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../_sources/modules/Closures.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul><div class="rst-breadcrumbs-buttons" role="navigation" aria-label="Sequential page navigation">
        <a href="IPythonParallel.html" class="btn btn-neutral float-left" title="IPython Parallel Quickstart" accesskey="p"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../exercises/lambda_magic.html" class="btn btn-neutral float-right" title="lambda and keyword Magic" accesskey="n">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
  <div class="section" id="closures-and-function-currying">
<span id="closures"></span><h1>Closures and Function Currying<a class="headerlink" href="#closures-and-function-currying" title="Permalink to this headline"></a></h1>
<p>Defining specialized functions on the fly.</p>
<p>A “Closure” is a fancy CS term that can be very hard to understand. Partly because they are expressed a little differently in every computer language that supports them But this is easiest definition I could find:</p>
<blockquote>
<div><p>Closure is when a function “remembers” its lexical scope even when the function is executed outside that lexical scope</p>
</div></blockquote>
<p>This definition is provided by Kyle Simpson
(by way of <a class="reference external" href="https://medium.com/beginners-guide-to-mobile-web-development/closures-in-functional-programming-and-javascript-3ed730e08fc2">an article about closures in Javascript</a>).</p>
<p>The basic idea behind the concept of a closure lies in the understanding of the fact that closure is a mechanism by which an inner function will have access to the variables defined in its outer function’s lexical scope even after the outer function has returned.</p>
<p>Which brings us to the key practical part of how this works in Python:</p>
<div class="section" id="scope">
<h2>Scope<a class="headerlink" href="#scope" title="Permalink to this headline"></a></h2>
<p>In order to get a handle on all this, it’s important to understand variable scoping rules in Python.</p>
<p>“Scope” is the word for <cite>where</cite> the names in your code are accessible. Another word for a scope is namespace.</p>
<div class="section" id="global-scope">
<h3>Global Scope<a class="headerlink" href="#global-scope" title="Permalink to this headline"></a></h3>
<p>The simplest is the global scope. This is where all the names defined right in your code file (module) are. When running in an interactive interpreter,  it is in the global namespace as well.</p>
<p>You can get the global namespace with the <code class="docutils literal notranslate"><span class="pre">globals()</span></code> function, but …</p>
<p>The Python interpreter defines a handful of names when it starts up, and iPython defines a whole bunch more.
Recall that a convention in Python is that names that start with an underscore are “special” in some way – double underscore names have a special meaning to Python, and single underscore names are considered “private”.
Most of the extra names defined by the Python interpreter or iPython that are designed for internal use start with an underscore. These can really “clutter up” the namespace, but they can be filtered out for a more reasonable result:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">print_globals</span><span class="p">():</span>
    <span class="n">ipy_names</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;In&#39;</span><span class="p">,</span> <span class="s1">&#39;Out&#39;</span><span class="p">,</span> <span class="s1">&#39;get_ipython&#39;</span><span class="p">,</span> <span class="s1">&#39;exit&#39;</span><span class="p">,</span> <span class="s1">&#39;quit&#39;</span><span class="p">]</span>
    <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="nb">globals</span><span class="p">()</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
        <span class="k">if</span> <span class="ow">not</span> <span class="p">(</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&quot;_&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">ipy_names</span><span class="p">):</span>
            <span class="nb">print</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
</pre></div>
<p>Try running that in a newly started interpreter:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [3]: </span>    <span class="k">def</span> <span class="nf">print_globals</span><span class="p">():</span>
<span class="gp">   ...: </span>        <span class="n">ipy_names</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;In&#39;</span><span class="p">,</span> <span class="s1">&#39;Out&#39;</span><span class="p">,</span> <span class="s1">&#39;get_ipython&#39;</span><span class="p">,</span> <span class="s1">&#39;exit&#39;</span><span class="p">,</span> <span class="s1">&#39;quit&#39;</span><span class="p">]</span>
<span class="gp">   ...: </span>        <span class="k">for</span> <span class="n">name</span> <span class="ow">in</span> <span class="nb">globals</span><span class="p">()</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
<span class="gp">   ...: </span>            <span class="k">if</span> <span class="ow">not</span> <span class="p">(</span><span class="n">name</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&quot;_&quot;</span><span class="p">)</span> <span class="ow">or</span> <span class="n">name</span> <span class="ow">in</span> <span class="n">ipy_names</span><span class="p">):</span>
<span class="gp">   ...: </span>                <span class="nb">print</span><span class="p">(</span><span class="n">name</span><span class="p">)</span>
<span class="gp">   ...:</span>
<span class="gp">In [4]: </span><span class="n">print_globals</span><span class="p">()</span>
<span class="go">print_globals</span>
</pre></div>
<p>The only name left is “print_globals” itself – created when we defined the function.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Try running <code class="docutils literal notranslate"><span class="pre">globals()</span></code> by itself to see all the cruft iPython adds. Also note that <code class="docutils literal notranslate"><span class="pre">globals</span></code> returns not just the names, but a dictionary, where the keys are the names, and the items are the values bound to those names.</p>
<p>If we add a name or two, they show up in the global scope:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [6]: </span><span class="n">x</span> <span class="o">=</span> <span class="mi">5</span>
<span class="gp">In [7]: </span><span class="n">this</span> <span class="o">=</span> <span class="s2">&quot;that&quot;</span>
<span class="gp">In [8]: </span><span class="n">print_globals</span><span class="p">()</span>
<span class="go">print_globals</span>
<span class="go">x</span>
<span class="go">this</span>
</pre></div>
<p>names are created by assignment, and by <code class="docutils literal notranslate"><span class="pre">def</span></code> and <code class="docutils literal notranslate"><span class="pre">class</span></code> statements. We already saw a <code class="docutils literal notranslate"><span class="pre">def</span></code>, here is a <code class="docutils literal notranslate"><span class="pre">class</span></code> definition.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [11]: </span><span class="k">class</span> <span class="nc">TestClass</span><span class="p">:</span>
<span class="go">    ...:     pass</span>
<span class="go">    ...:</span>
<span class="gp">In [12]: </span><span class="n">print_globals</span><span class="p">()</span>
<span class="go">print_globals</span>
<span class="go">x</span>
<span class="go">this</span>
<span class="go">test</span>
<span class="go">TestClass</span>
</pre></div>
<p>Always keep in mind that in Python, “global” means “global to the module”, <em>not</em> global to the entire program. In the case of the interactive interpreter, the module is the “__main__” module (remember <code class="docutils literal notranslate"><span class="pre">if</span> <span class="pre">__name__</span> <span class="pre">==</span> <span class="pre">__main__:</span></code>?). But in a particular python file (usually one file is one module), the global scope is global to that one file.</p>
<div class="section" id="local-scope">
<h3>Local Scope<a class="headerlink" href="#local-scope" title="Permalink to this headline"></a></h3>
<p>So that’s the global scope – what creates a new scope?</p>
<p>A new, “local” scope is created by a function or class definition:</p>
<p>There is a built-in function to get the names in the local scope, too, so we can use it to show us the names in a function’s local namespace. There isn’t a lot of cruft in the local namespace, so we don’t need a special function to print it.</p>
<p>Note that <code class="docutils literal notranslate"><span class="pre">locals()</span></code> and <code class="docutils literal notranslate"><span class="pre">globals()</span></code> returns a dict of the names and the objects they are bound to, so we can print the keys to get the names:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [15]: </span><span class="k">def</span> <span class="nf">test</span><span class="p">():</span>
<span class="go">...:     x = 5</span>
<span class="go">...:     y = 6</span>
<span class="go">...:     print(locals().keys())</span>
<span class="go">...:</span>
<span class="gp">In [16]: </span><span class="n">test</span><span class="p">()</span>
<span class="go">dict_keys([&#39;y&#39;, &#39;x&#39;])</span>
</pre></div>
<p>When a function is called, it creates a clean local namespace.</p>
<p>Similarly a class definition does the same thing:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [18]: </span><span class="k">class</span> <span class="nc">Test</span><span class="p">:</span>
<span class="go">    ...:     this = &quot;that&quot;</span>
<span class="go">    ...:     z = 54</span>
<span class="go">    ...:     def __init__(self):</span>
<span class="go">    ...:         pass</span>
<span class="go">    ...:     print(locals().keys())</span>
<span class="go">    ...:</span>
<span class="go">dict_keys([&#39;__module__&#39;, &#39;__qualname__&#39;, &#39;this&#39;, &#39;z&#39;, &#39;__init__&#39;])</span>
</pre></div>
<p>Interesting – that print statement ran when the class was defined…</p>
<p>But you see that class attributes are there, as is the <code class="docutils literal notranslate"><span class="pre">__init__</span></code> function.</p>
<p>So each function gets a local namespace (or scope), and so does each class. And it follows that each method (function) in the class gets its own namespace as well.</p>
<p>Turns out that this holds true for functions defined within functions also:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [23]: </span><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="go">    ...:     x = 5</span>
<span class="go">    ...:     y = 6</span>
<span class="go">    ...:     def inner():</span>
<span class="go">    ...:         w = 7</span>
<span class="go">    ...:         z = 8</span>
<span class="go">    ...:         print(&quot;inner scope:&quot;, locals().keys())</span>
<span class="go">    ...:     print(&quot;outer scope:&quot;, locals().keys())</span>
<span class="go">    ...:     inner()</span>
<span class="gp">In [24]: </span><span class="n">outer</span><span class="p">()</span>
<span class="go">outer scope: dict_keys([&#39;inner&#39;, &#39;y&#39;, &#39;x&#39;])</span>
<span class="go">inner scope: dict_keys([&#39;z&#39;, &#39;w&#39;])</span>
</pre></div>
<div class="section" id="function-parameters">
<h3>Function Parameters<a class="headerlink" href="#function-parameters" title="Permalink to this headline"></a></h3>
<p>The other way you can define names in a function’s local namespace is with function parameters:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [14]: </span><span class="k">def</span> <span class="nf">fun_with_parameters</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
<span class="go">    ...:     print(&quot;local names are:&quot;, locals().keys())</span>
<span class="go">    ...:</span>
<span class="go">    ...:</span>
<span class="gp">In [15]: </span><span class="n">fun_with_parameters</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span>
<span class="go">local names are: dict_keys([&#39;a&#39;, &#39;b&#39;])</span>
</pre></div>
<p>Notice that no other names have been defined in the function, but both of the parameters (positional and keyword) are local names.</p>
<div class="section" id="finding-names">
<h3>Finding Names<a class="headerlink" href="#finding-names" title="Permalink to this headline"></a></h3>
<p>At any point, there are multiple scopes in play: the local scope, and all the surrounding scopes.
When you use a name, python checks in the local scope first, then moves out one by one until it finds the name.
If you define a new name inside a function, it “overrides” the name in any of the outer scopes.
But any names not defined in an inner scope will be found by looking in the enclosing scopes.</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [33]: </span><span class="n">name1</span> <span class="o">=</span> <span class="s2">&quot;this is global&quot;</span>
<span class="gp">In [34]: </span><span class="n">name2</span> <span class="o">=</span> <span class="s2">&quot;this is global&quot;</span>
<span class="gp">In [35]: </span><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="go">    ...:     name2 = &quot;this is in outer&quot;</span>
<span class="go">    ...:     def inner():</span>
<span class="go">    ...:         name3 = &quot;this is in inner&quot;</span>
<span class="go">    ...:         print(name1)</span>
<span class="go">    ...:         print(name2)</span>
<span class="go">    ...:         print(name3)</span>
<span class="go">    ...:     inner()</span>
<span class="go">    ...:</span>
<span class="gp">In [36]: </span><span class="n">outer</span><span class="p">()</span>
<span class="go">this is global</span>
<span class="go">this is in outer</span>
<span class="go">this is in inner</span>
</pre></div>
<p>Look carefully to see where each of those names came from. All the print statements are in the inner function, so its local scope is searched first, and then the outer function’s scope, and then the global scope. <code class="docutils literal notranslate"><span class="pre">name1</span></code> is only defined in the global scope, so that one is found. but <code class="docutils literal notranslate"><span class="pre">name2</span></code> is redfined in the scope of the <code class="docutils literal notranslate"><span class="pre">outer</span></code> function, so that one is found. And <code class="docutils literal notranslate"><span class="pre">name3</span></code> is only defined in the <code class="docutils literal notranslate"><span class="pre">inner</span></code> function scope.</p>
<div class="section" id="the-global-keyword">
<h3>The <code class="docutils literal notranslate"><span class="pre">global</span></code> keyword<a class="headerlink" href="#the-global-keyword" title="Permalink to this headline"></a></h3>
<p>Global names can be accessed from within functions, but not if that same name is created in the local scope. So you can’t change an immutable object that is outside the local scope:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [37]: </span><span class="n">x</span> <span class="o">=</span> <span class="mi">5</span>
<span class="gp">In [38]: </span><span class="k">def</span> <span class="nf">increment_x</span><span class="p">():</span>
<span class="go">    ...:     x += 5</span>
<span class="go">    ...:</span>
<span class="gp">In [39]: </span><span class="n">increment_x</span><span class="p">()</span>
<span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">UnboundLocalError</span><span class="g g-Whitespace">                         </span>Traceback (most recent call last)
<span class="nn">&lt;ipython-input-39-c9a57e8c0d14&gt;</span> in <span class="ni">&lt;module&gt;</span><span class="nt">()</span>
<span class="ne">----&gt; </span><span class="mi">1</span> <span class="n">increment_x</span><span class="p">()</span>
<span class="nn">&lt;ipython-input-38-dc4f30fe2ac4&gt;</span> in <span class="ni">increment_x</span><span class="nt">()</span>
<span class="g g-Whitespace">      </span><span class="mi">1</span> <span class="k">def</span> <span class="nf">increment_x</span><span class="p">():</span>
<span class="ne">----&gt; </span><span class="mi">2</span>     <span class="n">x</span> <span class="o">+=</span> <span class="mi">5</span>
<span class="g g-Whitespace">      </span><span class="mi">3</span>
<span class="ne">UnboundLocalError</span>: local variable &#39;x&#39; referenced before assignment
</pre></div>
<p>The problem here is that <code class="docutils literal notranslate"><span class="pre">x</span> <span class="pre">+=</span> <span class="pre">5</span></code> is the same as <code class="docutils literal notranslate"><span class="pre">x</span> <span class="pre">=</span> <span class="pre">x</span> <span class="pre">+</span> <span class="pre">5</span></code>, so it is creating a local name, but it can’t be incremented, because it hasn’t had a value set yet.</p>
<p>How does the interpreter know that <code class="docutils literal notranslate"><span class="pre">x</span></code> is a local name? When it compiles the function definition, it marks all the names assigned in the function as local. So when the function runs, it knows that <code class="docutils literal notranslate"><span class="pre">x</span></code> is local, and thus it won’t go look in the global scope for it.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">global</span></code> keyword tells python that you want to use the global name, rather than create a new, local name:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [40]: </span><span class="k">def</span> <span class="nf">increment_x</span><span class="p">():</span>
<span class="go">    ...:     global x</span>
<span class="go">    ...:     x += 5</span>
<span class="go">    ...:</span>
<span class="go">    ...:</span>
<span class="gp">In [41]: </span><span class="n">increment_x</span><span class="p">()</span>
<span class="gp">In [42]: </span><span class="n">x</span>
<span class="gh">Out[42]: </span><span class="go">10</span>
</pre></div>
<p><strong>NOTE:</strong> The use of <code class="docutils literal notranslate"><span class="pre">global</span></code> is frowned upon – having global variables manipulated in arbitrary other scopes makes for buggy, hard to maintain code! You hardly ever need to use <code class="docutils literal notranslate"><span class="pre">global</span></code> – if a function needs to manipulate a value, you should pass that value into the function, or have it return a value that can then be used to change the global name.</p>
<div class="section" id="nonlocal-keyword">
<h3><code class="docutils literal notranslate"><span class="pre">nonlocal</span></code> keyword<a class="headerlink" href="#nonlocal-keyword" title="Permalink to this headline"></a></h3>
<p>The other limitation with <code class="docutils literal notranslate"><span class="pre">global</span></code> is that there is only one global namespace. What if you are in a nested scope, and want to get at the value outside the current scope, but not all the way up at the global scope:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [1]: </span><span class="n">x</span> <span class="o">=</span> <span class="mi">5</span>
<span class="gp">In [2]: </span><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="gp">   ...: </span>    <span class="n">x</span> <span class="o">=</span> <span class="mi">10</span>
<span class="gp">   ...: </span>    <span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="gp">   ...: </span>        <span class="n">x</span> <span class="o">+=</span> <span class="mi">5</span>
<span class="gp">   ...: </span>    <span class="n">inner</span><span class="p">()</span>
<span class="gp">   ...: </span>    <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;x in outer is:&quot;</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
</pre></div>
<p>That’s not going to work as the inner x hasn’t been initialized:</p>
<p><code class="docutils literal notranslate"><span class="pre">UnboundLocalError:</span> <span class="pre">local</span> <span class="pre">variable</span> <span class="pre">'x'</span> <span class="pre">referenced</span> <span class="pre">before</span> <span class="pre">assignment</span></code></p>
<p>But if we use <code class="docutils literal notranslate"><span class="pre">global</span></code>, we’ll get the global <code class="docutils literal notranslate"><span class="pre">x</span></code>:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [4]: </span><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="gp">   ...: </span>    <span class="n">x</span> <span class="o">=</span> <span class="mi">10</span>
<span class="gp">   ...: </span>    <span class="k">def</span> <span class="nf">inner</span><span class="p">():</span>
<span class="gp">   ...: </span>        <span class="k">global</span> <span class="n">x</span>
<span class="gp">   ...: </span>        <span class="n">x</span> <span class="o">+=</span> <span class="mi">5</span>
<span class="gp">   ...: </span>    <span class="n">inner</span><span class="p">()</span>
<span class="gp">   ...: </span>    <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;x in outer is:&quot;</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span>
<span class="gp">   ...:</span>
<span class="gp">In [5]: </span><span class="n">x</span>
<span class="gh">Out[5]: </span><span class="go">5</span>
<span class="gp">In [6]: </span><span class="n">outer</span><span class="p">()</span>
<span class="go">x in outer is: 10</span>
<span class="gp">In [7]: </span><span class="n">x</span>
<span class="gh">Out[7]: </span><span class="go">10</span>
<span class="gp">In [8]: </span><span class="n">outer</span><span class="p">()</span>
<span class="go">x in outer is: 10</span>
<span class="gp">In [9]: </span><span class="n">x</span>
<span class="gh">Out[9]: </span><span class="go">15</span>
</pre></div>
<p>This indicates that the global <code class="docutils literal notranslate"><span class="pre">x</span></code> is getting changed, but not the one in the <code class="docutils literal notranslate"><span class="pre">outer</span></code> scope.</p>
<p>This is enough of a limitation that Python 3 added a new keyword: <code class="docutils literal notranslate"><span class="pre">nonlocal</span></code>.
What it means is that the name should be looked for outside the local scope, but only as far as you need to go to find it:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [10]: </span><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="go">    ...:     x = 10</span>
<span class="go">    ...:     def inner():</span>
<span class="go">    ...:         nonlocal x</span>
<span class="go">    ...:         x += 5</span>
<span class="go">    ...:     inner()</span>
<span class="go">    ...:     print(&quot;x in outer is:&quot;, x)</span>
<span class="go">    ...:</span>
<span class="gp">In [11]: </span><span class="n">outer</span><span class="p">()</span>
<span class="go">x in outer is: 15</span>
</pre></div>
<p>So the <code class="docutils literal notranslate"><span class="pre">x</span></code> in the <code class="docutils literal notranslate"><span class="pre">outer</span></code> function scope is the one being changed.</p>
<p>While using <code class="docutils literal notranslate"><span class="pre">global</span></code> is discouraged, <code class="docutils literal notranslate"><span class="pre">nonlocal</span></code> is safer – as long as it is referring to a name in a scope that is closely defined like the above example. In fact, <code class="docutils literal notranslate"><span class="pre">nonlocal</span></code> will not go all the way up to the global scope to find a name:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [15]: </span><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="go">    ...:     def inner():</span>
<span class="go">    ...:         nonlocal x</span>
<span class="go">    ...:         x += 5</span>
<span class="go">    ...:     inner()</span>
<span class="go">    ...:     print(&quot;x in outer is:&quot;, x)</span>
<span class="go">    ...:</span>
<span class="gt">  File</span><span class="nn"> &quot;&lt;ipython-input-15-fc6f8de72dfc&gt;&quot;</span><span class="gt">, line </span><span class="mi">3</span>
    <span class="k">nonlocal</span> <span class="n">x</span>
    <span class="o">^</span>
<span class="ne">SyntaxError</span>: no binding for nonlocal &#39;x&#39; found
</pre></div>
<p>But it will go up multiple levels in nested scopes:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [16]: </span><span class="k">def</span> <span class="nf">outer</span><span class="p">():</span>
<span class="go">    ...:     x = 10</span>
<span class="go">    ...:     def inner():</span>
<span class="go">    ...:         def inner2():</span>
<span class="go">    ...:             nonlocal x</span>
<span class="go">    ...:             x += 10</span>
<span class="go">    ...:         inner2()</span>
<span class="go">    ...:     inner()</span>
<span class="go">    ...:     print(&quot;x in outer is:&quot;, x)</span>
<span class="go">    ...:</span>
<span class="gp">In [17]: </span><span class="n">outer</span><span class="p">()</span>
<span class="go">x in outer is: 20</span>
</pre></div>
<div class="section" id="id1">
<h2>Closures<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h2>
<p>Now that we have a good handle on namespace scope, we can get to see why this is all really useful.</p>
<p>“Closures” is a cool CS term for what is really just defining functions on the fly with some saved state. You can find a “proper” definition here:</p>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Closure_(computer_programming)">Closures on Wikipedia</a></p>
<p>But I have trouble following that, so we’ll look at real world examples to get the idea – it’s actually pretty logical, once you have idea about how scope works in Python.</p>
<div class="section" id="functions-within-functions">
<h3>Functions Within Functions<a class="headerlink" href="#functions-within-functions" title="Permalink to this headline"></a></h3>
<p>We’ve been defining functions within functions to explore namespace scope.  But functions are “first class objects” in python, so we can not only define them and call them, but we can assign names to them and pass them around like any other object.</p>
<p>So after we define a function within a function, we can actually return that function as an object:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">counter</span><span class="p">(</span><span class="n">start_at</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
    <span class="n">count</span> <span class="o">=</span> <span class="n">start_at</span>
    <span class="k">def</span> <span class="nf">incr</span><span class="p">():</span>
        <span class="k">nonlocal</span> <span class="n">count</span>
        <span class="n">count</span> <span class="o">+=</span> <span class="mi">1</span>
        <span class="k">return</span> <span class="n">count</span>
    <span class="k">return</span> <span class="n">incr</span>
</pre></div>
<p>This looks a lot like the previous examples, but we are returning the function that was defined inside the function. Which means is can be used elsewhere.</p>
<div class="section" id="what-s-going-on-here">
<h4>What’s going on here?<a class="headerlink" href="#what-s-going-on-here" title="Permalink to this headline"></a></h4>
<p>We have passed the <code class="docutils literal notranslate"><span class="pre">start_at</span></code> value into the <code class="docutils literal notranslate"><span class="pre">counter</span></code> function.</p>
<p>We have stored it in <code class="docutils literal notranslate"><span class="pre">counter</span></code>’s scope as a local variable: <code class="docutils literal notranslate"><span class="pre">count</span></code></p>
<p>Then we defined a function, <code class="docutils literal notranslate"><span class="pre">incr</span></code> that adds one to the value of count, and returns that value.</p>
<p>Note that we declared <code class="docutils literal notranslate"><span class="pre">count</span></code> to be nonlocal in <code class="docutils literal notranslate"><span class="pre">incr</span></code>’s scope, so that it would be the same <code class="docutils literal notranslate"><span class="pre">count</span></code> that’s in counter’s scope.</p>
<p>What type of object do you get when you call <code class="docutils literal notranslate"><span class="pre">counter()</span></code>?</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [37]: </span><span class="n">c</span> <span class="o">=</span> <span class="n">counter</span><span class="p">(</span><span class="n">start_at</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
<span class="gp">In [38]: </span><span class="nb">type</span><span class="p">(</span><span class="n">c</span><span class="p">)</span>
<span class="gh">Out[38]: </span><span class="go">function</span>
</pre></div>
<p>So we get a function back – makes sense. The <code class="docutils literal notranslate"><span class="pre">def</span></code> defines a function, and that function is what’s getting returned.</p>
<p>Being a function, we can, of course, call it:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [39]: </span><span class="n">c</span><span class="p">()</span>
<span class="gh">Out[39]: </span><span class="go">6</span>
<span class="gp">In [40]: </span><span class="n">c</span><span class="p">()</span>
<span class="gh">Out[40]: </span><span class="go">7</span>
</pre></div>
<p>Each time is it called, it increments the value by one – as you’d expect.</p>
<p>But what happens if we call <code class="docutils literal notranslate"><span class="pre">counter()</span></code> multiple times?</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [41]: </span><span class="n">c1</span> <span class="o">=</span> <span class="n">counter</span><span class="p">(</span><span class="mi">5</span><span class="p">)</span>
<span class="gp">In [42]: </span><span class="n">c2</span> <span class="o">=</span> <span class="n">counter</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="gp">In [43]: </span><span class="n">c1</span><span class="p">()</span>
<span class="gh">Out[43]: </span><span class="go">6</span>
<span class="gp">In [44]: </span><span class="n">c2</span><span class="p">()</span>
<span class="gh">Out[44]: </span><span class="go">11</span>
</pre></div>
<p>So each time <code class="docutils literal notranslate"><span class="pre">counter()</span></code> is called, a new <code class="docutils literal notranslate"><span class="pre">incr</span></code> function is created. Along with the new function, a new namespace is created that holds the <code class="docutils literal notranslate"><span class="pre">count</span></code> name. So the new <code class="docutils literal notranslate"><span class="pre">incr</span></code> function is holding a reference to that new <code class="docutils literal notranslate"><span class="pre">count</span></code> name.</p>
<p>This is what makes it a “closure” – it carries with it the scope in which it was created (or enclosed - I guess that’s where the word closure comes from).</p>
<p>The returned <code class="docutils literal notranslate"><span class="pre">incr</span></code> function is a “curried” function – a function with some parameters pre-specified.</p>
<p>Let’s experiment a bit more with these ideas:</p>
<p><a class="reference download internal" download="" href="../_downloads/b522fc8f0c034466a69d85fe72bc3a0d/play_with_scope.py"><code class="xref download docutils literal notranslate"><span class="pre">play_with_scope.py</span></code></a></p>
<div class="section" id="currying">
<h2>Currying<a class="headerlink" href="#currying" title="Permalink to this headline"></a></h2>
<p>“Currying” is a special case of closures:</p>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Currying">Currying on Wikipedia</a></p>
<p>The idea behind currying is that you may have a function with a number of parameters, and you want to make a specialized version of that function with a couple of parameters pre-set.</p>
<div class="section" id="real-world-example">
<h3>Real world Example<a class="headerlink" href="#real-world-example" title="Permalink to this headline"></a></h3>
<p>I was writing some code to compute the concentration of a contaminant in a river, as it was reduced by exponential decay, defined by a half-life:</p>
<p><a class="reference external" href="https://en.wikipedia.org/wiki/Half-life">https://en.wikipedia.org/wiki/Half-life</a></p>
<p>So I wanted a function that would compute how much the concentration would reduce as a function of time – that is:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">scale</span><span class="p">(</span><span class="n">time</span><span class="p">):</span>
    <span class="k">return</span> <span class="n">scale_factor</span>
</pre></div>
<p>The trick is, how much the concentration would be reduced depends on both time and the half life. And for a given material, and given flow conditions in the river, that half life is pre-determined.  Once you know the half-life, the scale is given by:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">scale</span> <span class="o">=</span> <span class="mf">0.5</span> <span class="o">**</span> <span class="p">(</span><span class="n">time</span> <span class="o">/</span> <span class="p">(</span><span class="n">half_life</span><span class="p">))</span>
</pre></div>
<p>So to compute the scale, I could pass that half-life in each time I called the function:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">scale</span><span class="p">(</span><span class="n">time</span><span class="p">,</span> <span class="n">half_life</span><span class="p">):</span>
    <span class="k">return</span> <span class="mf">0.5</span> <span class="o">**</span> <span class="p">(</span><span class="n">time</span> <span class="o">/</span> <span class="p">(</span><span class="n">half_life</span><span class="p">))</span>
</pre></div>
<p>But this is a bit klunky – I need to keep passing that <code class="docutils literal notranslate"><span class="pre">half_life</span></code> around, even though it isn’t changing. And there are places, like <code class="docutils literal notranslate"><span class="pre">map</span></code> that require a function that takes only one argument!</p>
<p>What if I could create a function, on the fly, that had a particular half-life “baked in”?</p>
<p><em>Enter Currying</em> – Currying is a technique where you reduce the number of parameters that function takes, creating a specialized function with one or more of the original parameters set to a particular value. Here is that technique, applied to the half-life decay problem:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_scale_fun</span><span class="p">(</span><span class="n">half_life</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">half_life_fun</span><span class="p">(</span><span class="n">time</span><span class="p">):</span>
        <span class="k">return</span> <span class="mf">0.5</span> <span class="o">**</span> <span class="p">(</span><span class="n">time</span> <span class="o">/</span> <span class="n">half_life</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">half_life_fun</span>
</pre></div>
<p><strong>NOTE:</strong> This is simple enough to use a lambda for a bit more compact code:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_scale_fun</span><span class="p">(</span><span class="n">half_life</span><span class="p">):</span>
    <span class="k">return</span> <span class="k">lambda</span> <span class="n">time</span><span class="p">:</span> <span class="mf">0.5</span> <span class="o">**</span> <span class="p">(</span><span class="n">time</span> <span class="o">/</span> <span class="n">half_life</span><span class="p">)</span>
</pre></div>
<div class="section" id="using-the-curried-function">
<h4>Using the Curried Function<a class="headerlink" href="#using-the-curried-function" title="Permalink to this headline"></a></h4>
<p>Create a scale function with a half-life of one hour:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [8]: </span><span class="n">scale</span> <span class="o">=</span> <span class="n">get_scale_fun</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="gp">In [9]: </span><span class="p">[</span><span class="n">scale</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">7</span><span class="p">)]</span>
<span class="gh">Out[9]: </span><span class="go">[1.0, 0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625]</span>
</pre></div>
<p>The value is reduced by half every hour.</p>
<p>Now create one with a half life of 2 hours:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [10]: </span><span class="n">scale</span> <span class="o">=</span> <span class="n">get_scale_fun</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span>
<span class="gp">In [11]: </span><span class="p">[</span><span class="n">scale</span><span class="p">(</span><span class="n">t</span><span class="p">)</span> <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">7</span><span class="p">)]</span>
<span class="gh">Out[11]:</span>
<span class="go">[1.0,</span>
<span class="go"> 0.7071067811865476,</span>
<span class="go"> 0.5,</span>
<span class="go"> 0.3535533905932738,</span>
<span class="go"> 0.25,</span>
<span class="go"> 0.1767766952966369,</span>
<span class="go"> 0.125]</span>
</pre></div>
<p>And the value is reduced by half every two hours…</p>
<p>And it can be used with <code class="docutils literal notranslate"><span class="pre">map</span></code>, too:</p>
<div class="highlight-ipython notranslate"><div class="highlight"><pre><span></span><span class="gp">In [13]: </span><span class="nb">list</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="n">scale</span><span class="p">,</span> <span class="nb">range</span><span class="p">(</span><span class="mi">7</span><span class="p">)))</span>
<span class="gh">Out[13]:</span>
<span class="go">[1.0,</span>
<span class="go"> 0.7071067811865476,</span>
<span class="go"> 0.5,</span>
<span class="go"> 0.3535533905932738,</span>
<span class="go"> 0.25,</span>
<span class="go"> 0.1767766952966369,</span>
<span class="go"> 0.125]</span>
</pre></div>
<div class="section" id="functools-partial">
<h3><code class="docutils literal notranslate"><span class="pre">functools.partial</span></code><a class="headerlink" href="#functools-partial" title="Permalink to this headline"></a></h3>
<p>The <code class="docutils literal notranslate"><span class="pre">functools</span></code> module in the standard library provides utilities for working with functions:</p>
<p><a class="reference external" href="https://docs.python.org/3.5/library/functools.html">https://docs.python.org/3.5/library/functools.html</a></p>
<p>Creating a curried function turns out to be common enough that the <code class="docutils literal notranslate"><span class="pre">functools.partial</span></code> function provides an optimized way to do it:</p>
<p>What <code class="docutils literal notranslate"><span class="pre">functools.partial</span></code> does is:</p>
<blockquote>
<div><ul class="simple">
<li><p>Makes a new version of a function with one or more arguments already filled in.</p></li>
<li><p>The new version of a function documents itself.</p></li>
</div></blockquote>
<p>Example:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">power</span><span class="p">(</span><span class="n">base</span><span class="p">,</span> <span class="n">exponent</span><span class="p">):</span>
    <span class="sd">&quot;&quot;&quot;returns based raised to the give exponent&quot;&quot;&quot;</span>
    <span class="k">return</span> <span class="n">base</span> <span class="o">**</span> <span class="n">exponent</span>
</pre></div>
<p>Simple enough. but what if we wanted a specialized <code class="docutils literal notranslate"><span class="pre">square</span></code> and <code class="docutils literal notranslate"><span class="pre">cube</span></code> function?</p>
<p>We can use <code class="docutils literal notranslate"><span class="pre">functools.partial</span></code> to <em>partially</em> evaluate the function, giving us a specialized version:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">square</span> <span class="o">=</span> <span class="n">partial</span><span class="p">(</span><span class="n">power</span><span class="p">,</span> <span class="n">exponent</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span>
<span class="n">cube</span> <span class="o">=</span> <span class="n">partial</span><span class="p">(</span><span class="n">power</span><span class="p">,</span> <span class="n">exponent</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
</pre></div>
           </div>
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="IPythonParallel.html" class="btn btn-neutral float-left" title="IPython Parallel Quickstart" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../exercises/lambda_magic.html" class="btn btn-neutral float-right" title="lambda and keyword Magic" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
  <div role="contentinfo">
    <p>&#169; Copyright 2020, University of Washington, Natasha Aleksandrova, Christopher Barker, Brian Dorsey, Cris Ewing, Christy Heaton, Jon Jacky, Maria McKinley, Andy Miles, Rick Riehle, Joseph Schilz, Joseph Sheedy, Hosung Song. Creative Commons Attribution-ShareAlike 4.0 license.</p>
  Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
    <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
    provided by <a href="https://readthedocs.org">Read the Docs</a>.
        </div>
      </div>
    </section>
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
  </script> 
Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FilesExpand file tree

Closures.html

Latest commit

History

Closures.html

File metadata and controls
