Illustrating the duality of closures and objects

I'd heard it before but this time it really sunk in. Let me explain a bit more.

I was recently reading the first few chapters of Software Design for Flexibility (SDF). Doing so reminded of the duality between closures in Functional Programming (FP) and object instances in Object-Oriented Programming (OOP). When I'd heard it previously this connection had seemed tenuous at best and irrelevant at worst. However when I considered this duality this last time, I was able to internalize its significance.

I further chatted about this realization with my friend Reuven Lerner who quoted one of his teachers saying that "closures are outside-in objects" and it blew my mind. As a concession to Reuven being one of the world's most successful Python trainers, code examples below are written in typed Python.

Let's dive in!

What is an object?

When you ask this question to different communities you will get different answers. For example a Smalltalker will likely leverage message passing in their definition while the Java kids' explanations might hinge on inheritance and encapsulation. Still, a Lisper will likely get very excited to tell you about generic functions and the importance of the metaobject protocol. None of these responses would be incorrect, strictly speaking.

For our purposes, we're going to use a more linguistically neutral definition and say that:

An object is a logical grouping of related data and behavior.

In other words, an object is the "what" and the "how" packaged together. In the words of Hanson and Sussman, the authors of SDF, the data and state of an object are "curated" together so that they can subsequently influence the actions of the object.

Assume our problem domain has the concept of a "pipeline job". How might we represent this entity using a classical object system like the one found in Python?

from dataclasses import dataclass


@dataclass
class Pipeline:
    job_id: int
    source: str
    destination: str

    def execute(self) -> str:
        message = f"PIPELINE[{self.job_id}] {self.source} -> {self.destination}"
        print(message)

        return message

Note that the string construction is merely a stand in for the business logic of our pipeline for the sake of simplicity.

That seems simple enough, right? With this approach we can also hold onto one or many Pipeline instances and execute them whenever we please!

# Conjure up a pipeline
my_pipeline = Pipeline(1, "src", "dest")
# Use it whenever
my_pipeline.execute()

What is a closure?

A closure is a special function which retains references to the environment in which it was defined. Let's look at a simple example!

external = 5

def vanilla_func(x: int) -> int:
    return x + 5

def closure_func(x: int) -> int
    return x + external

In the above example, we define three variables, where the first is a number and the latter two are functions. The first function, vanilla_func, simply adds 10 to its argument. The second one, closure, adds 5 to its argument. You'll also likely notice that closure_func references external which is defined outside its body. Since external is not in the parameter list of closure_func but appears in the body, it is a free variable.

Since closure_func references a free variable from the (lexical) environment in which it was defined, it is a closure as that reference persists for the life of closure_func. Hence, we say that a function is a closure because it closes over the variables in its environment! If my explanation wasn't quite right and this doesn't make sense to you yet, there's some great material out there to help

• An entire article, replete with examples, illustrating closures in Python using regular old functions.
• A fantastic explainer on closures with JavaScript examples over on the MDN Web Docs.
• An article using TypeScript examples with some excellent illustrations.

In languages which support FP, functions can be returned from other functions. So let's see what our pipeline conception from earlier would look like when modeled as a closure. Again, with a Python example:

def make_pipeline(job_id: int, source: str, destination: str) -> Callable[[], str]:

    # closure is here
    def execute_pipeline() -> str:
        message = f"PIPELINE[{job_id}] {source} -> {destination}"
        print(message)

        return message

    # return the closure as a thunk
    return execute_pipeline

In the above example, make_pipeline returns a closure, execute_pipeline which is enriched with the same data and behavior as the Pipeline object from the previous section.

# Conjure up a pipeline
my_pipeline = make_pipeline(1, "src", "dest")
# Use it whenever
my_pipeline()

Wrap-up discussion

We can immediately see some surface-level parallels between the closure and object approaches to expressing our domain's pipeline concept. The two Python code examples make it clear that you can achieve the same level of ahead-of-time data and behavior curation with closures as you can with objects. In short, we've highlighted a duality linking OOP object instances with FP closures. In my gut, I believe this should be explored and formalized further.

If you like this stuff, please reach out to me since I would love to discuss it further!