Just a quick overview of how a for loop works in Python:

x = [1, 2, 3, 4, 5]
for i in x:
    print(i)

OUTPUT : 

1
2
3
4
5

The code above is equivalent to the code given below:

x = [1, 2, 3, 4, 5]
y = iter(x)
try:
    while True:
        print(next(y))
except StopIteration as e:
    pass

OUTPUT :
1
2
3
4
5

What Are Coroutines?

Coroutines are basically functions whose execution can be paused/suspended at a particular point, and then we can resume the execution from that same point later whenever we want.

We need a mechanism — or to be more precise, a keyword — by which we can insert a checkpoint and tell the program that we want to pause the execution of the function here and return control to the point it is called from. We’ll resume the execution whenever we want.

In Python, we can pause the execution of a function using the yield keyword.

So here’s where things get interesting :

• We can think of a coroutine as a function that has one or more checkpoints where the execution will be paused and control will be returned to the point where it was called from.

• In essence, a coroutine is a function divided into many parts, and we can execute each part of a coroutine as we execute each iteration of a for loop using the next function.

Here’s a basic example :

def func():
    print('Function Starts')

    yield

    print('Function Ends')


try:
    y = func()
    print(type(y))
    next(y)  # First part of the function executed
    next(y)  # Second part of the function executed

except StopIteration as e:
    pass

OUTPUT :
<class 'generator'>
Function Starts
Function Ends

From the output, we notice a few things:

• First, we need to call the coroutine/function that will give us a generator object.

• That generator object will behave similarly to an iterator, but in the case of an iterator, we are traversing over an iterable. With a generator, we’re executing parts of the coroutine.

• Just as a StopIteration exception is thrown and caught behind the scenes of a for loop, the same happens in this case when the last part of the coroutine is executed.

Now, this pausing of the function in between is very interesting and opens up some possibilities:

• When the function is paused, we do nothing, which is the case that we just saw.

• Suppose a variable is being modified several times in a function and we want the value of that particular variable at a certain checkpoint. Then when we pause that function on that particular checkpoint, it returns the value of that variable.

Let’s see an example:

def func():
    x = 5
    print('Function Part 1')

    yield x
    x += 7
    print('Function part 2')

    yield x

    print('Function part 3')


try:
    y = func()
    z = next(y)        # Function part 1 executed
    print(z)

    z = next(y)        # Function part 2 executed
    print(z)

    z = next(y)        # Function part 3 executed and StopIteration exception raised
    print(z)          # This print will not be executed

except StopIteration as e:
    pass

OUTPUT :
Function Part 1
5
Function part 2
12
Function part 3

Here, the value of x is returned by yield at different checkpoints, as the function execution has been paused.

Whenever we are executing the last part of the function and there is no yield left in the function, after executing that last part, a StopIteration exception will be raised.

Much like when an iterator tries to execute the next function, but there are no more elements left in the iterable, it also raises the StopIteration exception.

• Suppose we want to send a value (which can be a constant or variable) at a certain checkpoint (i.e. at a certain state of a function). We can also do that using the yield keyword. When we want to send a value, we'll use the send function instead of next.

Let’s see an example:

def func():
    print('Function part 1')

    x = yield
    print(x)
    print('Function part 2')

    a = yield
    print(a)
    print('Function part 3')


try:

    y = func()

    next(y)            # Function part 1 executed, to reach the first yield we used next

    y.send(6)        # Function part 2 executed and value sent 6

    y.send(12)        # Function part 2 executed and value sent 12 and StopIteration raised

except StopIteration as e:
    pass

OUTPUT :
Function part 1
6
Function part 2
12
Function part 3

The reason we used next before using send is we can only use send when we are at the yield checkpoint, and yield is on the right side of the expression. So to reach that first yield, we have to use the next function.

Now here comes an interesting application of coroutines. Suppose we want to switch back and forth between two functions as we do in multithreading. In multithreading, until an interrupt is encountered by the OS, it will keep executing. In this case, we can switch whenever we want.

Let’s see an example :

def func1():
    print('Function 1 part 1')

    yield
    print('Function 1 part 2')

    yield
    print('Function 1 part 3')

    yield
    print('Function 1 part 4')

    yield
    print('Function 1 part 5')


def func2():
    print('Function 2 part 1')

    yield
    print('Function 2 part 2')

    yield
    print('Function 2 part 3')

    yield
    print('Function 2 part 4')

    yield
    print('Function 2 part 5')


try:

    a = func1()
    b = func2()

    next(a)      # Will execute Function 1 part 1
    next(b)      # Will execute Function 2 part 1
    next(a)      # Will execute Function 1 part 2
    next(a)      # Will execute Function 1 part 3
    next(b)      # Will execute Function 2 part 2
    next(b)      # Will execute Function 2 part 3
    next(b)      # Will execute Function 2 part 4
    next(a)      # Will execute Function 1 part 4
    next(a)      # Will execute Function 1 part 5 and raise StopIteration exception

except StopIteration as e:
    pass

OUTPUT :
Function 1 part 1
Function 2 part 1
Function 1 part 2
Function 1 part 3
Function 2 part 2
Function 2 part 3
Function 2 part 4
Function 1 part 4
Function 1 part 5

In this example, we can see that we can switch back and forth between coroutines whenever we want.

So if we write our own custom scheduler that handles the switching between multiple coroutines, we can achieve with single threading what we do with multithreading.

Coroutines have many applications such as concurrency and other programming patterns can also be implemented, like Producer-Consumer or Sender-Receiver in network programming. I’ll be exploring those in upcoming articles.

Coroutines are also the building blocks of many frameworks such as asyncio, twisted, aiohttp. They can also be chained together to make pipelines and solve problems.