Chapter 4: Async/Await

Teaching Objectives

By the end of this chapter, students should:

• Understand the concept of asynchronous programming
• Learn when and why to use async/await
• Master creating asynchronous functions with async def
• Comprehend how to await asynchronous results with await
• Manage multiple asynchronous tasks concurrently
• Apply async techniques to solve real-world problems

Preparation

Before teaching this chapter, ensure:

• Students have a solid understanding of functions and error handling
• You have examples of I/O-bound operations ready to demonstrate
• You have installed any necessary libraries (like aiohttp and asyncio)
• Students' systems are set up to run asynchronous code

Lesson Overview

1. Introduction to Asynchronous Programming (20 minutes)

Start by explaining the core concepts:

• Asynchronous programming allows code to continue running while waiting for operations to complete
• It's particularly useful for I/O-bound operations (network requests, file operations, etc.)
• It doesn't provide true parallelism, but rather concurrent execution on a single thread

Synchronous vs. Asynchronous:

import time

# Synchronous approach
def sync_example():
    print("Start")
    time.sleep(1)  # Simulates a blocking operation
    print("Middle")
    time.sleep(1)  # Simulates another blocking operation
    print("End")
    
# Let's time it
start = time.time()
sync_example()
print(f"Total time: {time.time() - start:.2f} seconds")

Teaching points:

• In synchronous code, operations block the entire program
• This is inefficient when operations involve waiting (like network requests)
• Asynchronous programming offers a solution to this problem

2. Understanding Asyncio (25 minutes)

Introduce Python's asyncio library:

import asyncio
import time

# Asynchronous function
async def async_example():
    print("Start")
    await asyncio.sleep(1)  # Non-blocking sleep
    print("Middle")
    await asyncio.sleep(1)  # Another non-blocking sleep
    print("End")

# Running the async function
start = time.time()
asyncio.run(async_example())
print(f"Total time: {time.time() - start:.2f} seconds")

Teaching points:

• async def defines an asynchronous function (a coroutine)
• await pauses the execution of the coroutine until the awaited operation completes
• asyncio.run() is the entry point for running asynchronous code
• asyncio.sleep() is a non-blocking alternative to time.sleep()

Concurrency with asyncio:

import asyncio
import time

async def task(name, delay):
    print(f"Task {name} starting")
    await asyncio.sleep(delay)  # Non-blocking sleep
    print(f"Task {name} completed after {delay} seconds")
    return f"Result from task {name}"

async def main():
    # Running multiple tasks concurrently
    start = time.time()
    
    # Create tasks
    task1 = asyncio.create_task(task("A", 2))
    task2 = asyncio.create_task(task("B", 1))
    task3 = asyncio.create_task(task("C", 3))
    
    # Await all tasks to complete
    results = await asyncio.gather(task1, task2, task3)
    
    print(f"All tasks completed in {time.time() - start:.2f} seconds")
    print("Results:", results)

# Run the main coroutine
asyncio.run(main())

Teaching points:

• Multiple tasks can run concurrently using asyncio.create_task()
• asyncio.gather() waits for multiple tasks to complete and collects their results
• Notice that the total time is determined by the longest task, not the sum of all tasks

3. Async/Await Syntax in Depth (20 minutes)

Explore the syntax and mechanics:

import asyncio

# Basic async function
async def fetch_data(delay):
    print(f"Fetching data with delay {delay}s...")
    await asyncio.sleep(delay)  # Simulate network delay
    return f"Data fetched after {delay}s"

# Function that awaits other async functions
async def process_data():
    # Await a single coroutine
    data = await fetch_data(1)
    print(f"Processing: {data}")
    
    # Create and await multiple tasks
    task1 = asyncio.create_task(fetch_data(2))
    task2 = asyncio.create_task(fetch_data(1))
    
    # Process results as they come in
    for task in [task1, task2]:
        result = await task
        print(f"Processed result: {result}")

# Execute everything
asyncio.run(process_data())

Teaching points:

• Async functions always return a coroutine object
• You must await coroutines to get their actual results
• Async/await is not about making code faster, but about making it more efficient by not blocking
• Any function that uses await must be defined with async def

Error handling in async code:

import asyncio

async def might_fail(success=True):
    await asyncio.sleep(1)
    if not success:
        raise ValueError("Operation failed")
    return "Success!"

async def handle_errors():
    try:
        # This will succeed
        result = await might_fail(True)
        print(f"Result: {result}")
        
        # This will fail
        result = await might_fail(False)
        print(f"Result: {result}")  # This won't execute
        
    except ValueError as e:
        print(f"Caught error: {e}")
    
    # Try/except with multiple tasks
    tasks = [
        asyncio.create_task(might_fail(True)),
        asyncio.create_task(might_fail(False))
    ]
    
    for task in tasks:
        try:
            result = await task
            print(f"Task result: {result}")
        except ValueError as e:
            print(f"Task error: {e}")

asyncio.run(handle_errors())

Teaching points:

• Error handling works similarly to synchronous code
• You can use try/except blocks around awaited expressions
• Uncaught exceptions in tasks can be tricky - always handle them appropriately

4. Practical Application: Web Scraping (25 minutes)

Demonstrate a real-world application using aiohttp:

import asyncio
import aiohttp
import time

async def fetch_url(session, url):
    print(f"Fetching {url}")
    async with session.get(url) as response:
        return await response.text()

async def fetch_multiple_urls():
    urls = [
        "http://python.org",
        "http://github.com",
        "http://stackoverflow.com"
    ]
    
    # Timing comparison
    # Synchronous approach (simulated)
    start = time.time()
    print("Simulating synchronous fetching...")
    for url in urls:
        print(f"Starting fetch for {url}")
        await asyncio.sleep(1)  # Simulate network delay
        print(f"Completed fetch for {url}")
    print(f"Synchronous time: {time.time() - start:.2f} seconds")
    
    # Asynchronous approach
    start = time.time()
    print("\nAsynchronous fetching...")
    async with aiohttp.ClientSession() as session:
        tasks = [fetch_url(session, url) for url in urls]
        results = await asyncio.gather(*tasks, return_exceptions=True)
        
        for url, result in zip(urls, results):
            if isinstance(result, Exception):
                print(f"Error fetching {url}: {result}")
            else:
                print(f"Successfully fetched {url}, size: {len(result)} bytes")
    
    print(f"Asynchronous time: {time.time() - start:.2f} seconds")

# Note: This needs to be installed with pip install aiohttp
# asyncio.run(fetch_multiple_urls())

Teaching points:

• Real-world asynchronous code often involves external libraries like aiohttp
• Session management is important for efficient HTTP requests
• The performance difference between synchronous and asynchronous approaches increases with the number of operations

5. Advanced Asyncio Features (20 minutes)

Cover more advanced concepts:

import asyncio
import random

# Timeouts
async def slow_operation():
    delay = random.uniform(1, 5)
    print(f"Starting slow operation (will take {delay:.1f}s)")
    await asyncio.sleep(delay)
    return f"Completed after {delay:.1f}s"

async def with_timeout():
    try:
        # Set a timeout of 2 seconds
        result = await asyncio.wait_for(slow_operation(), timeout=2.0)
        print(f"Success: {result}")
    except asyncio.TimeoutError:
        print("Operation timed out!")

# Cancellation
async def demonstrate_cancellation():
    # Start a task
    task = asyncio.create_task(slow_operation())
    
    # Wait a bit, then cancel it
    await asyncio.sleep(0.5)
    task.cancel()
    
    try:
        await task
    except asyncio.CancelledError:
        print("Task was cancelled!")

# Running multiple coroutines with different behaviors
async def demo_advanced_features():
    print("--- Timeout Demo ---")
    await with_timeout()
    print("\n--- Cancellation Demo ---")
    await demonstrate_cancellation()
    
    print("\n--- First Completed Demo ---")
    # Execute tasks and use the first result
    task1 = asyncio.create_task(slow_operation())
    task2 = asyncio.create_task(slow_operation())
    
    done, pending = await asyncio.wait(
        [task1, task2], 
        return_when=asyncio.FIRST_COMPLETED
    )
    
    # Print the first result
    for task in done:
        print(f"First completed: {task.result()}")
    
    # Cancel remaining tasks
    for task in pending:
        task.cancel()
        try:
            await task
        except asyncio.CancelledError:
            print("Cancelled pending task")

asyncio.run(demo_advanced_features())

Teaching points:

• Timeouts allow you to limit how long you wait for operations
• Cancellation lets you stop tasks that are no longer needed
• asyncio.wait() with return_when gives fine-grained control over task completion

6. Queues and Producers/Consumers (15 minutes)

Explain the producer/consumer pattern:

import asyncio
import random

async def producer(queue, items):
    for i in range(items):
        # Produce an item
        item = random.randint(1, 100)
        
        # Put it in the queue
        await queue.put(item)
        print(f"Produced {item}")
        
        # Simulate variable production time
        await asyncio.sleep(random.uniform(0.1, 0.5))
    
    # Signal the end of production
    await queue.put(None)
    print("Producer finished")

async def consumer(queue, name):
    while True:
        # Get an item from the queue
        item = await queue.get()
        
        # Check for end signal
        if item is None:
            print(f"Consumer {name} finished")
            # Put the signal back for other consumers
            await queue.put(None)
            break
        
        # Process the item
        print(f"Consumer {name} processing {item}")
        await asyncio.sleep(random.uniform(0.2, 0.8))
        
        # Mark task as done
        queue.task_done()

async def main():
    # Create a queue
    queue = asyncio.Queue(maxsize=5)
    
    # Create tasks
    producer_task = asyncio.create_task(producer(queue, 10))
    consumer_tasks = [
        asyncio.create_task(consumer(queue, f"Consumer-{i}"))
        for i in range(3)
    ]
    
    # Wait for the producer to finish
    await producer_task
    
    # Wait for all consumers to finish
    await asyncio.gather(*consumer_tasks)

asyncio.run(main())

Teaching points:

• Queues are useful for coordinating between producers and consumers
• Multiple producers and consumers can work with a single queue
• This pattern is ideal for workload distribution and parallel processing

7. Best Practices and Pitfalls (15 minutes)

Discuss common best practices and mistakes:

import asyncio

# BAD: Mixing sync and async code
async def bad_practice():
    print("Starting bad practice example")
    
    # BAD: Using a blocking sleep in async function
    # This blocks the event loop!
    import time
    time.sleep(1)  # Don't do this in async functions!
    
    # BETTER: Use asyncio.sleep
    await asyncio.sleep(1)
    
    # BAD: Not awaiting a coroutine
    asyncio.sleep(1)  # This does nothing! The coroutine is created but never awaited
    
    # CORRECT: Always await coroutines
    await asyncio.sleep(1)
    
    print("Bad practice example completed")

# GOOD: Proper async function design
async def good_practice():
    print("Starting good practice example")
    
    # Run CPU-bound work in a separate process/thread
    def cpu_intensive_work():
        result = 0
        for i in range(10_000_000):
            result += i
        return result
    
    # Use run_in_executor for CPU-bound tasks
    loop = asyncio.get_running_loop()
    result = await loop.run_in_executor(None, cpu_intensive_work)
    print(f"CPU work result: {result}")
    
    # Handle exceptions properly
    try:
        await asyncio.sleep(0.1)
        raise ValueError("Demo error")
    except ValueError as e:
        print(f"Caught error: {e}")
    
    print("Good practice example completed")

async def run_examples():
    await bad_practice()
    print("\n")
    await good_practice()

asyncio.run(run_examples())

Teaching points:

• Never use blocking calls in async functions
• Always await coroutines or you'll have silent bugs
• Use run_in_executor for CPU-bound tasks
• Handle exceptions properly
• Be careful about resource management (closing sessions, files, etc.)

8. Guided Practice (20 minutes)

Have students work through these exercises:

1. Basic async/await:

   # Exercise: Create an async function that simulates fetching user data
   # after a random delay, then process that data
   
   import asyncio
   import random
   
   async def fetch_user(user_id):
       await asyncio.sleep(random.uniform(0.5, 2.0))
       return {"id": user_id, "name": f"User {user_id}", "age": random.randint(18, 70)}
   
   async def process_user(user):
       print(f"Processing user {user['name']}")
       await asyncio.sleep(0.5)  # Simulate processing time
       return {**user, "processed": True}
   
   async def main():
       # TODO: Fetch data for users 1, 2, and 3 concurrently
       # TODO: Process each user's data as it becomes available
       # TODO: Print the final processed users
       pass
   
   asyncio.run(main())
   

2. Error handling:

   # Exercise: Implement proper error handling for async tasks
   
   import asyncio
   import random
   
   async def risky_operation(task_id):
       await asyncio.sleep(random.uniform(0.5, 1.5))
       if random.random() < 0.5:  # 50% chance of failure
           raise Exception(f"Task {task_id} failed!")
       return f"Task {task_id} completed successfully"
   
   async def main():
       tasks = [risky_operation(i) for i in range(5)]
       
       # TODO: Run all tasks and handle errors appropriately
       # TODO: For failed tasks, print the error message
       # TODO: For successful tasks, print the result
       pass
   
   asyncio.run(main())
   

9. Review and Discussion (10 minutes)

• Review the key concepts covered
• Ask students to explain in their own words:
  • What is asynchronous programming and when is it useful?
  • How does async/await work in Python?
  • What are some common pitfalls to avoid?

Common Challenges and Solutions

• Conceptual understanding: Many students struggle with the concept of asynchronous execution. Use visual diagrams to illustrate how operations interleave.
• Awaiting coroutines: Forgetting to await coroutines is a common mistake. Remind students that coroutines must be awaited to execute.
• Event loop: Students may be confused about the event loop. Explain it as the "heart" of async operations that manages the execution of tasks.
• Error handling: Errors in async code can be tricky. Emphasize proper error handling with try/except blocks.

Extension Activities

For students who finish early:

• Challenge them to implement a real-time chat application using asyncio and websockets
• Have them explore async file I/O operations with aiofiles
• Ask them to refactor a synchronous application to use async/await

Assessment

Look for these indicators of understanding:

• Students can correctly identify cases where async is beneficial
• They can write proper async functions with correct await syntax
• They understand how to manage multiple concurrent tasks
• They can handle errors appropriately in async code

Resources

• Python asyncio Documentation
• Real Python: Async IO in Python
• aiohttp Documentation
• Python Asyncio: The Complete Guide