Python Runtime Models

1# 1. Sync workers (WSGI). One request per worker at a time.
2gunicorn app:wsgi --workers 8
3#    concurrency = 8. A slow request costs exactly one worker.
4#    Easiest to debug; lowest concurrency per GB of memory.
5 
6# 2. Threaded workers (WSGI). Threads per worker for I/O overlap.
7gunicorn app:wsgi --workers 4 --threads 8 --worker-class gthread
8#    concurrency = 4 x 8 = 32 for I/O-bound work.
9#    CPU-bound work still serialises inside each process (GIL).
10#    Module-level state is now shared between threads: thread safety matters.
11 
12# 3. Greenlets. Cooperative scheduling via monkey-patching.
13gunicorn app:wsgi --workers 4 --worker-class gevent --worker-connections 1000
14#    concurrency = thousands, for I/O-bound work, with sync-looking code.
15#    Cost: the standard library is patched underneath you.
16 
17# 4. Async (ASGI). One event loop per worker.
18uvicorn app:asgi --workers 4
19#    concurrency = very high, IF the whole path is async.
20#    One blocking call collapses that worker to one request at a time.
21 
22# In every case, the number that reaches the database is:
23#   workers x pool_size_per_worker x instances
24# Check it against the database max_connections BEFORE deploying.

Four different services. Nothing in the application source distinguishes them, which is why "how many concurrent requests can one instance handle" is a question about your process manager configuration and not about your code.

import requests            # synchronous HTTP client

@app.get("/profile/{user_id}")
async def profile(user_id: str):
    # This blocks the ENTIRE event loop for this worker.
    # Every other in-flight request on this worker waits.
    r = requests.get(f"https://crm.example.com/users/{user_id}", timeout=2)
    return r.json()

import httpx                # async-capable client

client = httpx.AsyncClient(
    timeout=2.0,
    limits=httpx.Limits(max_connections=50),   # bounded concurrency
)

@app.get("/profile/{user_id}")
async def profile(user_id: str):
    r = await client.get(f"https://crm.example.com/users/{user_id}")
    return r.json()

# When a library has no async version, hand it to a thread
# explicitly rather than pretending:
#   result = await asyncio.to_thread(legacy_sdk.fetch, user_id)
# and bound the executor, or you have swapped one unbounded
# resource for another.

An async def handler is a coroutine on a shared loop, not a thread. A synchronous call inside it holds the loop for its full duration, so a worker that could have had hundreds of requests in flight has one. The shared AsyncClient also gets you connection reuse and an explicit connection cap, which the per-call requests.get form has neither of (Keep-Alive and Connection Reuse).