curl -sSL https://install.python-poetry.org | python3 -
poetry install
poetry run test- looks like ContextMiddleware intercepts each SSE response => bind each 2 yielded values
app.add_middleware(
ContextMiddleware,
plugins={ForwardedForPlugin()}
)- If there are only sync logics on async iterator (no await), cannot get yielded value until encounter next await
- In this case, looks like ContextMiddleware await on each 2 yields
- Without ContextMiddleware, it yields all values at once after executing all iterations
tests/test_main.py::test_stream_err
>>> rigt before yield: 1 / 5 (20.0%)
>>> rigt before yield: 2 / 5 (40.0%)
1 / 5 (20.0%)
2 / 5 (40.0%)
>>> rigt before yield: 3 / 5 (60.0%)
>>> rigt before yield: 4 / 5 (80.0%)
3 / 5 (60.0%)
4 / 5 (80.0%)
>>> rigt before yield: 5 / 5 (100.0%)
5 / 5 (100.0%)
- Workaround: do
await asyncio.sleep(0.01)right after yield- FastAPI executes
defroute on external threadpool(new thread, non-blocking) butasync defon internal threadpool(blocking) - Using
await, it returns the control to event loop so that it can execute another task
- FastAPI executes
- Direct: split sync and async logics, so that use sync generator
- What if there should be async logics in generator? Any other problems?
- With sync generator, can it handle multiple calls?
- if not, we should use gunicorn and multiple workers