Skip to content

devgony/play_python

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Play python

Getting started

curl -sSL https://install.python-poetry.org | python3 -
poetry install
poetry run test

Async generator with FastAPI middleware

Issue

  • looks like ContextMiddleware intercepts each SSE response => bind each 2 yielded values
app.add_middleware(
    ContextMiddleware,
    plugins={ForwardedForPlugin()}
)
  • If there are only sync logics on async iterator (no await), cannot get yielded value until encounter next await
    • In this case, looks like ContextMiddleware await on each 2 yields
    • Without ContextMiddleware, it yields all values at once after executing all iterations
tests/test_main.py::test_stream_err

>>> rigt before yield: 1 / 5 (20.0%)
>>> rigt before yield: 2 / 5 (40.0%)
1 / 5 (20.0%)
2 / 5 (40.0%)
>>> rigt before yield: 3 / 5 (60.0%)
>>> rigt before yield: 4 / 5 (80.0%)
3 / 5 (60.0%)
4 / 5 (80.0%)
>>> rigt before yield: 5 / 5 (100.0%)
5 / 5 (100.0%)

Solution

  • Workaround: do await asyncio.sleep(0.01) right after yield
    • FastAPI executes def route on external threadpool(new thread, non-blocking) but async def on internal threadpool(blocking)
    • Using await, it returns the control to event loop so that it can execute another task
  • Direct: split sync and async logics, so that use sync generator

Consideration

  • What if there should be async logics in generator? Any other problems?
  • With sync generator, can it handle multiple calls?
    • if not, we should use gunicorn and multiple workers

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages