domains / llm-infra / BerriAI/litellm
background task coordination
When working with background tasks in async applications, ensure proper task lifecycle management and coordination to prevent memory leaks, race conditions, and resource conflicts.
When working with background tasks in async applications, ensure proper task lifecycle management and coordination to prevent memory leaks, race conditions, and resource conflicts.
Key practices:
- Use asyncio.create_task for fire-and-forget operations to avoid blocking the main execution flow
- Implement proper task cleanup by having tasks remove themselves from tracking collections when complete
- Use controlled loop conditions instead of
while True:to allow graceful termination - Prevent overlapping executions by checking if a task is already running before starting a new one
- Always release resources in finally blocks to ensure cleanup even when exceptions occur
Example implementation:
# Global task tracking with automatic cleanup
BACKGROUND_TASKS = set()
async def start_background_operation():
# Use create_task for non-blocking execution
task = asyncio.create_task(background_health_check())
# Add to tracking set and auto-remove when done
BACKGROUND_TASKS.add(task)
task.add_done_callback(BACKGROUND_TASKS.discard)
async def background_health_check():
# Use controlled condition instead of while True
while use_background_health_checks:
try:
# Perform work
await perform_health_check()
finally:
# Always cleanup resources
await cleanup_resources()
This approach prevents memory leaks from orphaned tasks, allows controlled shutdown of background operations, and ensures resources are properly released even when errors occur.