Select data structures based on your specific access patterns and performance requirements. Using the right data structure can simplify your code and improve performance significantly.
For queue-like operations where you need to add/remove from both ends efficiently, use specialized collections like deque
instead of plain lists:
# Instead of this:
self._metrics_to_export = []
# ...later...
self._metrics_to_export.append(metric_records)
# ...and in another method...
self._metrics_to_export.remove(metric_batch) # Inefficient for large lists
# Use this:
from collections import deque
self._metrics_to_export = deque()
# ...later...
self._metrics_to_export.append(metric_records)
# ...and in another method...
metric_batch = self._metrics_to_export.popleft() # O(1) operation
When working with collections:
For example, when iterating and modifying:
# Instead of this:
for metric_batch in self._metrics_to_export:
# process metric_batch
self._metrics_to_export.remove(metric_batch) # DANGER! Modifying during iteration
# Use this:
while self._metrics_to_export:
metric_batch = self._metrics_to_export.popleft()
# process metric_batch
Enter the URL of a public GitHub repository