xingrin

mirror of https://github.com/yyhuni/xingrin.git synced 2026-01-31 11:46:16 +08:00

Files

yyhuni 2d2ec93626 perf(screenshot): optimize memory usage and add URL collection fallback logic

- Add iterator(chunk_size=50) to ScreenshotSnapshot query to prevent BinaryField data caching and reduce memory consumption
- Implement fallback logic in URL collection: WebSite → HostPortMapping → Default URL with priority handling
- Update _collect_urls_from_provider to return tuple with data source information for better logging and debugging
- Add detailed logging to track which data source was used during URL collection
- Improve code documentation with clear return type hints and fallback priority explanation
- Prevents memory spikes when processing large screenshot datasets with binary image data

2026-01-11 16:14:56 +08:00