ELO scraper: discover the embed link and retry transient failures
Read the current OneDrive link from the stat-check.com/elo iframe on each run, falling back to EMBED_URL if the page can't be read or its link 404s. Retry 429/5xx and network errors up to three times with backoff, after a one-off 503 from the Badger token endpoint failed a run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -11,7 +11,7 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from download_and_parse import download, xlsx_to_csv
|
||||
from pure_python_elo_scrape import EMBED_URL, download_url_for_embed
|
||||
from pure_python_elo_scrape import resolve_download_url
|
||||
|
||||
HERE = Path(__file__).parent
|
||||
ROOT = HERE.parent
|
||||
@@ -46,7 +46,7 @@ def main() -> None:
|
||||
json_path = Path(sys.argv[3]) if len(sys.argv) > 3 else ROOT / "public" / "elo-data.json"
|
||||
|
||||
print("Resolving OneDrive download URL…")
|
||||
url = download_url_for_embed(EMBED_URL)
|
||||
url = resolve_download_url()
|
||||
|
||||
print("Downloading XLSX…")
|
||||
download(url, xlsx_path)
|
||||
|
||||
Reference in New Issue
Block a user