mandalore/gateway-gamers.net
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ELO scraper: discover the embed link and retry transient failures

Browse Source 
Read the current OneDrive link from the stat-check.com/elo iframe on
each run, falling back to EMBED_URL if the page can't be read or its
link 404s. Retry 429/5xx and network errors up to three times with
backoff, after a one-off 503 from the Badger token endpoint failed a run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
mandalore
2026-09-22 16:40:45 -05:00
co-authored by Claude Opus 5.5
parent a8cb306f88
commit a6eea9c0ac
4 changed files with 71 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sites/40k-rankings/statcheck-elo-scraper/README.md
+3
View File
@@ -12,6 +12,7 @@ The Stat Check page embeds a public OneDrive Excel workbook through a `1drv.ms`
The scraper reproduces the useful part of that flow:
0. Read the current `1drv.ms` embed link from the Stat Check page's iframe. Stat Check replaces this link occasionally, so it is looked up on every run; `EMBED_URL` in `pure_python_elo_scrape.py` is only a fallback for when the page can't be read or its link returns 404.
1. Request an anonymous Microsoft "Badger" token from:
`https://api-badgerp.svc.ms/v1.0/token`
2. Convert the public OneDrive embed URL into Microsoft Graph's `shares/u!...` id format.
@@ -24,6 +25,8 @@ The scraper reproduces the useful part of that flow:
The Microsoft endpoints used here are undocumented and could change, but this currently works without browser automation.
Requests that fail with HTTP 429, a 5xx status, or a network error are retried up to three times (after 5, 15 and 45 seconds).
## Usage
```bash