feat: add Stat-Check ELO column with daily auto-refresh

- Scraper (statcheck-elo-scraper/refresh_elo.py) downloads the stat-check ELO workbook from OneDrive and builds public/elo-data.json - GitHub Actions workflow (.github/workflows/refresh-elo.yml) runs daily at 06:00 UTC and commits updated ELO data - Frontend fetches elo-data.json and matches players by name, using BCP career event history to disambiguate same-name candidates - ELO score shown in new sortable Stat-Check ELO column, visible on all screen sizes; unmatched players show — Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-29 00:00:02 -05:00
parent 7dd9ddfefe
commit ac223a6909
10 changed files with 445 additions and 6 deletions
@@ -0,0 +1,56 @@
+# Stat Check Elo Scraper
+
+This is a pure-Python scraper for the public Excel workbook embedded at:
+
+`https://www.stat-check.com/elo`
+
+It does not need Selenium, Playwright, Chromium, pandas, or openpyxl. It uses only the Python standard library.
+
+## How It Works
+
+The Stat Check page embeds a public OneDrive Excel workbook through a `1drv.ms` URL. Microsoft's web viewer does not expose a stable direct `.xlsx` URL in the page HTML, but it does use a public OneDrive metadata API behind the scenes.
+
+The scraper reproduces the useful part of that flow:
+
+1. Request an anonymous Microsoft "Badger" token from:
+   `https://api-badgerp.svc.ms/v1.0/token`
+2. Convert the public OneDrive embed URL into Microsoft Graph's `shares/u!...` id format.
+3. Call:
+   `https://my.microsoftpersonalcontent.com/_api/v2.0/shares/{share_id}/driveItem?action=EmbedView`
+   with `Authorization: Badger <token>`.
+4. Read `@content.downloadUrl` from the metadata response.
+5. Download the real `.xlsx`.
+6. Convert the first worksheet to CSV by reading the XLSX ZIP/XML structure directly.
+
+The Microsoft endpoints used here are undocumented and could change, but this currently works without browser automation.
+
+## Usage
+
+```bash
+python3 pure_python_elo_scrape.py
+```
+
+Default outputs:
+
+- `elo-pure.xlsx`
+- `elo-pure.csv`
+
+You can also pass output paths:
+
+```bash
+python3 pure_python_elo_scrape.py statcheck-elo.xlsx statcheck-elo.csv
+```
+
+## Files
+
+- `pure_python_elo_scrape.py` - gets the Badger token, resolves the workbook download URL, downloads the XLSX, writes CSV.
+- `download_and_parse.py` - helper functions for downloading and converting XLSX to CSV with stdlib only.
+- `sample-elo.csv` - sample output captured during testing.
+
+## Verification From Testing
+
+On June 28, 2026, this downloaded a `3,465,320` byte workbook and produced a `4,145,399` byte CSV with `40,945` rows. The CSV header was:
+
+```csv
+Rank,Delta,Name,W,L,D,G,WR%,Elo,Delta,Last Event,Country
+```