Refresh job
the data behind every page.
.github/workflows/refresh.yml ("refresh board") fetches Kalshi, Polymarket and Polygon on a GitHub-hosted runner, writes the JSON files the pages read, checks them for fabricated content and commits them to your repository.
01Why it runs on GitHub
- Kalshi refuses browser requests. Any request with an
Originheader gets HTTP 403, so the fetch must happen server-side. - Your network may block a venue. Some internet providers block Polymarket's domains. GitHub's runners are not behind your provider. Do not try to get around a block on your own network.
- Nothing to host. No server, no database, no keys.
02Run it by hand
The workflow ships with the schedule off; only a manual start (workflow_dispatch) runs it. Not tested here
- Open ActionsOn your repository on GitHub, click Actions. Enable workflows if GitHub asks.
- Pick "refresh board"In the list on the left.
- Run workflowKeep the branch
mainand confirm. - Pull the resultWhen the run is green, run
git pull. The commit is titledboard: refresh YYYY-MM-DDTHH:MMZand authored bymikodes-bot(the name is set in the workflow; change it if you like). On Vercel's Hobby plan such a commit is not deployed on its own; see Deployment.
The job pushes a commit, so the workflow declares permissions: contents: write. If the last step fails with a 403 on git push, check Settings → Actions → General → Workflow permissions.
03What each step does
| Step | What it does | Fails the run? |
|---|---|---|
| Probe venue reachability | Prints the HTTP status of Kalshi, Gamma and the CLOB host from the runner. | No |
| Probe CORS | Prints status and Access-Control-Allow-Origin with and without a foreign Origin. | No |
| Build board | node src/run.js in scanner/: both venues, matcher, spreads. | Yes, on a crash |
| Fail if the run fabricated anything | Stops if an unreachable venue reports markets, or cross-venue is not ok but carries rows. | Yes |
| Read the Terminal tape from Polygon | node src/onchain.js 40: the last 40 blocks of the exchange contract's logs, from your POLYGON_RPC_URL first if you set it (below), else the public RPCs. | Yes, on a crash |
| Attach market titles | node src/titles.js: token ids to questions through Gamma. If Gamma is unreachable it writes titles.status = "unavailable". | No |
| Fail if the tape carries titles it could not have fetched | Checks the titles block against its own status and counts. | Yes |
| Commit the new board | Commits the four data files if they changed and pushes. | Yes, if the push is refused |
04Use your own Polygon RPC
The Terminal tape is read from Polygon. Out of the box the job uses three public, keyless RPCs and falls through them in order. They are enough for a 40-block window; a paid or private endpoint gives you higher rate limits and one provider you chose.
- Get an endpointAn HTTPS Polygon mainnet (chain 137) URL from any RPC provider. It must allow
eth_getLogsover at least 40 blocks. - Add the secretIn your GitHub repository open Settings → Secrets and variables → Actions and add a repository secret named
POLYGON_RPC_URLwith the full URL. Several URLs may be given, separated by commas. - Run the workflowThe tape step passes the secret to
node src/onchain.js.
- Your endpoint is tried first; the public RPCs stay as the fallback. Set
POLYGON_RPC_ONLY=1as well to use only yours. - The URL is never written anywhere. Many providers put the key in the URL, so
tape.jsonand the Terminal footer name it only asown RPC (POLYGON_RPC_URL), and error messages carry that label instead of the URL. - A value that is not an
https://URL stops the job with an error that does not print the value. - The Polygon RPC URL in the admin's Integrations is separate: it feeds the admin's Status check only. Use the same URL in both places if you like.
Tested offline with node --test test/rpc.test.js in terminal/ (6 cases, a local stand-in RPC). Not tested here A GitHub Actions run with a real secret.
05Turn on the schedule
Edit .github/workflows/refresh.yml and remove the # in front of the two schedule lines, so the on: block reads:
on:
schedule:
- cron: '*/15 * * * *'
workflow_dispatch:
Commit and push. GitHub may delay scheduled runs at busy times and pauses schedules in repositories with no activity for a long time; re-enable them in the Actions tab.
06What it costs in Actions minutes
Recent runs on the project's own repository took 2 min 33 s to 3 min 51 s. GitHub bills each job rounded up to the whole minute. At 3 billed minutes per run:
| Schedule | cron | Runs / 30 days | Billed minutes / 30 days |
|---|---|---|---|
| Every 15 minutes | */15 * * * * | 2,880 | about 8,640 |
| Every 30 minutes | */30 * * * * | 1,440 | about 4,320 |
| Every hour | 0 * * * * | 720 | about 2,160 |
| Every 2 hours | 0 */2 * * * | 360 | about 1,080 |
| Every 6 hours | 0 */6 * * * | 120 | about 360 |
The comment in the workflow gives the same figure (2,880 runs, about 8,640 billed minutes a month at every 15 minutes). Check the free allowance and overage price for your own GitHub plan on GitHub's billing page before you turn the schedule on.
07Running the jobs locally
cd terminal
node src/onchain.js 40 # Polygon only; argument = blocks to read
POLYGON_RPC_URL=https://your-endpoint node src/onchain.js 40 # with your own RPC
cd scanner
node src/run.js # options: --contracts 1000 --max-events 1200 --no-depth --out public/board.json
cd ../terminal
node src/titles.js # needs Polymarket's Gamma API
run.js always rewrites board.json, crossvenue-diag.json and pairs.jsonl. From a network that blocks a venue, the run reports it unavailable, so committing it replaces the runner's two-venue board with a one-venue board. Keep the block window small: a few hundred blocks of exchange logs can be tens of megabytes. Local runs were not repeated for these docs.