Skip to content

Print citation counts at build time - #135

Merged
bits-bytes-nn merged 1 commit into
mainfrom
feat/build-time-citations
Nov 17, 2026
Merged

bits-bytes-nn merged 1 commit into
mainfrom
feat/build-time-citations

Conversation

@bits-bytes-nn

Copy link
Copy Markdown
Owner

Why

Paper posts showed their Semantic Scholar citation count only after each reader's browser asked the keyless API. In a check of all 34 paper posts, 21 got 429 even after retries, so most first-time readers saw no count. Two papers never showed one: Gemini 1.0 (arXiv:2312.11805 is not in the API) and DeepSeek-V3 (its arXiv id leads to a duplicate record with 0 citations).

What

  • script/fetch-citations.rb runs before the build in every jekyll.yml run, plus a new daily run (20:41 UTC). It sends one batch request for all papers, with backoff and a 240 s deadline, and writes _data/citations.json, which is not in Git.
    • If the request fails, it keeps the counts the live site published last (/citations.json, a new page).
    • With neither, it warns, and the build carries on.
    • If the API refuses SEMANTIC_SCHOLAR_API_KEY (now a repository secret), the run fails.
  • _plugins/citations.rb sets each post's Semantic Scholar id and its fetched count. _layouts/post.html renders the count on the server, so it shows without JavaScript.
  • _data/citation_ids.yml overrides the lookup id per DOI. It is keyed by DOI, so regenerating a post in scholar-lens keeps it. It has one entry, for DeepSeek-V3.
  • js/citations.js shows the newer of the build's count and the browser's count, and calls the API only when both are a day old (the build's count gets 6 h more). A cached zero never hides a count.
  • script/check-plugin-output.rb checks that each paper page has the field with the count that was fetched, or is hidden when there is none.
  • Docs: docs/architecture.md (how it works, known limits) and docs/operations.md (runbook).

No change to _data/post_conventions.yml and no new lint rule; scholar-lens is unaffected.

Verification

  • Unit tests (207, including new tests for the fetch script and the plugin), post lint, actionlint, strict build, htmlproofer and validate-site.sh: all pass.
  • Real batch fetch: 33 of 34 papers counted. Gemini 1.0 is still missing; the API has no record of it under any id.
  • The fallback paths were tested with the network blocked.
  • Headless Chrome, with the API blocked, showed the right count in all five cases: a fresh build count, a stale build count, a newer cached count, a cached zero, and a reader clock behind CI.
  • check-plugin-output.rb caught three deliberately broken pages.

Paper posts showed their Semantic Scholar citation count only after each
reader's browser asked the keyless API, which answered 429 to 21 of 34
papers in a test. CI now fetches every count in one batch request before
the build and the layout prints it, so it shows on a first view and
without JavaScript.

- script/fetch-citations.rb: one batch POST, retried with backoff within
  a deadline; on failure keeps the counts the live site published last
  (/citations.json); a refused SEMANTIC_SCHOLAR_API_KEY fails the run.
- _plugins/citations.rb: the Semantic Scholar id per post and its count.
- _data/citation_ids.yml: the record for a DOI whose arXiv id finds none
  or a zero-count duplicate (DeepSeek-V3); keyed by DOI, so a regenerated
  post keeps it.
- js/citations.js: shows the newer of the build's and the browser's count
  and asks only when both are a day old.
- jekyll.yml: the fetch step and a daily run to refresh the counts.
- check-plugin-output.rb: the field is on each paper page as fetched.
@bits-bytes-nn
bits-bytes-nn merged commit 2b1cdbd into main Nov 17, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant