Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/jekyll.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,10 @@ jobs:
# for the build to print. When it fails, the counts the live site
# published last are kept; with neither, it warns and the site builds
# without counts, which the browser then asks for. Run on PRs too, so
# the gates below check the pages as they will be published. It fails
# only on a refused SEMANTIC_SCHOLAR_API_KEY; its own timeouts and
# retries end it in about six minutes at worst.
# the gates below check the pages as they will be published. The API
# never fails the job: a refused SEMANTIC_SCHOLAR_API_KEY is an error
# (annotation and run summary) and a keyless retry, and its timeouts
# end it in about five minutes.
run: bundle exec ruby script/fetch-citations.rb
timeout-minutes: 8
env:
Expand Down
11 changes: 7 additions & 4 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,16 +68,19 @@ shellcheck script/*.sh assets/images/*.sh && actionlint .github/workflows/*.yml
count. They block nothing.
- **An API key**, for a higher rate limit: request one from Semantic Scholar
and save it as the repository secret `SEMANTIC_SCHOLAR_API_KEY`. A key the
API refuses fails every run, PRs from this repository included, until the
secret is renewed or deleted.
API refuses shows as an error (annotation and run summary) on every run, which then asks
without the key; renew or delete the secret.
- **Counts kept from the live site** for more than 7 days, because every
fetch failed, show as an error the same way.
- **A paper with no count, or the wrong one**: find its record on
semanticscholar.org and add `paper_doi: <paperId>` to
`_data/citation_ids.yml`. Check the paperId answers first
(`https://api.semanticscholar.org/graph/v1/paper/<paperId>?fields=citationCount`):
some records on the website are not in the API.
- **Counts in a local build**: run `bundle exec ruby script/fetch-citations.rb`
before `jekyll build`; `_data/citations.json` is not in Git, and a run that
gets no answer leaves an older one in place.
before `jekyll build`; `_data/citations.json` is not in Git. A run that gets
no answer writes the live site's counts instead, or, offline, leaves the
file as it was.
- **The daily run stopped**: GitHub turns schedules off after 60 days without
repository activity; turn it back on under Actions.

Expand Down
38 changes: 26 additions & 12 deletions script/fetch-citations.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@
# they can be read. With neither, it warns and the site builds without counts;
# the browser then asks for them (js/citations.js).
# SEMANTIC_SCHOLAR_API_KEY, when set, is sent as the API key; a key the API
# refuses fails the run, since only fixing the secret helps.
# refuses is reported as an error and the request is made again without it,
# so a lapsed secret never holds up a deploy.
#
# Usage: bundle exec ruby script/fetch-citations.rb

Expand All @@ -22,8 +23,10 @@
module CitationFetch
API = URI("https://api.semanticscholar.org/graph/v1/paper/batch?fields=citationCount")
WAITS = [5, 10, 20, 40].freeze # seconds before each retry, unless the API asks for longer
DEADLINE = 240 # no retry starts past it; with one last try and the fallback, about 6 minutes at worst
TIMEOUTS = { open_timeout: 10, ssl_timeout: 10, read_timeout: 30, write_timeout: 10, max_retries: 0 }.freeze
TRY = TIMEOUTS.values_at(:open_timeout, :ssl_timeout, :read_timeout, :write_timeout).sum # one try's longest
DEADLINE = 240 # the batch ends by then; with the fallback's one try, about 5 minutes at worst
STALE = 7 * 24 * 60 * 60 # published counts older than this are reported as an error

class KeyRefused < StandardError; end

Expand All @@ -36,21 +39,25 @@ def pause = @pause ||= ->(seconds) { sleep seconds }
def clock = @clock ||= -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) }
end

# The batch answer's body, or nil once every try has failed or the deadline
# is near. Raises KeyRefused when the API refuses the key.
def self.fetch(ids, key: "")
# The batch answer's body, or nil once every try has failed or `stop` (a
# clock time; DEADLINE from now by default) is near. Raises KeyRefused when
# the API refuses the key.
def self.fetch(ids, key: "", stop: clock.call + DEADLINE)
req = Net::HTTP::Post.new(API, "Content-Type" => "application/json")
req["x-api-key"] = key unless key.empty?
req.body = JSON.generate("ids" => ids)
stop = clock.call + DEADLINE
(WAITS + [nil]).each do |wait|
return nil if clock.call + TRY > stop # no try that could run past it
begin
response = http.call(API, req)
return response.body if response.is_a?(Net::HTTPSuccess)
status = response.code
raise KeyRefused, status if !key.empty? && %w[401 403].include?(status)
# Another 4xx is this request's fault; asking again will not help.
return nil if status.start_with?("4") && status != "429"
if status.start_with?("4") && status != "429"
warn("fetch-citations: #{status} #{response.body.to_s.lines.first.to_s.strip[0, 200]}")
return nil
end
retry_after = response["Retry-After"].to_i
rescue KeyRefused
raise
Expand All @@ -60,7 +67,7 @@ def self.fetch(ids, key: "")
end
return nil unless wait
wait = retry_after.clamp(wait, WAITS.max) if retry_after.positive?
return nil if clock.call + wait > stop
return nil if clock.call + wait + TRY > stop
warn("fetch-citations: #{status}, retrying in #{wait}s")
pause.call(wait)
end
Expand All @@ -78,8 +85,12 @@ def self.published(url)
end
end

# A warning or error: an annotation in CI, and an error also in the run's
# summary, where a scheduled run's shows without opening the log.
def note(message, level = "warning")
warn(ENV["GITHUB_ACTIONS"] ? "::#{level} title=Citation counts::#{message}" : "fetch-citations: #{message}")
summary = ENV["GITHUB_STEP_SUMMARY"]
File.write(summary, "**Citation counts:** #{message}\n", mode: "a") if level == "error" && summary && !summary.empty?
end

if $PROGRAM_NAME == __FILE__
Expand All @@ -97,11 +108,12 @@ def note(message, level = "warning")
abort "fetch-citations: no post has a citation_id; is _plugins/citations.rb loaded?" if ids.empty?

papers = nil
stop = CitationFetch.clock.call + CitationFetch::DEADLINE # one deadline for both tries below
begin
answer = CitationFetch.fetch(ids, key: ENV["SEMANTIC_SCHOLAR_API_KEY"].to_s)
answer = CitationFetch.fetch(ids, key: ENV["SEMANTIC_SCHOLAR_API_KEY"].to_s, stop: stop)
rescue CitationFetch::KeyRefused => e
note("Semantic Scholar refused SEMANTIC_SCHOLAR_API_KEY (#{e.message}); renew the secret", "error")
exit 1
note("Semantic Scholar refused SEMANTIC_SCHOLAR_API_KEY (#{e.message}); renew the secret. Asking without it", "error")
answer = CitationFetch.fetch(ids, stop: stop)
end
if answer
begin
Expand All @@ -118,7 +130,9 @@ def note(message, level = "warning")
note("no count for #{missing.join(", ")} (not found, or 0: see _data/citation_ids.yml)") unless missing.empty?
elsif (last = CitationFetch.published("#{site.config["url"]}#{ENV.fetch("BASE_PATH", site.baseurl)}/citations.json"))
File.write(out, JSON.pretty_generate(last) + "\n")
note("no usable answer from Semantic Scholar; kept the published counts of #{last["fetched_at"]}")
age = Time.now - (Time.iso8601(last["fetched_at"]) rescue Time.at(0))
note("no usable answer from Semantic Scholar; kept the published counts of #{last["fetched_at"]}",
age > CitationFetch::STALE ? "error" : "warning")
else
note("no usable answer from Semantic Scholar and no published counts; _data/citations.json is not updated")
end
Expand Down
16 changes: 13 additions & 3 deletions test/test_fetch_citations.rb
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,24 @@ def test_it_gives_up_after_the_last_try
def test_it_stops_before_the_deadline
answering(*Array.new(5) { response(429) }, took: 60)
assert_nil CitationFetch.fetch(%w[arXiv:1])
assert_operator @now, :<=, CitationFetch::DEADLINE + 60
assert_operator @now, :<=, CitationFetch::DEADLINE
assert_operator @asked.size, :<, 5
end

def test_another_4xx_is_not_retried
answering(response(400))
def test_a_shared_deadline_cuts_a_second_fetch_short
answering(*Array.new(10) { response(429) }, took: 60)
stop = @now + CitationFetch::DEADLINE
CitationFetch.fetch(%w[arXiv:1], stop: stop)
first = @asked.size
CitationFetch.fetch(%w[arXiv:1], stop: stop)
assert_equal first, @asked.size # no try that could run past the shared deadline
end

def test_another_4xx_is_not_retried_and_says_why
answering(response(400, "{\"error\":\"Unrecognized field\"}\n"))
assert_nil CitationFetch.fetch(%w[arXiv:1])
assert_equal 1, @asked.size
assert_includes $stderr.string, "400 {\"error\":\"Unrecognized field\"}"
end

def test_a_refused_key_raises_but_a_403_without_one_does_not
Expand Down
Loading