Download

Source response reliability

Reference docs/source-response-reliability.md

bioRxiv, medRxiv, ERIC, and DBLP can return a successful transport response whose body is empty, malformed, or an access-challenge page. Those replies are not valid empty search results.

Each affected client uses one cooperative 45-second search budget across pagination, limiter admission, response reads, and one bounded retry. A second unreadable reply records a parse failure and returns any results retained from earlier pages. A later successful retry remains a success. ERIC 504 responses remain upstream failures; the client-side handling is the shared deadline, single-retry limit, and truthful malformed-response outcome, not a claim that the provider is available. DBLP challenge pages are recorded without retry or challenge bypass. DBLP author searches deliberately use the official SPARQL service and dblp:authoredBy; this route is selected for author search and is never a fallback from a challenged REST request. REST keyword queries keep the caller’s query unchanged.

DBLP REST and SPARQL requests share a 0.5 request/second limiter, so starts are at least two seconds apart, and use the same truthful User-Agent. A valid Retry-After is honored only when it fits in the shared 45-second budget. Author lookup uses an exact escaped name label, may match several people with that label, and can miss aliases. DBLP dates have year precision: the stored YYYY-01-01 value does not claim a known month or day, and partial-year windows exclude boundary years that cannot be placed honestly.

The bioRxiv /details cursor advances by the documented 30-record page size in the official bioRxiv API help. The ERIC request preserves its documented search, start, rows, fields, and JSON format parameters described in ERIC’s official API example. The deadline and retry do not alter the bioRxiv, medRxiv, and ERIC query, author, date-window, or normalized identity behavior. DBLP SPARQL results use the conf/... or journals/... record key as external_id; the full DBLP record IRI is the URL.

OAPEN, now retired, used the same 45-second search budget and one bounded retry, with a per-request timeout of 20 seconds. Ten seconds was shorter than the provider’s own replies. Measured in September 2026 under the 30-second default, the live case passed in about 1.6 and 23.8 seconds, and OAPEN’s HTTP 500 error page took about 11.7 seconds to arrive. With two 20-second timeouts plus the one-second backoff, the worst case is 41 seconds, inside the budget. A longer timeout does not fix an HTTP 500 from the provider. It makes the 500 visible as a 500 instead of a timeout.

What a failed search records

Each search’s outcome keeps an ordered history of its failed attempts, both retried and terminal, for example http_500 then timeout. A new attempt does not erase the previous status, so “HTTP 500, then a timeout on the retry” is recorded as both, not as just timeout. The history holds at most eight entries, newest kept, and counts any older entries it dropped. Its words come from a fixed vocabulary (http_<status>, rate_limited, timeout, connect, request_error, operation_deadline, refused). refused is a source that answered HTTP 200 with an error in place of results, as HAL does for a query it cannot run. No URL, query string or response-body text is stored in the history. Several sources carry an API key in the query, and a reply body is the provider’s untrusted text. The history is not written to the database. A search’s recorded zero reason still comes from the last terminal failure alone.

Provider-outage quarantine in the weekly live suite

A single provider’s outage cannot fix or break resmon, so it should not decide whether the rest of the weekly live suite counts. resmon_scripts/verification_scripts/live_quarantine.json names at most two live cases. Each entry has a source, a failure signature from the vocabulary above, a first-observed date, an expiry no more than 30 days later, and the excused status captured from two vantage points. A quarantined case still runs and still asserts. Its failure is excused only when it is an assertion and the case’s recorded source outcome shows the search ended on a failed call whose history contains the signature. A case qualifies for an entry only once it records exactly one source outcome, because an excuse needs exactly one in total and exactly one for the entry’s source: a case that records none can be quarantined and never excused, and a case that records two does not say whose outage the failure was, so neither is excused. Any other failure fails the run: a different status, timeouts alone, a wrong answer after a successful retry, or a crash. A pass is reported as a recovery, so the entry can be lifted. An expired entry is ignored, and the case fails as it would without one. The run summary always prints the denominator from the collected suite, for example “91 of 92 asserted; 1 quarantined (oapen, since September 18, 2026, http_500, expires October 18, 2026)”. That example is the outage the mechanism was built for: from September 18, 2026, OAPEN’s server intermittently answered HTTP 500 with a database connection error, both to GitHub runners and to a workstation.

These controls bound cooperative HTTP work. They do not hard-preempt DNS shutdown, OS scheduling, CPU-bound work, or another non-cooperative dependency. Loopback tests establish local parser and timing behavior; they do not establish why a public provider emitted a particular response or guarantee future provider availability.