AnythingLLM applies unified SDK timeout and retry policy for OpenAI and Anthropic calls

FACTAI JUDGMENTDetected 39 days ago

Factual Description

AnythingLLM monkey-patches the OpenAI and Anthropic SDK layers so that all provider calls go through a single, user-configurable timeout (default 10 minutes) with retries disabled, letting the underlying socket error surface faster instead of hanging indefinitely.

Event TypeFeature

DetectedJun 02, 2026

TopicConsumer AI Applications

Core Technical Contributions

Unified SDK-level timeout and retry policy so LLM provider calls fail fast with a clear socket error instead of hanging for the default SDK retry/timeout window.

openai SDKanthropic SDKundici Agenttimeout configuration

AI Impact Judgment

Users of AnythingLLM-hosted LLM apps will see failed provider requests surface within minutes instead of hanging or silently retrying for long periods, making it easier to diagnose connectivity or provider-outage problems. Operators can now tune the timeout via environment variables for self-hosted deployments. The main risk is that disabling retries entirely (maxRetries=0) means any transient network glitch now immediately fails the request, so operators should monitor for a spike in timeout errors after upgrading and consider whether their network environment needs retries.

Confidence0%

Importance58

Evidence1

Raw Evidence Links

Github Pull RequestMintplex-Labs/anything-llm PR #5721: apply universal sdk timeout

Adds monkeypatch to `openai` and `anthropic` sdk layers to use a system level fetch `undici` Agent so the user can set a custom timeout. Base timeout bumped to 10min. Removes `maxRetries` and sets them globally to 0 to get socket timeouts back sooner. Adds these ENVs to `updateENV` for persistence.

{ "event_id": "event_8e2e1a956094992c", "topic_id": "consumer_ai_applications", "event_type": "Feature", "event_time": "2026-06-02T04:17:02Z", "title": "AnythingLLM applies unified SDK timeout and retry policy for OpenAI and Anthropic calls", "summary": "AnythingLLM monkey-patches the OpenAI and Anthropic SDK layers so that all provider calls go through a single, user-configurable timeout (default 10 minutes) with retries disabled, letting the underlying socket error surface faster instead of hanging indefinitely.", "contribution": "Unified SDK-level timeout and retry policy so LLM provider calls fail fast with a clear socket error instead of hanging for the default SDK retry/timeout window.", "impact": "Users of AnythingLLM-hosted LLM apps will see failed provider requests surface within minutes instead of hanging or silently retrying for long periods, making it easier to diagnose connectivity or provider-outage problems. Operators can now tune the timeout via environment variables for self-hosted deployments. The main risk is that disabling retries entirely (maxRetries=0) means any transient network glitch now immediately fails the request, so operators should monitor for a spike in timeout errors after upgrading and consider whether their network environment needs retries.", "maturity": "Code", "confidence": 0, "importance_score": 0.58, "risk_flags": [ "No Retries Means Transient Glitches Fail Immediately", "Timeout Value May Be Too Aggressive For Slow Providers" ], "evidence_count": 1 }