Timeline history

GPU Supply and Compute Market timeline history

Review how this topic changed over time through source-backed timeline events.

Timeline history

All tracked timeline events

85 total timeline events

Jun 2, 2026, 8:29 PM
dependency update
wandb Go SDK bumps go-git to 5.19.1 for packfile and worktree hardening
The wandb experimental Go SDK upgrades go-git from v5.19.0 to v5.19.1, pulling in packfile parser hardening, format input bounds validation, and worktree filesystem safety fixes.
ContributionUpgrades go-git to v5.19.1, which tightens delta chain depth limits, bounds format decoder inputs, validates idxfile offsets, and adds a worktreeFilesystem wrapper to reduce path-traversal and malformed-integer risks in the Go SDK's git operations.
ImpactDevelopers and CI pipelines using wandb's experimental Go SDK are less likely to hit panics or data-corruption paths when the SDK clones or inspects repositories, because the new go-git caps delta chain depth, rejects malformed variable-length integers, and bounds inflate sizes that previously could be exploited or cause uncontrolled memory growth. Watch for regressions in deeply-nested submodule cloning and SSH path-handling edge cases since the upstream release also changed symlink target validation and submodule relative-URL resolution.
Jun 2, 2026, 6:47 PM
pull request
Diffusers adds AWS Trainium/Inferentia as officially supported compute backend
The Diffusers library now treats AWS Neuron chips (Trainium and Inferentia) as a first-class compute backend alongside CUDA, MPS, XPU, and MLU, adding device registration, backend dispatch, and pipeline methods for eager-mode inference on these accelerators.
ContributionRegisters Neuron in all Diffusers backend dispatch tables and provides enable_neuron_compile() and neuron_warmup() pipeline methods for compiling and warming up models on Trainium/Inferentia hardware.
ImpactUsers of the Diffusers library can now run popular image generation models such as SDXL and PixArt on AWS Trainium and Inferentia chips without patching internals, opening a lower-cost inference option outside GPU clouds. Operators choosing AWS inf2 or trn instances get official support and warmup utilities that front-load neuronx-cc compilation so production latency is predictable. The initial validation covers only three models under eager mode; torch.compile support, tensor parallelism, and video-model sequence parallelism remain on the roadmap, so coverage gaps and performance regressions on other architectures need continued monitoring.
Jun 2, 2026, 12:49 PM
feature
LocalAI adds dynamic batching for GPU-bound audio transcription
LocalAI's parakeet-cpp backend now coalesces concurrent audio transcription requests into a single batched call, raising GPU utilization under load.
ContributionEnables dynamic batching for concurrent audio transcription requests to increase GPU throughput.
ImpactOperators running LocalAI for speech-to-text under concurrent load can get up to 1.84x higher transcription throughput on GPU by enabling dynamic batching, reducing the need for more GPUs to handle the same request volume. The feature requires raising the `batch_max_size` parameter and depends on a specific version of the parakeet.cpp library. Continued monitoring is needed to see how batching behaves with different model sizes, GPU architectures, and under extreme concurrency to ensure stability and predictable latency.
Jun 2, 2026, 12:47 PM
infrastructure change
TensorFlow ROCm CI image updated to ROCm 7.2.3
The CI infrastructure for TensorFlow on AMD ROCm was updated to use a newer container image, rocm/tensorflow-build:2.22-jammy-pythonall-rocm7.2.3-ci_official.
ContributionUpdates the AMD ROCm continuous integration environment for TensorFlow to the latest stable ROCm version.
ImpactTensorFlow developers targeting AMD GPUs will have a more reliable CI environment that matches the latest ROCm 7.2.3 release, reducing integration drift and potential build/test failures specific to AMD hardware. Operators and developers of AMD-based AI infrastructure should monitor subsequent CI stability to ensure the new image does not introduce regressions in testing or deployment pipelines.
Jun 2, 2026, 11:10 AM
pull request
ROCm collective latency model extended to AllGather and ReduceScatter
The analytical GPU-collective cost model in ROCm/XLA now covers AllGather and ReduceScatter operations instead of only AllReduce, with new tests mirroring the existing AllReduce case.
ContributionExtended the analytical collective performance model to AllGather and ReduceScatter operations and added corresponding ROCm-specific unit tests.
ImpactDevelopers and operators running distributed workloads on AMD GPUs get more accurate latency predictions for AllGather and ReduceScatter collectives, which can lead to better auto-sharding decisions and fewer performance surprises in multi-GPU training. The main follow-up is whether the new model predictions actually match real-world AMD hardware timings under different cluster sizes and communication patterns.
Jun 2, 2026, 9:45 AM
build configuration change
PaddlePaddle disables DeepEP GPU, DeepGemm, and NVSHMEM for non-XPU builds
The PaddlePaddle framework disables compilation of the DeepEP GPU collective communication library and the DeepGemm FP8 library for all non-XPU build targets, and forces NVSHMEM to be off since its only consumer was the now-retired DeepEP GPU.
ContributionReduces build complexity and potential fragility for non-XPU targets by removing optional GPU-specific collective communication and FP8 matrix multiplication libraries that were not actively used or maintained.
ImpactDevelopers building PaddlePaddle for standard GPU or CPU environments will have a simpler and potentially faster build process with fewer optional dependencies. Operators running Paddle on XPU hardware are unaffected as the DeepEP XPU path is preserved. The change signals a potential de-prioritization or internal archiving of the non-XPU DeepEP and DeepGemm paths within PaddlePaddle's mainline, which may affect researchers or integrators relying on those specific GPU acceleration features; further watch should confirm if alternative optimized communication or FP8 paths are provided or if these features are considered unsupported for standard GPU builds.
Jun 2, 2026, 4:20 AM
pull request
Milvus optimizes GC path for dropped segment indexes in 2.6 branch
Milvus backports a change to its 2.6 branch that integrates dropped segment index cleanup directly into the segment recycle path, ensuring index files and metadata are cleaned up together and reliably retried on failure.
ContributionConsolidates dropped segment index cleanup into the main segment recycle path with atomic, retryable steps to prevent orphaned index files and metadata.
ImpactOperators of Milvus vector databases on the 2.6 release line get more reliable storage reclamation after segment drops, reducing the risk of accumulating orphaned index files that waste disk space and complicate backups. This is a stability and operational cost improvement for AI infrastructure using Milvus as a retrieval backend. The next step is to monitor whether the retry mechanism sufficiently handles all failure modes in production under load.
Jun 2, 2026, 2:51 AM
pull request
SGLang parallelizes DeepGEMM JIT warmup across pipeline-parallel ranks to cut large-model startup time
SGLang adds per-PP-rank parallel compilation during startup warmup so all pipeline stages trigger DeepGEMM JIT compiles concurrently instead of sequentially, reducing large MoE model startup from roughly 9 minutes to under 4 minutes on multi-GPU setups.
ContributionParallelizes DeepGEMM JIT kernel compilation across pipeline-parallel ranks during model startup warmup, eliminating serial stage-by-stage compilation bottlenecks for large mixture-of-experts models.
ImpactOperators running large MoE models like DeepSeek-V4 across multiple GPUs can cut startup time by more than half, meaning fewer idle GPUs during scaling events and faster recovery after redeployments. The optimization is gated behind an environment variable (SGLANG_PP_PARALLEL_DEEPGEMM_WARMUP) and falls back gracefully on failure, but the concrete speedup depends on PP size and cache state, so teams should benchmark their specific cluster topology before assuming uniform gains.
Jun 1, 2026, 10:05 PM
optimization
Ray autoscaler v2 eliminates expensive debug logging for a 27x speedup in large-scale task scheduling
The Ray autoscaler v2's debug logging, which was proven to dominate a 20,000-task scheduling pass, is now skipped by default unless the debug level is explicitly enabled, cutting that operation's time from ~5.3 seconds to ~0.19 seconds.
ContributionConditionally skips expensive serialization in the autoscaler's debug logging, removing a major performance bottleneck in large task scheduling passes.
ImpactUsers and operators running large-scale Ray workloads with 20,000 or more tasks will see their autoscaler respond dramatically faster, reducing scheduling latency by over 27 times and allowing compute resources to be utilized more efficiently. The change achieves this by eliminating the costly serialization process (to_dict_list) for debug logs unless debugging is explicitly turned on. The main thing to watch is whether the default-off behavior for debug logging causes any unexpected issues when operators need to debug the autoscaler's scheduling decisions.
Jun 1, 2026, 5:44 PM
code change
Restored ROCm multi-GPU build flag for Windows in Ollama
This pull request reintroduces a previously dropped ROCm build flag that enables multi-GPU support on Windows, specifically by fixing the amdhip64_*.dll version detection for the 'latest' selection, ensuring AMD GPU users can reliably use multiple GPUs for local AI inference.
ContributionRe-enables multi-GPU ROCm inference on Windows by restoring a critical build flag and fixing dynamic library version selection.
ImpactWindows users with multiple AMD GPUs can now run Ollama without the multi-GPU feature silently breaking, which prevents crashes or fallback to slower single-GPU execution. Operators must continue to monitor for regressions in ROCm library detection across future Windows updates and AMD driver versions, as the 'latest' selection logic is sensitive to external changes.
May 29, 2026, 12:08 PM
release
llama.cpp adds pre-compiled binaries for CUDA 13.3 and ROCm 7.2
The b9405 release of llama.cpp introduces official pre-compiled binaries for Windows with CUDA 13.3 and Ubuntu with ROCm 7.2, while temporarily disabling SYCL and KleidiAI builds, aligning the inference engine with the latest GPU compute stacks.
ContributionProvides out-of-the-box inference binaries for the latest NVIDIA and AMD compute platforms and updates OpenVINO, reducing setup friction for users on newer hardware while pausing support for specific Intel and Apple optimization backends.
ImpactDevelopers and operators using the latest NVIDIA or AMD GPUs can now deploy llama.cpp using official pre-compiled binaries without building from source, avoiding complex toolchain setup and driver mismatches. This release adds CUDA 13.3 and ROCm 7.2 builds alongside OpenVINO 2026.0, but disables SYCL and KleidiAI backends, which may disrupt existing Intel XPU or optimized Apple Silicon workflows until those builds are restored.
May 29, 2026, 11:26 AM
code optimization
vLLM reduces ROCm decode latency for DeepSeek-V3.2 via CPU dispatch micro-optimizations
vLLM merged three CPU-side dispatch optimizations for the DeepSeek-V3.2 AITER decode path on AMD MI355X GPUs, eliminating redundant dtype casts, caching deterministic MLA metadata schedules, and trimming buffer fill operations to cut average decode step time by 3.1%.
ContributionEliminates 58 bf16-to-fp32 cast kernel launches per step, caches MLA metadata computation to skip 94% of redundant schedule generations, and replaces full-buffer zero-fills with targeted tail operations on the CPU dispatch path.
ImpactOperators running DeepSeek-V3.2 on AMD MI355X GPUs will see measurably faster token generation and lower CPU overhead, directly reducing per-request latency in high-concurrency production deployments. By removing unnecessary dtype conversions and caching deterministic attention metadata on the host side, the change trims ~1.3 ms per decode step without altering GPU kernels or model accuracy. Teams should monitor whether the CPU-side fingerprinting cache introduces edge-case misses under highly variable sequence lengths or dynamic batching, and verify that the bf16 router dispatch remains numerically stable across diverse prompt distributions.
May 29, 2026, 11:08 AM
bugfix
Fix AITER MXFP4 MoE weight loading and shuffle for DeepSeek-V4 on ROCm
vLLM corrected three weight loading and shuffling bugs in the AITER MXFP4 MoE backend that previously caused garbled outputs for DeepSeek-V4 on AMD gfx950 GPUs, restoring numerical correctness and unlocking significant latency and throughput gains.
ContributionCorrects tensor-parallel weight offset calculations, removes an incorrect de-interleave step for standard checkpoints, and aligns weight shuffling functions with AITER FlyDSL kernel expectations, enabling stable MXFP4 quantized MoE inference.
ImpactOperators running DeepSeek-V4 on AMD MI300-series GPUs can now deploy MXFP4-quantized models without encountering garbled text, reducing time-to-first-token by 46% and increasing request throughput by 34%. The fix resolves tensor-parallel offset miscalculations, checkpoint layout mismatches, and incorrect kernel shuffling routines that previously broke the AITER MoE backend. Teams should monitor GSM8K accuracy stability across different tensor-parallel sizes and watch for potential regression when switching between triton_unfused and aiter backends on other MoE architectures.
May 29, 2026, 11:04 AM
code change
Native W4A16 kernel for AMD RDNA3 enables fast bf16 quantized inference
vLLM now includes a native HIP W4A16 linear kernel optimized for AMD RDNA3 GPUs, replacing the slow Triton fallback and the fp16-only ExLlama path with a high-throughput implementation that natively supports bf16 and fp16.
ContributionImplements a device-specific HIP GEMM kernel that uses scalar dot-products for decode and WMMA instructions for prefill, eliminating the need to downcast bf16 models to fp16 and bypassing Triton's overhead on consumer AMD GPUs.
ImpactOperators running quantized LLMs on consumer AMD Radeon GPUs can now serve bf16 models at production speeds without encountering numerical instability or falling back to unusably slow fallbacks. By providing a native HIP kernel that handles bf16 natively at 2.5–4.2× the throughput of the previous Triton path, this change removes the trade-off between speed and output quality for RDNA3 deployments. Users no longer need to manually cast weights to fp16, which previously caused inf/NaN propagation and garbled tool-calling outputs under load. Teams should monitor register pressure and WMMA dispatch behavior on varying batch sizes, as the kernel currently restricts WMMA acceleration to bf16 and relies on scalar paths for fp16 decode.
May 29, 2026, 10:28 AM
release
vLLM introduces multi-tier KV cache offloading to extend serving capacity beyond CPU memory
vLLM v0.22.0 adds a multi-tier KV cache offloading framework that moves cached attention states to secondary storage like local filesystems or Mooncake disk tiers, allowing operators to serve longer contexts or larger batches without exhausting GPU or CPU RAM.
ContributionImplements a pluggable multi-tier KV cache management system that automatically spills and reloads attention key-value blocks to disk or network storage when GPU/CPU memory is constrained, with explicit support for DeepSeek V4 and Python filesystem backends.
ImpactOperators running long-context or high-throughput LLM services can now handle larger batches and longer prompts without hitting out-of-memory crashes or paying for additional GPU nodes. By spilling idle KV cache blocks to local disks or distributed storage like Mooncake, the engine reduces peak RAM pressure and keeps more requests in flight, though disk I/O bandwidth and latency will become the new bottleneck to monitor. Teams should watch for increased tail latencies during cache reloads under heavy churn, verify filesystem throughput matches their concurrency targets, and test the fallback behavior when secondary storage fills up or experiences network partitions.
May 29, 2026, 12:58 AM
pull request
[CINN] Optimize GroupSchedule for CustomDevice hardware like Iluvatar and MetaX
GPU Supply and Compute Market showed a tracked change with evidence attached, making the topic easier to monitor over time.
ContributionAdds evidence to the topic's change timeline.
ImpactHelps teams decide whether this direction deserves continued tracking.
May 28, 2026, 9:40 AM
developer tooling update
llama.cpp adds Dockerfile for ZenDNN backend to standardize AMD inference environments
llama.cpp introduced a dedicated Dockerfile for the ZenDNN backend, isolating dependencies and providing a reproducible containerized runtime for AMD-based development and benchmarking workflows.
ContributionProvides a pre-configured container image that eliminates manual dependency resolution for ZenDNN, enabling developers and benchmarkers to spin up consistent AMD inference environments without host-level configuration conflicts.
ImpactDevelopers and operators targeting AMD hardware can set up reproducible inference benchmarks faster without wrestling with host-level dependency conflicts. By containerizing the ZenDNN backend, the change isolates complex build requirements and ensures consistent runtime behavior across different AMD systems. Teams should monitor whether this Docker setup gains traction in CI pipelines or reveals hidden compatibility gaps when ZenDNN updates its underlying libraries.
May 27, 2026, 3:55 AM
release
Fix QNN export compatibility with built-in onnxruntime-qnn provider wheels
Ultralytics v8.4.56 resolves export failures on Linux x86-64 by detecting whether ONNX Runtime includes QNNExecutionProvider internally, avoiding plugin registration crashes and supporting both plugin-based and built-in QNN wheel packaging.
ContributionImproved QNN export robustness by dynamically detecting provider packaging style and adapting session initialization accordingly
ImpactEdge deployment engineers using YOLO models with Qualcomm QNN pipelines experience fewer export-time crashes and smoother integration across different `onnxruntime-qnn` releases — particularly on Linux — reducing manual debugging and environment-specific workarounds. Operators should monitor if future QNN runtime versions change internal provider detection logic, which could reintroduce compatibility gaps.
May 22, 2026, 8:05 PM
release
Metal GPU backend for Apple Silicon (IndexFlat)
FAISS added native Metal GPU backend support for Apple Silicon, starting with IndexFlat, enabling faster similarity search on M-series Macs without requiring CUDA or ROCm.
ContributionNative GPU acceleration via Metal for Apple Silicon devices, specifically enabling IndexFlat to run on M-series GPUs.
ImpactDevelopers using macOS with Apple Silicon can now run FAISS similarity search on GPU without relying on CUDA or ROCm drivers — reducing setup complexity and unlocking performance gains for local inference workloads. Operators deploying FAISS on M1/M2/M3 Macs will see lower latency and better throughput for vector search tasks. Watch for stability in multi-threaded or mixed-CPU/GPU scenarios and ensure compatibility with existing code that assumes CUDA backend.
Jan 31, 2026, 7:13 AM
release
New GPU Supply and Compute Market signal is ready for review
GPU Supply and Compute Market showed a tracked change with evidence attached, making the topic easier to monitor over time.
ContributionAdds evidence to the topic's change timeline.
ImpactHelps teams decide whether this direction deserves continued tracking.