---
title: WebGPU Cache API
tags: [asr, webgpu, client-side, caching, transformers-js]
last_updated: 2026-04-10
---

# WebGPU Cache API

The WebGPU Cache API strategy describes how the client-side ASR pipeline avoids redundant network downloads of ONNX model weights across page refreshes.

## Overview

The client uses `transformers.js` to run inference via **WebGPU**. Because ONNX model weights can be large, the system aggressively uses the **browser Cache API** to store downloaded weights locally. On subsequent page loads, the weights are served from the cache rather than re-fetched from the network.

## Why It Matters

- Eliminates startup latency on page refresh.
- Reduces bandwidth consumption significantly for repeat users.
- Enables faster time-to-first-word in the streaming pipeline, since the model is ready sooner.

## Relationship to Streaming

Once model weights are loaded from cache, inference begins and raw hypotheses flow into the [LocalAgreement-2](LocalAgreement-2.md) stabilization layer before being rendered in the UI.

## See Also

- [LocalAgreement-2](LocalAgreement-2.md) — streaming text stabilization running on top of WebGPU inference
- [Inference_Router_Strategy](../Architecture/Inference_Router_Strategy.md) — server-side counterpart for Cohere and Qwen routing
- [Chunked_Prefill](Chunked_Prefill.md) — server-side scheduling optimization