JetStream 3.0: 10 Key Innovations in Browser Benchmarking

In a joint announcement with Google and Mozilla, we introduced JetStream 3.0—a major update to the cross-browser benchmark suite. While the shared announcement covers the collaborative effort, this article dives deeper into the WebKit team's approach and the engineering work in JavaScriptCore. Benchmarks drive performance improvements, but the web evolves, and old benchmarks become outdated. JetStream 3 represents both a refresh and a fundamental shift in measuring performance, especially for WebAssembly and modern web applications. Here are ten things you need to know about this landmark release.

1. A Major Refresh for Modern Web

JetStream 3.0 is not just a minor update; it’s a comprehensive overhaul of the benchmark suite. The previous version, JetStream 2, was released when WebAssembly was still young. Since then, browser engines have advanced rapidly, and the workloads on the web have grown more complex. This refresh ensures that the benchmarks accurately reflect the performance challenges of today’s web applications, from complex JavaScript frameworks to real-time WebAssembly modules used in libraries, image decoders, and UI components.

JetStream 3.0: 10 Key Innovations in Browser Benchmarking — Source: webkit.org

2. WebAssembly Takes Center Stage

One of the most significant changes in JetStream 3 is how we measure WebAssembly (Wasm) workloads. In JetStream 2, Wasm was scored in two separate phases: Startup and Runtime. This distinction made sense when large C/C++ applications dominated Wasm usage, tolerating long startup times for high throughput. But today, Wasm is in the critical path for many page loads, requiring fast startup and efficient execution. JetStream 3 integrates these phases into a unified score that reflects real-world performance.

3. The Infinity Problem

As browser engines optimized WebAssembly instantiation, startup times shrank dramatically. WebKit, for instance, reduced startup effectively to zero milliseconds for some smaller workloads. In JetStream 2, each iteration’s time was measured with Date.now(), which rounds down—so any sub-1 ms time became 0 ms. The scoring formula Score = 5000 / Time then produced an infinite score. While this sounds like a victory, it signified that the benchmark was no longer useful for measuring improvements.

4. Clamping Scores: A Temporary Fix

To salvage JetStream 2, the benchmark harness was patched in version 2.2. The fix clamped the per-test score to a maximum of 5,000, preventing infinite scores from rendering other results meaningless. This patch was a stopgap measure—it acknowledged that the benchmarks had been outgrown. Yet it also highlighted the need for a more robust system, leading directly to the design of JetStream 3’s new scoring methodology that avoids such pathological cases.

5. Outgrowing Old Benchmarks

The infinite score problem was a clear signal that browser engines had surpassed JetStream 2’s Wasm subtests. When the most accessible optimizations have been exhausted, subsequent optimizations become less general and more specific to the exact workload. JetStream 3 was designed to reset the baseline, using larger, more diverse workloads that challenge engines in ways that reflect current web development practices, ensuring that future performance gains are meaningful and broadly applicable.

6. Real-World Workloads at Scale

JetStream 3 includes benchmarks that mimic the scale of modern web applications. Instead of isolated microbenchmarks, the suite uses comprehensive simulations of actual scenarios: loading full web pages, processing images, and executing Wasm modules in sequence. This shift ensures that performance metrics correlate directly with user experience. For example, a Wasm image decoder is now tested not just on decoding speed but also on how quickly it integrates with the rest of the page lifecycle.

7. Collaborative Effort: WebKit, Google, and Mozilla

JetStream 3 is the result of a cross-industry collaboration among browser vendors. Engineers from WebKit, Chrome’s V8 team, and Firefox’s SpiderMonkey worked together to design benchmarks that are fair and representative. This partnership ensures no single engine is advantaged and that the suite remains relevant as all browsers evolve. The shared ownership also means faster updates and a more transparent development process, with input from the wider web community.

8. Engineering in JavaScriptCore

The WebKit team invested heavily in JavaScriptCore improvements to excel in JetStream 3. Key optimizations include a new WebAssembly instantiation pipeline that reduces overhead, faster compilation tiers, and better memory management for long-running Wasm modules. These changes were guided by the benchmark’s emphasis on startup and runtime integration. The result is a JavaScriptCore that not only scores well but also delivers tangible performance benefits for real websites using WebAssembly.

9. Benchmarking Startup vs. Runtime: Now Unified

In JetStream 2, the separate scoring of Startup and Runtime could mislead developers into optimizing one at the expense of the other. JetStream 3 unifies these metrics into a single composite score that reflects the total user-perceived performance. This change encourages balanced optimization: improving startup without degrading runtime, and vice versa. The scoring formula now uses geometric mean of normalized times, avoiding the infinity problem and providing a more stable ranking.

10. Future-Proofing the Benchmark Suite

JetStream 3 is designed to evolve. The suite includes a modular structure that allows new workloads to be added as the web changes. The team has also built in automated regression detection to catch performance regressions early. By keeping the benchmarks aligned with real-world usage, JetStream 3 will remain a valuable tool for years. This forward-looking approach ensures that browser engines continue to optimize for the web of tomorrow, not just the web of yesterday.

Conclusion

JetStream 3.0 marks a pivotal moment in browser performance benchmarking. By addressing the shortcomings of its predecessor—most notably the infinity problem and outdated Wasm metrics—it provides a more accurate and actionable picture of browser capability. The collaborative effort behind it guarantees that the suite meets the needs of all major engines, while the focus on real-world workloads ensures that improvements translate directly to better user experiences. For developers and browser engineers alike, JetStream 3 is the new standard for measuring what matters most: performance on the modern web.

Tags: