WebAssembly (Wasm) is an open standard that enables the execution of binary code on the web. This format allows developers to leverage the performance of languages like C, C++, and Rust in web development. Currently, it is primarily used for demanding operations in the browser. For instance, Figma is powered by Wasm.

I recently discovered that Wasm is also being used outside of browsers. Companies like Vercel, Fastly, Shopify, and Cloudflare support running Wasm code at the edge.

Fourth Language of the Web

WebAssembly is considered the fourth language of the web, alongside HTML, CSS, and JavaScript. Browsers constantly render as they don’t have a concept of windowing. With WebAssembly, you use binaries to bypass HTML, resulting in faster performance, as demonstrated by Figma’s approach for Photoshop. Another example is Quadratic - The Data Science Spreadsheet, which scales spreadsheets using WebAssembly. Shawn Swyx Wang

An exciting development is Docker exploring WebAssembly as an alternative to Linux containers. It will be interesting to see how this evolves! Docker+Wasm (Beta) | Docker Documentation

For a great explanation of why WebAssembly is so powerful, check out this video: Rust & Wasm (Safe and fast web development) [RUST-10] - YouTube

Usually, databases are abstracted away and take longish roundtrips to 1. send the query 2. fetch the data 3. send it back to the network. What if there’s another way, directly in your application? This is where WebAssembly, an open standard that enables the execution of binary code, comes into play, and this blog is all about. As DuckDB is a single binary, we have now the opportunity to place the full database and its data inside the browser. This eliminates any latency on the network or round trip, plus with DuckDB we have super fast response times, even, or especially for analytical data queries. Some use cases can be

Ad-hoc queries on data lakes, such as schema exploration or data previews.Dynamic querying in dashboards by adjusting filters on the fly.Educational tools for SQL learning or in-browser SQL IDEs.

Check more on What Happens When You Put a Database in Your Browser? and make sure to check the source code too, if this sparked your interest.

Below are features from WebAssembly.

The Wasm stack machine is designed to be encoded in a size- and load-time-efficient binary format. WebAssembly aims to execute at native speed by taking advantage of common hardware capabilities available on a wide range of platforms.

WebAssembly describes a memory-safe, sandboxed execution environment that may even be implemented inside existing JavaScript virtual machines. When embedded in the web, WebAssembly will enforce the same-origin and permissions security policies of the browser.

WebAssembly is designed to be pretty-printed in a textual format for debugging, testing, experimenting, optimizing, learning, teaching, and writing programs by hand. The textual format will be used when viewing the source of Wasm modules on the web.j

WebAssembly is designed to maintain the versionless, feature-tested, and backward-compatible nature of the web. WebAssembly modules will be able to call into and out of the JavaScript context and access browser functionality through the same Web APIs accessible from JavaScript. WebAssembly also supports non-web embeddings.

Popular applications like Figma use Wasm to run complex software written in languages such as C++ or Rust directly in the browser. This allows for fast, lightweight applications that are easy to deploy.

Reading the schema of the parquet is not optimal, as you need to download the file and read it with a notebook or similar.

But with DuckDB and WASM, you can do that entirely inside the browser with a mouse hover, e.g. Christophe Blefari did a POC with Google BigQuery to do exactly that:

DuckDB Experiments: Peeking into the Future of Analytics ft. Christophe Blefari - YouTube

Count-co uses DuckDB with WASM when your database can’t keep up. They used querying locally with DuckDB. With the full integration of DuckDB-WASM into the canvas, you have the option to execute any cell or visual on your database or locally in the browser. This flexibility allows you to get results faster by eliminating latency—DuckDB Labs has developed one of the quickest engines available. You can also visualize hundreds of thousands of points interactively and instantaneously, join your CSV files with database tables or other sources seamlessly, and reduce your database load by executing more queries in the browser. For more details on how our unique query architecture can optimize your team’s data exploration, check out the full article on How we evolved our query architecture with DuckDB. Origin.

Tweet

Experimenting with DuckDB and Wasm—running DuckDB in the browser. The results are impressive: reading a 500 MB public parquet file and performing a groupby operation in less than a second. If this isn’t the future of data, what else could it be?

More on What has WebAssembly (WASM) to do with DuckDB and data engineering and [[Newsletter MotherDuck#[WASM What Happens When You Put a Database in Your Browser?](https //motherduck.com/blog/olap-database-in-browser/)]].

MotherDuck WASM SDK

See Evidence.

LakeFS has another integration with DuckDB WASM for ad-hoc queries within their Web UI to enhance performance.

1-5-Tier Architecture

TigerBeetle

Our F3 File Format files embed small WASM programs to decode data. If somebody creates a new encoding and the DBMS does not have native impl, it can still read data using WASM passing Arrow buffers. Our experiments show WASM is 15-20% slower than native. We use SpiralDB’s Vortex encoding impls. Bsky

Origin:
References:
Created 2022-10-20