We’ve been working hard at Polybase to continue to innovate on our core architecture, and we’re excited to announce v2 of our whitepaper draft.

Here’s an extract from the whitepaper showing our new high-level architecture:

What’s Changed?

Modular storage and indexing protocol

Extraction of data and indexing functionality into a modular pluggable system. This allows other providers and protocols to work with Polybase, enabling new types of indexing and data storage use cases. External services/protocols can be created for the many different requirements and trade-offs (e.g. security, durability, and performance) required by different types of applications.

For storage, this could include local storage on the client's device (for ultra-sensitive data), a decentralized storage network such as Filecoin/Arweave (for collaborative data), or even a centralized storage provider such as S3.

For indexing, services could be created to serve, search, aggregation, or machine learning use cases. As the root state proofs can be verified, indexers and aggregators can build on top of the Polybase state proofs, to provide proof that the aggregation was performed correctly.

Polybase will continue to offer its own data storage and index provider so that everything still works out of the box, but we’re excited to offer this new level of flexibility.

Another benefit this modularity provides is reducing the scope for our first mainnet launch. Now we have separated the components we can first decentralize our core state rollup, and then decentralize the other components thereafter. That moves our decentralized mainnet launch forward by a quarter.

Computation on the client

We’ve moved the State Transition process and proof generation to the client.

This has a number of benefits. Firstly, because the client is creating the proofs we can move a large portion of the work from the Polybase nodes to the client. That dramatically reduces the cost of the service, whilst helping to improve performance and scalability.

Secondly, it enables massive improvements in privacy and censorship resistance. Because the computation is moved to the client, the data representing the transaction no longer needs to leave the device, and so the Polybase nodes have no knowledge of what is being transacted on the network. This can help prevent censorship of transactions (as happened with Tornado Cash on Ethereum), enhances privacy, limits front-running, and protects against attacks on specific datasets as an attacker will not know where a specific dataset resides.

More to come

We have some more excited changes coming in the near future, as we learn from our initial customers and continue to improve the protocol. Follow us on Twitter to stay up to date!