Proposal: Enabling BNB smart contracts to execute tamperproof SQL against Greenfield data

[ebruengwall]

OVERVIEW

Space and Time (SxT) is a Web3 data warehouse that introduces a novel approach for smart contracts to interact with off-chain data. Unlike traditional centralized databases, Space and Time’s decentralized node network allows smart contracts on BNB chain to execute low-latency SQL requests joining hundreds of terabytes of on-chain and off-chain data. This data can queried from Greenfield, as well as loaded into SxT directly via our indexer (or from Web2 systems off-chain). Today we index BNB Chain, Ethereum, and a wealth of other popular L1s/L2s, but we are currently adding indexing support and retrieval of data in Greenfield.

The most innovative feature of SxT is its ability to generate zk-Proofs for queries, which verifies the integrity of the SQL query execution and the underlying data. This allows smart contracts to not only retrieve datasets from Greenfield, but also run complex SQL aggregations, filters, joins, arithmetic, etc., against the data before returning it on-chain in a tamperproof / cryptographically secure manner. Proposing native integration with BNB Chain.

MOTIVATION
Space and Time can enable smart contracts on BNB chain not only to store and retrieve simple data from Greenfield but actually do SQL query processing on that data. By enabling SQL query processing on structured or semi-structured data in Greenfield, smart contracts can perform complex operations such as joins, aggregations, filters, sorts, arithmetic operations, and other standard SQL functions. This means that dApp developers can perform SQL operations off-chain, while smart contracts execute on-chain.

Similar to how Snowflake sits on top of AWS S3 for SQL processing, SxT can sit on top of Greenfield to process the data and return it to the smart contract. SxT’s advantage is that it allows for complex SQL operations on the data in Greenfield, which opens up opportunities for dApp developers to build more powerful decentralized applications.

The current limitation of smart contracts on BNB chain is that they can only store and retrieve simple data from Greenfield. To enable more complex operations such as filtering, sorting, aggregation, and arithmetic calculations, SxT offers a solution to process structured or semi-structured data in Greenfield using SQL queries. This allows for more efficient processing of data and opens up new possibilities for dApp developers to build sophisticated decentralized applications. By running SQL off-chain and executing smart contracts on-chain, SxT ensures that the data is processed efficiently and securely while offering more flexibility for developers to create robust and powerful dApps.

To give some examples to what dApp developers can accomplish:

• DeFi lending protocols can store and retrieve supplementary information about user activities on-chain.
• On-chain gaming can store large amounts of data on Greenfield, offloading it from the smart contract and enabling complex SQL queries to be run on that data.
• Social networks can store additional information about wallets, offloading it from the smart contract to Greenfield, and use SxT to retrieve all users that meet specific criteria, such as those who posted three times in a week.
• SxT’s indexed on-chain and off-chain data, both structured and semi-structured, empowers new use cases in gaming, DeFi, and social networks. For example, a DeFi protocol could query the system to show the total number of transactions a user has made in a specific liquidity pool.

CHALLENGE
The main challenge in implementing a data processing solution like Space and Time (SxT) in Smart Contracts is the need for additional layers of architecture complexity between Greenfield and the smart contracts. However, Space and Time is designed to minimize this complexity by indexing the structured and semi-structured data in Greenfield through a relational organization of the data. The challenge is to ensure the accuracy of the query results before they are put on-chain. This requires an external party to verify the accuracy of the queries. There are two possible solutions to this challenge: a decentralized middleware layer or the smart contract on BNB Chain can perform the verification. Space and Time proposes using the latter approach to make the process more native, seamless, low-latency, and affordable for BNB Chain users.

SOLUTION

Currently, a decentralized middleware layer sits between Space and Time and BNB Chain, verifying the zk-Proofs of SQL queries before coming to consensus off-chain and returning the query result to a smart
contract. However, we are proposing that the Space and Time team performs heavy R&D ourselves, to
tightly integrate our decentralized data warehouse directly with BNB validators (or execution layer, etc)
and remove the need for external, 3rd-party middleware layer altogether. This layer adds extra latency, transactions costs, architecture complexity, and trust properties.

Our collaboration with BNB directly would enable SxT to operate entirely on-chain, providing an
unprecedented level of transparency and trustlessness. This would allow developers to create even more
sophisticated and secure smart contracts that can execute complex queries against Greenfield data in a
low-latency, native, and secure manner.

Figure 1: Naïve approach with middleware layer between BNB Chain and SxT data warehouse nodes (le. side) vs proposed next-gen approach removing the middleware layer via native integration with BNB Chain validators.

Querying on-chain vs off-chain data

Let’s look at two types of access enabled by SxT for smart contracts on BNB Chain:

Off-chain Web2 data as well as on-chain BNB data indexed by SxT is :

1. loaded directly into SxT, then
2. commitments on data created by SxT (for Proof of SQL verification), then
3. data archived to Greenfield and made available for smart contracts to query

Greenfield data (created by a Smart Contract or Dapp) is

1. Indexed by SxT for commitment creation and queries against metadata or content addressing, and
2. Can be read directly into SxT at SQL runtime, when smart contracts on BNB chain request SQL execution against objects in Greenfield
3. Path of Smart Contract’s SQL execution against Greenfield Data

What does a BNB smart contract “SQL job” against Greenfield data look like?

First the smart contract issues an opcode or event on-chain which requests Greenfield data from SxT and points to the contentaddressed location, providing a SQL statement (or an SxT API endpoint which executes specific preconfigured SQL statements). Then, Space and Time reads in structured or semi-structured data from Greenfield, verifies content hashes, and executes the SQL request on that data, genera7ng a Proof of SQL. Finally, the payload (query result and zk-Proof) is handed off to BNB Chain validators to verify the proofs redundantly, come to consensus, and hand the payload (tamperproof query result) to the smart contract which requested it.

Figure 2: Architecture considerations for BNB Chain serving as consensus layer for zk-Proofs of SQL

CONCLUSION
Space and Time offers a powerful solution to process structured and semi-structured data in Greenfield using SQL queries, enabling more complex operations such as filtering, sorting, aggregation, and arithmetic calculations. By removing the need for an external middleware layer and tightly integrating with BNB validators, SxT can operate entirely on-chain, providing unique way to provide transparency and trustlessness. This new approach with SxT, opens up new possibilities for dApp developers to build sophisticated and secure smart contracts that can execute complex queries against Greenfield data in a low-latency, native, and secure manner. With the potential for decentralized applications in gaming, DeFi, and social networks, SxT’s capabilities are poised to drive innovation and growth in the blockchain space.

[ArnoB]

Hi,

Space and Time (SxT) is an innovative Web3 data warehouse that could revolutionize how smart contracts on BNB chain interact with off-chain data. The integration of low-latency SQL requests and zk-Proofs ensures secure and efficient data processing, making it a powerful tool for dApp developers.

The proposed native integration with BNB Chain validators to remove the middleware layer is ambitious and would provide greater transparency and trustlessness. Your examples of DeFi lending protocols, on-chain gaming, and social networks demonstrate the immense potential of SxT in the blockchain ecosystem.

I have a couple of follow-up questions:

1. How do you plan to handle the scalability of SxT, especially with the growing amount of data and increasing complexity of smart contracts in the future?
2. What measures will ensure end-user data privacy and security, particularly when dealing with sensitive information in DeFi applications or social networks?

I’m excited to see the impact of SxT on the BNB Chain and the future of decentralized applications. Great work!

[vgg]

The idea is interesting, but it seems to be more appropriate for the Greenfield validators - not BSC validators. This way it will be totally inside the Greenfield ecosystem. BSC’s smart contracts will still be able to send a job request through the relayer between BSC<->Greenfield validators, but everything will be performed within Greenfield. This is because BSC validators don’t have access to the data and it doesn’t make sense to couple the 2 systems together. Greenfield validators, however, do have access to the data, and already perform proof-of-availability verifications, so extending the functionality of Greenfield validators will increase added value for Greenfield dApps and rewards for Greenfield validators.

Let me know if this makes sense

[ebruengwall]

@ArnoB, sorry for the late response. And thank you for your great questions. Here is the information about scalability and privacy:

Scalability:

• SxT uses big terabyte-scale clusters to handle the growing amount of data.
• The clusters autoscale horizontally, meaning more compute is added to the cluster on the fly whenever possible.
• The decentralized network adds more clusters, meaning each cluster can handle tens of TBs.
• The entire network can handle petabytes, given the decentralized nature of having 100+ clusters working together around the globe.

Performance:

• SxT is a hybrid transactional, analytical processing (HTAP), which means it’s a low-latency in-memory cache for transactional queries that power apps and quick smart contract requests (OLTP). It also handles scalable analytics in the cluster (OLAP) analytic queries that combine large volumes of on-chain and off-chain data.

Indexing:

Additionally, SxT uses an enterprise-grade verifiable Indexer, which:

• Proves/double-checks that the events that are indexed are accurate.
• Grabs every blockchain data, every event from every transaction, and every block from every major chain.
• Or grabs only the events from the contracts the user cares about and loads them into their database environment in SxT.

By leveraging these capabilities, SxT can ensure the scalability and performance needed to handle the growing data and increasing complexity of smart contracts.

Privacy and security:

1. Decentralized auth with role-based access control (RBAC), where fine-tuned security configs can be set up for each dataset.

2. In-database encryption: SxT uses column-level encryption in its database. Before loading any data, the columns are encrypted to protect sensitive information. The information remains secure even when running SQL queries against the encrypted data.

For example, the “credit score” column can be encrypted, and SxT can still run a query against it, such as “Does wallet XYZ have a credit score greater than 650?” SxT can provide the answer “yes” or “no” (along with a zk-proof that the query was executed correctly) without ever revealing any credit scores or decrypting the credit score column.

3. zk-SQL (Proof of SQL): SxT enables Proof of SQL, which ensures that a query has been executed correctly without revealing the underlying data. This provides an additional layer of security for sensitive information in DeFi applications or social networks.

Like the previous example, SxT uses zk-proof to prove that the query was executed correctly to answer a question about a user’s credit score without revealing sensitive information. This ensures data privacy and security for sensitive information.

[ebruengwall]

Hi @vgg,

Thank you for your feedback. I understand that it may make sense for SxT to be an extension of a Greenfield validator’s service. However, we believe many use cases exist beyond just data in Greenfield. The historical state of Ethereum, tradefi markets, protocols on other chains, price feeds, and video gaming server data are just a few examples.

Assuming Greenfield validators do perform proof of storage verifications and have access to the data SxT can allow greenfield validators to run Proof of SQL. However, we propose connecting Proof of SQL natively to BSC validators because the data that smart contracts are querying against extends far beyond just Greenfield.

We are excited about Greenfield, which will be the primary use case for SxT. But also, extending the functionality of BSC validators to include SxT will increase added value on both Greenfield and BSC.

Thank you for your feedback, I’m happy to discuss further if you have any additional questions.