Decentralized Storage

How is data stored on the traditional internet?

The data storage behind the traditional internet(Web2) is predominantly servers—physical or virtual—in massive server farms or cloud platforms such as AWS or Google Cloud. What happens if the server farm is infected with a nasty virus and the entire platform goes offline? These are centralized, potential points of failure which we are “trusting” to stay online and continue to render service.

Decentralized web networks such as IPFS/Filecoin and Arweave solve this centralization problem by distributing the data to nodes and eliminate a central point of failure.

Web3 Storage

Among the many use cases for these great technologies, the most common are for storage of image, audio and video NFTs as well as deploying web dApps. Here are two protocols that specialize in decentralized data storage.

Arweave

Arweave is a distributed, permanent storage network. Built on top of the network sits the Permaweb, which is a decentralized, permissionless web. The two layers enable users and developers to:

– store data permanently

– deploy dynamic and permissionless dApps to the Permaweb

– write smart contracts in JavaScript, Rust, Solidity and more

– create profit sharing communities

How does Arweave work?

The workings of the Arweave protocol can be best understood by looking at the two layers involved: the Arweave network and the Permaweb.

The Arweave network

A P2P protocol that allows miners (nodes) with large storage spaces on their hard drives to connect with the network and store data for Arweave users. Arweave rewards these miners for offering their storage services through sustainable and perpetual endowments, coming from the user's one time payment to store data permanently. As the volume of data stored in a particular system increases, the amount of hashing needed for the consensus decreases, resulting in more cost-effective data storage.

The network uses a blockchain like structure called blockweave. The blockweave is different from the typical blockchain that many cryptos are built upon because apart from storing transactional information, the Arweave network can also store custom data on each block. Though similar to other blockchains whereby each block refers to its previous, Arweave attaches a randomly selected block known as the recall block to the transaction.

This Proof of Access consensus rewards the participating nodes in the network for storing all of Arweave’s transactional history. Say you want to manage a node on the Arweave blockchain. To receive block rewards, you must prove to blockweave that you have complete access to the data in the previous block, along with some randomly selected recall block. If you don’t have access to the recall block, the network will know that you aren’t storing all the transactional data on the Arweave network, and will thus deny you any block reward.

The Permaweb

Another main component in the Arweave network is the permaweb, a community-owned, decentralized web layer built on top of the Arweave network. Not only does the Permaweb host permissionless, decentralized and censor resistant applications permanently, it gives us the ability to create queryable transactions and smart contracts to build interactive and dynamic user interfaces with tools like GraphQL.

IPFS/Filecoin

The textbook definition: IPFS is a distributed system for storing and accessing files, websites, applications, and data. Filecoin is a peer-to-peer network that stores files, with built-in economic incentives to ensure files are stored reliably over time.

IPFS

The IPFS decentralized web is made up of all the computers connected to it, known as nodes. Nodes can store data and make it accessible to anyone who requests it.

If someone requests a file or a webpage, a copy of the file is cached on their node. As more and more people request that data, more and more cached copies will exist. Subsequent requests for that file can be fulfilled by any node or combination of nodes that has the file on it. The burden of delivering the data and fulfilling the request is gradually shared out amongst many nodes, so the more popular the file, the deeper the distribution of the file will be. For IPFS, caching recall is shorter-term, unless using a gateway provider such as https://www.pinata.cloud/, who provides over 200 caching locations allowing for speedy retrieval.

Enter Filecoin

For longer-term storage, this is where Filecoin comes in. Filecoin is a decentralized blockchain where miners are incentivized by payment in FIL, the protocol’s native cryptocurrency, to store medium files such as videos, audios, NFT images, NFT collections, scientific data. The Filecoin network allows for verification of storage for users by accessing “proofs” on the Filecoin blockchain.