cryptopolitan.com
Storing files on the blockchain is a crucial topic of discussion among blockchain enthusiasts and developers. While it is technically possible to store files directly on the blockchain, it is not a practical approach due to high costs and limitations in storage capacity. Let’s explore alternative solutions for storing files in a decentralized and cost-effective manner using blockchain technology.
Why storing files directly on the blockchain is not feasible
Storing files directly on the blockchain may seem like a logical solution for decentralized file storage, but it is not a practical one. There are several reasons why storing files directly on the blockchain is not feasible.
First, storing files on the blockchain can be incredibly expensive. Blockchain transactions are processed using “gas,” which is a fee paid to the network to execute a transaction. The cost of storing a file on the blockchain would be proportionate to the size of the file. For example, storing a 1 MB file on the Ethereum blockchain would cost around $57,332.91 in gas fees alone at the time of writing of this article, which is not a viable option for most users.
In Ethereum, the block size is not determined by the maximum size limit, but rather by the gas limit. The gas limit is the maximum amount of gas (or energy units) a user needs to complete the transaction. A higher gas limit means more work is required to complete a transaction. This cost varies with the transactional workload on the Ethereum network.
For example, it takes about 640,000 gas to store 1 kilobyte of data in a block (at a standard gas price of 20,000). Ethereum has a current block limit of 15 million gas, which can be expanded to 30 million based on demand. Therefore, the block size in Ethereum can be up to 46 kilobytes, depending on the gas limit.
To provide an example of the high cost of storing files on the blockchain, let’s take the Ethereum blockchain as an example.
The Cost of Storing 1kb on Ethereum
Ethereum stores data in slots. Each slot is the size of 256 bits.
According to Ethereum’s yellow paper, It requires 20,000 gas to store a single slot of 256 bits.
8 bits make one byte, so one slot is 32 bytes (256/8= 32 bytes).
1kb equals 1024 bytes, so it has 32 slots (1024 bytes/32 bytes = 32 slots).
The amount of gas required to store 1kb is 32 slots*20,000 gas = 640,000 gas
To include the file in the call data transaction’s field, it will cost additional gas.
Each byte costs 16 gas so the total will be 32 slots * 32 bytes * 16 gas = 16,384 gas
The base gas fee for any transaction is 21,000 gas
So the total gas required to store 1kb = 640,000 + 16,384 + 21,000 = 677,384 gas
1 gas is equal to 0.00000005 ETH
So to store 1kb it costs 677,384 gas* 0.00000005 = 0.0338692 ETH
The current price of ETH is $1,787.57 so the cost of storing 1kb will be $60.66
The Cost of Storing 1MB on Ethereum
1MB = 1,000,000 byte
1,000,000 byte / 32 bytes = 31,250 slots
31,250 slots * 20,000 gas = 625,000,000 gas
31,250 slots * 32 bytes * 16 gas = 16,000,000 gas for call data
Base gas cost is 21,000 gas
Total gas = 625,000,000 + 16,000,000 + 21,000 = 641,021,000 gas
1 gas is equal to 0.00000005 ETH
So store 1MB it costs 641,021,000 gas * 0.00000005 ETH = 32.05105 ETH = $57,332.91
Clearly, this cost is prohibitive for most users and businesses, making storing files directly on the blockchain an unrealistic option. However, there are alternative methods for decentralized file storage that are more practical and cost-effective, which we will explore in the next section.
Storing files in a decentralized way
Decentralization is a fundamental concept in the blockchain world, and the same goes for file storage. Decentralized file storage ensures that data is not stored in one central location, which can be a single point of failure. In a decentralized storage system, data is distributed across multiple nodes or computers, making it more secure and less vulnerable to attack or manipulation. This distributed architecture also ensures that no single entity has complete control over the data, ensuring greater transparency and accountability.
In a decentralized file storage system, metadata plays a crucial role. Metadata is data that describes other data and provides context, making it easier to manage and locate specific files. In the context of decentralized file storage, metadata typically includes information about the file’s location, access rights, and other attributes. The metadata is stored on the blockchain, making it immutable and transparent, which ensures that the data cannot be tampered with.
While storing files directly on the blockchain is not feasible due to high costs and technical limitations, storing a document’s hash on-chain while keeping the whole document elsewhere is a practical solution. The document could be stored in a centralized database or on a distributed file storage system, while the hash is stored on the blockchain. By putting the document through a secure hash algorithm like SHA-256 and storing the hash in a block, the hash value becomes a unique digital fingerprint of the document. This approach saves a huge amount of space and cost, as hash values are far smaller than whole documents. Moreover, it ensures the integrity and authenticity of the original document, as any change in input would result in a completely new hash value, different from the original document.
Decentralized storage networks
Decentralized storage networks offer a way to store data in a distributed and secure manner. Unlike traditional centralized storage services, which store data on a single server or data center, decentralized storage networks store data on a global network of nodes. This makes them less susceptible to data breaches and data loss, as data is redundantly stored in multiple locations.
Examples of decentralized storage networks include Arweave, Filecoin, and Storj. These networks work by allowing users to rent out their unused storage space in exchange for tokens or other rewards. When a user wants to store a file, it is broken up into multiple pieces and distributed across the network of nodes. The data is encrypted and replicated across multiple nodes, ensuring that even if one node fails, the data is still available. Let’s explore Arweave and Filecoin more in the following sections.
Arweave
Arweave’s innovative approach to decentralized storage is called the permaweb, a permanent and decentralized web built on top of the Arweave. The permaweb consists of a set of modular and interchangeable protocols. It addresses the critical flaw of the traditional web, which is the lack of memory. When documents are removed from one server in the network, they are lost from the knowledge base. The permaweb on Arweave allows developers and users to share a resilient knowledge base that is replicated in vast numbers across the world, perpetually.
The base protocol of the permaweb is Arweave, which is powered by two fundamental components: a scalable blockchain-like structure that uses a new mining system and a sustainable endowment to ensure the availability and permanence of arbitrarily-tagged data. This approach makes it possible to store large amounts of data in a decentralized manner without the high costs associated with blockchain storage. On top of the Arweave protocol is a decentralized network of gateway servers that makes the data easily accessible to web browsers without the need for modification or specialist software.
By combining these elements, the permaweb on Arweave provides a secure and decentralized storage solution that is designed to last for centuries. Unlike centralized storage solutions, which can be vulnerable to hacking and data breaches, the permaweb provides a resilient and immutable storage system that is resistant to tampering and censorship. It’s no wonder that Arweave is quickly gaining recognition as one of the most promising decentralized storage networks on the market today.
Filecoin
Filecoin is a unique decentralized storage network that offers built-in economic incentives and cryptography to ensure files are stored reliably over time. This system is built on top of the InterPlanetary File System (IPFS) protocol, which uses content addressing to allow permanent references to data and avoids relying on specific devices or cloud servers for addressing content. Filecoin extends the capabilities of IPFS by adding an incentive layer on top, incentivizing content to be reliably stored and accessed.
Filecoin operates through a peer-to-peer network in which users pay to store their files on storage providers, which are computers responsible for storing files and proving they have stored them correctly over time. The available storage and the price of that storage are not controlled by any single company. Instead, Filecoin facilitates open markets for storing and retrieving files in that anyone can participate in. This ensures that the storage costs are distributed evenly across the network, with no central authority controlling the prices.
Filecoin has multiple use cases, including Web3 native NFT and metaverse/game assets storage, incentivized permanent storage, and archiving Web2 datasets as a cheaper alternative to cloud storage. For instance, NFT. Storage uses Filecoin to provide a simple decentralized storage solution for NFT contents and metadata, while Shoah Foundation and Internet Archive leverage Filecoin to back up their contents. Moreover, Filecoin supports a wide range of formats of data, including audio and video files, allowing Web3 platforms such as Audius and Huddle01 to leverage Filecoin as the decentralized storage back end for music streaming and video conferencing.
Best practices for decentralized file storage
When it comes to decentralized file storage, there are several best practices that should be followed to ensure data reliability, privacy, and security.
Firstly, it is recommended to store the metadata on the blockchain, while storing the actual file in a decentralized storage network like Arweave or Filecoin. This way, the metadata can be publicly accessible and verifiable on the blockchain, while the actual file is stored in a secure and decentralized manner.
Secondly, data redundancy is crucial to ensure that the data is always available and reliable. By storing multiple copies of the file in different locations, the risk of losing the data due to a single point of failure is greatly reduced. Decentralized storage networks often have built-in redundancy mechanisms to ensure data is always available.
Thirdly, protecting data privacy and security is of utmost importance. Encryption should be used to protect the data while in transit and at rest. Access to the data should also be controlled to ensure that only authorized parties can access it. Decentralized storage networks often offer built-in encryption and access control mechanisms.
Best practices for decentralized file storage include storing metadata on the blockchain, utilizing a decentralized storage network for the actual file, ensuring data redundancy for increased reliability, and protecting data privacy and security. By following these best practices, users can ensure that their data is stored in a secure and reliable manner.
Conclusion
While it is possible to store files directly on the blockchain, the high cost and limitations in the amount of data that can be stored make it an impractical approach. Decentralized file storage networks such as Arweave and Filecoin offer a much more reliable and cost-effective method for storing files in a decentralized way. By utilizing a combination of storing metadata on the blockchain and the actual file on a decentralized storage network, data redundancy can be ensured for increased reliability. It is also important to prioritize data privacy and security in the storage process.
As the blockchain industry continues to grow, decentralized file storage is becoming more and more prevalent. It is important to understand the limitations of traditional file storage methods and to explore and utilize the various decentralized storage networks available for secure and reliable file storage. With the right approach and tools, decentralized file storage can provide a more robust and secure storage solution for individuals and businesses alike.